-- Logs begin at Thu 2022-05-19 13:33:25 UTC, end at Thu 2022-05-19 15:07:13 UTC. -- May 19 14:09:05.969809 dv-cinder-842541-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... May 19 14:09:05.977560 dv-cinder-842541-1-iscsi devstack@keystone.service[80189]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini May 19 14:09:05.984824 dv-cinder-842541-1-iscsi devstack@keystone.service[80189]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu May 19 14:09:05 2022] *** May 19 14:09:05.984824 dv-cinder-842541-1-iscsi devstack@keystone.service[80189]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 May 19 14:09:05.984824 dv-cinder-842541-1-iscsi devstack@keystone.service[80189]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 May 19 14:09:05.984824 dv-cinder-842541-1-iscsi devstack@keystone.service[80189]: nodename: dv-cinder-842541-1-iscsi May 19 14:09:05.984824 dv-cinder-842541-1-iscsi devstack@keystone.service[80189]: machine: x86_64 May 19 14:09:05.984824 dv-cinder-842541-1-iscsi devstack@keystone.service[80189]: clock source: unix May 19 14:09:05.984824 dv-cinder-842541-1-iscsi devstack@keystone.service[80189]: pcre jit disabled May 19 14:09:05.984824 dv-cinder-842541-1-iscsi devstack@keystone.service[80189]: detected number of CPU cores: 4 May 19 14:09:05.984824 dv-cinder-842541-1-iscsi devstack@keystone.service[80189]: current working directory: / May 19 14:09:05.984824 dv-cinder-842541-1-iscsi devstack@keystone.service[80189]: detected binary path: /usr/bin/uwsgi-core May 19 14:09:05.984824 dv-cinder-842541-1-iscsi devstack@keystone.service[80189]: your processes number limit is 47680 May 19 14:09:05.984824 dv-cinder-842541-1-iscsi devstack@keystone.service[80189]: your memory page size is 4096 bytes May 19 14:09:05.984824 dv-cinder-842541-1-iscsi devstack@keystone.service[80189]: detected max file descriptor number: 2048 May 19 14:09:05.984824 dv-cinder-842541-1-iscsi devstack@keystone.service[80189]: lock engine: pthread robust mutexes May 19 14:09:05.984824 dv-cinder-842541-1-iscsi devstack@keystone.service[80189]: thunder lock: enabled May 19 14:09:05.984824 dv-cinder-842541-1-iscsi devstack@keystone.service[80189]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 May 19 14:09:05.984824 dv-cinder-842541-1-iscsi devstack@keystone.service[80189]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] May 19 14:09:06.027607 dv-cinder-842541-1-iscsi devstack@keystone.service[80189]: Python main interpreter initialized at 0x5605b44449c0 May 19 14:09:06.027607 dv-cinder-842541-1-iscsi devstack@keystone.service[80189]: python threads support enabled May 19 14:09:06.027607 dv-cinder-842541-1-iscsi devstack@keystone.service[80189]: your server socket listen backlog is limited to 100 connections May 19 14:09:06.027607 dv-cinder-842541-1-iscsi devstack@keystone.service[80189]: your mercy for graceful operations on workers is 90 seconds May 19 14:09:06.027976 dv-cinder-842541-1-iscsi devstack@keystone.service[80189]: mapped 403077 bytes (393 KB) for 2 cores May 19 14:09:06.028053 dv-cinder-842541-1-iscsi devstack@keystone.service[80189]: *** Operational MODE: preforking *** May 19 14:09:06.028303 dv-cinder-842541-1-iscsi devstack@keystone.service[80189]: *** uWSGI is running in multiple interpreter mode *** May 19 14:09:06.028303 dv-cinder-842541-1-iscsi devstack@keystone.service[80189]: spawned uWSGI master process (pid: 80189) May 19 14:09:06.028511 dv-cinder-842541-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. May 19 14:09:06.029596 dv-cinder-842541-1-iscsi devstack@keystone.service[80189]: spawned uWSGI worker 1 (pid: 80202, cores: 1) May 19 14:09:06.029596 dv-cinder-842541-1-iscsi devstack@keystone.service[80189]: spawned uWSGI worker 2 (pid: 80203, cores: 1) May 19 14:09:06.029596 dv-cinder-842541-1-iscsi devstack@keystone.service[80189]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... May 19 14:09:11.785939 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 14:09:11.785939 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: from cryptography.utils import int_from_bytes May 19 14:09:11.787342 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 14:09:11.787342 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: from cryptography.utils import int_from_bytes May 19 14:09:11.787469 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 14:09:11.787469 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: from cryptography.utils import int_from_bytes May 19 14:09:11.787469 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 14:09:11.787469 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: from cryptography.utils import int_from_bytes May 19 14:09:11.837998 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 19 14:09:11.838177 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 19 14:09:11.838364 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] command line args: [] {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 19 14:09:11.838461 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 19 14:09:11.838567 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 19 14:09:11.838656 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 19 14:09:11.838762 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] command line args: [] {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 19 14:09:11.838848 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ================================================================================ {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 19 14:09:11.839041 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] admin_token = **** {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:09:11.839125 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 19 14:09:11.839222 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ================================================================================ {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 19 14:09:11.839312 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] config_dir = [] {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:09:11.839458 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:09:11.839542 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] admin_token = **** {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:09:11.839640 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] config_source = [] {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:09:11.839723 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] config_dir = [] {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:09:11.839820 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] debug = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:09:11.839909 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:09:11.840022 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:09:11.840330 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] config_source = [] {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:09:11.840330 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] debug = True {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:09:11.840526 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:09:11.840625 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:09:11.840752 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:09:11.840860 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:09:11.840961 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:09:11.841066 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:09:11.841169 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:09:11.841265 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:09:11.841372 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] list_limit = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:09:11.841476 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:09:11.841578 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] log_config_append = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:09:11.841682 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] list_limit = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:09:11.841785 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:09:11.841918 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] log_config_append = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:09:11.842021 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] log_dir = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:09:11.842123 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:09:11.842236 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] log_file = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:09:11.842354 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] log_dir = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:09:11.842461 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:09:11.842559 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] log_file = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:09:11.842638 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:09:11.842755 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:09:11.842836 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:09:11.842960 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:09:11.843037 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:09:11.843162 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:09:11.843238 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:09:11.843361 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:09:11.843439 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:09:11.843566 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:09:11.843652 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:09:11.843812 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:09:11.843899 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(user_domain)s %(project_domain)s {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:09:11.843978 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:09:11.844057 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:09:11.844130 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(user_domain)s %(project_domain)s {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:09:11.844246 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:09:11.844326 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:09:11.844398 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:09:11.844512 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:09:11.844600 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:09:11.844672 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:09:11.844794 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:09:11.844885 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:09:11.844962 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:09:11.845038 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:09:11.845210 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:09:11.845291 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:09:11.845501 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:09:11.845700 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.57/identity {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:09:11.845902 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] publish_errors = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:09:11.846018 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.57/identity {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:09:11.846144 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:09:11.846258 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] publish_errors = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:09:11.846376 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:09:11.846522 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:09:11.846629 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:09:11.846763 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:09:11.846874 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:09:11.846980 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:09:11.847149 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:09:11.847269 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:09:11.847350 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] standard_threads = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:09:11.847454 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:09:11.847536 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:09:11.847618 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] standard_threads = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:09:11.847692 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:09:11.847762 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:09:11.847868 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:09:11.847942 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:09:11.848019 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] use_journal = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:09:11.848099 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:09:11.848172 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] use_json = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:09:11.848248 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] use_journal = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:09:11.848325 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] use_stderr = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:09:11.848402 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] use_json = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:09:11.848474 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] use_syslog = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:09:11.848589 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] use_stderr = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:09:11.848672 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:09:11.848744 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] use_syslog = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:09:11.848931 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.849015 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:09:11.849238 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.849365 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.849481 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.849602 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.849724 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.849876 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.850012 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.850129 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.850244 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.850358 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.850472 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.850586 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.850775 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.850894 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.851008 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.851122 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.851243 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.851359 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.851485 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.851601 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.851715 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.851828 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.851949 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.852065 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.852191 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.852322 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.852447 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.852602 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.852754 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.852875 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.853001 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.853119 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.853233 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.853353 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.853468 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.853582 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.853704 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.853821 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.853954 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.854070 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.854183 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.854296 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.854421 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.854538 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.854657 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.854802 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.854918 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.855033 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.855183 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.855301 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.855422 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.855537 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.855651 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.855765 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.855887 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.856003 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.856118 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.856238 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.856358 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.856474 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.856664 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.856781 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.856895 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.857014 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.857129 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.857243 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.857367 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.857467 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.857562 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.857657 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.857777 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.857929 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.857929 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.858155 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.858155 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.858379 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.858379 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.858599 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.858716 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.858829 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.858943 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.859056 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.859169 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.859292 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.859408 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.859527 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.859649 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.859764 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.859764 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.860003 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.860003 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.860232 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.860354 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.860454 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.860550 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.860652 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.860751 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.860852 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.860948 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.861049 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.861145 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.861252 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.861357 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.861504 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.861629 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.861745 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.861917 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.862045 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.862163 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.862278 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.862395 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.862509 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.862634 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.862753 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.862869 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.862983 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.863097 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.863211 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.863332 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.863458 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.863583 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.863706 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.863831 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.863956 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.864073 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.864178 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.864300 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.864426 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.864562 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.864731 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.864861 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.864978 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.865078 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.865201 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.865318 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.865434 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.865550 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.865664 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.865779 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.865932 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.866052 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.866168 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.866284 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.866646 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.866765 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.866889 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.867007 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.867123 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.867247 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.867364 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.867486 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.867614 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.867733 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.867849 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.867965 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.868088 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.868195 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.868370 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.868489 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.868611 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.868729 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.868845 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.868972 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.869090 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.869205 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.869327 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.869444 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.869560 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.869683 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.869801 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.869971 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.870088 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.870203 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.870318 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.870445 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.870564 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.870679 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.870802 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.870925 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.871041 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.871168 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.871298 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.871427 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.871544 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.871659 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.871774 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.871890 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.872012 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.872138 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.872255 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.872370 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.872494 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.872615 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.872738 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.872863 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.872982 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.873098 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.873220 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.873337 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.873452 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.873576 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.873694 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.873809 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.873954 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.874070 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.874185 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.874317 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] auth.external = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.874435 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.874550 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.874665 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] auth.external = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.874780 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.874896 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.875051 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.875179 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.875307 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] auth.password = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.875432 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.875550 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] auth.token = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.875666 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] auth.password = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.875788 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.875905 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] auth.token = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.876045 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.876209 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.876328 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.876498 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.876617 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.876743 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.876862 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.876979 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.877097 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.877212 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.877329 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.877453 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.877572 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] credential.caching = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.877688 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.877805 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.877961 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] credential.caching = True {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.878088 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.878217 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.878342 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.878459 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.878576 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.878692 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.878808 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.878932 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.879051 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.879168 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.879284 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.879400 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.879517 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.879641 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.879767 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.879886 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.880002 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.880636 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.880750 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.880857 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.880970 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.881087 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.881204 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.881321 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.881438 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.881562 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.881682 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.881814 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.881974 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] federation.caching = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.882091 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.882208 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.882338 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] federation.caching = True {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.882458 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.882576 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.882695 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.882811 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.882928 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.883120 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.883250 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.883370 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.883488 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.883488 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.883706 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.883831 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.883949 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.884066 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.884182 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.884299 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.884416 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.884541 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.884659 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.884783 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] identity.caching = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.884783 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.885002 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.885118 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.885243 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.885362 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.885479 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] identity.caching = True {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.885595 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.885710 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.885826 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.886004 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.886124 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.886240 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.886356 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.886471 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.886589 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.886714 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.886831 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.886947 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.887063 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.887178 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.887303 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.887420 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.887537 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.887537 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.887754 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.887754 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.887979 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.888097 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.888213 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.888327 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.888442 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.888542 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.888657 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.888774 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.888892 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.889007 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.889123 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.889239 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.889363 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.889481 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.889598 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.889598 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.889814 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.889958 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.890087 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.890206 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.890323 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.890438 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.890554 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.890676 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.890803 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.890922 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.891038 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.891154 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.891270 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.891386 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.891511 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.891629 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.891746 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.891862 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.891978 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.892094 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.892220 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.892344 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.892462 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.892582 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.892697 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.892814 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.892939 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.893057 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.893173 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.893288 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.893404 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.893520 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.893645 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.893762 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.893917 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.894036 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.894152 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.894269 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.894400 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.894520 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.894637 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.894752 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.894868 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.894984 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.895109 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.895226 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.895350 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.895468 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.895584 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.895700 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.896384 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.896511 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.896629 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.896745 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.896862 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.896979 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.897104 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.897222 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.user = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.897338 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.897454 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.897570 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.897686 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.897812 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.897971 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.898091 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.898208 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.898324 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.user = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.898440 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.898573 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.898693 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.898809 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.898926 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.899042 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.899159 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.899284 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.899402 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.899519 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.899636 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.899751 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.899876 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.899995 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.900111 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.900228 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.900344 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.900460 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.900585 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.900703 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.900820 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.900936 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.901053 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.901170 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.901294 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.901412 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.901529 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.901645 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.901761 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.901889 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.902020 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.902140 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.902257 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.902374 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.902491 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.902611 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.902736 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.902855 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.902971 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.903087 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.903203 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.903320 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.903449 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.903573 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.903691 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.903808 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.903924 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.904040 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.904165 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.904283 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.904401 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.904518 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.904634 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.904750 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.904874 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.904993 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.905109 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.905232 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.905350 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.905468 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.905593 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.905711 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.905828 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.905993 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.906109 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] resource.caching = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.906225 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.906425 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.906548 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.906664 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.906786 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.906905 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.907021 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.907146 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.907264 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] resource.caching = True {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.907380 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.907542 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.907662 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.907784 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.907910 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.908029 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.908145 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.908262 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.908378 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.908494 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.908619 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] role.caching = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.908738 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.908855 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] role.driver = sql {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.908970 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.909087 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.909202 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.909327 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.909445 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.909562 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.909678 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] role.caching = True {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.909794 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.909928 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] role.driver = sql {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.910057 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.910176 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.910292 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.910408 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.910524 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.910640 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.910766 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.910884 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.911001 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.911117 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.911233 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.911372 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.911495 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.911613 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.911731 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.911848 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.911964 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.912089 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.912208 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.912324 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.912324 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.912546 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.912691 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.912818 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.912935 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.913050 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.913165 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.913286 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.913286 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.913510 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.913627 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.913742 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.913885 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.914003 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.914118 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.914249 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.914367 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.914482 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.914596 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.914710 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.914825 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.914948 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.915065 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.915180 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.915180 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.915394 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.915515 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.915639 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.915755 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.915872 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.915986 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.916100 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.916214 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.916336 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.916453 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.916567 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.916681 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.916795 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.916910 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.917033 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] token.caching = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.917155 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.917270 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.917384 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.917498 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.917620 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.917737 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.917870 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] token.caching = True {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.917989 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.918103 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.918217 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.918343 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.918460 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.918575 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.918688 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.918802 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.918916 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.919039 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.919156 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.919270 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.919385 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.919504 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.919645 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.920021 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.920144 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.920260 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.920377 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.920493 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.920610 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.920734 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.920852 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.920968 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.921092 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.921210 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.921326 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.921451 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.921570 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] database.connection = **** {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.921695 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.921821 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.921993 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.922112 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.922229 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.922346 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.922475 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] database.connection = **** {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.922608 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.922735 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.922853 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.922969 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.923086 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.923203 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.923328 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.923446 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.923563 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.923679 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.923795 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.923911 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.924040 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.924158 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.924277 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.924393 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.924509 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.924631 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.924758 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.924876 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.924992 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.925107 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.925223 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.925340 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.925529 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.925649 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.925766 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.925895 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.926013 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.926013 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 19 14:09:11.926244 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.926244 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.926244 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.926244 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:09:11.926244 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 19 14:09:11.954317 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:09:11.954534 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:09:11.954625 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:09:11.954756 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:09:11.954842 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:09:11.954981 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:09:11.964096 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:09:11.964420 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:09:11.992462 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:09:11.992688 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:09:11.993246 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:09:11.993488 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:09:12.046199 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:09:12.046472 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:09:12.098062 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:09:12.098293 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:09:12.098490 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:09:12.098730 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:09:12.165259 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:09:12.165484 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:09:12.165577 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:09:12.165706 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:09:12.165794 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:09:12.165947 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:09:12.166033 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:09:12.166166 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:09:12.166265 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:09:12.166383 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:09:12.166467 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:09:12.166597 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:09:12.189923 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:09:12.190093 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:09:12.190182 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:09:12.190263 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:09:12.190373 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:09:12.190489 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:09:12.190597 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:09:12.190718 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:09:12.190802 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:09:12.190942 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:09:12.191024 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:09:12.191156 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:09:12.191247 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:09:12.191370 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:09:12.203070 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:09:12.203371 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:09:12.217707 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:09:12.217959 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:09:12.218100 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:09:12.218188 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:09:12.218337 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:09:12.218421 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:09:12.218563 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:09:12.218660 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:09:12.218784 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:09:12.218867 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:09:12.218998 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:09:12.219079 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:09:12.219210 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:09:12.219421 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:09:12.236402 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.236612 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 14:09:12.236950 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.237107 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.237193 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 14:09:12.237271 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 14:09:12.237407 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.237493 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.237667 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.237761 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 14:09:12.237950 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.238036 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.238207 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.238294 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.238465 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.238724 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.250203 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.250477 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.250632 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.250744 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.250913 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.251022 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.251185 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.251468 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.259869 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80203) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:09:12.260169 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80202) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:09:12.263176 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80203) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:09:12.263460 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.263549 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80202) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:09:12.263813 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.263919 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.264082 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.264165 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.264399 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.264480 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.264657 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.264748 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.264924 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.265008 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.265194 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.265277 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.265462 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.265545 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.265734 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.265815 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.266024 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.266123 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.266326 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.292462 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.293513 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.294102 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80203) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:09:12.294383 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.295287 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80202) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:09:12.295577 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.299673 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80203) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:09:12.299914 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.300194 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.300912 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80202) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:09:12.301196 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.301478 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.308021 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80203) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:09:12.308300 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.309611 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80202) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:09:12.309919 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.312920 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80203) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:09:12.313191 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.313473 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.313756 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.314054 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.314329 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.314543 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80202) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:09:12.314625 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.314833 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.314926 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.315118 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.315403 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.315678 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.315956 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.316224 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.316491 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.334170 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.334446 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.335879 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.336166 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.339028 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80203) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:09:12.340696 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80202) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:09:12.342680 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.342976 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.344341 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.344637 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.347991 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80203) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:09:12.349713 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80202) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:09:12.351648 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80203) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:09:12.353323 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80202) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:09:12.355540 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.355840 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.356121 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.356409 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.356688 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.356972 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.357084 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.357387 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.357674 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.357983 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.358264 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.358578 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.378024 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80203) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:09:12.378399 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.378670 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.378935 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.379020 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80202) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:09:12.379325 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.379597 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.379866 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.387508 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.388592 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.389213 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.389496 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.390363 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.390658 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.392551 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80203) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:09:12.392830 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.393117 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.393412 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.393698 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.393781 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80202) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:09:12.394068 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.394366 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.394664 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.394948 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.407114 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80203) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:09:12.407402 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.407685 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.407965 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.408244 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.408516 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.408624 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80202) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:09:12.408796 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.408920 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.409207 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.409499 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.409785 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.410087 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.410370 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.427639 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80203) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:09:12.429109 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80202) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:09:12.430910 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80203) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:09:12.432386 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80202) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:09:12.434248 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.435735 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.435837 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.437324 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.437445 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80203) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:09:12.437732 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.438036 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.439002 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80202) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:09:12.439290 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.439575 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.446134 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.447716 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.447802 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80203) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:09:12.449339 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80202) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:09:12.450989 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.451274 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.451553 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.451832 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.452594 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.452889 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.453184 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.453467 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.461219 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80203) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:09:12.461553 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.461901 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.462864 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80202) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:09:12.463200 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.463528 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.469768 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80203) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:09:12.470081 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.470363 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.470641 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.470913 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.471183 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.471464 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80202) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:09:12.471557 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.471735 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.471817 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.472008 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.472090 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.472275 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.472359 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.472546 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.472634 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.472804 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.472888 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.473079 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.473161 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.473337 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.473419 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.473622 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.474499 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.474780 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.475045 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.475330 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.475594 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:09:12.513461 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5605b44449c0 pid: 80203 (default app) May 19 14:09:12.515298 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5605b44449c0 pid: 80202 (default app) May 19 14:09:12.516823 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8590ee-ba49-4269-b515-ddbc791acee4 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:12.517135 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8590ee-ba49-4269-b515-ddbc791acee4 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:12.517418 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8590ee-ba49-4269-b515-ddbc791acee4 None None] PATH_INFO: `/v3` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:12.521717 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1/1] 10.222.0.57 () {50 vars in 791 bytes} [Thu May 19 14:09:12 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:09:20.560897 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd3aa2e-5d98-45d0-8af1-5ee750a95d93 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:20.561458 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd3aa2e-5d98-45d0-8af1-5ee750a95d93 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:20.561458 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd3aa2e-5d98-45d0-8af1-5ee750a95d93 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:20.565754 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1/2] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:09:20 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:09:20.571267 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab183d3-3b8b-4f5d-b7c6-f731ec171d76 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:20.571674 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab183d3-3b8b-4f5d-b7c6-f731ec171d76 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:20.571823 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab183d3-3b8b-4f5d-b7c6-f731ec171d76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:20.641374 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: INFO dbcounter [None req-7ab183d3-3b8b-4f5d-b7c6-f731ec171d76 None None] Registered counter for database keystone May 19 14:09:20.664274 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG oslo_db.sqlalchemy.engines [None req-7ab183d3-3b8b-4f5d-b7c6-f731ec171d76 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80203) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 19 14:09:20.670581 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writer thread running {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 19 14:09:20.850723 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG passlib.handlers.bcrypt [None req-7ab183d3-3b8b-4f5d-b7c6-f731ec171d76 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80203) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 19 14:09:20.851072 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG passlib.handlers.bcrypt [None req-7ab183d3-3b8b-4f5d-b7c6-f731ec171d76 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80203) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 19 14:09:20.908643 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-7ab183d3-3b8b-4f5d-b7c6-f731ec171d76 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:09:20.910299 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-7ab183d3-3b8b-4f5d-b7c6-f731ec171d76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:09:21.041591 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 2/3] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:09:20 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 471 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 19 14:09:21.180903 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd3aa2e-5d98-45d0-8af1-5ee750a95d93 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:21.251223 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: INFO dbcounter [None req-4fd3aa2e-5d98-45d0-8af1-5ee750a95d93 None None] Registered counter for database keystone May 19 14:09:21.270891 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG oslo_db.sqlalchemy.engines [None req-4fd3aa2e-5d98-45d0-8af1-5ee750a95d93 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80202) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 19 14:09:21.277381 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writer thread running {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 19 14:09:21.395653 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd3aa2e-5d98-45d0-8af1-5ee750a95d93 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:21.400003 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4475c8e7-e55d-4e14-86f0-05c8a45cd310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc6295ed5b1f4894a42c16df5aa39db4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:21.400660 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4475c8e7-e55d-4e14-86f0-05c8a45cd310 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:21.400945 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4475c8e7-e55d-4e14-86f0-05c8a45cd310 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:21.401222 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4475c8e7-e55d-4e14-86f0-05c8a45cd310 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:21.409378 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4475c8e7-e55d-4e14-86f0-05c8a45cd310 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:21.461334 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: WARNING keystone.common.rbac_enforcer.enforcer [None req-4475c8e7-e55d-4e14-86f0-05c8a45cd310 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 19 14:09:21.461666 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4475c8e7-e55d-4e14-86f0-05c8a45cd310 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:21.471316 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 2/4] 10.222.0.57 () {58 vars in 1324 bytes} [Thu May 19 14:09:21 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 14:09:21.478485 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab183d3-3b8b-4f5d-b7c6-f731ec171d76 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:21.494437 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab183d3-3b8b-4f5d-b7c6-f731ec171d76 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:21.498885 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb43364-5268-4991-95b9-e1ccef655f0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc6295ed5b1f4894a42c16df5aa39db4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:21.500013 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb43364-5268-4991-95b9-e1ccef655f0b admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:21.500304 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb43364-5268-4991-95b9-e1ccef655f0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:21.500585 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb43364-5268-4991-95b9-e1ccef655f0b admin admin] PATH_INFO: `/v3/projects/dc6295ed5b1f4894a42c16df5aa39db4` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:21.502147 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb43364-5268-4991-95b9-e1ccef655f0b admin admin] RBAC: Authorizing `identity:get_project(project_id=dc6295ed5b1f4894a42c16df5aa39db4)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:21.660880 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: WARNING keystone.common.rbac_enforcer.enforcer [None req-4fb43364-5268-4991-95b9-e1ccef655f0b admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 19 14:09:21.661500 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb43364-5268-4991-95b9-e1ccef655f0b admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:21.666540 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 3/5] 10.222.0.57 () {56 vars in 1207 bytes} [Thu May 19 14:09:21 2022] GET /identity/v3/projects/dc6295ed5b1f4894a42c16df5aa39db4 => generated 344 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:09:23.858334 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad99e1db-dfba-4a42-b517-e3f35e1f2ef1 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:23.858889 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad99e1db-dfba-4a42-b517-e3f35e1f2ef1 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:23.858889 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad99e1db-dfba-4a42-b517-e3f35e1f2ef1 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:23.863032 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 3/6] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:09:23 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:09:23.868934 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcda9a79-2c9c-4f4b-9b19-6db699368439 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:23.869356 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcda9a79-2c9c-4f4b-9b19-6db699368439 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:23.869494 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcda9a79-2c9c-4f4b-9b19-6db699368439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:23.889604 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-fcda9a79-2c9c-4f4b-9b19-6db699368439 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:09:23.891170 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-fcda9a79-2c9c-4f4b-9b19-6db699368439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:09:23.899643 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 4/7] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:09:23 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 19 14:09:23.924060 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad99e1db-dfba-4a42-b517-e3f35e1f2ef1 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:23.936201 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad99e1db-dfba-4a42-b517-e3f35e1f2ef1 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:23.940438 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2bf897-ff71-4b06-a387-384ea5e83ca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc6295ed5b1f4894a42c16df5aa39db4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:23.941070 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2bf897-ff71-4b06-a387-384ea5e83ca6 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:23.941356 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2bf897-ff71-4b06-a387-384ea5e83ca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:23.941634 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2bf897-ff71-4b06-a387-384ea5e83ca6 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:23.949993 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2bf897-ff71-4b06-a387-384ea5e83ca6 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:23.951778 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2bf897-ff71-4b06-a387-384ea5e83ca6 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:23.961299 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 4/8] 10.222.0.57 () {58 vars in 1324 bytes} [Thu May 19 14:09:23 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 14:09:23.966740 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcda9a79-2c9c-4f4b-9b19-6db699368439 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:23.975766 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcda9a79-2c9c-4f4b-9b19-6db699368439 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:23.980183 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a502cab-e78a-4edc-af40-2ae6996d6f71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc6295ed5b1f4894a42c16df5aa39db4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:23.981045 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a502cab-e78a-4edc-af40-2ae6996d6f71 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:23.981332 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a502cab-e78a-4edc-af40-2ae6996d6f71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:23.981611 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a502cab-e78a-4edc-af40-2ae6996d6f71 admin admin] PATH_INFO: `/v3/users/d4608701c57147b39ea0df421d2e5029` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:23.983035 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a502cab-e78a-4edc-af40-2ae6996d6f71 admin admin] RBAC: Authorizing `identity:get_user(user_id=d4608701c57147b39ea0df421d2e5029)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:23.984876 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a502cab-e78a-4edc-af40-2ae6996d6f71 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:23.986487 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 5/9] 10.222.0.57 () {56 vars in 1198 bytes} [Thu May 19 14:09:23 2022] GET /identity/v3/users/d4608701c57147b39ea0df421d2e5029 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:09:26.261396 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b47f8b-5b34-40ca-9588-d23e166245cd None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:26.261940 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b47f8b-5b34-40ca-9588-d23e166245cd None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:26.262045 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b47f8b-5b34-40ca-9588-d23e166245cd None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:26.263331 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 5/10] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:09:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:09:26.268866 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30386ddc-e11c-4127-9a67-390a5c901eb2 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:26.269308 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30386ddc-e11c-4127-9a67-390a5c901eb2 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:26.269439 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30386ddc-e11c-4127-9a67-390a5c901eb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:26.286171 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-30386ddc-e11c-4127-9a67-390a5c901eb2 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:09:26.287871 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-30386ddc-e11c-4127-9a67-390a5c901eb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:09:26.297761 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d421ec-282c-46be-ae9d-163e91591d80 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:26.298084 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d421ec-282c-46be-ae9d-163e91591d80 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:26.298367 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d421ec-282c-46be-ae9d-163e91591d80 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:26.299711 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 6/11] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:09:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:09:26.305616 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa395d0-48a4-4990-801b-05d29572502c None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:26.305933 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa395d0-48a4-4990-801b-05d29572502c None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:26.306216 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa395d0-48a4-4990-801b-05d29572502c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:26.346978 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG passlib.handlers.bcrypt [None req-afa395d0-48a4-4990-801b-05d29572502c None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80202) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 19 14:09:26.347434 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG passlib.handlers.bcrypt [None req-afa395d0-48a4-4990-801b-05d29572502c None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80202) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 19 14:09:26.369775 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 6/12] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:09:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:09:26.392510 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30386ddc-e11c-4127-9a67-390a5c901eb2 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:26.394513 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-afa395d0-48a4-4990-801b-05d29572502c None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:09:26.396711 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-afa395d0-48a4-4990-801b-05d29572502c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:09:26.417311 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:26.498298 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 7/13] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:09:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:09:26.499034 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30386ddc-e11c-4127-9a67-390a5c901eb2 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:26.520733 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa395d0-48a4-4990-801b-05d29572502c None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:26.535129 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eab5b90-1899-4f07-bf95-98a16ff02969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:26.535931 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eab5b90-1899-4f07-bf95-98a16ff02969 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:26.536218 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eab5b90-1899-4f07-bf95-98a16ff02969 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:26.536501 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eab5b90-1899-4f07-bf95-98a16ff02969 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:26.548615 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eab5b90-1899-4f07-bf95-98a16ff02969 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:09:26.559963 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-7eab5b90-1899-4f07-bf95-98a16ff02969 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:09:26.559963 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:09:26.559963 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:09:26.559963 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:09:26.559963 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:09:26.559963 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:09:26.559963 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:09:26.559963 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:09:26.559963 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:09:26.559963 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:09:26.559963 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:09:26.559963 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:09:26.559963 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:09:26.559963 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:09:26.559963 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:09:26.559963 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:09:26.561171 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:09:26.561171 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:09:26.561171 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:09:26.561171 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:09:26.561171 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:09:26.561171 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:09:26.561171 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:09:26.561171 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:09:26.561171 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:09:26.561171 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:09:26.561171 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:09:26.561171 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:09:26.561171 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:09:26.561171 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:09:26.561171 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:09:26.561171 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:09:26.562226 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:09:26.562226 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:09:26.562226 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:09:26.562226 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:09:26.562226 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:09:26.562226 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:09:26.562226 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:09:26.562226 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:09:26.562226 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:09:26.562226 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:09:26.562226 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:09:26.562226 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:09:26.562226 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 7/14] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:09:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:09:26.569124 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eab5b90-1899-4f07-bf95-98a16ff02969 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:26.661530 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:26.684080 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eab5b90-1899-4f07-bf95-98a16ff02969 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:26.718981 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eb1085-a5ca-4a9c-8229-7134e50b927b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:26.719633 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eb1085-a5ca-4a9c-8229-7134e50b927b None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:26.719917 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eb1085-a5ca-4a9c-8229-7134e50b927b None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:26.720186 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eb1085-a5ca-4a9c-8229-7134e50b927b None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:26.720954 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eb1085-a5ca-4a9c-8229-7134e50b927b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:09:26.721264 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eb1085-a5ca-4a9c-8229-7134e50b927b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:26.722710 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eb1085-a5ca-4a9c-8229-7134e50b927b None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:26.730124 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa395d0-48a4-4990-801b-05d29572502c None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:26.732450 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:26.735204 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 8/15] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:09:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:09:26.740627 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eb1085-a5ca-4a9c-8229-7134e50b927b None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:26.770970 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d968c53-a398-401c-b3f4-4367c4e6eab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:26.772464 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d968c53-a398-401c-b3f4-4367c4e6eab4 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:26.772754 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d968c53-a398-401c-b3f4-4367c4e6eab4 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:26.773024 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d968c53-a398-401c-b3f4-4367c4e6eab4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:26.842043 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d968c53-a398-401c-b3f4-4367c4e6eab4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:26.842237 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eb1085-a5ca-4a9c-8229-7134e50b927b None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:26.843592 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d968c53-a398-401c-b3f4-4367c4e6eab4 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:26.877121 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2ff82f-4a0f-4c71-8ce5-e10dd15459e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:26.878028 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2ff82f-4a0f-4c71-8ce5-e10dd15459e4 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:26.878327 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2ff82f-4a0f-4c71-8ce5-e10dd15459e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:26.878596 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2ff82f-4a0f-4c71-8ce5-e10dd15459e4 None admin] PATH_INFO: `/v3/users/d4608701c57147b39ea0df421d2e5029` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:26.879812 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2ff82f-4a0f-4c71-8ce5-e10dd15459e4 None admin] RBAC: Authorizing `identity:get_user(user_id=d4608701c57147b39ea0df421d2e5029)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:26.882195 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2ff82f-4a0f-4c71-8ce5-e10dd15459e4 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:26.883835 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 9/16] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:09:26 2022] GET /identity/v3/users/d4608701c57147b39ea0df421d2e5029 => generated 247 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:09:26.889537 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2ff82f-4a0f-4c71-8ce5-e10dd15459e4 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:26.904693 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:26.945296 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 8/17] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:09:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 14:09:26.950798 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d968c53-a398-401c-b3f4-4367c4e6eab4 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:26.953539 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:26.999468 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2ff82f-4a0f-4c71-8ce5-e10dd15459e4 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:27.033832 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066b0c92-91cf-4962-8551-1f3beccb8cee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:27.034663 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066b0c92-91cf-4962-8551-1f3beccb8cee None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:27.034951 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066b0c92-91cf-4962-8551-1f3beccb8cee None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:27.035230 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066b0c92-91cf-4962-8551-1f3beccb8cee None admin] PATH_INFO: `/v3/domains/default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:27.036419 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066b0c92-91cf-4962-8551-1f3beccb8cee None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:27.038113 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066b0c92-91cf-4962-8551-1f3beccb8cee None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:27.039734 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 10/18] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:09:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:09:27.045365 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066b0c92-91cf-4962-8551-1f3beccb8cee None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:27.052709 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d968c53-a398-401c-b3f4-4367c4e6eab4 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:27.087202 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c8af53-4ae0-4336-9b42-739d18e7c375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:27.087998 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c8af53-4ae0-4336-9b42-739d18e7c375 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:27.088282 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c8af53-4ae0-4336-9b42-739d18e7c375 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:27.088560 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c8af53-4ae0-4336-9b42-739d18e7c375 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:27.107148 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c8af53-4ae0-4336-9b42-739d18e7c375 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:27.109126 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c8af53-4ae0-4336-9b42-739d18e7c375 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:27.130070 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-23c8af53-4ae0-4336-9b42-739d18e7c375 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:09:27.130070 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:09:27.130070 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:09:27.130070 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:09:27.130070 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:09:27.130070 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:09:27.130070 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:09:27.130070 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:09:27.130070 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:09:27.130070 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:09:27.130070 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:09:27.130070 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:09:27.130070 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:09:27.130070 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:09:27.130070 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:09:27.131510 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:09:27.131510 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:09:27.131510 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:09:27.131510 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:09:27.131510 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:09:27.131510 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:09:27.131510 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:09:27.131510 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:09:27.131510 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:09:27.131510 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:09:27.131510 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:09:27.131510 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:09:27.131510 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:09:27.131510 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:09:27.131510 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:09:27.131510 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:09:27.131510 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:09:27.132310 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:09:27.132310 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:09:27.132310 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:09:27.132310 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:09:27.132587 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:27.133060 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 9/19] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:09:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:09:27.138806 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c8af53-4ae0-4336-9b42-739d18e7c375 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:27.151463 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066b0c92-91cf-4962-8551-1f3beccb8cee None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:27.186670 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715235fe-9f0d-4d2c-834e-4a43fd2373c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:27.187329 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715235fe-9f0d-4d2c-834e-4a43fd2373c2 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:27.187616 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715235fe-9f0d-4d2c-834e-4a43fd2373c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:27.187899 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715235fe-9f0d-4d2c-834e-4a43fd2373c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:27.188739 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715235fe-9f0d-4d2c-834e-4a43fd2373c2 None admin] RBAC: Adding query filter params (role.id=4946cf0ff7164307b36efeee7c787146, scope.domain.id=default, user.id=d4608701c57147b39ea0df421d2e5029) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:09:27.189062 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715235fe-9f0d-4d2c-834e-4a43fd2373c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:27.190559 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715235fe-9f0d-4d2c-834e-4a43fd2373c2 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:27.198003 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 11/20] 10.222.0.57 () {56 vars in 1322 bytes} [Thu May 19 14:09:27 2022] GET /identity/v3/role_assignments?user.id=d4608701c57147b39ea0df421d2e5029&scope.domain.id=default&role.id=4946cf0ff7164307b36efeee7c787146 => generated 233 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:09:27.214344 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:27.248048 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c8af53-4ae0-4336-9b42-739d18e7c375 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:27.286497 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0573b01a-21af-405e-b19d-ddc134ba221f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:27.287344 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0573b01a-21af-405e-b19d-ddc134ba221f None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:27.287640 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0573b01a-21af-405e-b19d-ddc134ba221f None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:27.287920 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0573b01a-21af-405e-b19d-ddc134ba221f None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:27.288669 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0573b01a-21af-405e-b19d-ddc134ba221f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:09:27.288982 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0573b01a-21af-405e-b19d-ddc134ba221f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:27.290444 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0573b01a-21af-405e-b19d-ddc134ba221f None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:27.310722 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 10/21] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:09:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:09:29.890505 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e1cdc8-76dc-4968-998a-bc3f5117d892 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:29.891538 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e1cdc8-76dc-4968-998a-bc3f5117d892 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:29.892020 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e1cdc8-76dc-4968-998a-bc3f5117d892 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:29.893587 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 12/22] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:09:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:09:29.900420 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277bf8df-ff5e-4b5d-b224-fc00ac5086f3 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:29.900871 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277bf8df-ff5e-4b5d-b224-fc00ac5086f3 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:29.900966 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277bf8df-ff5e-4b5d-b224-fc00ac5086f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:29.926845 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-277bf8df-ff5e-4b5d-b224-fc00ac5086f3 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:09:29.928704 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-277bf8df-ff5e-4b5d-b224-fc00ac5086f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:09:30.006618 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fda692-1cb8-44d4-b7e8-0f393df9a4cd None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:30.007255 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fda692-1cb8-44d4-b7e8-0f393df9a4cd None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:30.007740 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fda692-1cb8-44d4-b7e8-0f393df9a4cd None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:30.010649 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 13/23] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:09:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:09:30.019447 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf42b474-4845-44d4-8bfb-0d6b8c349070 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:30.020183 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf42b474-4845-44d4-8bfb-0d6b8c349070 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:30.020783 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf42b474-4845-44d4-8bfb-0d6b8c349070 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:30.033965 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 11/24] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:09:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:09:30.054556 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-cf42b474-4845-44d4-8bfb-0d6b8c349070 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:09:30.056118 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-cf42b474-4845-44d4-8bfb-0d6b8c349070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:09:30.065021 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277bf8df-ff5e-4b5d-b224-fc00ac5086f3 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:30.162327 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:30.225871 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 14/25] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:09:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:09:30.236539 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41ae2d2-f2cc-446d-9155-20ac40016522 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:30.236823 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41ae2d2-f2cc-446d-9155-20ac40016522 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:30.237102 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41ae2d2-f2cc-446d-9155-20ac40016522 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:30.238641 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 15/26] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:09:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:09:30.244921 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4176c099-0160-4a3e-979f-0cde0dab9a65 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:30.245479 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4176c099-0160-4a3e-979f-0cde0dab9a65 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:30.245981 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4176c099-0160-4a3e-979f-0cde0dab9a65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:30.275971 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-4176c099-0160-4a3e-979f-0cde0dab9a65 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:09:30.278192 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-4176c099-0160-4a3e-979f-0cde0dab9a65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:09:30.286196 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277bf8df-ff5e-4b5d-b224-fc00ac5086f3 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:30.348054 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e329e44c-01f9-4007-b25c-00f539590ecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:30.349556 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e329e44c-01f9-4007-b25c-00f539590ecf None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:30.350064 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e329e44c-01f9-4007-b25c-00f539590ecf None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:30.350530 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e329e44c-01f9-4007-b25c-00f539590ecf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:30.361629 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e329e44c-01f9-4007-b25c-00f539590ecf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:30.363486 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e329e44c-01f9-4007-b25c-00f539590ecf None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:30.383575 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-e329e44c-01f9-4007-b25c-00f539590ecf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:09:30.383575 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:09:30.383575 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:09:30.383575 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:09:30.383575 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:09:30.383575 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:09:30.383575 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:09:30.383575 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:09:30.383575 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:09:30.383575 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:09:30.383575 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:09:30.383575 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:09:30.383575 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:09:30.383575 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:09:30.383575 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:09:30.384428 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:09:30.384428 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:09:30.384428 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:09:30.384428 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:09:30.384428 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:09:30.384428 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:09:30.384428 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:09:30.384428 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:09:30.384428 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:09:30.384428 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:09:30.384428 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:09:30.384428 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:09:30.384428 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:09:30.384428 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:09:30.384428 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:09:30.384428 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:09:30.384428 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:09:30.385227 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:09:30.385227 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:09:30.385227 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:09:30.385227 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:09:30.387574 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 12/27] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:09:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:09:30.388846 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e329e44c-01f9-4007-b25c-00f539590ecf None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:30.401412 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 16/28] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:09:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) May 19 14:09:30.402291 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4176c099-0160-4a3e-979f-0cde0dab9a65 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:30.411046 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:30.556330 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:30.560596 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e329e44c-01f9-4007-b25c-00f539590ecf None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:30.606604 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c651b9-09b1-4fc6-a50a-e03fde76cf06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:30.607443 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c651b9-09b1-4fc6-a50a-e03fde76cf06 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:30.607741 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c651b9-09b1-4fc6-a50a-e03fde76cf06 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:30.608027 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c651b9-09b1-4fc6-a50a-e03fde76cf06 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:30.616039 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4176c099-0160-4a3e-979f-0cde0dab9a65 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:30.637946 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c651b9-09b1-4fc6-a50a-e03fde76cf06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:09:30.651835 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-46c651b9-09b1-4fc6-a50a-e03fde76cf06 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:09:30.651835 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:09:30.651835 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:09:30.651835 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:09:30.651835 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:09:30.651835 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:09:30.651835 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:09:30.651835 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:09:30.651835 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:09:30.651835 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:09:30.651835 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:09:30.651835 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:09:30.651835 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:09:30.651835 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:09:30.651835 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:09:30.651835 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:09:30.653000 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:09:30.653000 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:09:30.653000 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:09:30.653000 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:09:30.653000 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:09:30.653000 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:09:30.653000 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:09:30.653000 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:09:30.653000 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:09:30.653000 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:09:30.653000 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:09:30.653000 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:09:30.653000 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:09:30.653000 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:09:30.653000 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:09:30.653000 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:09:30.653769 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:09:30.653769 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:09:30.653769 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:09:30.653769 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:09:30.653769 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:09:30.653769 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:09:30.653769 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:09:30.653769 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:09:30.653769 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:09:30.653769 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:09:30.653769 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:09:30.653769 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:09:30.656310 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 13/29] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:09:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:09:30.657904 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c651b9-09b1-4fc6-a50a-e03fde76cf06 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:30.663450 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d829c98-4875-4dca-a872-bb66beddec08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:30.664343 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d829c98-4875-4dca-a872-bb66beddec08 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:30.664776 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d829c98-4875-4dca-a872-bb66beddec08 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:30.665212 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d829c98-4875-4dca-a872-bb66beddec08 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:30.666129 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d829c98-4875-4dca-a872-bb66beddec08 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:09:30.666579 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d829c98-4875-4dca-a872-bb66beddec08 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:30.668182 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d829c98-4875-4dca-a872-bb66beddec08 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:30.710041 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 17/30] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:09:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:09:30.713071 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d829c98-4875-4dca-a872-bb66beddec08 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:30.810278 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:30.876853 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d829c98-4875-4dca-a872-bb66beddec08 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:30.930222 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980d071d-0707-40ba-aa08-a3b382efc672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:30.931617 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980d071d-0707-40ba-aa08-a3b382efc672 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:30.932081 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980d071d-0707-40ba-aa08-a3b382efc672 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:30.932509 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980d071d-0707-40ba-aa08-a3b382efc672 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:30.933448 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980d071d-0707-40ba-aa08-a3b382efc672 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:09:30.933951 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980d071d-0707-40ba-aa08-a3b382efc672 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:30.935596 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980d071d-0707-40ba-aa08-a3b382efc672 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:30.947051 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c651b9-09b1-4fc6-a50a-e03fde76cf06 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:30.953525 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 18/31] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:09:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:09:30.962558 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980d071d-0707-40ba-aa08-a3b382efc672 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:31.013946 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e409963f-69a3-41cc-a74c-a4dbeab3f10e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:31.014868 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e409963f-69a3-41cc-a74c-a4dbeab3f10e None admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:31.015323 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e409963f-69a3-41cc-a74c-a4dbeab3f10e None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:31.015744 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e409963f-69a3-41cc-a74c-a4dbeab3f10e None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:31.016646 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e409963f-69a3-41cc-a74c-a4dbeab3f10e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:31.018230 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e409963f-69a3-41cc-a74c-a4dbeab3f10e None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:31.157951 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980d071d-0707-40ba-aa08-a3b382efc672 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:31.187999 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:31.191523 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 14/32] 10.222.0.57 () {60 vars in 1131 bytes} [Thu May 19 14:09:30 2022] POST /identity/v3/roles => generated 219 bytes in 534 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:09:31.193321 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e409963f-69a3-41cc-a74c-a4dbeab3f10e None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:31.205295 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8996cbbd-ea44-48ac-9bca-5fa6dd07f226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:31.206392 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8996cbbd-ea44-48ac-9bca-5fa6dd07f226 None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:31.206948 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8996cbbd-ea44-48ac-9bca-5fa6dd07f226 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:31.207417 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8996cbbd-ea44-48ac-9bca-5fa6dd07f226 None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:31.208466 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8996cbbd-ea44-48ac-9bca-5fa6dd07f226 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:31.210935 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8996cbbd-ea44-48ac-9bca-5fa6dd07f226 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:31.224938 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:31.328711 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e409963f-69a3-41cc-a74c-a4dbeab3f10e None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:31.364436 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 19/33] 10.222.0.57 () {60 vars in 1141 bytes} [Thu May 19 14:09:30 2022] POST /identity/v3/projects => generated 301 bytes in 402 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:09:31.367923 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba15ea76-a2a6-4506-a530-b2df3fbb72f9 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:31.368509 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba15ea76-a2a6-4506-a530-b2df3fbb72f9 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:31.368995 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba15ea76-a2a6-4506-a530-b2df3fbb72f9 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:31.371143 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 20/34] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:09:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:09:31.372918 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62d1dab-1e4c-49ad-94e2-4b48920a382b None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:31.373446 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62d1dab-1e4c-49ad-94e2-4b48920a382b None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:31.373972 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62d1dab-1e4c-49ad-94e2-4b48920a382b None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:31.375639 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 21/35] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:09:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:09:31.378581 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0f667d-5644-4d18-965b-dbd5e79541c0 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:31.379124 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0f667d-5644-4d18-965b-dbd5e79541c0 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:31.379735 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0f667d-5644-4d18-965b-dbd5e79541c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:31.385603 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5313dd4e-732a-4e7d-9582-394d8bf2aff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:31.387130 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5313dd4e-732a-4e7d-9582-394d8bf2aff4 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:31.387616 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5313dd4e-732a-4e7d-9582-394d8bf2aff4 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:31.389105 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5313dd4e-732a-4e7d-9582-394d8bf2aff4 None admin] PATH_INFO: `/v3/users/d4608701c57147b39ea0df421d2e5029` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:31.391362 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5313dd4e-732a-4e7d-9582-394d8bf2aff4 None admin] RBAC: Authorizing `identity:get_user(user_id=d4608701c57147b39ea0df421d2e5029)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:31.393251 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5313dd4e-732a-4e7d-9582-394d8bf2aff4 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:31.397104 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 15/36] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:09:31 2022] GET /identity/v3/users/d4608701c57147b39ea0df421d2e5029 => generated 247 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:09:31.404941 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf311a5-46c1-49ae-ad21-1773f8be05a2 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:31.405500 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf311a5-46c1-49ae-ad21-1773f8be05a2 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:31.406021 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf311a5-46c1-49ae-ad21-1773f8be05a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:31.407181 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-7e0f667d-5644-4d18-965b-dbd5e79541c0 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:09:31.409146 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-7e0f667d-5644-4d18-965b-dbd5e79541c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:09:31.428807 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-8bf311a5-46c1-49ae-ad21-1773f8be05a2 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:09:31.430722 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-8bf311a5-46c1-49ae-ad21-1773f8be05a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:09:31.502954 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 22/37] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:09:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:09:31.504561 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca17186-4f3a-4b55-8447-51ea09eeb681 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:31.505000 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca17186-4f3a-4b55-8447-51ea09eeb681 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:31.505997 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca17186-4f3a-4b55-8447-51ea09eeb681 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:31.507534 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 23/38] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:09:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:09:31.508021 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca17186-4f3a-4b55-8447-51ea09eeb681 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:31.527506 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 16/39] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:09:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:09:31.529099 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac33abfa-36ea-4fa1-91dc-8267f13d3ac4 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:31.529396 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac33abfa-36ea-4fa1-91dc-8267f13d3ac4 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:31.529683 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac33abfa-36ea-4fa1-91dc-8267f13d3ac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:31.548172 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-ac33abfa-36ea-4fa1-91dc-8267f13d3ac4 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:09:31.549425 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-ac33abfa-36ea-4fa1-91dc-8267f13d3ac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:09:31.564087 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:31.631579 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 17/40] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:09:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:09:31.632519 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac33abfa-36ea-4fa1-91dc-8267f13d3ac4 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:31.657198 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca17186-4f3a-4b55-8447-51ea09eeb681 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:31.668354 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:31.693111 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfb05db-2a86-4606-adc3-bb138f321cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:31.693945 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfb05db-2a86-4606-adc3-bb138f321cf3 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:31.694236 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfb05db-2a86-4606-adc3-bb138f321cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:31.694517 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfb05db-2a86-4606-adc3-bb138f321cf3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:31.695699 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfb05db-2a86-4606-adc3-bb138f321cf3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:31.697343 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfb05db-2a86-4606-adc3-bb138f321cf3 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:31.703558 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 24/41] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:09:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:09:31.703558 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfb05db-2a86-4606-adc3-bb138f321cf3 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:31.739186 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac33abfa-36ea-4fa1-91dc-8267f13d3ac4 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:31.780151 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17164f4b-6cd0-47cc-b8cd-bc0ba002654d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:31.780818 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17164f4b-6cd0-47cc-b8cd-bc0ba002654d None admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:31.781109 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17164f4b-6cd0-47cc-b8cd-bc0ba002654d None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:31.781390 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17164f4b-6cd0-47cc-b8cd-bc0ba002654d None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:31.782183 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17164f4b-6cd0-47cc-b8cd-bc0ba002654d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:31.783581 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17164f4b-6cd0-47cc-b8cd-bc0ba002654d None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:31.799889 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 18/42] 10.222.0.57 () {60 vars in 1131 bytes} [Thu May 19 14:09:31 2022] POST /identity/v3/roles => generated 225 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:09:31.800494 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17164f4b-6cd0-47cc-b8cd-bc0ba002654d None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:31.804163 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfb05db-2a86-4606-adc3-bb138f321cf3 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:31.810180 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:31.848545 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7939639f-7150-4074-87cc-5223275144ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:31.849957 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7939639f-7150-4074-87cc-5223275144ce None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:31.850302 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7939639f-7150-4074-87cc-5223275144ce None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:31.850584 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7939639f-7150-4074-87cc-5223275144ce None admin] PATH_INFO: `/v3/domains/default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:31.851861 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7939639f-7150-4074-87cc-5223275144ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:31.853395 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7939639f-7150-4074-87cc-5223275144ce None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:31.855343 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 25/43] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:09:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:09:31.855957 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7939639f-7150-4074-87cc-5223275144ce None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:31.911081 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:31.927407 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17164f4b-6cd0-47cc-b8cd-bc0ba002654d None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:31.967292 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f973707-8c03-41d5-8731-168d4835f05c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:31.968037 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f973707-8c03-41d5-8731-168d4835f05c None admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:31.968307 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f973707-8c03-41d5-8731-168d4835f05c None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:31.968596 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f973707-8c03-41d5-8731-168d4835f05c None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:31.969363 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f973707-8c03-41d5-8731-168d4835f05c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:31.970827 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f973707-8c03-41d5-8731-168d4835f05c None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:31.977339 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7939639f-7150-4074-87cc-5223275144ce None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:31.986988 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 19/44] 10.222.0.57 () {60 vars in 1131 bytes} [Thu May 19 14:09:31 2022] POST /identity/v3/roles => generated 223 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:09:31.988105 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f973707-8c03-41d5-8731-168d4835f05c None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:32.041076 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b69a12-a57f-4cdb-933d-c737bdd1b3b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:32.042012 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b69a12-a57f-4cdb-933d-c737bdd1b3b9 None admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:32.042308 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b69a12-a57f-4cdb-933d-c737bdd1b3b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:32.042600 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b69a12-a57f-4cdb-933d-c737bdd1b3b9 None admin] PATH_INFO: `/v3/domains/default/users/d4608701c57147b39ea0df421d2e5029/roles/4946cf0ff7164307b36efeee7c787146` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:32.044769 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b69a12-a57f-4cdb-933d-c737bdd1b3b9 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=d4608701c57147b39ea0df421d2e5029, role_id=4946cf0ff7164307b36efeee7c787146)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:32.046327 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b69a12-a57f-4cdb-933d-c737bdd1b3b9 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:32.068568 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 26/45] 10.222.0.57 () {58 vars in 1361 bytes} [Thu May 19 14:09:31 2022] PUT /identity/v3/domains/default/users/d4608701c57147b39ea0df421d2e5029/roles/4946cf0ff7164307b36efeee7c787146 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:09:32.103772 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:32.151948 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f973707-8c03-41d5-8731-168d4835f05c None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:32.193252 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e769a038-8486-4186-a12b-0090bdfe65b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:32.194534 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e769a038-8486-4186-a12b-0090bdfe65b8 None admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:32.195071 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e769a038-8486-4186-a12b-0090bdfe65b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:32.195353 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e769a038-8486-4186-a12b-0090bdfe65b8 None admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:32.196094 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e769a038-8486-4186-a12b-0090bdfe65b8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:32.197664 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e769a038-8486-4186-a12b-0090bdfe65b8 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:32.230328 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 20/46] 10.222.0.57 () {60 vars in 1141 bytes} [Thu May 19 14:09:31 2022] POST /identity/v3/projects => generated 312 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:09:34.325619 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8600580-a039-4d20-a32b-f937cb3604ca None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:34.326166 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8600580-a039-4d20-a32b-f937cb3604ca None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:34.326278 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8600580-a039-4d20-a32b-f937cb3604ca None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:34.327600 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 27/47] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:09:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:09:34.332832 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da863028-fc5a-466a-8662-37c01b0e8184 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:34.333223 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da863028-fc5a-466a-8662-37c01b0e8184 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:34.333761 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da863028-fc5a-466a-8662-37c01b0e8184 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:34.351149 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-da863028-fc5a-466a-8662-37c01b0e8184 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:09:34.352322 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-da863028-fc5a-466a-8662-37c01b0e8184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:09:34.418452 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 21/48] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:09:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:09:34.441731 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8600580-a039-4d20-a32b-f937cb3604ca None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:34.447454 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f785c6cd-30cf-4dee-9f76-d3e92106c5db None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:34.447749 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f785c6cd-30cf-4dee-9f76-d3e92106c5db None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:34.448033 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f785c6cd-30cf-4dee-9f76-d3e92106c5db None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:34.449409 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 22/49] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:09:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:09:34.454381 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb725b5d-e94f-4500-ba15-dd511e19a8f5 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:34.454677 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb725b5d-e94f-4500-ba15-dd511e19a8f5 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:34.455179 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb725b5d-e94f-4500-ba15-dd511e19a8f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:34.471815 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-cb725b5d-e94f-4500-ba15-dd511e19a8f5 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:09:34.473032 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-cb725b5d-e94f-4500-ba15-dd511e19a8f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:09:34.518559 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:34.543452 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 23/50] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:09:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:09:34.544861 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8600580-a039-4d20-a32b-f937cb3604ca None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:34.547867 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:34.566869 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb725b5d-e94f-4500-ba15-dd511e19a8f5 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:34.580272 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39d80ba-d4e6-451e-809e-bbae32782eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:34.581190 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39d80ba-d4e6-451e-809e-bbae32782eca None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:34.581486 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39d80ba-d4e6-451e-809e-bbae32782eca None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:34.581773 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39d80ba-d4e6-451e-809e-bbae32782eca None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:34.592543 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39d80ba-d4e6-451e-809e-bbae32782eca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:09:34.600998 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-a39d80ba-d4e6-451e-809e-bbae32782eca None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:09:34.600998 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:09:34.600998 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:09:34.600998 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:09:34.600998 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:09:34.600998 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:09:34.600998 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:09:34.600998 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:09:34.600998 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:09:34.600998 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:09:34.600998 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:09:34.600998 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:09:34.600998 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:09:34.600998 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:09:34.600998 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:09:34.600998 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:09:34.602052 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:09:34.602052 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:09:34.602052 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:09:34.602052 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:09:34.602052 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:09:34.602052 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:09:34.602052 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:09:34.602052 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:09:34.602052 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:09:34.602052 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:09:34.602052 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:09:34.602052 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:09:34.602052 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:09:34.602052 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:09:34.602052 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:09:34.602052 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:09:34.602872 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:09:34.602872 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:09:34.602872 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:09:34.602872 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:09:34.602872 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:09:34.602872 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:09:34.602872 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:09:34.602872 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:09:34.602872 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:09:34.602872 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:09:34.602872 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:09:34.602872 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:09:34.602872 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 28/51] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:09:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:09:34.607867 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39d80ba-d4e6-451e-809e-bbae32782eca None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:34.667540 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb725b5d-e94f-4500-ba15-dd511e19a8f5 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:34.705364 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2c47a0-1428-47ec-9bd9-be2bd4902a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:34.706535 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2c47a0-1428-47ec-9bd9-be2bd4902a51 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:34.706908 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2c47a0-1428-47ec-9bd9-be2bd4902a51 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:34.707257 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2c47a0-1428-47ec-9bd9-be2bd4902a51 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:34.708793 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39d80ba-d4e6-451e-809e-bbae32782eca None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:34.709043 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2c47a0-1428-47ec-9bd9-be2bd4902a51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:34.710541 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2c47a0-1428-47ec-9bd9-be2bd4902a51 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:34.712185 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 24/52] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:09:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:09:34.717612 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2c47a0-1428-47ec-9bd9-be2bd4902a51 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:34.742871 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af49e015-f2fd-4a07-af55-0ea8461ba12c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:34.743573 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af49e015-f2fd-4a07-af55-0ea8461ba12c None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:34.743866 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af49e015-f2fd-4a07-af55-0ea8461ba12c None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:34.744145 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af49e015-f2fd-4a07-af55-0ea8461ba12c None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:34.744890 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af49e015-f2fd-4a07-af55-0ea8461ba12c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:09:34.745199 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af49e015-f2fd-4a07-af55-0ea8461ba12c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:34.746624 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af49e015-f2fd-4a07-af55-0ea8461ba12c None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:34.754712 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 29/53] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:09:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:09:34.760138 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af49e015-f2fd-4a07-af55-0ea8461ba12c None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:34.763808 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:34.787955 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:34.815918 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2c47a0-1428-47ec-9bd9-be2bd4902a51 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:34.852728 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2964857e-71fd-483a-8309-3962c88d74c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:34.853437 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2964857e-71fd-483a-8309-3962c88d74c5 None admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:34.853751 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2964857e-71fd-483a-8309-3962c88d74c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:34.854116 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2964857e-71fd-483a-8309-3962c88d74c5 None admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:34.854954 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2964857e-71fd-483a-8309-3962c88d74c5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:34.856424 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2964857e-71fd-483a-8309-3962c88d74c5 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:34.862512 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af49e015-f2fd-4a07-af55-0ea8461ba12c None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:34.884319 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 25/54] 10.222.0.57 () {60 vars in 1140 bytes} [Thu May 19 14:09:34 2022] POST /identity/v3/projects => generated 298 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 14:09:34.898663 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f98300-ae1a-40ac-a4ef-6b8836be881b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:34.899571 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f98300-ae1a-40ac-a4ef-6b8836be881b None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:34.899860 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f98300-ae1a-40ac-a4ef-6b8836be881b None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:34.900142 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f98300-ae1a-40ac-a4ef-6b8836be881b None admin] PATH_INFO: `/v3/users/d4608701c57147b39ea0df421d2e5029` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:34.901501 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f98300-ae1a-40ac-a4ef-6b8836be881b None admin] RBAC: Authorizing `identity:get_user(user_id=d4608701c57147b39ea0df421d2e5029)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:34.903077 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f98300-ae1a-40ac-a4ef-6b8836be881b None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:34.904832 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 30/55] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:09:34 2022] GET /identity/v3/users/d4608701c57147b39ea0df421d2e5029 => generated 247 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:09:34.910586 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2964857e-71fd-483a-8309-3962c88d74c5 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:35.018183 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:35.023221 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2964857e-71fd-483a-8309-3962c88d74c5 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:35.062437 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff8c807-931d-4ecc-b9d2-d50b0b94d0a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:35.063245 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff8c807-931d-4ecc-b9d2-d50b0b94d0a9 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:35.063529 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff8c807-931d-4ecc-b9d2-d50b0b94d0a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:35.063806 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff8c807-931d-4ecc-b9d2-d50b0b94d0a9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:35.064990 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff8c807-931d-4ecc-b9d2-d50b0b94d0a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:35.066479 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff8c807-931d-4ecc-b9d2-d50b0b94d0a9 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:35.069283 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 26/56] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:09:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:09:35.075063 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f98300-ae1a-40ac-a4ef-6b8836be881b None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:35.172434 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:35.179913 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f98300-ae1a-40ac-a4ef-6b8836be881b None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:35.213324 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d8abcd-35f7-48f4-be2b-1ce30b18ca41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:35.214024 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d8abcd-35f7-48f4-be2b-1ce30b18ca41 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:35.214314 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d8abcd-35f7-48f4-be2b-1ce30b18ca41 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:35.214593 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d8abcd-35f7-48f4-be2b-1ce30b18ca41 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:35.215428 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d8abcd-35f7-48f4-be2b-1ce30b18ca41 None admin] RBAC: Adding query filter params (role.id=4946cf0ff7164307b36efeee7c787146, scope.domain.id=default, user.id=d4608701c57147b39ea0df421d2e5029) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:09:35.215743 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d8abcd-35f7-48f4-be2b-1ce30b18ca41 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:35.217217 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d8abcd-35f7-48f4-be2b-1ce30b18ca41 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:35.223016 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 31/57] 10.222.0.57 () {56 vars in 1322 bytes} [Thu May 19 14:09:35 2022] GET /identity/v3/role_assignments?user.id=d4608701c57147b39ea0df421d2e5029&scope.domain.id=default&role.id=4946cf0ff7164307b36efeee7c787146 => generated 530 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:09:37.152892 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3df1d9-6007-43b4-832d-e8cfbe5e34a6 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:37.153385 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3df1d9-6007-43b4-832d-e8cfbe5e34a6 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:37.153471 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3df1d9-6007-43b4-832d-e8cfbe5e34a6 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:37.154874 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 27/58] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:09:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:09:37.160017 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cb37c3-cbc3-40e0-9631-d3608890eae9 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:37.160421 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cb37c3-cbc3-40e0-9631-d3608890eae9 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:37.160587 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cb37c3-cbc3-40e0-9631-d3608890eae9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:37.179696 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-32cb37c3-cbc3-40e0-9631-d3608890eae9 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:09:37.180874 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-32cb37c3-cbc3-40e0-9631-d3608890eae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:09:37.247453 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 32/59] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:09:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:09:37.272832 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3df1d9-6007-43b4-832d-e8cfbe5e34a6 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:37.373397 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3df1d9-6007-43b4-832d-e8cfbe5e34a6 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:37.407034 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89408d6c-f572-440b-bdbe-3eb000b29074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:37.407823 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89408d6c-f572-440b-bdbe-3eb000b29074 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:37.408114 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89408d6c-f572-440b-bdbe-3eb000b29074 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:37.408398 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89408d6c-f572-440b-bdbe-3eb000b29074 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:37.409618 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89408d6c-f572-440b-bdbe-3eb000b29074 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:37.411125 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89408d6c-f572-440b-bdbe-3eb000b29074 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:37.412925 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 28/60] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:09:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:09:37.418430 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cb37c3-cbc3-40e0-9631-d3608890eae9 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:37.501785 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:37.515008 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cb37c3-cbc3-40e0-9631-d3608890eae9 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:37.548217 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4236e9db-2298-47ed-9e21-b192fffdb89d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:37.548886 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4236e9db-2298-47ed-9e21-b192fffdb89d None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:37.549176 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4236e9db-2298-47ed-9e21-b192fffdb89d None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:37.549461 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4236e9db-2298-47ed-9e21-b192fffdb89d None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:37.550211 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4236e9db-2298-47ed-9e21-b192fffdb89d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:37.551697 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4236e9db-2298-47ed-9e21-b192fffdb89d None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:37.576818 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 33/61] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 19 14:09:37 2022] POST /identity/v3/users => generated 275 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 14:09:40.917354 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7b1253-98af-4767-bcc2-549c5b85d46b None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:40.917354 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7b1253-98af-4767-bcc2-549c5b85d46b None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:40.917354 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7b1253-98af-4767-bcc2-549c5b85d46b None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:40.917354 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 29/62] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:09:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:09:40.936381 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1b4cb9-4ad8-4e14-b905-d1e5fb5cb0ed None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:40.937270 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1b4cb9-4ad8-4e14-b905-d1e5fb5cb0ed None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:40.937739 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1b4cb9-4ad8-4e14-b905-d1e5fb5cb0ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:40.962989 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-ae1b4cb9-4ad8-4e14-b905-d1e5fb5cb0ed None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:09:40.964445 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-ae1b4cb9-4ad8-4e14-b905-d1e5fb5cb0ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:09:41.017537 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3b93d5-7ea5-467a-ae52-c0c0c3e67407 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:41.017537 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3b93d5-7ea5-467a-ae52-c0c0c3e67407 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:41.017537 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3b93d5-7ea5-467a-ae52-c0c0c3e67407 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:41.019330 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 30/63] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:09:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:09:41.025898 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3356b509-8329-454b-9a17-2af4e5ebfe9b None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:41.026523 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3356b509-8329-454b-9a17-2af4e5ebfe9b None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:41.027018 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3356b509-8329-454b-9a17-2af4e5ebfe9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:41.055440 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-3356b509-8329-454b-9a17-2af4e5ebfe9b None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:09:41.057867 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-3356b509-8329-454b-9a17-2af4e5ebfe9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:09:41.088694 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 34/64] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:09:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:09:41.111179 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1b4cb9-4ad8-4e14-b905-d1e5fb5cb0ed None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:41.130499 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:41.209184 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 31/65] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:09:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:09:41.219803 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f60648-5b5c-4fd9-8818-bcf33a5a581c None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:41.220104 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f60648-5b5c-4fd9-8818-bcf33a5a581c None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:41.220378 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f60648-5b5c-4fd9-8818-bcf33a5a581c None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:41.222080 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 32/66] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:09:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:09:41.231624 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8864f08-7caa-4868-9767-b1b7d536c8ec None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:41.232171 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8864f08-7caa-4868-9767-b1b7d536c8ec None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:41.232645 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8864f08-7caa-4868-9767-b1b7d536c8ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:41.253476 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-a8864f08-7caa-4868-9767-b1b7d536c8ec None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:09:41.256153 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-a8864f08-7caa-4868-9767-b1b7d536c8ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:09:41.258982 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:41.290080 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1b4cb9-4ad8-4e14-b905-d1e5fb5cb0ed None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:41.333879 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544e566a-7c4e-4c9a-9919-08d105087851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:41.335044 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544e566a-7c4e-4c9a-9919-08d105087851 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:41.335561 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544e566a-7c4e-4c9a-9919-08d105087851 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:41.335898 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544e566a-7c4e-4c9a-9919-08d105087851 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:41.349991 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 33/67] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:09:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:09:41.351340 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8864f08-7caa-4868-9767-b1b7d536c8ec None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:41.351991 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544e566a-7c4e-4c9a-9919-08d105087851 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:09:41.368180 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-544e566a-7c4e-4c9a-9919-08d105087851 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:09:41.368180 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:09:41.368180 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:09:41.368180 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:09:41.368180 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:09:41.368180 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:09:41.368180 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:09:41.368180 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:09:41.368180 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:09:41.368180 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:09:41.368180 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:09:41.368180 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:09:41.368180 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:09:41.368180 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:09:41.368180 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:09:41.368180 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:09:41.369243 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:09:41.369243 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:09:41.369243 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:09:41.369243 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:09:41.369243 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:09:41.369243 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:09:41.369243 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:09:41.369243 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:09:41.369243 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:09:41.369243 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:09:41.369243 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:09:41.369243 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:09:41.369243 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:09:41.369243 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:09:41.369243 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:09:41.369243 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:09:41.370240 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:09:41.370240 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:09:41.370240 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:09:41.370240 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:09:41.370240 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:09:41.370240 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:09:41.370240 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:09:41.370240 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:09:41.370240 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:09:41.370240 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:09:41.370240 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:09:41.370240 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:09:41.372218 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 35/68] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:09:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:09:41.375043 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544e566a-7c4e-4c9a-9919-08d105087851 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:41.490708 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544e566a-7c4e-4c9a-9919-08d105087851 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:41.499334 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8864f08-7caa-4868-9767-b1b7d536c8ec None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:41.502190 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:41.536902 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2022f224-87c1-4c12-94f2-68d7beb5083b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:41.538134 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62386ab7-7968-42d8-92e5-60fb73a0c461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:41.538687 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2022f224-87c1-4c12-94f2-68d7beb5083b None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:41.538970 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62386ab7-7968-42d8-92e5-60fb73a0c461 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:41.539265 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62386ab7-7968-42d8-92e5-60fb73a0c461 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:41.539551 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62386ab7-7968-42d8-92e5-60fb73a0c461 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:41.539809 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2022f224-87c1-4c12-94f2-68d7beb5083b None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:41.540300 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2022f224-87c1-4c12-94f2-68d7beb5083b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:41.542014 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:41.555336 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2022f224-87c1-4c12-94f2-68d7beb5083b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:09:41.555613 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62386ab7-7968-42d8-92e5-60fb73a0c461 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:09:41.563692 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-2022f224-87c1-4c12-94f2-68d7beb5083b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:09:41.563692 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:09:41.563692 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:09:41.563692 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:09:41.563692 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:09:41.563692 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:09:41.563692 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:09:41.563692 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:09:41.563692 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:09:41.563692 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:09:41.563692 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:09:41.563692 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:09:41.563692 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:09:41.563692 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:09:41.563692 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:09:41.563692 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:09:41.564605 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:09:41.564605 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:09:41.564605 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:09:41.564605 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:09:41.564605 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:09:41.564605 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:09:41.564605 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:09:41.564605 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:09:41.564605 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:09:41.564605 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:09:41.564605 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:09:41.564605 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:09:41.564605 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:09:41.564605 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:09:41.564605 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:09:41.564605 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:09:41.565387 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:09:41.565387 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:09:41.565387 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:09:41.565387 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:09:41.565387 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:09:41.565387 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:09:41.565387 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:09:41.565387 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:09:41.565387 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:09:41.565387 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:09:41.565387 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:09:41.565387 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:09:41.565986 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-62386ab7-7968-42d8-92e5-60fb73a0c461 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:09:41.565986 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:09:41.565986 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:09:41.565986 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:09:41.565986 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:09:41.565986 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:09:41.565986 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:09:41.565986 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:09:41.565986 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:09:41.565986 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:09:41.565986 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:09:41.565986 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:09:41.565986 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:09:41.565986 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:09:41.565986 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:09:41.565986 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:09:41.566770 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:09:41.566770 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:09:41.566770 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:09:41.566770 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:09:41.566770 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:09:41.566770 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:09:41.566770 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:09:41.566770 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:09:41.566770 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:09:41.566770 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:09:41.566770 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:09:41.566770 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:09:41.566770 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:09:41.566770 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:09:41.566770 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:09:41.566770 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:09:41.567587 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:09:41.567587 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:09:41.567587 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:09:41.567587 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:09:41.567587 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:09:41.567587 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:09:41.567587 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:09:41.567587 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:09:41.567587 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:09:41.567587 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:09:41.567587 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:09:41.567587 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:09:41.567587 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 36/69] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 19 14:09:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:09:41.567587 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62386ab7-7968-42d8-92e5-60fb73a0c461 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:41.570684 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 34/70] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:09:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:09:41.573063 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2022f224-87c1-4c12-94f2-68d7beb5083b None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:41.690590 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62386ab7-7968-42d8-92e5-60fb73a0c461 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:41.703998 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2022f224-87c1-4c12-94f2-68d7beb5083b None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:41.730187 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbf674d-2da9-4a7f-9b4e-8de76e199625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:41.731367 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbf674d-2da9-4a7f-9b4e-8de76e199625 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:41.731854 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbf674d-2da9-4a7f-9b4e-8de76e199625 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:41.732356 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbf674d-2da9-4a7f-9b4e-8de76e199625 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:41.733337 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbf674d-2da9-4a7f-9b4e-8de76e199625 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:09:41.733874 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbf674d-2da9-4a7f-9b4e-8de76e199625 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:41.735534 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbf674d-2da9-4a7f-9b4e-8de76e199625 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:41.745144 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 37/71] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:09:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:09:41.746387 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbf674d-2da9-4a7f-9b4e-8de76e199625 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:41.746856 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4c4a51-d458-4b47-8fcc-b7af002fcf4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:41.747651 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4c4a51-d458-4b47-8fcc-b7af002fcf4d None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:41.747940 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4c4a51-d458-4b47-8fcc-b7af002fcf4d None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:41.748387 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4c4a51-d458-4b47-8fcc-b7af002fcf4d None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:41.749316 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4c4a51-d458-4b47-8fcc-b7af002fcf4d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:09:41.749787 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4c4a51-d458-4b47-8fcc-b7af002fcf4d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:41.751336 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4c4a51-d458-4b47-8fcc-b7af002fcf4d None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:41.764900 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 35/72] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:09:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:09:41.766077 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e8c302-a47a-4893-9f6d-236291d52cc2 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:41.766543 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e8c302-a47a-4893-9f6d-236291d52cc2 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:41.766951 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e8c302-a47a-4893-9f6d-236291d52cc2 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:41.768753 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 36/73] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:09:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:09:41.769604 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e8c302-a47a-4893-9f6d-236291d52cc2 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:41.790618 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:41.806476 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:41.859110 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbf674d-2da9-4a7f-9b4e-8de76e199625 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:41.881368 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e8c302-a47a-4893-9f6d-236291d52cc2 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:41.896636 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65216c66-b7ca-4cab-a1cf-be2b818429b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:41.897429 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65216c66-b7ca-4cab-a1cf-be2b818429b9 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:41.897728 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65216c66-b7ca-4cab-a1cf-be2b818429b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:41.898060 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65216c66-b7ca-4cab-a1cf-be2b818429b9 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:41.898934 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65216c66-b7ca-4cab-a1cf-be2b818429b9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:09:41.899248 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65216c66-b7ca-4cab-a1cf-be2b818429b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:41.900645 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65216c66-b7ca-4cab-a1cf-be2b818429b9 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:41.908854 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 38/74] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:09:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:09:41.909719 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65216c66-b7ca-4cab-a1cf-be2b818429b9 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:41.916354 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ebbc8f-6143-4e3d-93fb-a43855732667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:41.917230 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ebbc8f-6143-4e3d-93fb-a43855732667 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:41.917515 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ebbc8f-6143-4e3d-93fb-a43855732667 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:41.918064 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ebbc8f-6143-4e3d-93fb-a43855732667 None admin] PATH_INFO: `/v3/users/d4608701c57147b39ea0df421d2e5029` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:41.919376 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ebbc8f-6143-4e3d-93fb-a43855732667 None admin] RBAC: Authorizing `identity:get_user(user_id=d4608701c57147b39ea0df421d2e5029)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:41.920487 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ebbc8f-6143-4e3d-93fb-a43855732667 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:41.922182 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 37/75] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:09:41 2022] GET /identity/v3/users/d4608701c57147b39ea0df421d2e5029 => generated 247 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:09:41.923382 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aeaf7c-a60e-4d69-97d5-11e9331217ec None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:41.923819 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aeaf7c-a60e-4d69-97d5-11e9331217ec None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:41.924263 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aeaf7c-a60e-4d69-97d5-11e9331217ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:41.941227 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-e3aeaf7c-a60e-4d69-97d5-11e9331217ec None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:09:41.942543 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-e3aeaf7c-a60e-4d69-97d5-11e9331217ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:09:42.009875 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 38/76] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:09:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:09:42.011100 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d890b5b5-8cf1-46a2-9e35-8db938b7d6b2 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:42.011389 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d890b5b5-8cf1-46a2-9e35-8db938b7d6b2 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:42.011667 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d890b5b5-8cf1-46a2-9e35-8db938b7d6b2 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:42.012924 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 39/77] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:09:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:09:42.017281 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d890b5b5-8cf1-46a2-9e35-8db938b7d6b2 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:42.017387 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65216c66-b7ca-4cab-a1cf-be2b818429b9 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:42.033478 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:42.035196 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:42.055332 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660e1c26-cc3a-4a59-8add-57d83c89fcad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:42.056212 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660e1c26-cc3a-4a59-8add-57d83c89fcad None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:42.056505 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660e1c26-cc3a-4a59-8add-57d83c89fcad None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:42.056789 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660e1c26-cc3a-4a59-8add-57d83c89fcad None admin] PATH_INFO: `/v3/users/47b5316234e94937b78cedeab15c486e` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:42.070494 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660e1c26-cc3a-4a59-8add-57d83c89fcad None admin] RBAC: Authorizing `identity:get_user(user_id=47b5316234e94937b78cedeab15c486e)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:42.071988 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660e1c26-cc3a-4a59-8add-57d83c89fcad None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:42.073708 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 39/78] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:09:41 2022] GET /identity/v3/users/47b5316234e94937b78cedeab15c486e => generated 275 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:09:42.074514 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660e1c26-cc3a-4a59-8add-57d83c89fcad None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:42.125501 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d890b5b5-8cf1-46a2-9e35-8db938b7d6b2 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:42.161776 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4c61ad-5154-44e9-a298-e42056b8ce5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:42.162685 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4c61ad-5154-44e9-a298-e42056b8ce5d None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:42.162977 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4c61ad-5154-44e9-a298-e42056b8ce5d None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:42.163258 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4c61ad-5154-44e9-a298-e42056b8ce5d None admin] PATH_INFO: `/v3/users/47b5316234e94937b78cedeab15c486e` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:42.164492 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4c61ad-5154-44e9-a298-e42056b8ce5d None admin] RBAC: Authorizing `identity:get_user(user_id=47b5316234e94937b78cedeab15c486e)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:42.165994 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4c61ad-5154-44e9-a298-e42056b8ce5d None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:42.167652 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 40/79] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:09:42 2022] GET /identity/v3/users/47b5316234e94937b78cedeab15c486e => generated 275 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:09:42.168810 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182191cc-949e-4421-b633-1d1423b49ef0 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:42.169098 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182191cc-949e-4421-b633-1d1423b49ef0 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:42.169395 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182191cc-949e-4421-b633-1d1423b49ef0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:42.176773 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660e1c26-cc3a-4a59-8add-57d83c89fcad None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:42.187351 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-182191cc-949e-4421-b633-1d1423b49ef0 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:09:42.188573 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-182191cc-949e-4421-b633-1d1423b49ef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:09:42.210327 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9e7b3d-f4c1-4806-9948-1887f750b9a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:42.210984 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9e7b3d-f4c1-4806-9948-1887f750b9a6 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:42.211279 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9e7b3d-f4c1-4806-9948-1887f750b9a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:42.211562 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9e7b3d-f4c1-4806-9948-1887f750b9a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:42.255625 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 41/80] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:09:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:09:42.256465 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182191cc-949e-4421-b633-1d1423b49ef0 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:42.256611 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:42.284806 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9e7b3d-f4c1-4806-9948-1887f750b9a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:42.286927 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9e7b3d-f4c1-4806-9948-1887f750b9a6 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:42.289870 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:42.375950 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182191cc-949e-4421-b633-1d1423b49ef0 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:42.386104 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 40/81] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:09:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:09:42.387141 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9e7b3d-f4c1-4806-9948-1887f750b9a6 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:42.410118 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4e71d3-1268-4a34-968d-afb35f3fea56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:42.410921 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4e71d3-1268-4a34-968d-afb35f3fea56 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:42.411851 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4e71d3-1268-4a34-968d-afb35f3fea56 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:42.412132 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4e71d3-1268-4a34-968d-afb35f3fea56 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:42.422248 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4e71d3-1268-4a34-968d-afb35f3fea56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:09:42.429867 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-dd4e71d3-1268-4a34-968d-afb35f3fea56 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:09:42.429867 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:09:42.429867 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:09:42.429867 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:09:42.429867 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:09:42.429867 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:09:42.429867 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:09:42.429867 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:09:42.429867 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:09:42.429867 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:09:42.429867 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:09:42.429867 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:09:42.429867 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:09:42.429867 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:09:42.429867 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:09:42.429867 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:09:42.430705 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:09:42.430705 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:09:42.430705 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:09:42.430705 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:09:42.430705 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:09:42.430705 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:09:42.430705 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:09:42.430705 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:09:42.430705 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:09:42.430705 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:09:42.430705 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:09:42.430705 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:09:42.430705 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:09:42.430705 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:09:42.430705 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:09:42.430705 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:09:42.431491 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:09:42.431491 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:09:42.431491 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:09:42.431491 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:09:42.431491 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:09:42.431491 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:09:42.431491 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:09:42.431491 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:09:42.431491 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:09:42.431491 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:09:42.431491 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:09:42.431491 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:09:42.431491 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 42/82] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:09:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:09:42.432118 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4e71d3-1268-4a34-968d-afb35f3fea56 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:42.468244 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:42.492333 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9e7b3d-f4c1-4806-9948-1887f750b9a6 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:42.501864 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:42.526141 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daeabc5b-6e76-402a-a96e-d1a06311441a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:42.526789 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeabc5b-6e76-402a-a96e-d1a06311441a None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:42.527069 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeabc5b-6e76-402a-a96e-d1a06311441a None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:42.527343 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeabc5b-6e76-402a-a96e-d1a06311441a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:42.533446 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4e71d3-1268-4a34-968d-afb35f3fea56 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:42.567514 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08b689b-0874-4118-aae8-3602cf6a5879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:42.568093 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08b689b-0874-4118-aae8-3602cf6a5879 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:42.568370 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08b689b-0874-4118-aae8-3602cf6a5879 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:42.568657 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08b689b-0874-4118-aae8-3602cf6a5879 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:42.594230 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daeabc5b-6e76-402a-a96e-d1a06311441a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:42.595740 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daeabc5b-6e76-402a-a96e-d1a06311441a None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:42.635414 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08b689b-0874-4118-aae8-3602cf6a5879 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:42.636894 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08b689b-0874-4118-aae8-3602cf6a5879 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:42.671994 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:42.698028 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 41/83] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:09:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:09:42.698874 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daeabc5b-6e76-402a-a96e-d1a06311441a None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:42.711472 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:42.745752 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 43/84] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:09:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:09:42.750498 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08b689b-0874-4118-aae8-3602cf6a5879 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:42.805309 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daeabc5b-6e76-402a-a96e-d1a06311441a None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:42.838962 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ff9a43-80cc-4f79-bb3c-2cb52203f5f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:42.839749 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ff9a43-80cc-4f79-bb3c-2cb52203f5f4 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:42.840043 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ff9a43-80cc-4f79-bb3c-2cb52203f5f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:42.840328 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ff9a43-80cc-4f79-bb3c-2cb52203f5f4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:42.841515 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ff9a43-80cc-4f79-bb3c-2cb52203f5f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:42.843019 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ff9a43-80cc-4f79-bb3c-2cb52203f5f4 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:42.844655 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 42/85] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:09:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:09:42.846999 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ff9a43-80cc-4f79-bb3c-2cb52203f5f4 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:42.849783 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08b689b-0874-4118-aae8-3602cf6a5879 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:42.883523 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422c91bc-4ec1-4e91-b4a3-391b3158e1e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:42.884378 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422c91bc-4ec1-4e91-b4a3-391b3158e1e4 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:42.884662 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422c91bc-4ec1-4e91-b4a3-391b3158e1e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:42.884937 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422c91bc-4ec1-4e91-b4a3-391b3158e1e4 None admin] PATH_INFO: `/v3/projects/ed85b374a73a4c94a310482b7ba566f3` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:42.886155 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422c91bc-4ec1-4e91-b4a3-391b3158e1e4 None admin] RBAC: Authorizing `identity:get_project(project_id=ed85b374a73a4c94a310482b7ba566f3)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:42.887698 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422c91bc-4ec1-4e91-b4a3-391b3158e1e4 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:42.889404 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 44/86] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:09:42 2022] GET /identity/v3/projects/ed85b374a73a4c94a310482b7ba566f3 => generated 298 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:09:42.890207 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422c91bc-4ec1-4e91-b4a3-391b3158e1e4 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:42.895338 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:42.935413 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:42.948630 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ff9a43-80cc-4f79-bb3c-2cb52203f5f4 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:42.982744 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3713b1a6-887d-4dd8-b6ea-575ce4cfc10e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:42.983412 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3713b1a6-887d-4dd8-b6ea-575ce4cfc10e None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:42.983697 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3713b1a6-887d-4dd8-b6ea-575ce4cfc10e None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:42.983971 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3713b1a6-887d-4dd8-b6ea-575ce4cfc10e None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:42.984709 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3713b1a6-887d-4dd8-b6ea-575ce4cfc10e None admin] RBAC: Adding query filter params (name=member) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:09:42.985010 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3713b1a6-887d-4dd8-b6ea-575ce4cfc10e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:42.986427 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3713b1a6-887d-4dd8-b6ea-575ce4cfc10e None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:42.996081 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 43/87] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:09:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:09:42.996912 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3713b1a6-887d-4dd8-b6ea-575ce4cfc10e None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:42.997173 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422c91bc-4ec1-4e91-b4a3-391b3158e1e4 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:43.035359 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b75190-d6db-4fd1-9155-8cc70cdadf6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:43.036787 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b75190-d6db-4fd1-9155-8cc70cdadf6d None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:43.037070 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b75190-d6db-4fd1-9155-8cc70cdadf6d None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:43.037347 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b75190-d6db-4fd1-9155-8cc70cdadf6d None admin] PATH_INFO: `/v3/projects/ed85b374a73a4c94a310482b7ba566f3` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:43.038579 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b75190-d6db-4fd1-9155-8cc70cdadf6d None admin] RBAC: Authorizing `identity:get_project(project_id=ed85b374a73a4c94a310482b7ba566f3)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:43.040138 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b75190-d6db-4fd1-9155-8cc70cdadf6d None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:43.041855 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 45/88] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:09:42 2022] GET /identity/v3/projects/ed85b374a73a4c94a310482b7ba566f3 => generated 298 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:09:43.042628 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b75190-d6db-4fd1-9155-8cc70cdadf6d None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:43.102670 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3713b1a6-887d-4dd8-b6ea-575ce4cfc10e None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:43.109697 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:43.140067 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab00ad97-3613-4fb3-8c36-4adb00823ed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:43.140928 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab00ad97-3613-4fb3-8c36-4adb00823ed9 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:43.141211 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab00ad97-3613-4fb3-8c36-4adb00823ed9 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:43.141489 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab00ad97-3613-4fb3-8c36-4adb00823ed9 None admin] PATH_INFO: `/v3/projects/ed85b374a73a4c94a310482b7ba566f3` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:43.142791 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab00ad97-3613-4fb3-8c36-4adb00823ed9 None admin] RBAC: Authorizing `identity:get_project(project_id=ed85b374a73a4c94a310482b7ba566f3)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:43.144266 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab00ad97-3613-4fb3-8c36-4adb00823ed9 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:43.144441 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b75190-d6db-4fd1-9155-8cc70cdadf6d None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:43.146527 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 44/89] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:09:42 2022] GET /identity/v3/projects/ed85b374a73a4c94a310482b7ba566f3 => generated 298 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:09:43.147295 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab00ad97-3613-4fb3-8c36-4adb00823ed9 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:43.155285 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:43.182141 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddb910f-5973-4864-b58d-789cc42042f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:43.182823 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddb910f-5973-4864-b58d-789cc42042f1 None admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:43.183114 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddb910f-5973-4864-b58d-789cc42042f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:43.183397 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddb910f-5973-4864-b58d-789cc42042f1 None admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:43.184117 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddb910f-5973-4864-b58d-789cc42042f1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:43.185821 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddb910f-5973-4864-b58d-789cc42042f1 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:43.221874 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 46/90] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 19 14:09:43 2022] POST /identity/v3/users => generated 289 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:09:43.221874 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddb910f-5973-4864-b58d-789cc42042f1 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:43.253359 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab00ad97-3613-4fb3-8c36-4adb00823ed9 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:43.291960 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacd0af2-81a7-47a7-be36-81c3f7d1db90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:43.292638 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacd0af2-81a7-47a7-be36-81c3f7d1db90 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:43.292924 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacd0af2-81a7-47a7-be36-81c3f7d1db90 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:43.293206 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacd0af2-81a7-47a7-be36-81c3f7d1db90 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:43.294104 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacd0af2-81a7-47a7-be36-81c3f7d1db90 None admin] RBAC: Adding query filter params (role.id=4946cf0ff7164307b36efeee7c787146, scope.project.id=ed85b374a73a4c94a310482b7ba566f3, user.id=d4608701c57147b39ea0df421d2e5029) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:09:43.294420 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacd0af2-81a7-47a7-be36-81c3f7d1db90 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:43.295949 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacd0af2-81a7-47a7-be36-81c3f7d1db90 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:43.302361 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 45/91] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:09:43 2022] GET /identity/v3/role_assignments?user.id=d4608701c57147b39ea0df421d2e5029&scope.project.id=ed85b374a73a4c94a310482b7ba566f3&role.id=4946cf0ff7164307b36efeee7c787146 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:09:43.303087 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacd0af2-81a7-47a7-be36-81c3f7d1db90 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:43.351843 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:43.355889 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddb910f-5973-4864-b58d-789cc42042f1 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:43.403170 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66711674-9b68-4155-a59c-2c19dd5fb5b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:43.404524 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66711674-9b68-4155-a59c-2c19dd5fb5b5 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:43.405058 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66711674-9b68-4155-a59c-2c19dd5fb5b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:43.405579 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66711674-9b68-4155-a59c-2c19dd5fb5b5 None admin] PATH_INFO: `/v3/users/47b5316234e94937b78cedeab15c486e` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:43.407455 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66711674-9b68-4155-a59c-2c19dd5fb5b5 None admin] RBAC: Authorizing `identity:get_user(user_id=47b5316234e94937b78cedeab15c486e)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:43.409225 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66711674-9b68-4155-a59c-2c19dd5fb5b5 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:43.413429 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 47/92] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:09:43 2022] GET /identity/v3/users/47b5316234e94937b78cedeab15c486e => generated 275 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:09:43.414498 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66711674-9b68-4155-a59c-2c19dd5fb5b5 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:43.425132 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacd0af2-81a7-47a7-be36-81c3f7d1db90 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:43.468176 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:43.481607 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db60fac-6f83-4e3c-864e-4d26e5c77935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:43.482369 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db60fac-6f83-4e3c-864e-4d26e5c77935 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:43.483281 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db60fac-6f83-4e3c-864e-4d26e5c77935 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:43.483576 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db60fac-6f83-4e3c-864e-4d26e5c77935 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:43.484441 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db60fac-6f83-4e3c-864e-4d26e5c77935 None admin] RBAC: Adding query filter params (role.id=23d9dd31632340bdba9cf3093c095dbb, scope.project.id=ed85b374a73a4c94a310482b7ba566f3, user.id=47b5316234e94937b78cedeab15c486e) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:09:43.485139 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db60fac-6f83-4e3c-864e-4d26e5c77935 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:43.488741 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db60fac-6f83-4e3c-864e-4d26e5c77935 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:43.496051 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 46/93] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:09:43 2022] GET /identity/v3/role_assignments?user.id=47b5316234e94937b78cedeab15c486e&scope.project.id=ed85b374a73a4c94a310482b7ba566f3&role.id=23d9dd31632340bdba9cf3093c095dbb => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:09:43.497693 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db60fac-6f83-4e3c-864e-4d26e5c77935 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:43.570223 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66711674-9b68-4155-a59c-2c19dd5fb5b5 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:43.628451 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7f7573-5cfa-456b-bd3b-b12acd6a6084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:43.630451 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7f7573-5cfa-456b-bd3b-b12acd6a6084 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:43.630948 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7f7573-5cfa-456b-bd3b-b12acd6a6084 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:43.631871 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7f7573-5cfa-456b-bd3b-b12acd6a6084 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:43.632949 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7f7573-5cfa-456b-bd3b-b12acd6a6084 None admin] RBAC: Adding query filter params (role.id=7d0e162805b54c019ce6c954cbef3818, scope.project.id=ed85b374a73a4c94a310482b7ba566f3, user.id=47b5316234e94937b78cedeab15c486e) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:09:43.633468 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7f7573-5cfa-456b-bd3b-b12acd6a6084 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:43.636042 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7f7573-5cfa-456b-bd3b-b12acd6a6084 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:43.651544 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 48/94] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:09:43 2022] GET /identity/v3/role_assignments?user.id=47b5316234e94937b78cedeab15c486e&scope.project.id=ed85b374a73a4c94a310482b7ba566f3&role.id=7d0e162805b54c019ce6c954cbef3818 => generated 259 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:09:43.656734 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db60fac-6f83-4e3c-864e-4d26e5c77935 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:43.659599 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:43.711414 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339dd98c-160f-46e7-91bf-50eeeaf8e419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:43.712468 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339dd98c-160f-46e7-91bf-50eeeaf8e419 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:43.712959 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339dd98c-160f-46e7-91bf-50eeeaf8e419 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:43.713435 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339dd98c-160f-46e7-91bf-50eeeaf8e419 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:43.797009 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339dd98c-160f-46e7-91bf-50eeeaf8e419 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:43.798905 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339dd98c-160f-46e7-91bf-50eeeaf8e419 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:43.918889 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 47/95] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:09:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:09:43.924338 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7f7573-5cfa-456b-bd3b-b12acd6a6084 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:44.044598 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:44.055616 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7f7573-5cfa-456b-bd3b-b12acd6a6084 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:44.105389 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657b695e-9148-4bc6-9bd8-4feb184cc1f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:44.106306 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657b695e-9148-4bc6-9bd8-4feb184cc1f7 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:44.106589 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657b695e-9148-4bc6-9bd8-4feb184cc1f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:44.106877 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657b695e-9148-4bc6-9bd8-4feb184cc1f7 None admin] PATH_INFO: `/v3/projects/5d9dd3087a0e490682a3b6c6a27f9361` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:44.108144 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657b695e-9148-4bc6-9bd8-4feb184cc1f7 None admin] RBAC: Authorizing `identity:get_project(project_id=5d9dd3087a0e490682a3b6c6a27f9361)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:44.109630 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657b695e-9148-4bc6-9bd8-4feb184cc1f7 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:44.111711 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 49/96] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:09:43 2022] GET /identity/v3/projects/5d9dd3087a0e490682a3b6c6a27f9361 => generated 312 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:09:44.118491 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339dd98c-160f-46e7-91bf-50eeeaf8e419 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:44.121233 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:44.267941 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339dd98c-160f-46e7-91bf-50eeeaf8e419 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:44.314784 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf9cb78-3eb2-4854-a0ac-341f76e95313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:44.315825 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf9cb78-3eb2-4854-a0ac-341f76e95313 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:44.316303 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf9cb78-3eb2-4854-a0ac-341f76e95313 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:44.316769 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf9cb78-3eb2-4854-a0ac-341f76e95313 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:44.317871 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf9cb78-3eb2-4854-a0ac-341f76e95313 None admin] RBAC: Adding query filter params (role.id=7d0e162805b54c019ce6c954cbef3818, scope.project.id=5d9dd3087a0e490682a3b6c6a27f9361, user.id=47b5316234e94937b78cedeab15c486e) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:09:44.318403 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf9cb78-3eb2-4854-a0ac-341f76e95313 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:44.320068 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf9cb78-3eb2-4854-a0ac-341f76e95313 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:44.330161 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 48/97] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:09:44 2022] GET /identity/v3/role_assignments?user.id=47b5316234e94937b78cedeab15c486e&scope.project.id=5d9dd3087a0e490682a3b6c6a27f9361&role.id=7d0e162805b54c019ce6c954cbef3818 => generated 259 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:09:46.266089 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93087647-04d9-4966-97d7-ae98ba374acd None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:46.267141 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93087647-04d9-4966-97d7-ae98ba374acd None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:46.267660 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93087647-04d9-4966-97d7-ae98ba374acd None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:46.269685 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 50/98] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:09:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:09:46.275532 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a486f5-6930-469d-9854-f562975efca2 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:46.276412 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a486f5-6930-469d-9854-f562975efca2 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:46.276908 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a486f5-6930-469d-9854-f562975efca2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:46.295493 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-02a486f5-6930-469d-9854-f562975efca2 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:09:46.297044 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-02a486f5-6930-469d-9854-f562975efca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:09:46.361518 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:46.370553 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 49/99] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:09:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:09:46.393610 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93087647-04d9-4966-97d7-ae98ba374acd None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:46.507099 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93087647-04d9-4966-97d7-ae98ba374acd None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:46.559492 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dbfac7-7576-41c3-bd6b-b27d2edaeb4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:46.560324 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dbfac7-7576-41c3-bd6b-b27d2edaeb4b None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:46.560620 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dbfac7-7576-41c3-bd6b-b27d2edaeb4b None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:46.560908 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dbfac7-7576-41c3-bd6b-b27d2edaeb4b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:46.577024 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dbfac7-7576-41c3-bd6b-b27d2edaeb4b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:09:46.585490 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-f3dbfac7-7576-41c3-bd6b-b27d2edaeb4b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:09:46.585490 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:09:46.585490 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:09:46.585490 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:09:46.585490 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:09:46.585490 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:09:46.585490 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:09:46.585490 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:09:46.585490 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:09:46.585490 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:09:46.585490 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:09:46.585490 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:09:46.585490 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:09:46.585490 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:09:46.585490 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:09:46.585490 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:09:46.586476 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:09:46.586476 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:09:46.586476 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:09:46.586476 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:09:46.586476 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:09:46.586476 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:09:46.586476 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:09:46.586476 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:09:46.586476 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:09:46.586476 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:09:46.586476 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:09:46.586476 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:09:46.586476 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:09:46.586476 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:09:46.586476 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:09:46.586476 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:09:46.587272 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:09:46.587272 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:09:46.587272 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:09:46.587272 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:09:46.587272 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:09:46.587272 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:09:46.587272 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:09:46.587272 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:09:46.587272 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:09:46.587272 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:09:46.587272 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:09:46.587272 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:09:46.587272 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 51/100] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:09:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:09:46.592696 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a486f5-6930-469d-9854-f562975efca2 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:46.751324 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a486f5-6930-469d-9854-f562975efca2 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:46.798786 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4854ef9-df0e-43d8-9731-066f239185c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:46.799876 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4854ef9-df0e-43d8-9731-066f239185c5 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:46.800364 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4854ef9-df0e-43d8-9731-066f239185c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:46.800907 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4854ef9-df0e-43d8-9731-066f239185c5 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:46.802518 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4854ef9-df0e-43d8-9731-066f239185c5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:09:46.803038 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4854ef9-df0e-43d8-9731-066f239185c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:46.804650 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4854ef9-df0e-43d8-9731-066f239185c5 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:46.818137 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 50/101] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:09:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:09:46.826976 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dbfac7-7576-41c3-bd6b-b27d2edaeb4b None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:46.837660 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457805d0-1930-4ea2-a1e7-37f8b06c143c None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:46.838042 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457805d0-1930-4ea2-a1e7-37f8b06c143c None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:46.838327 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457805d0-1930-4ea2-a1e7-37f8b06c143c None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:46.839761 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 51/102] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:09:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:09:46.845174 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00c31da-8fc0-424a-8a30-f18e664ad14f None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:46.845469 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00c31da-8fc0-424a-8a30-f18e664ad14f None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:46.845753 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00c31da-8fc0-424a-8a30-f18e664ad14f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:46.868294 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-e00c31da-8fc0-424a-8a30-f18e664ad14f None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:09:46.869504 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-e00c31da-8fc0-424a-8a30-f18e664ad14f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:09:46.895270 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:46.943570 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:46.964670 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 52/103] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:09:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:09:46.968231 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb856c2-344d-49bf-8866-349882c0182c None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:46.968231 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb856c2-344d-49bf-8866-349882c0182c None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:46.968231 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb856c2-344d-49bf-8866-349882c0182c None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:46.968791 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 53/104] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:09:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:09:46.970175 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac702757-6c3f-4965-a855-4d6c9a3fa0bb None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:46.970464 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac702757-6c3f-4965-a855-4d6c9a3fa0bb None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:46.970750 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac702757-6c3f-4965-a855-4d6c9a3fa0bb None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:46.972110 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 54/105] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:09:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:09:46.982068 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cfda18-1366-4f30-9f11-b3bb76266bb5 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:46.982434 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cfda18-1366-4f30-9f11-b3bb76266bb5 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:46.982780 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cfda18-1366-4f30-9f11-b3bb76266bb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:47.003218 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-e4cfda18-1366-4f30-9f11-b3bb76266bb5 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:09:47.004733 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-e4cfda18-1366-4f30-9f11-b3bb76266bb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:09:47.061011 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dbfac7-7576-41c3-bd6b-b27d2edaeb4b None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:47.092731 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 55/106] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:09:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:09:47.094938 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ef7ed0-f447-4f76-89a3-9940f66dd535 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:47.095333 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ef7ed0-f447-4f76-89a3-9940f66dd535 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:47.095958 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ef7ed0-f447-4f76-89a3-9940f66dd535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:47.115346 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97cd5b5-bf85-4f58-920c-acec58894770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:47.117479 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97cd5b5-bf85-4f58-920c-acec58894770 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:47.117667 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-13ef7ed0-f447-4f76-89a3-9940f66dd535 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:09:47.118224 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97cd5b5-bf85-4f58-920c-acec58894770 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:47.118710 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97cd5b5-bf85-4f58-920c-acec58894770 None admin] PATH_INFO: `/v3/users/d4608701c57147b39ea0df421d2e5029` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:47.118905 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-13ef7ed0-f447-4f76-89a3-9940f66dd535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:09:47.123920 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97cd5b5-bf85-4f58-920c-acec58894770 None admin] RBAC: Authorizing `identity:get_user(user_id=d4608701c57147b39ea0df421d2e5029)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:47.125599 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97cd5b5-bf85-4f58-920c-acec58894770 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:47.127564 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 52/107] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:09:46 2022] GET /identity/v3/users/d4608701c57147b39ea0df421d2e5029 => generated 247 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:09:47.128240 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97cd5b5-bf85-4f58-920c-acec58894770 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:47.200157 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 56/108] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:09:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:09:47.210743 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ef7ed0-f447-4f76-89a3-9940f66dd535 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:47.230925 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:47.243628 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97cd5b5-bf85-4f58-920c-acec58894770 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:47.270109 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:47.284501 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79820890-d270-458d-9b7c-da36b327e208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:47.286691 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79820890-d270-458d-9b7c-da36b327e208 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:47.287189 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79820890-d270-458d-9b7c-da36b327e208 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:47.287645 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79820890-d270-458d-9b7c-da36b327e208 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:47.299403 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79820890-d270-458d-9b7c-da36b327e208 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:09:47.308221 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-79820890-d270-458d-9b7c-da36b327e208 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:09:47.308221 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:09:47.308221 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:09:47.308221 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:09:47.308221 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:09:47.308221 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:09:47.308221 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:09:47.308221 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:09:47.308221 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:09:47.308221 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:09:47.308221 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:09:47.308221 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:09:47.308221 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:09:47.308221 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:09:47.308221 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:09:47.308221 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:09:47.309244 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:09:47.309244 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:09:47.309244 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:09:47.309244 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:09:47.309244 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:09:47.309244 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:09:47.309244 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:09:47.309244 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:09:47.309244 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:09:47.309244 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:09:47.309244 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:09:47.309244 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:09:47.309244 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:09:47.309244 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:09:47.309244 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:09:47.309244 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:09:47.310060 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:09:47.310060 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:09:47.310060 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:09:47.310060 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:09:47.310060 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:09:47.310060 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:09:47.310060 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:09:47.310060 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:09:47.310060 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:09:47.310060 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:09:47.310060 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:09:47.310060 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:09:47.310060 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 53/109] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:09:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:09:47.314748 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79820890-d270-458d-9b7c-da36b327e208 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:47.327717 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ef7ed0-f447-4f76-89a3-9940f66dd535 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:47.365670 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c53186a-9ad8-4149-bb3d-698e7c708f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:47.366762 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c53186a-9ad8-4149-bb3d-698e7c708f27 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:47.367052 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c53186a-9ad8-4149-bb3d-698e7c708f27 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:47.367330 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c53186a-9ad8-4149-bb3d-698e7c708f27 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:47.378121 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c53186a-9ad8-4149-bb3d-698e7c708f27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:09:47.386048 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-9c53186a-9ad8-4149-bb3d-698e7c708f27 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:09:47.386048 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:09:47.386048 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:09:47.386048 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:09:47.386048 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:09:47.386048 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:09:47.386048 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:09:47.386048 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:09:47.386048 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:09:47.386048 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:09:47.386048 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:09:47.386048 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:09:47.386048 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:09:47.386048 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:09:47.386048 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:09:47.386048 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:09:47.386916 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:09:47.386916 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:09:47.386916 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:09:47.386916 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:09:47.386916 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:09:47.386916 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:09:47.386916 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:09:47.386916 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:09:47.386916 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:09:47.386916 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:09:47.386916 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:09:47.386916 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:09:47.386916 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:09:47.386916 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:09:47.386916 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:09:47.386916 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:09:47.387681 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:09:47.387681 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:09:47.387681 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:09:47.387681 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:09:47.387681 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:09:47.387681 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:09:47.387681 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:09:47.387681 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:09:47.387681 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:09:47.387681 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:09:47.387681 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:09:47.387681 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:09:47.387681 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 57/110] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:09:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:09:47.388833 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2005e825-af3b-4598-8f01-4cb0a3b80036 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:47.388833 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2005e825-af3b-4598-8f01-4cb0a3b80036 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:47.389113 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2005e825-af3b-4598-8f01-4cb0a3b80036 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:47.390542 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 58/111] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:09:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:09:47.395968 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2005e825-af3b-4598-8f01-4cb0a3b80036 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:47.427845 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79820890-d270-458d-9b7c-da36b327e208 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:47.465229 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3578909b-3111-48bf-8af6-505848713ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:47.466178 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3578909b-3111-48bf-8af6-505848713ae5 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:47.466462 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3578909b-3111-48bf-8af6-505848713ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:47.466741 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3578909b-3111-48bf-8af6-505848713ae5 None admin] PATH_INFO: `/v3/projects/ed85b374a73a4c94a310482b7ba566f3` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:47.467928 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3578909b-3111-48bf-8af6-505848713ae5 None admin] RBAC: Authorizing `identity:get_project(project_id=ed85b374a73a4c94a310482b7ba566f3)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:47.469393 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3578909b-3111-48bf-8af6-505848713ae5 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:47.471135 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 54/112] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:09:47 2022] GET /identity/v3/projects/ed85b374a73a4c94a310482b7ba566f3 => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:09:47.471896 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3578909b-3111-48bf-8af6-505848713ae5 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:47.497265 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:47.512524 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2005e825-af3b-4598-8f01-4cb0a3b80036 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:47.522560 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:47.553473 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec6babc-7778-4a21-af8d-7faae646a382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:47.554524 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec6babc-7778-4a21-af8d-7faae646a382 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:47.554972 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec6babc-7778-4a21-af8d-7faae646a382 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:47.555404 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec6babc-7778-4a21-af8d-7faae646a382 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:47.556721 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec6babc-7778-4a21-af8d-7faae646a382 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:47.558230 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec6babc-7778-4a21-af8d-7faae646a382 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:47.564178 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 59/113] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:09:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:09:47.564178 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec6babc-7778-4a21-af8d-7faae646a382 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:47.589359 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3578909b-3111-48bf-8af6-505848713ae5 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:47.627009 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0bd7f2-af86-49fa-b6d1-cbfc13a6c02e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:47.627685 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0bd7f2-af86-49fa-b6d1-cbfc13a6c02e None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:47.627976 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0bd7f2-af86-49fa-b6d1-cbfc13a6c02e None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:47.628258 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0bd7f2-af86-49fa-b6d1-cbfc13a6c02e None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:47.629013 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0bd7f2-af86-49fa-b6d1-cbfc13a6c02e None admin] RBAC: Adding query filter params (name=member) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:09:47.629328 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0bd7f2-af86-49fa-b6d1-cbfc13a6c02e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:47.630788 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0bd7f2-af86-49fa-b6d1-cbfc13a6c02e None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:47.639596 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 55/114] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:09:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:09:47.640812 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203252a8-67bf-496b-9650-fd631e8c821f None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:47.641107 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203252a8-67bf-496b-9650-fd631e8c821f None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:47.641391 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203252a8-67bf-496b-9650-fd631e8c821f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:47.661885 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-203252a8-67bf-496b-9650-fd631e8c821f None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:09:47.663233 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-203252a8-67bf-496b-9650-fd631e8c821f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:09:47.683143 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec6babc-7778-4a21-af8d-7faae646a382 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:47.721278 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88cbbe1-6209-4452-ac8a-43d06992013a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:47.722141 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88cbbe1-6209-4452-ac8a-43d06992013a None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:47.722434 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88cbbe1-6209-4452-ac8a-43d06992013a None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:47.722717 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88cbbe1-6209-4452-ac8a-43d06992013a None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:47.723576 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88cbbe1-6209-4452-ac8a-43d06992013a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:09:47.724052 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88cbbe1-6209-4452-ac8a-43d06992013a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:47.725611 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88cbbe1-6209-4452-ac8a-43d06992013a None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:47.734287 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 60/115] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:09:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:09:47.734287 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88cbbe1-6209-4452-ac8a-43d06992013a None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:47.739039 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 56/116] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:09:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:09:47.740092 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203252a8-67bf-496b-9650-fd631e8c821f None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:47.760835 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:47.786974 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:47.857948 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203252a8-67bf-496b-9650-fd631e8c821f None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:47.858522 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88cbbe1-6209-4452-ac8a-43d06992013a None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:47.894978 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2801fdab-18b9-42cd-9b76-fa6ca3ce51c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:47.896588 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2801fdab-18b9-42cd-9b76-fa6ca3ce51c3 None admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:47.896942 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2801fdab-18b9-42cd-9b76-fa6ca3ce51c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:47.897287 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2801fdab-18b9-42cd-9b76-fa6ca3ce51c3 None admin] PATH_INFO: `/v3/projects/ed85b374a73a4c94a310482b7ba566f3/users/d4608701c57147b39ea0df421d2e5029/roles/4946cf0ff7164307b36efeee7c787146` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:47.898387 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeeeefff-0bb5-4d8a-8c02-071cfc4bf9a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:47.899125 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeeefff-0bb5-4d8a-8c02-071cfc4bf9a7 None admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:47.899481 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeeefff-0bb5-4d8a-8c02-071cfc4bf9a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:47.899569 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2801fdab-18b9-42cd-9b76-fa6ca3ce51c3 None admin] RBAC: Authorizing `identity:create_grant(project_id=ed85b374a73a4c94a310482b7ba566f3, user_id=d4608701c57147b39ea0df421d2e5029, role_id=4946cf0ff7164307b36efeee7c787146)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:47.899892 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeeefff-0bb5-4d8a-8c02-071cfc4bf9a7 None admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:47.900678 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeeeefff-0bb5-4d8a-8c02-071cfc4bf9a7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:47.901212 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2801fdab-18b9-42cd-9b76-fa6ca3ce51c3 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:47.902292 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeeeefff-0bb5-4d8a-8c02-071cfc4bf9a7 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:47.915345 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 61/117] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 14:09:47 2022] PUT /identity/v3/projects/ed85b374a73a4c94a310482b7ba566f3/users/d4608701c57147b39ea0df421d2e5029/roles/4946cf0ff7164307b36efeee7c787146 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:09:47.916297 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2801fdab-18b9-42cd-9b76-fa6ca3ce51c3 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:47.947272 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 57/118] 10.222.0.57 () {60 vars in 1141 bytes} [Thu May 19 14:09:47 2022] POST /identity/v3/projects => generated 302 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:09:47.948574 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeeeefff-0bb5-4d8a-8c02-071cfc4bf9a7 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:48.044633 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:48.079147 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:48.085113 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2801fdab-18b9-42cd-9b76-fa6ca3ce51c3 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:48.103027 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeeeefff-0bb5-4d8a-8c02-071cfc4bf9a7 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:48.141331 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73326f94-987c-4c03-a303-8b02e2447eee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:48.142302 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73326f94-987c-4c03-a303-8b02e2447eee None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:48.142598 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73326f94-987c-4c03-a303-8b02e2447eee None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:48.142880 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73326f94-987c-4c03-a303-8b02e2447eee None admin] PATH_INFO: `/v3/users/47b5316234e94937b78cedeab15c486e` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:48.144240 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73326f94-987c-4c03-a303-8b02e2447eee None admin] RBAC: Authorizing `identity:get_user(user_id=47b5316234e94937b78cedeab15c486e)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:48.146121 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73326f94-987c-4c03-a303-8b02e2447eee None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:48.148243 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 62/119] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:09:47 2022] GET /identity/v3/users/47b5316234e94937b78cedeab15c486e => generated 275 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:09:48.148894 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73326f94-987c-4c03-a303-8b02e2447eee None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:48.152749 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162e325f-7942-4f00-aae2-22a2a743dec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:48.153732 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162e325f-7942-4f00-aae2-22a2a743dec0 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:48.154116 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162e325f-7942-4f00-aae2-22a2a743dec0 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:48.154397 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162e325f-7942-4f00-aae2-22a2a743dec0 None admin] PATH_INFO: `/v3/users/d21fb41db72f4c15b0261fc2028ed8e2` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:48.172698 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162e325f-7942-4f00-aae2-22a2a743dec0 None admin] RBAC: Authorizing `identity:get_user(user_id=d21fb41db72f4c15b0261fc2028ed8e2)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:48.174221 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162e325f-7942-4f00-aae2-22a2a743dec0 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:48.176177 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 58/120] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:09:47 2022] GET /identity/v3/users/d21fb41db72f4c15b0261fc2028ed8e2 => generated 289 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:09:48.178327 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39478201-35df-4f92-926f-bb690616f435 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:48.178619 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39478201-35df-4f92-926f-bb690616f435 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:48.178957 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39478201-35df-4f92-926f-bb690616f435 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:48.180244 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 59/121] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:09:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:09:48.181164 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39478201-35df-4f92-926f-bb690616f435 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:48.280089 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73326f94-987c-4c03-a303-8b02e2447eee None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:48.309741 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39478201-35df-4f92-926f-bb690616f435 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:48.326676 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3295701a-dd24-4600-84e2-ce12632ce594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:48.327756 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3295701a-dd24-4600-84e2-ce12632ce594 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:48.328509 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3295701a-dd24-4600-84e2-ce12632ce594 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:48.328804 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3295701a-dd24-4600-84e2-ce12632ce594 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:48.338554 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:48.344045 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3295701a-dd24-4600-84e2-ce12632ce594 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:09:48.352851 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-3295701a-dd24-4600-84e2-ce12632ce594 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:09:48.352851 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:09:48.352851 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:09:48.352851 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:09:48.352851 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:09:48.352851 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:09:48.352851 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:09:48.352851 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:09:48.352851 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:09:48.352851 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:09:48.352851 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:09:48.352851 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:09:48.352851 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:09:48.352851 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:09:48.352851 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:09:48.352851 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:09:48.355375 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:09:48.355375 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:09:48.355375 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:09:48.355375 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:09:48.355375 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:09:48.355375 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:09:48.355375 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:09:48.355375 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:09:48.355375 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:09:48.355375 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:09:48.355375 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:09:48.355375 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:09:48.355375 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:09:48.355375 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:09:48.355375 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:09:48.355375 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:09:48.357286 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:09:48.357286 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:09:48.357286 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:09:48.357286 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:09:48.357286 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:09:48.357286 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:09:48.357286 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:09:48.357286 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:09:48.357286 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:09:48.357286 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:09:48.357286 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:09:48.357286 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:09:48.358494 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed335df6-53b5-45a4-8e0b-bd7ed1b97cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:48.358494 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed335df6-53b5-45a4-8e0b-bd7ed1b97cab None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:48.358669 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 63/122] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 19 14:09:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:09:48.358747 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed335df6-53b5-45a4-8e0b-bd7ed1b97cab None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:48.358747 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed335df6-53b5-45a4-8e0b-bd7ed1b97cab None admin] PATH_INFO: `/v3/projects/ed85b374a73a4c94a310482b7ba566f3` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:48.358900 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3295701a-dd24-4600-84e2-ce12632ce594 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:48.358988 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed335df6-53b5-45a4-8e0b-bd7ed1b97cab None admin] RBAC: Authorizing `identity:get_project(project_id=ed85b374a73a4c94a310482b7ba566f3)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:48.358988 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed335df6-53b5-45a4-8e0b-bd7ed1b97cab None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:48.366292 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 60/123] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:09:48 2022] GET /identity/v3/projects/ed85b374a73a4c94a310482b7ba566f3 => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:09:48.366292 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5630590a-c907-44f7-8c6c-cce91e23c559 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:48.366292 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5630590a-c907-44f7-8c6c-cce91e23c559 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:48.366292 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5630590a-c907-44f7-8c6c-cce91e23c559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:48.387330 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-5630590a-c907-44f7-8c6c-cce91e23c559 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:09:48.388878 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-5630590a-c907-44f7-8c6c-cce91e23c559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:09:48.391895 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:48.471242 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3295701a-dd24-4600-84e2-ce12632ce594 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:48.488802 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 61/124] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:09:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:09:48.490480 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5630590a-c907-44f7-8c6c-cce91e23c559 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:48.511083 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c395dad3-fbab-4cba-8758-d7b89ce659fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:48.511545 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c395dad3-fbab-4cba-8758-d7b89ce659fd None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:48.511836 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c395dad3-fbab-4cba-8758-d7b89ce659fd None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:48.512118 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c395dad3-fbab-4cba-8758-d7b89ce659fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:48.589163 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c395dad3-fbab-4cba-8758-d7b89ce659fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:48.590666 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c395dad3-fbab-4cba-8758-d7b89ce659fd None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:48.621103 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:48.627687 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5630590a-c907-44f7-8c6c-cce91e23c559 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:48.654323 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:48.674720 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad1deba-90e3-4ae0-892b-ad45afb6e69c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:48.675753 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad1deba-90e3-4ae0-892b-ad45afb6e69c None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:48.676237 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad1deba-90e3-4ae0-892b-ad45afb6e69c None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:48.677267 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad1deba-90e3-4ae0-892b-ad45afb6e69c None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:48.678280 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad1deba-90e3-4ae0-892b-ad45afb6e69c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:09:48.678829 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad1deba-90e3-4ae0-892b-ad45afb6e69c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:48.680989 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad1deba-90e3-4ae0-892b-ad45afb6e69c None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:48.692632 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 62/125] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:09:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:09:48.694278 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad1deba-90e3-4ae0-892b-ad45afb6e69c None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:48.707795 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 64/126] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:09:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:09:48.708894 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c395dad3-fbab-4cba-8758-d7b89ce659fd None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:48.804368 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad1deba-90e3-4ae0-892b-ad45afb6e69c None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:48.822535 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c395dad3-fbab-4cba-8758-d7b89ce659fd None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:48.840602 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053bfec9-d75d-4db8-952d-fb6074d18e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:48.841975 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053bfec9-d75d-4db8-952d-fb6074d18e71 None admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:48.842530 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053bfec9-d75d-4db8-952d-fb6074d18e71 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:48.843002 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053bfec9-d75d-4db8-952d-fb6074d18e71 None admin] PATH_INFO: `/v3/projects/ed85b374a73a4c94a310482b7ba566f3/users/47b5316234e94937b78cedeab15c486e/roles/7d0e162805b54c019ce6c954cbef3818` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:48.845532 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053bfec9-d75d-4db8-952d-fb6074d18e71 None admin] RBAC: Authorizing `identity:create_grant(project_id=ed85b374a73a4c94a310482b7ba566f3, user_id=47b5316234e94937b78cedeab15c486e, role_id=7d0e162805b54c019ce6c954cbef3818)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:48.847380 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053bfec9-d75d-4db8-952d-fb6074d18e71 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:48.852273 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:48.863042 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b74b00d-2881-498a-a924-07cb7a42c217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:48.863944 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b74b00d-2881-498a-a924-07cb7a42c217 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:48.864286 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b74b00d-2881-498a-a924-07cb7a42c217 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:48.864560 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b74b00d-2881-498a-a924-07cb7a42c217 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:48.871073 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 63/127] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 14:09:48 2022] PUT /identity/v3/projects/ed85b374a73a4c94a310482b7ba566f3/users/47b5316234e94937b78cedeab15c486e/roles/7d0e162805b54c019ce6c954cbef3818 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:09:48.872330 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053bfec9-d75d-4db8-952d-fb6074d18e71 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:48.884626 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b74b00d-2881-498a-a924-07cb7a42c217 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:09:48.900637 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-0b74b00d-2881-498a-a924-07cb7a42c217 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:09:48.900637 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:09:48.900637 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:09:48.900637 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:09:48.900637 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:09:48.900637 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:09:48.900637 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:09:48.900637 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:09:48.900637 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:09:48.900637 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:09:48.900637 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:09:48.900637 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:09:48.900637 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:09:48.900637 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:09:48.900637 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:09:48.900637 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:09:48.901585 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:09:48.901585 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:09:48.901585 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:09:48.901585 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:09:48.901585 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:09:48.901585 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:09:48.901585 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:09:48.901585 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:09:48.901585 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:09:48.901585 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:09:48.901585 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:09:48.901585 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:09:48.901585 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:09:48.901585 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:09:48.901585 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:09:48.901585 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:09:48.902403 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:09:48.902403 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:09:48.902403 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:09:48.902403 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:09:48.902403 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:09:48.902403 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:09:48.902403 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:09:48.902403 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:09:48.902403 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:09:48.902403 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:09:48.902403 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:09:48.902403 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:09:48.902403 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 65/128] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:09:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:09:48.903047 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b74b00d-2881-498a-a924-07cb7a42c217 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:48.926779 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:48.993138 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053bfec9-d75d-4db8-952d-fb6074d18e71 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:49.031353 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e22ddb-41ac-4b6e-961d-0dc3e09ec2d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:49.032640 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e22ddb-41ac-4b6e-961d-0dc3e09ec2d9 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:49.033128 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e22ddb-41ac-4b6e-961d-0dc3e09ec2d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:49.033604 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e22ddb-41ac-4b6e-961d-0dc3e09ec2d9 None admin] PATH_INFO: `/v3/users/47b5316234e94937b78cedeab15c486e` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:49.035121 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e22ddb-41ac-4b6e-961d-0dc3e09ec2d9 None admin] RBAC: Authorizing `identity:get_user(user_id=47b5316234e94937b78cedeab15c486e)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:49.037518 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e22ddb-41ac-4b6e-961d-0dc3e09ec2d9 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:49.039645 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 64/129] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:09:48 2022] GET /identity/v3/users/47b5316234e94937b78cedeab15c486e => generated 275 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:09:49.040706 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e22ddb-41ac-4b6e-961d-0dc3e09ec2d9 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:49.174958 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e22ddb-41ac-4b6e-961d-0dc3e09ec2d9 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:49.176120 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b74b00d-2881-498a-a924-07cb7a42c217 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:49.188190 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:49.223048 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aae0f7c-d5be-421c-9e0f-bed491c33638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:49.224157 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aae0f7c-d5be-421c-9e0f-bed491c33638 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:49.224441 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aae0f7c-d5be-421c-9e0f-bed491c33638 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:49.224721 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aae0f7c-d5be-421c-9e0f-bed491c33638 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:49.225481 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aae0f7c-d5be-421c-9e0f-bed491c33638 None admin] RBAC: Adding query filter params (name=member) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:09:49.225794 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aae0f7c-d5be-421c-9e0f-bed491c33638 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:49.227249 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aae0f7c-d5be-421c-9e0f-bed491c33638 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:49.236763 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 65/130] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:09:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:09:49.237714 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aae0f7c-d5be-421c-9e0f-bed491c33638 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:49.248444 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0d4918-8ac0-4aab-a2b2-a1d40272ba3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:49.249649 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0d4918-8ac0-4aab-a2b2-a1d40272ba3e None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:49.250634 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0d4918-8ac0-4aab-a2b2-a1d40272ba3e None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:49.251126 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0d4918-8ac0-4aab-a2b2-a1d40272ba3e None admin] PATH_INFO: `/v3/projects/ed85b374a73a4c94a310482b7ba566f3` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:49.255520 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0d4918-8ac0-4aab-a2b2-a1d40272ba3e None admin] RBAC: Authorizing `identity:get_project(project_id=ed85b374a73a4c94a310482b7ba566f3)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:49.258042 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0d4918-8ac0-4aab-a2b2-a1d40272ba3e None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:49.260596 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 66/131] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:09:48 2022] GET /identity/v3/projects/ed85b374a73a4c94a310482b7ba566f3 => generated 298 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:09:49.261941 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0d4918-8ac0-4aab-a2b2-a1d40272ba3e None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:49.359748 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:49.420422 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aae0f7c-d5be-421c-9e0f-bed491c33638 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:49.429621 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0d4918-8ac0-4aab-a2b2-a1d40272ba3e None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:49.456511 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05f0e1a-00c6-4053-832c-59b1bfd008d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:49.457388 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05f0e1a-00c6-4053-832c-59b1bfd008d6 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:49.457674 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05f0e1a-00c6-4053-832c-59b1bfd008d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:49.459915 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05f0e1a-00c6-4053-832c-59b1bfd008d6 None admin] PATH_INFO: `/v3/projects/ed85b374a73a4c94a310482b7ba566f3` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:49.460722 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05f0e1a-00c6-4053-832c-59b1bfd008d6 None admin] RBAC: Authorizing `identity:get_project(project_id=ed85b374a73a4c94a310482b7ba566f3)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:49.462218 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05f0e1a-00c6-4053-832c-59b1bfd008d6 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:49.464124 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 66/132] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:09:49 2022] GET /identity/v3/projects/ed85b374a73a4c94a310482b7ba566f3 => generated 298 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:09:49.464722 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05f0e1a-00c6-4053-832c-59b1bfd008d6 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:49.473957 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95937ce-ae9f-47ad-b3e2-3c9eb8db5e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:49.475498 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95937ce-ae9f-47ad-b3e2-3c9eb8db5e71 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:49.476004 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95937ce-ae9f-47ad-b3e2-3c9eb8db5e71 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:49.476513 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95937ce-ae9f-47ad-b3e2-3c9eb8db5e71 None admin] PATH_INFO: `/v3/users/47b5316234e94937b78cedeab15c486e` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:49.478091 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95937ce-ae9f-47ad-b3e2-3c9eb8db5e71 None admin] RBAC: Authorizing `identity:get_user(user_id=47b5316234e94937b78cedeab15c486e)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:49.479854 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95937ce-ae9f-47ad-b3e2-3c9eb8db5e71 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:49.482037 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 67/133] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:09:49 2022] GET /identity/v3/users/47b5316234e94937b78cedeab15c486e => generated 275 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:09:49.483046 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95937ce-ae9f-47ad-b3e2-3c9eb8db5e71 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:49.515527 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:49.578224 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05f0e1a-00c6-4053-832c-59b1bfd008d6 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:49.592515 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95937ce-ae9f-47ad-b3e2-3c9eb8db5e71 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:49.618031 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3592389-02db-4c3c-869a-377ce00ca149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:49.618857 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3592389-02db-4c3c-869a-377ce00ca149 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:49.619265 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3592389-02db-4c3c-869a-377ce00ca149 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:49.619974 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:49.620623 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3592389-02db-4c3c-869a-377ce00ca149 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:49.621986 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3592389-02db-4c3c-869a-377ce00ca149 None admin] RBAC: Adding query filter params (role.id=b0ae35f3e09648d0bbac778d23fa3463, scope.project.id=ed85b374a73a4c94a310482b7ba566f3, user.id=d21fb41db72f4c15b0261fc2028ed8e2) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:09:49.622567 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3592389-02db-4c3c-869a-377ce00ca149 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:49.625247 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3592389-02db-4c3c-869a-377ce00ca149 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:49.632713 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 67/134] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:09:49 2022] GET /identity/v3/role_assignments?user.id=d21fb41db72f4c15b0261fc2028ed8e2&scope.project.id=ed85b374a73a4c94a310482b7ba566f3&role.id=b0ae35f3e09648d0bbac778d23fa3463 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:09:49.633694 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3592389-02db-4c3c-869a-377ce00ca149 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:49.644562 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913770cb-d09f-4271-bfa7-8454a7c2703b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:49.646258 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913770cb-d09f-4271-bfa7-8454a7c2703b None admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:49.646694 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913770cb-d09f-4271-bfa7-8454a7c2703b None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:49.647138 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913770cb-d09f-4271-bfa7-8454a7c2703b None admin] PATH_INFO: `/v3/projects/ed85b374a73a4c94a310482b7ba566f3/users/47b5316234e94937b78cedeab15c486e/roles/23d9dd31632340bdba9cf3093c095dbb` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:49.655456 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913770cb-d09f-4271-bfa7-8454a7c2703b None admin] RBAC: Authorizing `identity:create_grant(project_id=ed85b374a73a4c94a310482b7ba566f3, user_id=47b5316234e94937b78cedeab15c486e, role_id=23d9dd31632340bdba9cf3093c095dbb)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:49.657149 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913770cb-d09f-4271-bfa7-8454a7c2703b None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:49.680075 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 68/135] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 14:09:49 2022] PUT /identity/v3/projects/ed85b374a73a4c94a310482b7ba566f3/users/47b5316234e94937b78cedeab15c486e/roles/23d9dd31632340bdba9cf3093c095dbb => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:09:49.784392 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3592389-02db-4c3c-869a-377ce00ca149 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:49.788554 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:49.839310 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dc6399-2812-41b6-85e7-a9ad92cc8f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:49.840648 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dc6399-2812-41b6-85e7-a9ad92cc8f04 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:49.841162 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dc6399-2812-41b6-85e7-a9ad92cc8f04 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:49.841661 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dc6399-2812-41b6-85e7-a9ad92cc8f04 None admin] PATH_INFO: `/v3/projects/5d9dd3087a0e490682a3b6c6a27f9361` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:49.846776 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dc6399-2812-41b6-85e7-a9ad92cc8f04 None admin] RBAC: Authorizing `identity:get_project(project_id=5d9dd3087a0e490682a3b6c6a27f9361)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:49.848723 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dc6399-2812-41b6-85e7-a9ad92cc8f04 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:49.851317 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 68/136] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:09:49 2022] GET /identity/v3/projects/5d9dd3087a0e490682a3b6c6a27f9361 => generated 312 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:09:49.858505 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913770cb-d09f-4271-bfa7-8454a7c2703b None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:49.995169 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913770cb-d09f-4271-bfa7-8454a7c2703b None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:50.043425 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c07798-9798-4527-8483-ca95f8e68991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:50.044674 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c07798-9798-4527-8483-ca95f8e68991 None admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:50.045139 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c07798-9798-4527-8483-ca95f8e68991 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:50.045598 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c07798-9798-4527-8483-ca95f8e68991 None admin] PATH_INFO: `/v3/projects/5d9dd3087a0e490682a3b6c6a27f9361/users/47b5316234e94937b78cedeab15c486e/roles/7d0e162805b54c019ce6c954cbef3818` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:50.051314 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c07798-9798-4527-8483-ca95f8e68991 None admin] RBAC: Authorizing `identity:create_grant(project_id=5d9dd3087a0e490682a3b6c6a27f9361, user_id=47b5316234e94937b78cedeab15c486e, role_id=7d0e162805b54c019ce6c954cbef3818)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:50.054040 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c07798-9798-4527-8483-ca95f8e68991 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:50.082283 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 69/137] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 14:09:49 2022] PUT /identity/v3/projects/5d9dd3087a0e490682a3b6c6a27f9361/users/47b5316234e94937b78cedeab15c486e/roles/7d0e162805b54c019ce6c954cbef3818 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:09:51.019980 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463264a8-688e-4236-8c09-5f666264a4f4 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:51.021577 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463264a8-688e-4236-8c09-5f666264a4f4 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:51.022108 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463264a8-688e-4236-8c09-5f666264a4f4 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:51.024083 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 69/138] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:09:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:09:51.029681 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65776e5-6015-4035-9f03-19d3d1a43e4c None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:51.030591 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65776e5-6015-4035-9f03-19d3d1a43e4c None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:51.031100 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65776e5-6015-4035-9f03-19d3d1a43e4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:51.054860 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-d65776e5-6015-4035-9f03-19d3d1a43e4c None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:09:51.056395 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-d65776e5-6015-4035-9f03-19d3d1a43e4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:09:51.108694 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:51.149320 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 70/139] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:09:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:09:51.185915 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463264a8-688e-4236-8c09-5f666264a4f4 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:51.212203 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558fe43e-6dc9-4230-badc-6164de88e066 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:51.212843 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558fe43e-6dc9-4230-badc-6164de88e066 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:51.213340 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558fe43e-6dc9-4230-badc-6164de88e066 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:51.214951 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 71/140] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:09:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:09:51.220353 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceb554b-c906-4c72-81e7-95592d0f9c76 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:51.220918 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceb554b-c906-4c72-81e7-95592d0f9c76 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:51.221422 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceb554b-c906-4c72-81e7-95592d0f9c76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:51.244164 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-3ceb554b-c906-4c72-81e7-95592d0f9c76 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:09:51.247260 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-3ceb554b-c906-4c72-81e7-95592d0f9c76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:09:51.383346 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463264a8-688e-4236-8c09-5f666264a4f4 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:51.411954 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 72/141] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:09:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:09:51.421571 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2ffefa-b24c-4c1b-bda9-2ce088dd9dc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:51.422396 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2ffefa-b24c-4c1b-bda9-2ce088dd9dc7 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:51.422681 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2ffefa-b24c-4c1b-bda9-2ce088dd9dc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:51.422959 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2ffefa-b24c-4c1b-bda9-2ce088dd9dc7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:51.424122 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2ffefa-b24c-4c1b-bda9-2ce088dd9dc7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:51.425586 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2ffefa-b24c-4c1b-bda9-2ce088dd9dc7 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:51.428034 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 70/142] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:09:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:09:51.433196 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceb554b-c906-4c72-81e7-95592d0f9c76 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:51.452360 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2ffefa-b24c-4c1b-bda9-2ce088dd9dc7 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:51.504627 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:51.572913 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2ffefa-b24c-4c1b-bda9-2ce088dd9dc7 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:51.613308 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1966c4-6472-48df-b926-66568359d7fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:51.615180 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1966c4-6472-48df-b926-66568359d7fc None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:51.615606 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1966c4-6472-48df-b926-66568359d7fc None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:51.616529 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1966c4-6472-48df-b926-66568359d7fc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:51.628993 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1966c4-6472-48df-b926-66568359d7fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:09:51.638243 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-fc1966c4-6472-48df-b926-66568359d7fc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:09:51.638243 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:09:51.638243 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:09:51.638243 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:09:51.638243 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:09:51.638243 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:09:51.638243 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:09:51.638243 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:09:51.638243 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:09:51.638243 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:09:51.638243 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:09:51.638243 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:09:51.638243 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:09:51.638243 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:09:51.638243 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:09:51.638243 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:09:51.639115 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:09:51.639115 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:09:51.639115 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:09:51.639115 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:09:51.639115 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:09:51.639115 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:09:51.639115 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:09:51.639115 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:09:51.639115 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:09:51.639115 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:09:51.639115 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:09:51.639115 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:09:51.639115 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:09:51.639115 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:09:51.639115 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:09:51.639115 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:09:51.639886 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:09:51.639886 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:09:51.639886 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:09:51.639886 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:09:51.639886 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:09:51.639886 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:09:51.639886 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:09:51.639886 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:09:51.639886 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:09:51.639886 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:09:51.639886 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:09:51.639886 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:09:51.642027 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 71/143] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:09:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:09:51.647868 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1966c4-6472-48df-b926-66568359d7fc None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:51.658298 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceb554b-c906-4c72-81e7-95592d0f9c76 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:51.666388 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:51.710866 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f08aa5-6e4e-4b48-a558-dc13cf3bbb53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:51.711885 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f08aa5-6e4e-4b48-a558-dc13cf3bbb53 None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:51.712361 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f08aa5-6e4e-4b48-a558-dc13cf3bbb53 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:51.712826 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f08aa5-6e4e-4b48-a558-dc13cf3bbb53 None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:51.713786 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f08aa5-6e4e-4b48-a558-dc13cf3bbb53 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:51.715772 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f08aa5-6e4e-4b48-a558-dc13cf3bbb53 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:51.743249 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 73/144] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 19 14:09:51 2022] POST /identity/v3/users => generated 283 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 14:09:51.828349 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:51.833661 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1966c4-6472-48df-b926-66568359d7fc None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:51.883443 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7b2d57-73bc-4767-bd80-4496cedaf663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:51.884137 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7b2d57-73bc-4767-bd80-4496cedaf663 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:51.884430 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7b2d57-73bc-4767-bd80-4496cedaf663 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:51.884715 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7b2d57-73bc-4767-bd80-4496cedaf663 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:51.885482 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7b2d57-73bc-4767-bd80-4496cedaf663 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:09:51.885799 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7b2d57-73bc-4767-bd80-4496cedaf663 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:51.887218 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7b2d57-73bc-4767-bd80-4496cedaf663 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:51.899373 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 72/145] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:09:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:09:51.905187 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f08aa5-6e4e-4b48-a558-dc13cf3bbb53 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:52.074444 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f08aa5-6e4e-4b48-a558-dc13cf3bbb53 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:52.118179 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1910d880-68a7-4ee7-88f5-6a22a6c92356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:52.119078 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1910d880-68a7-4ee7-88f5-6a22a6c92356 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:52.119364 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1910d880-68a7-4ee7-88f5-6a22a6c92356 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:52.119645 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1910d880-68a7-4ee7-88f5-6a22a6c92356 None admin] PATH_INFO: `/v3/users/d4608701c57147b39ea0df421d2e5029` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:52.122598 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1910d880-68a7-4ee7-88f5-6a22a6c92356 None admin] RBAC: Authorizing `identity:get_user(user_id=d4608701c57147b39ea0df421d2e5029)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:52.122598 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1910d880-68a7-4ee7-88f5-6a22a6c92356 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:52.125133 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 74/146] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:09:51 2022] GET /identity/v3/users/d4608701c57147b39ea0df421d2e5029 => generated 247 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:09:52.130257 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7b2d57-73bc-4767-bd80-4496cedaf663 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:52.255950 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7b2d57-73bc-4767-bd80-4496cedaf663 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:52.285115 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6750e00a-36a4-4cff-b79e-55d3de59309a None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:52.285443 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6750e00a-36a4-4cff-b79e-55d3de59309a None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:52.285685 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6750e00a-36a4-4cff-b79e-55d3de59309a None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:52.287787 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 75/147] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:09:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:09:52.299215 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a96a4cd-d903-482f-a47f-5c0dd03bea36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:52.300308 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a96a4cd-d903-482f-a47f-5c0dd03bea36 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:52.300805 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a96a4cd-d903-482f-a47f-5c0dd03bea36 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:52.301286 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a96a4cd-d903-482f-a47f-5c0dd03bea36 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:52.311442 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e46007-8ea3-4c26-9355-b3616af55c1f None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:52.311756 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e46007-8ea3-4c26-9355-b3616af55c1f None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:52.312017 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e46007-8ea3-4c26-9355-b3616af55c1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:52.338132 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-64e46007-8ea3-4c26-9355-b3616af55c1f None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:09:52.339350 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-64e46007-8ea3-4c26-9355-b3616af55c1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:09:52.365578 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:52.368001 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:52.445254 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a96a4cd-d903-482f-a47f-5c0dd03bea36 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:52.446782 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a96a4cd-d903-482f-a47f-5c0dd03bea36 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:52.456102 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 76/148] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:09:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:09:52.489442 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774795be-1a1f-4a36-b369-d7017971ab43 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:52.489778 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774795be-1a1f-4a36-b369-d7017971ab43 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:52.489983 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774795be-1a1f-4a36-b369-d7017971ab43 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:52.491692 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 77/149] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:09:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:09:52.497141 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b6b30c-c428-4018-800b-5190d1e9977d None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:52.497432 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b6b30c-c428-4018-800b-5190d1e9977d None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:52.497713 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b6b30c-c428-4018-800b-5190d1e9977d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:52.517176 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-b2b6b30c-c428-4018-800b-5190d1e9977d None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:09:52.518754 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-b2b6b30c-c428-4018-800b-5190d1e9977d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:09:52.593358 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 73/150] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:09:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:09:52.598336 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a96a4cd-d903-482f-a47f-5c0dd03bea36 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:52.608395 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 78/151] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:09:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:09:52.609576 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b6b30c-c428-4018-800b-5190d1e9977d None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:52.672185 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:52.719225 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a96a4cd-d903-482f-a47f-5c0dd03bea36 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:52.743815 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:52.759202 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2357ead6-d984-4b61-a2d9-def89861156d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:52.760012 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2357ead6-d984-4b61-a2d9-def89861156d None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:52.760302 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2357ead6-d984-4b61-a2d9-def89861156d None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:52.760584 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2357ead6-d984-4b61-a2d9-def89861156d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:52.766766 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b6b30c-c428-4018-800b-5190d1e9977d None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:52.787784 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2357ead6-d984-4b61-a2d9-def89861156d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:09:52.795576 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-2357ead6-d984-4b61-a2d9-def89861156d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:09:52.795576 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:09:52.795576 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:09:52.795576 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:09:52.795576 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:09:52.795576 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:09:52.795576 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:09:52.795576 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:09:52.795576 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:09:52.795576 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:09:52.795576 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:09:52.795576 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:09:52.795576 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:09:52.795576 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:09:52.795576 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:09:52.795576 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:09:52.796532 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:09:52.796532 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:09:52.796532 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:09:52.796532 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:09:52.796532 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:09:52.796532 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:09:52.796532 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:09:52.796532 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:09:52.796532 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:09:52.796532 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:09:52.796532 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:09:52.796532 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:09:52.796532 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:09:52.796532 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:09:52.796532 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:09:52.796532 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:09:52.797290 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:09:52.797290 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:09:52.797290 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:09:52.797290 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:09:52.797290 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:09:52.797290 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:09:52.797290 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:09:52.797290 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:09:52.797290 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:09:52.797290 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:09:52.797290 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:09:52.797290 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:09:52.797290 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 74/152] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:09:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:09:52.797934 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2357ead6-d984-4b61-a2d9-def89861156d None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:52.824668 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12dbe36-0f88-4448-9b17-eead16c82d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:52.826200 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12dbe36-0f88-4448-9b17-eead16c82d53 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:52.826487 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12dbe36-0f88-4448-9b17-eead16c82d53 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:52.826763 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12dbe36-0f88-4448-9b17-eead16c82d53 None admin] PATH_INFO: `/v3/projects/ed85b374a73a4c94a310482b7ba566f3` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:52.828468 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12dbe36-0f88-4448-9b17-eead16c82d53 None admin] RBAC: Authorizing `identity:get_project(project_id=ed85b374a73a4c94a310482b7ba566f3)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:52.830963 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12dbe36-0f88-4448-9b17-eead16c82d53 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:52.834099 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 79/153] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:09:52 2022] GET /identity/v3/projects/ed85b374a73a4c94a310482b7ba566f3 => generated 298 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:09:52.835051 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12dbe36-0f88-4448-9b17-eead16c82d53 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:52.936799 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2357ead6-d984-4b61-a2d9-def89861156d None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:52.964008 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12dbe36-0f88-4448-9b17-eead16c82d53 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:52.969201 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:52.985769 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92433075-eabc-457e-bf38-c355a7114c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:52.986832 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92433075-eabc-457e-bf38-c355a7114c55 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:52.987280 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92433075-eabc-457e-bf38-c355a7114c55 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:52.987695 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92433075-eabc-457e-bf38-c355a7114c55 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:53.008850 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f4f656-5f8e-4b77-a949-57ed1caf93c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:53.009464 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f4f656-5f8e-4b77-a949-57ed1caf93c2 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:53.009755 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f4f656-5f8e-4b77-a949-57ed1caf93c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:53.010077 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f4f656-5f8e-4b77-a949-57ed1caf93c2 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:53.010424 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92433075-eabc-457e-bf38-c355a7114c55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:09:53.010834 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f4f656-5f8e-4b77-a949-57ed1caf93c2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:09:53.011141 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f4f656-5f8e-4b77-a949-57ed1caf93c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:53.012534 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f4f656-5f8e-4b77-a949-57ed1caf93c2 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:53.028939 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-92433075-eabc-457e-bf38-c355a7114c55 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:09:53.028939 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:09:53.028939 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:09:53.028939 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:09:53.028939 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:09:53.028939 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:09:53.028939 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:09:53.028939 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:09:53.028939 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:09:53.028939 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:09:53.028939 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:09:53.028939 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:09:53.028939 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:09:53.028939 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:09:53.028939 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:09:53.028939 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:09:53.029877 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:09:53.029877 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:09:53.029877 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:09:53.029877 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:09:53.029877 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:09:53.029877 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:09:53.029877 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:09:53.029877 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:09:53.029877 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:09:53.029877 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:09:53.029877 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:09:53.029877 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:09:53.029877 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:09:53.029877 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:09:53.029877 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:09:53.029877 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:09:53.030817 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:09:53.030817 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:09:53.030817 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:09:53.030817 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:09:53.030817 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:09:53.030817 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:09:53.030817 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:09:53.030817 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:09:53.030817 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:09:53.030817 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:09:53.030817 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:09:53.030817 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:09:53.030817 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 75/154] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:09:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:09:53.031546 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92433075-eabc-457e-bf38-c355a7114c55 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:53.035937 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 80/155] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:09:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:09:53.041169 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f4f656-5f8e-4b77-a949-57ed1caf93c2 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:53.078792 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:53.177638 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92433075-eabc-457e-bf38-c355a7114c55 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:53.219716 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4e8c4b-82e2-4fe6-a33a-0544571d3ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:53.220402 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4e8c4b-82e2-4fe6-a33a-0544571d3ce4 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:53.220686 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4e8c4b-82e2-4fe6-a33a-0544571d3ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:53.220974 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4e8c4b-82e2-4fe6-a33a-0544571d3ce4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:53.221835 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4e8c4b-82e2-4fe6-a33a-0544571d3ce4 None admin] RBAC: Adding query filter params (role.id=4946cf0ff7164307b36efeee7c787146, scope.project.id=ed85b374a73a4c94a310482b7ba566f3, user.id=d4608701c57147b39ea0df421d2e5029) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:09:53.223042 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4e8c4b-82e2-4fe6-a33a-0544571d3ce4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:53.224820 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4e8c4b-82e2-4fe6-a33a-0544571d3ce4 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:53.231932 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 76/156] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:09:53 2022] GET /identity/v3/role_assignments?user.id=d4608701c57147b39ea0df421d2e5029&scope.project.id=ed85b374a73a4c94a310482b7ba566f3&role.id=4946cf0ff7164307b36efeee7c787146 => generated 608 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:09:53.232791 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4e8c4b-82e2-4fe6-a33a-0544571d3ce4 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:53.282298 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f4f656-5f8e-4b77-a949-57ed1caf93c2 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:53.285537 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:53.342808 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7787e8-9d4d-4df3-92bd-2bc1086dfe5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:53.343639 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7787e8-9d4d-4df3-92bd-2bc1086dfe5b None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:53.343924 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7787e8-9d4d-4df3-92bd-2bc1086dfe5b None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:53.344200 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7787e8-9d4d-4df3-92bd-2bc1086dfe5b None admin] PATH_INFO: `/v3/users/47b5316234e94937b78cedeab15c486e` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:53.350815 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7787e8-9d4d-4df3-92bd-2bc1086dfe5b None admin] RBAC: Authorizing `identity:get_user(user_id=47b5316234e94937b78cedeab15c486e)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:53.352512 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7787e8-9d4d-4df3-92bd-2bc1086dfe5b None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:53.354395 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 81/157] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:09:53 2022] GET /identity/v3/users/47b5316234e94937b78cedeab15c486e => generated 275 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:09:53.361749 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7787e8-9d4d-4df3-92bd-2bc1086dfe5b None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:53.372692 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4e8c4b-82e2-4fe6-a33a-0544571d3ce4 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:53.434160 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b941ca9e-f42f-4299-8ef7-96898f7ea1ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:53.434843 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b941ca9e-f42f-4299-8ef7-96898f7ea1ad None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:53.435126 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b941ca9e-f42f-4299-8ef7-96898f7ea1ad None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:53.435402 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b941ca9e-f42f-4299-8ef7-96898f7ea1ad None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:53.436166 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b941ca9e-f42f-4299-8ef7-96898f7ea1ad None admin] RBAC: Adding query filter params (name=reader) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:09:53.436475 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b941ca9e-f42f-4299-8ef7-96898f7ea1ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:53.445979 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b941ca9e-f42f-4299-8ef7-96898f7ea1ad None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:53.457894 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 77/158] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:09:53 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:09:53.461306 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b941ca9e-f42f-4299-8ef7-96898f7ea1ad None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:53.486425 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:53.508952 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7787e8-9d4d-4df3-92bd-2bc1086dfe5b None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:53.546686 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac781ea-b9e7-4561-9170-a669136ee6d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:53.547438 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac781ea-b9e7-4561-9170-a669136ee6d4 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:53.547733 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac781ea-b9e7-4561-9170-a669136ee6d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:53.548016 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac781ea-b9e7-4561-9170-a669136ee6d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:53.626835 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac781ea-b9e7-4561-9170-a669136ee6d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:53.628345 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac781ea-b9e7-4561-9170-a669136ee6d4 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:53.634311 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b941ca9e-f42f-4299-8ef7-96898f7ea1ad None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:53.644457 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:53.685658 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae0b063-485c-4952-8a5f-692ca79351a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:53.686584 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae0b063-485c-4952-8a5f-692ca79351a7 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:53.686866 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae0b063-485c-4952-8a5f-692ca79351a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:53.687141 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae0b063-485c-4952-8a5f-692ca79351a7 None admin] PATH_INFO: `/v3/users/d21fb41db72f4c15b0261fc2028ed8e2` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:53.688404 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae0b063-485c-4952-8a5f-692ca79351a7 None admin] RBAC: Authorizing `identity:get_user(user_id=d21fb41db72f4c15b0261fc2028ed8e2)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:53.694008 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae0b063-485c-4952-8a5f-692ca79351a7 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:53.696239 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 78/159] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:09:53 2022] GET /identity/v3/users/d21fb41db72f4c15b0261fc2028ed8e2 => generated 289 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:09:53.697577 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5ba7aa-6858-4fe2-9ed9-c17c6c823775 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:53.697883 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5ba7aa-6858-4fe2-9ed9-c17c6c823775 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:53.698157 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5ba7aa-6858-4fe2-9ed9-c17c6c823775 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:53.699425 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 79/160] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:09:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:09:53.707259 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b749101-5209-4fe5-bc0a-cb39654a5715 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:53.707584 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b749101-5209-4fe5-bc0a-cb39654a5715 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:53.707882 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b749101-5209-4fe5-bc0a-cb39654a5715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:53.745227 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-6b749101-5209-4fe5-bc0a-cb39654a5715 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:09:53.746867 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-6b749101-5209-4fe5-bc0a-cb39654a5715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:09:53.749078 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:53.769963 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 82/161] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:09:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:09:53.770916 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac781ea-b9e7-4561-9170-a669136ee6d4 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:53.874832 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 80/162] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:09:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:09:53.875535 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b749101-5209-4fe5-bc0a-cb39654a5715 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:54.018180 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac781ea-b9e7-4561-9170-a669136ee6d4 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:54.100468 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b749101-5209-4fe5-bc0a-cb39654a5715 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:54.109765 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc97d0b2-4f76-4efd-a877-b584a21d9390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:54.111182 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc97d0b2-4f76-4efd-a877-b584a21d9390 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:54.111700 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc97d0b2-4f76-4efd-a877-b584a21d9390 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:54.112171 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc97d0b2-4f76-4efd-a877-b584a21d9390 None admin] PATH_INFO: `/v3/projects/ed85b374a73a4c94a310482b7ba566f3` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:54.115611 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:54.116855 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc97d0b2-4f76-4efd-a877-b584a21d9390 None admin] RBAC: Authorizing `identity:get_project(project_id=ed85b374a73a4c94a310482b7ba566f3)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:54.118598 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc97d0b2-4f76-4efd-a877-b584a21d9390 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:54.130274 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 83/163] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:09:53 2022] GET /identity/v3/projects/ed85b374a73a4c94a310482b7ba566f3 => generated 298 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:09:54.131585 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc97d0b2-4f76-4efd-a877-b584a21d9390 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:54.177639 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150bff53-882a-4f2c-883a-572100eb84c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:54.179215 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150bff53-882a-4f2c-883a-572100eb84c4 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:54.179695 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150bff53-882a-4f2c-883a-572100eb84c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:54.180206 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150bff53-882a-4f2c-883a-572100eb84c4 None admin] PATH_INFO: `/v3/projects/ed85b374a73a4c94a310482b7ba566f3` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:54.181782 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150bff53-882a-4f2c-883a-572100eb84c4 None admin] RBAC: Authorizing `identity:get_project(project_id=ed85b374a73a4c94a310482b7ba566f3)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:54.184662 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150bff53-882a-4f2c-883a-572100eb84c4 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:54.193864 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 81/164] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:09:53 2022] GET /identity/v3/projects/ed85b374a73a4c94a310482b7ba566f3 => generated 298 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:09:54.197216 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150bff53-882a-4f2c-883a-572100eb84c4 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:54.228155 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:54.317582 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150bff53-882a-4f2c-883a-572100eb84c4 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:54.329965 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc97d0b2-4f76-4efd-a877-b584a21d9390 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:54.356913 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b85b32-cdd9-4377-85c0-2d92356c814e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:54.357810 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b85b32-cdd9-4377-85c0-2d92356c814e None admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:54.359706 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b85b32-cdd9-4377-85c0-2d92356c814e None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:54.359990 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b85b32-cdd9-4377-85c0-2d92356c814e None admin] PATH_INFO: `/v3/projects/ed85b374a73a4c94a310482b7ba566f3/users/d21fb41db72f4c15b0261fc2028ed8e2/roles/b0ae35f3e09648d0bbac778d23fa3463` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:54.363563 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b85b32-cdd9-4377-85c0-2d92356c814e None admin] RBAC: Authorizing `identity:create_grant(project_id=ed85b374a73a4c94a310482b7ba566f3, user_id=d21fb41db72f4c15b0261fc2028ed8e2, role_id=b0ae35f3e09648d0bbac778d23fa3463)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:54.365635 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b85b32-cdd9-4377-85c0-2d92356c814e None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:54.390821 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 82/165] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 14:09:54 2022] PUT /identity/v3/projects/ed85b374a73a4c94a310482b7ba566f3/users/d21fb41db72f4c15b0261fc2028ed8e2/roles/b0ae35f3e09648d0bbac778d23fa3463 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:09:54.392136 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0443af93-3dd0-45ce-9529-bd18139dbcdb None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:54.393097 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0443af93-3dd0-45ce-9529-bd18139dbcdb None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:54.393378 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0443af93-3dd0-45ce-9529-bd18139dbcdb None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:54.394940 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 83/166] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:09:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:09:54.395786 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0443af93-3dd0-45ce-9529-bd18139dbcdb None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:54.400020 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d074b2c-6b7f-4601-afcd-e2c5efab99b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:54.400856 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d074b2c-6b7f-4601-afcd-e2c5efab99b3 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:54.401140 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d074b2c-6b7f-4601-afcd-e2c5efab99b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:54.401419 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d074b2c-6b7f-4601-afcd-e2c5efab99b3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:54.425932 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d074b2c-6b7f-4601-afcd-e2c5efab99b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:09:54.442709 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-1d074b2c-6b7f-4601-afcd-e2c5efab99b3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:09:54.442709 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:09:54.442709 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:09:54.442709 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:09:54.442709 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:09:54.442709 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:09:54.442709 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:09:54.442709 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:09:54.442709 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:09:54.442709 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:09:54.442709 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:09:54.442709 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:09:54.442709 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:09:54.442709 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:09:54.442709 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:09:54.442709 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:09:54.443690 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:09:54.443690 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:09:54.443690 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:09:54.443690 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:09:54.443690 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:09:54.443690 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:09:54.443690 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:09:54.443690 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:09:54.443690 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:09:54.443690 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:09:54.443690 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:09:54.443690 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:09:54.443690 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:09:54.443690 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:09:54.443690 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:09:54.443690 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:09:54.444733 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:09:54.444733 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:09:54.444733 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:09:54.444733 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:09:54.444733 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:09:54.444733 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:09:54.444733 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:09:54.444733 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:09:54.444733 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:09:54.444733 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:09:54.444733 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:09:54.444733 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:09:54.445914 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 84/167] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 19 14:09:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:09:54.447073 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32db800c-a30f-4299-a9aa-8597f082350f None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:54.447073 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32db800c-a30f-4299-a9aa-8597f082350f None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:54.447073 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32db800c-a30f-4299-a9aa-8597f082350f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:54.466408 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:54.485264 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-32db800c-a30f-4299-a9aa-8597f082350f None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:09:54.486514 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-32db800c-a30f-4299-a9aa-8597f082350f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:09:54.606594 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 85/168] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:09:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:09:54.607386 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32db800c-a30f-4299-a9aa-8597f082350f None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:54.685169 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:54.698534 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0443af93-3dd0-45ce-9529-bd18139dbcdb None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:54.747898 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32db800c-a30f-4299-a9aa-8597f082350f None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:54.789015 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a790315-d25d-47cc-a686-8c098660b7db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:54.789691 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a790315-d25d-47cc-a686-8c098660b7db None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:54.790001 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a790315-d25d-47cc-a686-8c098660b7db None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:54.790279 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a790315-d25d-47cc-a686-8c098660b7db None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:54.791490 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a790315-d25d-47cc-a686-8c098660b7db None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:09:54.791948 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a790315-d25d-47cc-a686-8c098660b7db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:54.793403 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a790315-d25d-47cc-a686-8c098660b7db None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:54.804664 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 86/169] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:09:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:09:54.805473 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a790315-d25d-47cc-a686-8c098660b7db None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:54.818071 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437a49ac-0976-4ca5-9c75-f5758bace2dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:54.818751 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437a49ac-0976-4ca5-9c75-f5758bace2dc None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:54.819034 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437a49ac-0976-4ca5-9c75-f5758bace2dc None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:54.819312 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437a49ac-0976-4ca5-9c75-f5758bace2dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:54.820155 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437a49ac-0976-4ca5-9c75-f5758bace2dc None admin] RBAC: Adding query filter params (role.id=7d0e162805b54c019ce6c954cbef3818, scope.project.id=ed85b374a73a4c94a310482b7ba566f3, user.id=47b5316234e94937b78cedeab15c486e) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:09:54.820521 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437a49ac-0976-4ca5-9c75-f5758bace2dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:54.825074 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437a49ac-0976-4ca5-9c75-f5758bace2dc None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:54.836298 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 84/170] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:09:54 2022] GET /identity/v3/role_assignments?user.id=47b5316234e94937b78cedeab15c486e&scope.project.id=ed85b374a73a4c94a310482b7ba566f3&role.id=7d0e162805b54c019ce6c954cbef3818 => generated 608 bytes in 441 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:09:54.843832 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437a49ac-0976-4ca5-9c75-f5758bace2dc None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:55.066065 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a790315-d25d-47cc-a686-8c098660b7db None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:55.075133 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:55.081030 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:55.089034 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437a49ac-0976-4ca5-9c75-f5758bace2dc None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:55.179283 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb8b729-fc79-4b4f-8be9-bb7c6eb15fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:55.180676 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb8b729-fc79-4b4f-8be9-bb7c6eb15fa2 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:55.181135 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb8b729-fc79-4b4f-8be9-bb7c6eb15fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:55.181561 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823ed7f2-af97-453e-a39c-1fb34c7ee520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:55.181988 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb8b729-fc79-4b4f-8be9-bb7c6eb15fa2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:55.185863 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823ed7f2-af97-453e-a39c-1fb34c7ee520 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:55.185863 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823ed7f2-af97-453e-a39c-1fb34c7ee520 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:55.185863 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823ed7f2-af97-453e-a39c-1fb34c7ee520 None admin] PATH_INFO: `/v3/users/47b5316234e94937b78cedeab15c486e` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:55.186821 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823ed7f2-af97-453e-a39c-1fb34c7ee520 None admin] RBAC: Authorizing `identity:get_user(user_id=47b5316234e94937b78cedeab15c486e)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:55.188524 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823ed7f2-af97-453e-a39c-1fb34c7ee520 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:55.190473 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 85/171] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:09:54 2022] GET /identity/v3/users/47b5316234e94937b78cedeab15c486e => generated 275 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:09:55.196996 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823ed7f2-af97-453e-a39c-1fb34c7ee520 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:55.205041 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb8b729-fc79-4b4f-8be9-bb7c6eb15fa2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:09:55.213940 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-6cb8b729-fc79-4b4f-8be9-bb7c6eb15fa2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:09:55.213940 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:09:55.213940 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:09:55.213940 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:09:55.213940 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:09:55.213940 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:09:55.213940 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:09:55.213940 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:09:55.213940 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:09:55.213940 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:09:55.213940 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:09:55.213940 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:09:55.213940 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:09:55.213940 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:09:55.213940 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:09:55.213940 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:09:55.214898 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:09:55.214898 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:09:55.214898 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:09:55.214898 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:09:55.214898 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:09:55.214898 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:09:55.214898 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:09:55.214898 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:09:55.214898 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:09:55.214898 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:09:55.214898 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:09:55.214898 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:09:55.214898 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:09:55.214898 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:09:55.214898 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:09:55.214898 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:09:55.215702 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:09:55.215702 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:09:55.215702 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:09:55.215702 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:09:55.215702 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:09:55.215702 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:09:55.215702 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:09:55.215702 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:09:55.215702 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:09:55.215702 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:09:55.215702 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:09:55.215702 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:09:55.223481 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 87/172] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:09:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 418 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:09:55.230573 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb8b729-fc79-4b4f-8be9-bb7c6eb15fa2 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:55.388264 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823ed7f2-af97-453e-a39c-1fb34c7ee520 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:55.405504 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb8b729-fc79-4b4f-8be9-bb7c6eb15fa2 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:55.436670 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791a81b7-d1f2-4557-a9e4-9420cef27cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:55.437624 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791a81b7-d1f2-4557-a9e4-9420cef27cff None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:55.442703 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791a81b7-d1f2-4557-a9e4-9420cef27cff None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:55.443076 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791a81b7-d1f2-4557-a9e4-9420cef27cff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:55.446700 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d84e48-e377-44dc-8fed-9f160b2f6b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:55.448535 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d84e48-e377-44dc-8fed-9f160b2f6b8b None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:55.448970 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d84e48-e377-44dc-8fed-9f160b2f6b8b None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:55.449415 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d84e48-e377-44dc-8fed-9f160b2f6b8b None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:55.450386 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d84e48-e377-44dc-8fed-9f160b2f6b8b None admin] RBAC: Adding query filter params (name=member) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:09:55.450853 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d84e48-e377-44dc-8fed-9f160b2f6b8b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:55.452458 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d84e48-e377-44dc-8fed-9f160b2f6b8b None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:55.478065 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 88/173] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:09:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:09:55.482233 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d84e48-e377-44dc-8fed-9f160b2f6b8b None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:55.533215 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:55.537777 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:55.589385 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791a81b7-d1f2-4557-a9e4-9420cef27cff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:55.590928 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791a81b7-d1f2-4557-a9e4-9420cef27cff None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:55.603881 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d84e48-e377-44dc-8fed-9f160b2f6b8b None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:55.642432 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4755c7-67ca-4d57-be9f-4c797fedfebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:55.643299 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4755c7-67ca-4d57-be9f-4c797fedfebb None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:55.643585 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4755c7-67ca-4d57-be9f-4c797fedfebb None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:55.643862 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4755c7-67ca-4d57-be9f-4c797fedfebb None admin] PATH_INFO: `/v3/users/47b5316234e94937b78cedeab15c486e` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:55.646636 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4755c7-67ca-4d57-be9f-4c797fedfebb None admin] RBAC: Authorizing `identity:get_user(user_id=47b5316234e94937b78cedeab15c486e)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:55.648127 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4755c7-67ca-4d57-be9f-4c797fedfebb None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:55.649805 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 89/174] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:09:55 2022] GET /identity/v3/users/47b5316234e94937b78cedeab15c486e => generated 275 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:09:55.656985 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4755c7-67ca-4d57-be9f-4c797fedfebb None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:55.749420 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 86/175] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:09:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 553 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:09:55.755083 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791a81b7-d1f2-4557-a9e4-9420cef27cff None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:55.803102 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4755c7-67ca-4d57-be9f-4c797fedfebb None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:55.806455 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:55.846133 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71f1372-fb56-4051-878a-83fe35901be4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:55.847133 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71f1372-fb56-4051-878a-83fe35901be4 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:55.847636 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71f1372-fb56-4051-878a-83fe35901be4 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:55.848152 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71f1372-fb56-4051-878a-83fe35901be4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:55.904911 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:55.972171 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791a81b7-d1f2-4557-a9e4-9420cef27cff None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:55.990947 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71f1372-fb56-4051-878a-83fe35901be4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:55.992721 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71f1372-fb56-4051-878a-83fe35901be4 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:56.041134 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dc57bb-6bb8-4d57-9880-cc8748d47466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:56.042359 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dc57bb-6bb8-4d57-9880-cc8748d47466 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:56.043901 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dc57bb-6bb8-4d57-9880-cc8748d47466 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:56.044364 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dc57bb-6bb8-4d57-9880-cc8748d47466 None admin] PATH_INFO: `/v3/projects/ed85b374a73a4c94a310482b7ba566f3` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:56.047970 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dc57bb-6bb8-4d57-9880-cc8748d47466 None admin] RBAC: Authorizing `identity:get_project(project_id=ed85b374a73a4c94a310482b7ba566f3)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:56.049627 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dc57bb-6bb8-4d57-9880-cc8748d47466 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:56.054109 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 87/176] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:09:55 2022] GET /identity/v3/projects/ed85b374a73a4c94a310482b7ba566f3 => generated 298 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:09:56.057895 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ae8633-28cb-4409-87eb-15c9822271c7 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:56.061186 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ae8633-28cb-4409-87eb-15c9822271c7 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:56.061650 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ae8633-28cb-4409-87eb-15c9822271c7 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:56.063199 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 88/177] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:09:56 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:09:56.065565 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ae8633-28cb-4409-87eb-15c9822271c7 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:56.182300 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:56.197993 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 90/178] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:09:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 541 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:09:56.199494 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2c1eab-a0a9-47b7-b5b0-51f633ed546a None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:56.199793 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2c1eab-a0a9-47b7-b5b0-51f633ed546a None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:56.200080 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2c1eab-a0a9-47b7-b5b0-51f633ed546a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:56.239902 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-2e2c1eab-a0a9-47b7-b5b0-51f633ed546a None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:09:56.241607 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-2e2c1eab-a0a9-47b7-b5b0-51f633ed546a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:09:56.292971 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ae8633-28cb-4409-87eb-15c9822271c7 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:56.375737 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd3406b-f1ff-4cd2-857f-68d7ed1866ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:56.376404 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd3406b-f1ff-4cd2-857f-68d7ed1866ab None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:56.376691 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd3406b-f1ff-4cd2-857f-68d7ed1866ab None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:56.376963 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd3406b-f1ff-4cd2-857f-68d7ed1866ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:56.377808 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd3406b-f1ff-4cd2-857f-68d7ed1866ab None admin] RBAC: Adding query filter params (role.id=23d9dd31632340bdba9cf3093c095dbb, scope.project.id=ed85b374a73a4c94a310482b7ba566f3, user.id=47b5316234e94937b78cedeab15c486e) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:09:56.378144 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd3406b-f1ff-4cd2-857f-68d7ed1866ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:56.379653 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd3406b-f1ff-4cd2-857f-68d7ed1866ab None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:56.390260 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:56.392836 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 91/179] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:09:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:09:56.394828 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2c1eab-a0a9-47b7-b5b0-51f633ed546a None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:56.398203 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 89/180] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:09:56 2022] GET /identity/v3/role_assignments?user.id=47b5316234e94937b78cedeab15c486e&scope.project.id=ed85b374a73a4c94a310482b7ba566f3&role.id=23d9dd31632340bdba9cf3093c095dbb => generated 608 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:09:56.422668 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd3406b-f1ff-4cd2-857f-68d7ed1866ab None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:56.631334 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2c1eab-a0a9-47b7-b5b0-51f633ed546a None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:56.641076 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd3406b-f1ff-4cd2-857f-68d7ed1866ab None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:56.660625 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:56.674363 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3100273c-b6e5-4541-b860-598d4d2a1719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:56.675678 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3100273c-b6e5-4541-b860-598d4d2a1719 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:56.676206 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3100273c-b6e5-4541-b860-598d4d2a1719 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:56.676809 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3100273c-b6e5-4541-b860-598d4d2a1719 None admin] PATH_INFO: `/v3/projects/5d9dd3087a0e490682a3b6c6a27f9361` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:56.678490 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3100273c-b6e5-4541-b860-598d4d2a1719 None admin] RBAC: Authorizing `identity:get_project(project_id=5d9dd3087a0e490682a3b6c6a27f9361)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:56.680223 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3100273c-b6e5-4541-b860-598d4d2a1719 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:56.682185 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 92/181] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:09:56 2022] GET /identity/v3/projects/5d9dd3087a0e490682a3b6c6a27f9361 => generated 312 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:09:56.683801 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed59b0b0-5d7e-430d-9c6f-bf19ec94dab2 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:56.684283 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed59b0b0-5d7e-430d-9c6f-bf19ec94dab2 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:56.685005 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed59b0b0-5d7e-430d-9c6f-bf19ec94dab2 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:56.686736 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 93/182] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:09:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:09:56.692413 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359ba977-08e0-410f-ae5c-279ce37f812e None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:56.693122 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359ba977-08e0-410f-ae5c-279ce37f812e None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:56.693601 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359ba977-08e0-410f-ae5c-279ce37f812e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:56.700626 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac3b54e-2962-4b56-a03d-a056f7ed18d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:56.701434 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac3b54e-2962-4b56-a03d-a056f7ed18d3 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:56.701718 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac3b54e-2962-4b56-a03d-a056f7ed18d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:56.702065 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac3b54e-2962-4b56-a03d-a056f7ed18d3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:56.712112 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-359ba977-08e0-410f-ae5c-279ce37f812e None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:09:56.713287 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-359ba977-08e0-410f-ae5c-279ce37f812e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:09:56.713704 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac3b54e-2962-4b56-a03d-a056f7ed18d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:09:56.723699 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-9ac3b54e-2962-4b56-a03d-a056f7ed18d3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:09:56.723699 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:09:56.723699 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:09:56.723699 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:09:56.723699 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:09:56.723699 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:09:56.723699 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:09:56.723699 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:09:56.723699 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:09:56.723699 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:09:56.723699 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:09:56.723699 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:09:56.723699 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:09:56.723699 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:09:56.723699 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:09:56.723699 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:09:56.725077 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:09:56.725077 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:09:56.725077 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:09:56.725077 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:09:56.725077 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:09:56.725077 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:09:56.725077 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:09:56.725077 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:09:56.725077 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:09:56.725077 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:09:56.725077 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:09:56.725077 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:09:56.725077 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:09:56.725077 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:09:56.725077 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:09:56.725077 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:09:56.726349 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:09:56.726349 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:09:56.726349 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:09:56.726349 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:09:56.726349 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:09:56.726349 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:09:56.726349 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:09:56.726349 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:09:56.726349 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:09:56.726349 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:09:56.726349 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:09:56.726349 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:09:56.726349 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 90/183] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:09:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:09:56.726349 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac3b54e-2962-4b56-a03d-a056f7ed18d3 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:56.802362 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 94/184] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:09:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:09:56.802362 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359ba977-08e0-410f-ae5c-279ce37f812e None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:56.808606 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:56.856361 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac3b54e-2962-4b56-a03d-a056f7ed18d3 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:56.896081 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1521f494-710f-4323-8290-cf945a8ffdb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:56.897298 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1521f494-710f-4323-8290-cf945a8ffdb8 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:56.897757 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1521f494-710f-4323-8290-cf945a8ffdb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:56.898276 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1521f494-710f-4323-8290-cf945a8ffdb8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:56.899187 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1521f494-710f-4323-8290-cf945a8ffdb8 None admin] RBAC: Adding query filter params (role.id=7d0e162805b54c019ce6c954cbef3818, scope.project.id=5d9dd3087a0e490682a3b6c6a27f9361, user.id=47b5316234e94937b78cedeab15c486e) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:09:56.899503 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1521f494-710f-4323-8290-cf945a8ffdb8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:56.901167 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1521f494-710f-4323-8290-cf945a8ffdb8 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:56.917869 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 91/185] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:09:56 2022] GET /identity/v3/role_assignments?user.id=47b5316234e94937b78cedeab15c486e&scope.project.id=5d9dd3087a0e490682a3b6c6a27f9361&role.id=7d0e162805b54c019ce6c954cbef3818 => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:09:56.917869 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a427e7a1-126a-4c93-a448-a88ee416ae17 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:56.918178 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359ba977-08e0-410f-ae5c-279ce37f812e None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:56.918268 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a427e7a1-126a-4c93-a448-a88ee416ae17 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:56.918268 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a427e7a1-126a-4c93-a448-a88ee416ae17 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:56.918268 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 92/186] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:09:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:09:56.918268 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a427e7a1-126a-4c93-a448-a88ee416ae17 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:56.944112 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:56.957483 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b656abe5-04cc-480a-b75b-10146a2af9e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:56.959027 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b656abe5-04cc-480a-b75b-10146a2af9e4 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:56.959396 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b656abe5-04cc-480a-b75b-10146a2af9e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:56.959752 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b656abe5-04cc-480a-b75b-10146a2af9e4 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:56.960598 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b656abe5-04cc-480a-b75b-10146a2af9e4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:09:56.961062 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b656abe5-04cc-480a-b75b-10146a2af9e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:56.962854 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b656abe5-04cc-480a-b75b-10146a2af9e4 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:56.972847 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 95/187] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:09:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:09:56.979036 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50967108-0dd4-43a1-8edb-be3503a5ccf9 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:56.979408 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50967108-0dd4-43a1-8edb-be3503a5ccf9 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:56.979768 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50967108-0dd4-43a1-8edb-be3503a5ccf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:56.999981 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-50967108-0dd4-43a1-8edb-be3503a5ccf9 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:09:57.001469 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-50967108-0dd4-43a1-8edb-be3503a5ccf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:09:57.062511 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a427e7a1-126a-4c93-a448-a88ee416ae17 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:57.078140 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 96/188] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:09:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:09:57.083647 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50967108-0dd4-43a1-8edb-be3503a5ccf9 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:57.098580 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:57.113431 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1334cecb-4132-41f5-a83a-0758f026aabf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:57.122252 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1334cecb-4132-41f5-a83a-0758f026aabf None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:57.122252 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1334cecb-4132-41f5-a83a-0758f026aabf None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:57.122252 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1334cecb-4132-41f5-a83a-0758f026aabf None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:57.141636 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1334cecb-4132-41f5-a83a-0758f026aabf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:09:57.149992 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-1334cecb-4132-41f5-a83a-0758f026aabf None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:09:57.149992 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:09:57.149992 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:09:57.149992 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:09:57.149992 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:09:57.149992 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:09:57.149992 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:09:57.149992 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:09:57.149992 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:09:57.149992 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:09:57.149992 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:09:57.149992 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:09:57.149992 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:09:57.149992 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:09:57.149992 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:09:57.149992 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:09:57.150891 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:09:57.150891 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:09:57.150891 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:09:57.150891 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:09:57.150891 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:09:57.150891 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:09:57.150891 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:09:57.150891 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:09:57.150891 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:09:57.150891 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:09:57.150891 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:09:57.150891 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:09:57.150891 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:09:57.150891 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:09:57.150891 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:09:57.150891 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:09:57.151677 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:09:57.151677 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:09:57.151677 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:09:57.151677 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:09:57.151677 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:09:57.151677 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:09:57.151677 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:09:57.151677 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:09:57.151677 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:09:57.151677 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:09:57.151677 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:09:57.151677 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:09:57.151677 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 93/189] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 19 14:09:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:09:57.152333 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1334cecb-4132-41f5-a83a-0758f026aabf None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:57.253556 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50967108-0dd4-43a1-8edb-be3503a5ccf9 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:57.272621 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:57.285895 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1334cecb-4132-41f5-a83a-0758f026aabf None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:57.298743 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d36961a-4e77-4051-82e4-19811d5167c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:57.299791 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d36961a-4e77-4051-82e4-19811d5167c1 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:57.300081 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d36961a-4e77-4051-82e4-19811d5167c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:57.300717 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d36961a-4e77-4051-82e4-19811d5167c1 None admin] PATH_INFO: `/v3/users/6e59e9f7740a4dc28ef53b32621e82df` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:57.319673 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d36961a-4e77-4051-82e4-19811d5167c1 None admin] RBAC: Authorizing `identity:get_user(user_id=6e59e9f7740a4dc28ef53b32621e82df)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:57.321144 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d36961a-4e77-4051-82e4-19811d5167c1 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:57.321378 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0eb83d-dc7f-46e7-aba3-cba6eff9ecef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:57.322207 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0eb83d-dc7f-46e7-aba3-cba6eff9ecef None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:57.322490 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0eb83d-dc7f-46e7-aba3-cba6eff9ecef None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:57.322763 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0eb83d-dc7f-46e7-aba3-cba6eff9ecef None admin] PATH_INFO: `/v3/domains/default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:57.327285 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 97/190] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:09:57 2022] GET /identity/v3/users/6e59e9f7740a4dc28ef53b32621e82df => generated 283 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:09:57.327285 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d36961a-4e77-4051-82e4-19811d5167c1 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:57.327907 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0eb83d-dc7f-46e7-aba3-cba6eff9ecef None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:57.329363 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0eb83d-dc7f-46e7-aba3-cba6eff9ecef None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:57.331171 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 94/191] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:09:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:09:57.331804 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0eb83d-dc7f-46e7-aba3-cba6eff9ecef None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:57.391643 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:57.437900 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d36961a-4e77-4051-82e4-19811d5167c1 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:57.447143 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0eb83d-dc7f-46e7-aba3-cba6eff9ecef None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:57.472488 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dd4cf3-1605-4b34-b9bf-c3ee02d20822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:57.473779 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dd4cf3-1605-4b34-b9bf-c3ee02d20822 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:57.474251 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dd4cf3-1605-4b34-b9bf-c3ee02d20822 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:57.474527 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dd4cf3-1605-4b34-b9bf-c3ee02d20822 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:57.475267 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dd4cf3-1605-4b34-b9bf-c3ee02d20822 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:09:57.475571 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dd4cf3-1605-4b34-b9bf-c3ee02d20822 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:57.476952 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dd4cf3-1605-4b34-b9bf-c3ee02d20822 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:57.480841 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12874784-73d6-4270-ba05-75bd491f11ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:57.481483 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12874784-73d6-4270-ba05-75bd491f11ef None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:57.481768 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12874784-73d6-4270-ba05-75bd491f11ef None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:57.482085 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12874784-73d6-4270-ba05-75bd491f11ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:57.484848 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 98/192] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:09:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:09:57.485617 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dd4cf3-1605-4b34-b9bf-c3ee02d20822 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:57.523166 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:57.553616 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12874784-73d6-4270-ba05-75bd491f11ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:57.555142 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12874784-73d6-4270-ba05-75bd491f11ef None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:57.588795 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dd4cf3-1605-4b34-b9bf-c3ee02d20822 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:57.603411 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:57.623621 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ca8f7b-380a-4f94-9b85-bd65b5d1b988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:57.624320 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ca8f7b-380a-4f94-9b85-bd65b5d1b988 None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:57.624607 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ca8f7b-380a-4f94-9b85-bd65b5d1b988 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:57.624885 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ca8f7b-380a-4f94-9b85-bd65b5d1b988 None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:57.625592 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ca8f7b-380a-4f94-9b85-bd65b5d1b988 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:57.627068 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ca8f7b-380a-4f94-9b85-bd65b5d1b988 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:57.656119 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 99/193] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 19 14:09:57 2022] POST /identity/v3/users => generated 297 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:09:57.656119 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196b82f3-f867-4cca-a824-022606fe5543 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:57.656119 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196b82f3-f867-4cca-a824-022606fe5543 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:57.656119 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196b82f3-f867-4cca-a824-022606fe5543 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:57.656119 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 100/194] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:09:57 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:09:57.656119 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196b82f3-f867-4cca-a824-022606fe5543 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:57.670498 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 95/195] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:09:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:09:57.677217 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd116f4-2ea2-41d7-999e-468a148d7c53 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:57.677514 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd116f4-2ea2-41d7-999e-468a148d7c53 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:57.677789 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd116f4-2ea2-41d7-999e-468a148d7c53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:57.696178 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-bfd116f4-2ea2-41d7-999e-468a148d7c53 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:09:57.697315 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-bfd116f4-2ea2-41d7-999e-468a148d7c53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:09:57.769154 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:57.772019 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 96/196] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:09:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:09:57.782802 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196b82f3-f867-4cca-a824-022606fe5543 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:57.782994 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd116f4-2ea2-41d7-999e-468a148d7c53 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:57.826442 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c047a51e-3a13-497e-bad1-275619a8b4e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:57.827316 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c047a51e-3a13-497e-bad1-275619a8b4e8 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:57.827598 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c047a51e-3a13-497e-bad1-275619a8b4e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:57.827874 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c047a51e-3a13-497e-bad1-275619a8b4e8 None admin] PATH_INFO: `/v3/users/6e59e9f7740a4dc28ef53b32621e82df` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:57.829108 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c047a51e-3a13-497e-bad1-275619a8b4e8 None admin] RBAC: Authorizing `identity:get_user(user_id=6e59e9f7740a4dc28ef53b32621e82df)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:57.830605 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c047a51e-3a13-497e-bad1-275619a8b4e8 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:57.832310 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 101/197] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:09:57 2022] GET /identity/v3/users/6e59e9f7740a4dc28ef53b32621e82df => generated 283 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:09:57.833590 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c047a51e-3a13-497e-bad1-275619a8b4e8 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:57.886179 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:57.914771 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd116f4-2ea2-41d7-999e-468a148d7c53 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:57.963887 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17052df-971b-463c-8c74-07458280b638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:57.965365 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17052df-971b-463c-8c74-07458280b638 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:57.966117 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17052df-971b-463c-8c74-07458280b638 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:57.966593 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17052df-971b-463c-8c74-07458280b638 None admin] PATH_INFO: `/v3/projects/3c9e67fcd5b14da2a3775ef2b2cad6b2` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:57.967973 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17052df-971b-463c-8c74-07458280b638 None admin] RBAC: Authorizing `identity:get_project(project_id=3c9e67fcd5b14da2a3775ef2b2cad6b2)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:57.968101 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c047a51e-3a13-497e-bad1-275619a8b4e8 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:57.969580 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17052df-971b-463c-8c74-07458280b638 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:57.971370 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 97/198] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:09:57 2022] GET /identity/v3/projects/3c9e67fcd5b14da2a3775ef2b2cad6b2 => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:09:57.972142 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17052df-971b-463c-8c74-07458280b638 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:58.005422 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306906e6-3784-45e7-936b-c2f082b74210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:58.006258 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306906e6-3784-45e7-936b-c2f082b74210 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:58.006547 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306906e6-3784-45e7-936b-c2f082b74210 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:58.006827 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306906e6-3784-45e7-936b-c2f082b74210 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:58.019154 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306906e6-3784-45e7-936b-c2f082b74210 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:09:58.027536 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-306906e6-3784-45e7-936b-c2f082b74210 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:09:58.027536 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:09:58.027536 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:09:58.027536 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:09:58.027536 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:09:58.027536 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:09:58.027536 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:09:58.027536 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:09:58.027536 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:09:58.027536 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:09:58.027536 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:09:58.027536 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:09:58.027536 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:09:58.027536 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:09:58.027536 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:09:58.027536 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:09:58.028551 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:09:58.028551 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:09:58.028551 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:09:58.028551 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:09:58.028551 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:09:58.028551 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:09:58.028551 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:09:58.028551 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:09:58.028551 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:09:58.028551 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:09:58.028551 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:09:58.028551 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:09:58.028551 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:09:58.028551 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:09:58.028551 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:09:58.028551 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:09:58.029336 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:09:58.029336 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:09:58.029336 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:09:58.029336 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:09:58.029336 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:09:58.029336 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:09:58.029336 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:09:58.029336 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:09:58.029336 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:09:58.029336 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:09:58.029336 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:09:58.029336 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:09:58.029336 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 102/199] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:09:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:09:58.030029 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306906e6-3784-45e7-936b-c2f082b74210 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:58.055228 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:58.104700 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17052df-971b-463c-8c74-07458280b638 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:58.152163 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa3c73d-2d1e-4dde-a56a-08bb512d75c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:58.152823 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa3c73d-2d1e-4dde-a56a-08bb512d75c1 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:58.153110 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa3c73d-2d1e-4dde-a56a-08bb512d75c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:58.153391 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa3c73d-2d1e-4dde-a56a-08bb512d75c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:58.167484 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306906e6-3784-45e7-936b-c2f082b74210 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:58.178464 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:58.211552 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d75e2ed-681e-4d23-a55c-0f7c2d86d7b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:58.212218 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d75e2ed-681e-4d23-a55c-0f7c2d86d7b5 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:58.212552 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d75e2ed-681e-4d23-a55c-0f7c2d86d7b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:58.212836 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d75e2ed-681e-4d23-a55c-0f7c2d86d7b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:58.213679 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d75e2ed-681e-4d23-a55c-0f7c2d86d7b5 None admin] RBAC: Adding query filter params (role.id=4946cf0ff7164307b36efeee7c787146, scope.project.id=3c9e67fcd5b14da2a3775ef2b2cad6b2, user.id=6e59e9f7740a4dc28ef53b32621e82df) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:09:58.214092 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d75e2ed-681e-4d23-a55c-0f7c2d86d7b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:58.215517 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d75e2ed-681e-4d23-a55c-0f7c2d86d7b5 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:58.225869 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 103/200] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:09:58 2022] GET /identity/v3/role_assignments?user.id=6e59e9f7740a4dc28ef53b32621e82df&scope.project.id=3c9e67fcd5b14da2a3775ef2b2cad6b2&role.id=4946cf0ff7164307b36efeee7c787146 => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:09:58.225869 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d75e2ed-681e-4d23-a55c-0f7c2d86d7b5 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:58.278641 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa3c73d-2d1e-4dde-a56a-08bb512d75c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:58.280161 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa3c73d-2d1e-4dde-a56a-08bb512d75c1 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:58.402239 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:58.440003 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d75e2ed-681e-4d23-a55c-0f7c2d86d7b5 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:58.482177 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 98/201] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:09:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 507 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:09:58.485606 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa3c73d-2d1e-4dde-a56a-08bb512d75c1 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:58.491152 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dda103-5061-487a-858c-e69fe448c7db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:58.491831 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dda103-5061-487a-858c-e69fe448c7db None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:58.492117 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dda103-5061-487a-858c-e69fe448c7db None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:58.492393 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dda103-5061-487a-858c-e69fe448c7db None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:58.494630 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dda103-5061-487a-858c-e69fe448c7db None admin] RBAC: Adding query filter params (name=reader) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:09:58.494630 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dda103-5061-487a-858c-e69fe448c7db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:58.494997 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dda103-5061-487a-858c-e69fe448c7db None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:58.515635 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 104/202] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:09:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:09:58.521402 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dda103-5061-487a-858c-e69fe448c7db None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:58.543010 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:58.597508 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa3c73d-2d1e-4dde-a56a-08bb512d75c1 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:58.662311 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dda103-5061-487a-858c-e69fe448c7db None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:58.683178 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790ae27e-4e4e-4b59-a2e4-a0787eac9fd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:58.684394 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790ae27e-4e4e-4b59-a2e4-a0787eac9fd8 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:58.684887 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790ae27e-4e4e-4b59-a2e4-a0787eac9fd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:58.685362 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790ae27e-4e4e-4b59-a2e4-a0787eac9fd8 None admin] PATH_INFO: `/v3/projects/3c9e67fcd5b14da2a3775ef2b2cad6b2` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:58.690590 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790ae27e-4e4e-4b59-a2e4-a0787eac9fd8 None admin] RBAC: Authorizing `identity:get_project(project_id=3c9e67fcd5b14da2a3775ef2b2cad6b2)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:58.692876 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790ae27e-4e4e-4b59-a2e4-a0787eac9fd8 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:58.696134 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 99/203] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:09:58 2022] GET /identity/v3/projects/3c9e67fcd5b14da2a3775ef2b2cad6b2 => generated 302 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:09:58.699698 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9df2100-12f8-409d-b4d9-ddc8e623d84d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:58.700584 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9df2100-12f8-409d-b4d9-ddc8e623d84d None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:58.700871 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9df2100-12f8-409d-b4d9-ddc8e623d84d None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:58.701151 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9df2100-12f8-409d-b4d9-ddc8e623d84d None admin] PATH_INFO: `/v3/users/d21fb41db72f4c15b0261fc2028ed8e2` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:58.702088 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790ae27e-4e4e-4b59-a2e4-a0787eac9fd8 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:58.702392 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9df2100-12f8-409d-b4d9-ddc8e623d84d None admin] RBAC: Authorizing `identity:get_user(user_id=d21fb41db72f4c15b0261fc2028ed8e2)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:58.703961 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9df2100-12f8-409d-b4d9-ddc8e623d84d None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:58.706014 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 105/204] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:09:58 2022] GET /identity/v3/users/d21fb41db72f4c15b0261fc2028ed8e2 => generated 289 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:09:58.711903 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9df2100-12f8-409d-b4d9-ddc8e623d84d None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:58.758289 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:58.823947 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9df2100-12f8-409d-b4d9-ddc8e623d84d None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:58.892510 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d80a2d2-6d90-4733-a92a-2fb9adb85c3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:58.893167 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d80a2d2-6d90-4733-a92a-2fb9adb85c3a None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:58.893448 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d80a2d2-6d90-4733-a92a-2fb9adb85c3a None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:58.893719 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d80a2d2-6d90-4733-a92a-2fb9adb85c3a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:58.948867 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790ae27e-4e4e-4b59-a2e4-a0787eac9fd8 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:58.967049 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:58.992321 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f95570-b91f-4a0b-a238-54dae3c625ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:58.993320 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f95570-b91f-4a0b-a238-54dae3c625ab None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:58.993785 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f95570-b91f-4a0b-a238-54dae3c625ab None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:58.994283 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f95570-b91f-4a0b-a238-54dae3c625ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:58.995299 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f95570-b91f-4a0b-a238-54dae3c625ab None admin] RBAC: Adding query filter params (role.id=23d9dd31632340bdba9cf3093c095dbb, scope.project.id=3c9e67fcd5b14da2a3775ef2b2cad6b2, user.id=6e59e9f7740a4dc28ef53b32621e82df) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:09:58.995790 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f95570-b91f-4a0b-a238-54dae3c625ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:58.997454 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f95570-b91f-4a0b-a238-54dae3c625ab None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:59.007796 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 100/205] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:09:58 2022] GET /identity/v3/role_assignments?user.id=6e59e9f7740a4dc28ef53b32621e82df&scope.project.id=3c9e67fcd5b14da2a3775ef2b2cad6b2&role.id=23d9dd31632340bdba9cf3093c095dbb => generated 259 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:09:59.030241 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d80a2d2-6d90-4733-a92a-2fb9adb85c3a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:59.032020 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d80a2d2-6d90-4733-a92a-2fb9adb85c3a None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:59.056285 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:59.172530 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 106/206] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:09:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:09:59.178976 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f95570-b91f-4a0b-a238-54dae3c625ab None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:59.289822 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f95570-b91f-4a0b-a238-54dae3c625ab None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:59.327005 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdef917b-2197-496a-ae78-26da6f5adefb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:59.328171 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdef917b-2197-496a-ae78-26da6f5adefb None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:59.328661 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdef917b-2197-496a-ae78-26da6f5adefb None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:59.329112 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdef917b-2197-496a-ae78-26da6f5adefb None admin] PATH_INFO: `/v3/projects/ed85b374a73a4c94a310482b7ba566f3` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:59.330513 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdef917b-2197-496a-ae78-26da6f5adefb None admin] RBAC: Authorizing `identity:get_project(project_id=ed85b374a73a4c94a310482b7ba566f3)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:59.332149 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdef917b-2197-496a-ae78-26da6f5adefb None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:59.334912 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 101/207] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:09:59 2022] GET /identity/v3/projects/ed85b374a73a4c94a310482b7ba566f3 => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:09:59.340701 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d80a2d2-6d90-4733-a92a-2fb9adb85c3a None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:59.474817 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d80a2d2-6d90-4733-a92a-2fb9adb85c3a None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:59.492317 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:59.525256 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f03c560-8777-4daf-a4f3-b25fca3e236b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:59.525943 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f03c560-8777-4daf-a4f3-b25fca3e236b None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:59.526229 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f03c560-8777-4daf-a4f3-b25fca3e236b None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:59.526508 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f03c560-8777-4daf-a4f3-b25fca3e236b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:59.527522 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f03c560-8777-4daf-a4f3-b25fca3e236b None admin] RBAC: Adding query filter params (role.id=b0ae35f3e09648d0bbac778d23fa3463, scope.project.id=ed85b374a73a4c94a310482b7ba566f3, user.id=d21fb41db72f4c15b0261fc2028ed8e2) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:09:59.527833 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f03c560-8777-4daf-a4f3-b25fca3e236b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:59.529283 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f03c560-8777-4daf-a4f3-b25fca3e236b None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:59.540639 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 107/208] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:09:59 2022] GET /identity/v3/role_assignments?user.id=d21fb41db72f4c15b0261fc2028ed8e2&scope.project.id=ed85b374a73a4c94a310482b7ba566f3&role.id=b0ae35f3e09648d0bbac778d23fa3463 => generated 608 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:10:00.175944 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf78c367-6460-42b2-aad7-8d80214cbf1c None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:00.176959 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf78c367-6460-42b2-aad7-8d80214cbf1c None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:00.177274 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf78c367-6460-42b2-aad7-8d80214cbf1c None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:00.179081 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 102/209] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:10:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:10:00.184126 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e87fa1-1b5a-4949-9f47-6e0962931dd3 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:00.184482 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e87fa1-1b5a-4949-9f47-6e0962931dd3 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:00.184772 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e87fa1-1b5a-4949-9f47-6e0962931dd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:00.206328 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-14e87fa1-1b5a-4949-9f47-6e0962931dd3 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:10:00.207568 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-14e87fa1-1b5a-4949-9f47-6e0962931dd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:10:00.291996 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 108/210] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:10:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:10:00.315172 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf78c367-6460-42b2-aad7-8d80214cbf1c None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:00.369457 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:00.423541 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf78c367-6460-42b2-aad7-8d80214cbf1c None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:00.460027 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2099935-39f4-43e7-9655-c4911f226446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:00.460828 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2099935-39f4-43e7-9655-c4911f226446 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:00.461115 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2099935-39f4-43e7-9655-c4911f226446 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:00.461396 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2099935-39f4-43e7-9655-c4911f226446 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:00.472286 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2099935-39f4-43e7-9655-c4911f226446 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:10:00.475392 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479b878f-b07f-4062-b0da-15fc40cdcae4 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:00.476248 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479b878f-b07f-4062-b0da-15fc40cdcae4 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:00.476724 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479b878f-b07f-4062-b0da-15fc40cdcae4 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:00.478331 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 109/211] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:10:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:10:00.481708 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-a2099935-39f4-43e7-9655-c4911f226446 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:10:00.481708 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:00.481708 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:00.481708 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:00.481708 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:00.481708 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:00.481708 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:00.481708 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:00.481708 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:00.481708 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:00.481708 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:00.481708 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:00.481708 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:10:00.481708 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:10:00.481708 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:10:00.481708 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:10:00.482650 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:10:00.482650 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:10:00.482650 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:10:00.482650 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:10:00.482650 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:00.482650 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:00.482650 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:00.482650 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:00.482650 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:00.482650 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:00.482650 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:00.482650 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:10:00.482650 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:00.482650 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:00.482650 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:00.482650 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:00.483429 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:00.483429 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:00.483429 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:00.483429 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:00.483429 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:10:00.483429 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:10:00.483429 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:10:00.483429 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:10:00.483429 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:10:00.483429 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:10:00.483429 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:10:00.483429 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:10:00.483429 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 103/212] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:10:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:10:00.486798 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ca415e-fa31-48d5-8820-22511ea44745 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:00.487334 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ca415e-fa31-48d5-8820-22511ea44745 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:00.487809 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ca415e-fa31-48d5-8820-22511ea44745 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:00.490030 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2099935-39f4-43e7-9655-c4911f226446 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:00.510311 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-a4ca415e-fa31-48d5-8820-22511ea44745 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:10:00.511909 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-a4ca415e-fa31-48d5-8820-22511ea44745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:10:00.589926 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 110/213] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:10:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:10:00.600362 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2099935-39f4-43e7-9655-c4911f226446 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:00.610293 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:00.612053 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ca415e-fa31-48d5-8820-22511ea44745 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:00.638154 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994820a3-cb44-4bf4-9d37-627fa0de90ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:00.638841 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994820a3-cb44-4bf4-9d37-627fa0de90ba None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:00.639116 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994820a3-cb44-4bf4-9d37-627fa0de90ba None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:00.639560 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994820a3-cb44-4bf4-9d37-627fa0de90ba None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:00.640481 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994820a3-cb44-4bf4-9d37-627fa0de90ba None admin] RBAC: Adding query filter params (name=member) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:00.640934 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994820a3-cb44-4bf4-9d37-627fa0de90ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:00.642495 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994820a3-cb44-4bf4-9d37-627fa0de90ba None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:00.651788 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 104/214] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:10:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:00.653081 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:00.657532 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994820a3-cb44-4bf4-9d37-627fa0de90ba None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:00.742670 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ca415e-fa31-48d5-8820-22511ea44745 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:00.782597 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60936dae-8a2c-49fe-a770-964e9c2e6a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:00.783954 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60936dae-8a2c-49fe-a770-964e9c2e6a09 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:00.784419 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60936dae-8a2c-49fe-a770-964e9c2e6a09 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:00.784955 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60936dae-8a2c-49fe-a770-964e9c2e6a09 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:00.787108 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60936dae-8a2c-49fe-a770-964e9c2e6a09 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:00.787299 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994820a3-cb44-4bf4-9d37-627fa0de90ba None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:00.789097 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60936dae-8a2c-49fe-a770-964e9c2e6a09 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:00.791532 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 111/215] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:10:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:10:00.799485 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60936dae-8a2c-49fe-a770-964e9c2e6a09 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:00.830497 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab42b58-a0f7-452b-8756-b9bd19c82043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:00.831382 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab42b58-a0f7-452b-8756-b9bd19c82043 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:00.831676 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab42b58-a0f7-452b-8756-b9bd19c82043 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:00.831939 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab42b58-a0f7-452b-8756-b9bd19c82043 None admin] PATH_INFO: `/v3/users/349808e5d90e41e7961267c3ab752d9c` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:00.854131 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab42b58-a0f7-452b-8756-b9bd19c82043 None admin] RBAC: Authorizing `identity:get_user(user_id=349808e5d90e41e7961267c3ab752d9c)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:00.855606 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab42b58-a0f7-452b-8756-b9bd19c82043 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:00.858104 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 105/216] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:10:00 2022] GET /identity/v3/users/349808e5d90e41e7961267c3ab752d9c => generated 297 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:00.873874 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab42b58-a0f7-452b-8756-b9bd19c82043 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:00.903257 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:00.928734 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60936dae-8a2c-49fe-a770-964e9c2e6a09 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:00.932162 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:00.968330 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70497f44-f503-427b-be5b-28c15a60b74f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:00.968996 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70497f44-f503-427b-be5b-28c15a60b74f None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:00.969277 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70497f44-f503-427b-be5b-28c15a60b74f None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:00.969547 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70497f44-f503-427b-be5b-28c15a60b74f None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:00.970301 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70497f44-f503-427b-be5b-28c15a60b74f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:00.971781 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70497f44-f503-427b-be5b-28c15a60b74f None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:00.993051 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab42b58-a0f7-452b-8756-b9bd19c82043 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:00.994453 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 112/217] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 19 14:10:00 2022] POST /identity/v3/users => generated 297 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:10:00.995870 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6458b8-df18-4665-96c3-60f54c694340 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:00.996160 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6458b8-df18-4665-96c3-60f54c694340 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:00.996457 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6458b8-df18-4665-96c3-60f54c694340 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:00.997897 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 113/218] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:10:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:10:01.003479 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817ac760-0fa3-4d56-805c-bfe49ea4ed37 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:01.003815 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817ac760-0fa3-4d56-805c-bfe49ea4ed37 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:01.004155 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817ac760-0fa3-4d56-805c-bfe49ea4ed37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:01.024025 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-817ac760-0fa3-4d56-805c-bfe49ea4ed37 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:10:01.025253 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-817ac760-0fa3-4d56-805c-bfe49ea4ed37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:10:01.045904 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3f8789-ee00-4f9f-9041-2c5f6d306436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:01.047108 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3f8789-ee00-4f9f-9041-2c5f6d306436 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:01.047650 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3f8789-ee00-4f9f-9041-2c5f6d306436 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:01.048120 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3f8789-ee00-4f9f-9041-2c5f6d306436 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:01.106586 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 114/219] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:10:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:10:01.139611 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817ac760-0fa3-4d56-805c-bfe49ea4ed37 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:01.165765 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3f8789-ee00-4f9f-9041-2c5f6d306436 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:01.167676 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3f8789-ee00-4f9f-9041-2c5f6d306436 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:01.201173 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:01.261233 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:01.275802 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817ac760-0fa3-4d56-805c-bfe49ea4ed37 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:01.296559 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 106/220] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:10:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 14:10:01.302817 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3f8789-ee00-4f9f-9041-2c5f6d306436 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:01.327854 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7cccb3-93d0-4a7b-a9b2-82e34a2a1e1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:01.329038 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7cccb3-93d0-4a7b-a9b2-82e34a2a1e1d None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:01.329517 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7cccb3-93d0-4a7b-a9b2-82e34a2a1e1d None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:01.330061 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7cccb3-93d0-4a7b-a9b2-82e34a2a1e1d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:01.343393 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7cccb3-93d0-4a7b-a9b2-82e34a2a1e1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:10:01.352486 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-cc7cccb3-93d0-4a7b-a9b2-82e34a2a1e1d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:10:01.352486 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:01.352486 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:01.352486 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:01.352486 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:01.352486 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:01.352486 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:01.352486 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:01.352486 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:01.352486 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:01.352486 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:01.352486 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:01.352486 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:10:01.352486 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:10:01.352486 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:10:01.352486 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:10:01.353527 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:10:01.353527 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:10:01.353527 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:10:01.353527 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:10:01.353527 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:01.353527 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:01.353527 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:01.353527 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:01.353527 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:01.353527 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:01.353527 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:01.353527 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:10:01.353527 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:01.353527 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:01.353527 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:01.353527 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:01.354557 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:01.354557 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:01.354557 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:01.354557 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:01.354557 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:10:01.354557 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:10:01.354557 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:10:01.354557 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:10:01.354557 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:10:01.354557 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:10:01.354557 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:10:01.354557 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:10:01.356505 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 115/221] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:10:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:10:01.362471 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7cccb3-93d0-4a7b-a9b2-82e34a2a1e1d None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:01.428513 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3f8789-ee00-4f9f-9041-2c5f6d306436 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:01.467397 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:01.478877 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ffda0f-804c-41ab-9db1-6c0529f72300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:01.480460 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ffda0f-804c-41ab-9db1-6c0529f72300 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:01.480976 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ffda0f-804c-41ab-9db1-6c0529f72300 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:01.481437 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ffda0f-804c-41ab-9db1-6c0529f72300 None admin] PATH_INFO: `/v3/projects/3c9e67fcd5b14da2a3775ef2b2cad6b2` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:01.483118 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ffda0f-804c-41ab-9db1-6c0529f72300 None admin] RBAC: Authorizing `identity:get_project(project_id=3c9e67fcd5b14da2a3775ef2b2cad6b2)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:01.485953 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ffda0f-804c-41ab-9db1-6c0529f72300 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:01.490118 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 107/222] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:10:01 2022] GET /identity/v3/projects/3c9e67fcd5b14da2a3775ef2b2cad6b2 => generated 302 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:10:01.496242 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ffda0f-804c-41ab-9db1-6c0529f72300 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:01.499593 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7cccb3-93d0-4a7b-a9b2-82e34a2a1e1d None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:01.545652 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a69220-dc59-4b57-a5e2-63304290a637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:01.546441 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a69220-dc59-4b57-a5e2-63304290a637 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:01.546791 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a69220-dc59-4b57-a5e2-63304290a637 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:01.547137 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a69220-dc59-4b57-a5e2-63304290a637 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:01.548479 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a69220-dc59-4b57-a5e2-63304290a637 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:01.548790 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a69220-dc59-4b57-a5e2-63304290a637 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:01.550369 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a69220-dc59-4b57-a5e2-63304290a637 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:01.564437 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 116/223] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:10:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:01.570519 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a69220-dc59-4b57-a5e2-63304290a637 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:01.606191 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:01.644083 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ffda0f-804c-41ab-9db1-6c0529f72300 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:01.681563 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e77647-e3e3-4505-a173-840cb0838c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:01.682186 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e77647-e3e3-4505-a173-840cb0838c8a None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:01.682475 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e77647-e3e3-4505-a173-840cb0838c8a None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:01.682757 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e77647-e3e3-4505-a173-840cb0838c8a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:01.683601 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e77647-e3e3-4505-a173-840cb0838c8a None admin] RBAC: Adding query filter params (role.id=7d0e162805b54c019ce6c954cbef3818, scope.project.id=3c9e67fcd5b14da2a3775ef2b2cad6b2, user.id=349808e5d90e41e7961267c3ab752d9c) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:01.683917 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e77647-e3e3-4505-a173-840cb0838c8a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:01.685580 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e77647-e3e3-4505-a173-840cb0838c8a None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:01.692325 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 108/224] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:10:01 2022] GET /identity/v3/role_assignments?user.id=349808e5d90e41e7961267c3ab752d9c&scope.project.id=3c9e67fcd5b14da2a3775ef2b2cad6b2&role.id=7d0e162805b54c019ce6c954cbef3818 => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:01.693906 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c519ef7a-0137-4f15-ac50-253578ec2c2b None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:01.694400 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c519ef7a-0137-4f15-ac50-253578ec2c2b None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:01.694783 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c519ef7a-0137-4f15-ac50-253578ec2c2b None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:01.696700 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 109/225] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:10:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:10:01.702486 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b09683-77e7-4d76-8eee-51cf8959df78 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:01.702779 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b09683-77e7-4d76-8eee-51cf8959df78 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:01.703050 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b09683-77e7-4d76-8eee-51cf8959df78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:01.737332 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-29b09683-77e7-4d76-8eee-51cf8959df78 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:10:01.738757 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-29b09683-77e7-4d76-8eee-51cf8959df78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:10:01.790551 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a69220-dc59-4b57-a5e2-63304290a637 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:01.847056 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:01.860155 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a1428d-4dfa-4876-892a-fdc500c4ddf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:01.861070 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a1428d-4dfa-4876-892a-fdc500c4ddf4 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:01.861355 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a1428d-4dfa-4876-892a-fdc500c4ddf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:01.861678 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a1428d-4dfa-4876-892a-fdc500c4ddf4 None admin] PATH_INFO: `/v3/users/6e59e9f7740a4dc28ef53b32621e82df` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:01.863082 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a1428d-4dfa-4876-892a-fdc500c4ddf4 None admin] RBAC: Authorizing `identity:get_user(user_id=6e59e9f7740a4dc28ef53b32621e82df)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:01.864714 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a1428d-4dfa-4876-892a-fdc500c4ddf4 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:01.871082 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 110/226] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:10:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:10:01.871333 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 117/227] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:10:01 2022] GET /identity/v3/users/6e59e9f7740a4dc28ef53b32621e82df => generated 283 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:01.885335 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b09683-77e7-4d76-8eee-51cf8959df78 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:01.902746 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a1428d-4dfa-4876-892a-fdc500c4ddf4 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:01.998551 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:02.014271 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b09683-77e7-4d76-8eee-51cf8959df78 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:02.021561 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a1428d-4dfa-4876-892a-fdc500c4ddf4 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:02.052693 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e341b1e-5b86-44e6-9d99-263bcff9d6dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:02.053962 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e341b1e-5b86-44e6-9d99-263bcff9d6dc None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:02.054444 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e341b1e-5b86-44e6-9d99-263bcff9d6dc None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:02.054935 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e341b1e-5b86-44e6-9d99-263bcff9d6dc None admin] PATH_INFO: `/v3/projects/3c9e67fcd5b14da2a3775ef2b2cad6b2` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:02.056336 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e341b1e-5b86-44e6-9d99-263bcff9d6dc None admin] RBAC: Authorizing `identity:get_project(project_id=3c9e67fcd5b14da2a3775ef2b2cad6b2)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:02.058029 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e341b1e-5b86-44e6-9d99-263bcff9d6dc None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:02.058384 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9730e04a-ef9a-41b3-9000-8f7588525c81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:02.059194 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9730e04a-ef9a-41b3-9000-8f7588525c81 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:02.059475 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9730e04a-ef9a-41b3-9000-8f7588525c81 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:02.059750 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9730e04a-ef9a-41b3-9000-8f7588525c81 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:02.061134 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 111/228] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:10:01 2022] GET /identity/v3/projects/3c9e67fcd5b14da2a3775ef2b2cad6b2 => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:02.067268 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e341b1e-5b86-44e6-9d99-263bcff9d6dc None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:02.072174 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9730e04a-ef9a-41b3-9000-8f7588525c81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:10:02.080214 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-9730e04a-ef9a-41b3-9000-8f7588525c81 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:10:02.080214 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:02.080214 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:02.080214 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:02.080214 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:02.080214 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:02.080214 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:02.080214 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:02.080214 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:02.080214 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:02.080214 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:02.080214 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:02.080214 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:10:02.080214 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:10:02.080214 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:10:02.080214 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:10:02.081110 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:10:02.081110 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:10:02.081110 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:10:02.081110 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:10:02.081110 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:02.081110 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:02.081110 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:02.081110 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:02.081110 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:02.081110 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:02.081110 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:02.081110 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:10:02.081110 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:02.081110 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:02.081110 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:02.081110 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:02.081943 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:02.081943 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:02.081943 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:02.081943 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:02.081943 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:10:02.081943 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:10:02.081943 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:10:02.081943 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:10:02.081943 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:10:02.081943 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:10:02.081943 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:10:02.081943 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:10:02.081943 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 118/229] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 19 14:10:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:10:02.088434 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9730e04a-ef9a-41b3-9000-8f7588525c81 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:02.146632 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:02.192743 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e341b1e-5b86-44e6-9d99-263bcff9d6dc None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:02.199295 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9730e04a-ef9a-41b3-9000-8f7588525c81 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:02.230264 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d108fe15-3ffc-40bc-a3bb-1614add3bbb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:02.231900 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d108fe15-3ffc-40bc-a3bb-1614add3bbb2 None admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:02.232382 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d108fe15-3ffc-40bc-a3bb-1614add3bbb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:02.232870 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d108fe15-3ffc-40bc-a3bb-1614add3bbb2 None admin] PATH_INFO: `/v3/projects/3c9e67fcd5b14da2a3775ef2b2cad6b2/users/6e59e9f7740a4dc28ef53b32621e82df/roles/4946cf0ff7164307b36efeee7c787146` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:02.235253 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d108fe15-3ffc-40bc-a3bb-1614add3bbb2 None admin] RBAC: Authorizing `identity:create_grant(project_id=3c9e67fcd5b14da2a3775ef2b2cad6b2, user_id=6e59e9f7740a4dc28ef53b32621e82df, role_id=4946cf0ff7164307b36efeee7c787146)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:02.236922 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d108fe15-3ffc-40bc-a3bb-1614add3bbb2 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:02.238041 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2d30b5-e904-4088-bc8e-5bf8c02f375e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:02.238800 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2d30b5-e904-4088-bc8e-5bf8c02f375e None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:02.239711 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2d30b5-e904-4088-bc8e-5bf8c02f375e None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:02.239995 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2d30b5-e904-4088-bc8e-5bf8c02f375e None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:02.241169 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2d30b5-e904-4088-bc8e-5bf8c02f375e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:02.241343 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2d30b5-e904-4088-bc8e-5bf8c02f375e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:02.242789 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2d30b5-e904-4088-bc8e-5bf8c02f375e None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:02.252102 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 119/230] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:10:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:10:02.258679 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2d30b5-e904-4088-bc8e-5bf8c02f375e None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:02.261501 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:02.262143 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 112/231] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 14:10:02 2022] PUT /identity/v3/projects/3c9e67fcd5b14da2a3775ef2b2cad6b2/users/6e59e9f7740a4dc28ef53b32621e82df/roles/4946cf0ff7164307b36efeee7c787146 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 14:10:02.376018 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2d30b5-e904-4088-bc8e-5bf8c02f375e None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:02.414181 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a3cd21-82b9-4728-8764-eca5446d24dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:02.415081 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a3cd21-82b9-4728-8764-eca5446d24dc None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:02.415400 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a3cd21-82b9-4728-8764-eca5446d24dc None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:02.415635 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a3cd21-82b9-4728-8764-eca5446d24dc None admin] PATH_INFO: `/v3/users/6e59e9f7740a4dc28ef53b32621e82df` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:02.416882 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a3cd21-82b9-4728-8764-eca5446d24dc None admin] RBAC: Authorizing `identity:get_user(user_id=6e59e9f7740a4dc28ef53b32621e82df)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:02.418522 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a3cd21-82b9-4728-8764-eca5446d24dc None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:02.420528 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 120/232] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:10:02 2022] GET /identity/v3/users/6e59e9f7740a4dc28ef53b32621e82df => generated 283 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:02.427168 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d108fe15-3ffc-40bc-a3bb-1614add3bbb2 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:02.563082 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d108fe15-3ffc-40bc-a3bb-1614add3bbb2 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:02.582760 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:02.604132 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26275e3c-357a-403b-865e-57610ef2fb31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:02.605323 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26275e3c-357a-403b-865e-57610ef2fb31 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:02.605804 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26275e3c-357a-403b-865e-57610ef2fb31 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:02.606304 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26275e3c-357a-403b-865e-57610ef2fb31 None admin] PATH_INFO: `/v3/projects/3c9e67fcd5b14da2a3775ef2b2cad6b2` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:02.607700 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26275e3c-357a-403b-865e-57610ef2fb31 None admin] RBAC: Authorizing `identity:get_project(project_id=3c9e67fcd5b14da2a3775ef2b2cad6b2)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:02.609369 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26275e3c-357a-403b-865e-57610ef2fb31 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:02.611617 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 113/233] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:10:02 2022] GET /identity/v3/projects/3c9e67fcd5b14da2a3775ef2b2cad6b2 => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:02.618446 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a3cd21-82b9-4728-8764-eca5446d24dc None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:02.694983 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:02.744733 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a3cd21-82b9-4728-8764-eca5446d24dc None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:02.789567 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fcbfbc-1b08-497a-94a6-0ac94e67f88a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:02.790500 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fcbfbc-1b08-497a-94a6-0ac94e67f88a None admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:02.790782 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fcbfbc-1b08-497a-94a6-0ac94e67f88a None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:02.791054 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fcbfbc-1b08-497a-94a6-0ac94e67f88a None admin] PATH_INFO: `/v3/projects/3c9e67fcd5b14da2a3775ef2b2cad6b2/users/6e59e9f7740a4dc28ef53b32621e82df/roles/23d9dd31632340bdba9cf3093c095dbb` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:02.793362 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fcbfbc-1b08-497a-94a6-0ac94e67f88a None admin] RBAC: Authorizing `identity:create_grant(project_id=3c9e67fcd5b14da2a3775ef2b2cad6b2, user_id=6e59e9f7740a4dc28ef53b32621e82df, role_id=23d9dd31632340bdba9cf3093c095dbb)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:02.794841 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fcbfbc-1b08-497a-94a6-0ac94e67f88a None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:02.824348 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 121/234] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 14:10:02 2022] PUT /identity/v3/projects/3c9e67fcd5b14da2a3775ef2b2cad6b2/users/6e59e9f7740a4dc28ef53b32621e82df/roles/23d9dd31632340bdba9cf3093c095dbb => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:10:04.082492 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a80d34-bb6a-492b-9d1e-0bcf54bc9e4a None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:04.083009 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a80d34-bb6a-492b-9d1e-0bcf54bc9e4a None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:04.083281 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a80d34-bb6a-492b-9d1e-0bcf54bc9e4a None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:04.084901 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 114/235] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:10:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:10:04.090704 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c14821f-159d-4f30-afe9-034c13b428a1 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:04.091169 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c14821f-159d-4f30-afe9-034c13b428a1 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:04.091245 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c14821f-159d-4f30-afe9-034c13b428a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:04.094220 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a60cb8-dfa7-42d3-bfe0-77434e2bc574 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:04.094742 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a60cb8-dfa7-42d3-bfe0-77434e2bc574 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:04.095216 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a60cb8-dfa7-42d3-bfe0-77434e2bc574 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:04.098420 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 115/236] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:10:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:10:04.110608 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9e43cb-89eb-4877-8b7a-35c0a21c6b88 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:04.111138 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9e43cb-89eb-4877-8b7a-35c0a21c6b88 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:04.111612 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9e43cb-89eb-4877-8b7a-35c0a21c6b88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:04.114659 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-0c14821f-159d-4f30-afe9-034c13b428a1 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:10:04.116129 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-0c14821f-159d-4f30-afe9-034c13b428a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:10:04.139959 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-3b9e43cb-89eb-4877-8b7a-35c0a21c6b88 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:10:04.142856 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-3b9e43cb-89eb-4877-8b7a-35c0a21c6b88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:10:04.225572 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 116/237] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:10:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:10:04.260601 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 122/238] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:10:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:10:04.288402 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9e43cb-89eb-4877-8b7a-35c0a21c6b88 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:04.289568 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c14821f-159d-4f30-afe9-034c13b428a1 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:04.350541 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:04.393384 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c14821f-159d-4f30-afe9-034c13b428a1 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:04.413312 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9e43cb-89eb-4877-8b7a-35c0a21c6b88 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:04.432324 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70349493-c0ae-4f16-8e05-50348d77eb11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:04.433226 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70349493-c0ae-4f16-8e05-50348d77eb11 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:04.433589 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70349493-c0ae-4f16-8e05-50348d77eb11 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:04.433965 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70349493-c0ae-4f16-8e05-50348d77eb11 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:04.435046 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:04.435314 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70349493-c0ae-4f16-8e05-50348d77eb11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:04.436885 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70349493-c0ae-4f16-8e05-50348d77eb11 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:04.438690 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 123/239] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:10:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:10:04.440235 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7785b8c6-d224-474a-8a13-4b757001d89a None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:04.440627 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7785b8c6-d224-474a-8a13-4b757001d89a None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:04.441156 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7785b8c6-d224-474a-8a13-4b757001d89a None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:04.442631 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 124/240] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:10:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:10:04.444405 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7785b8c6-d224-474a-8a13-4b757001d89a None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:04.461247 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449090e1-f4cc-4b30-b8a1-c7ffcf7b5898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:04.462188 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449090e1-f4cc-4b30-b8a1-c7ffcf7b5898 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:04.462642 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449090e1-f4cc-4b30-b8a1-c7ffcf7b5898 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:04.462995 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449090e1-f4cc-4b30-b8a1-c7ffcf7b5898 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:04.475604 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449090e1-f4cc-4b30-b8a1-c7ffcf7b5898 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:10:04.483839 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-449090e1-f4cc-4b30-b8a1-c7ffcf7b5898 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:10:04.483839 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:04.483839 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:04.483839 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:04.483839 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:04.483839 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:04.483839 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:04.483839 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:04.483839 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:04.483839 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:04.483839 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:04.483839 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:04.483839 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:10:04.483839 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:10:04.483839 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:10:04.483839 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:10:04.484733 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:10:04.484733 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:10:04.484733 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:10:04.484733 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:10:04.484733 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:04.484733 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:04.484733 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:04.484733 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:04.484733 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:04.484733 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:04.484733 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:04.484733 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:10:04.484733 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:04.484733 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:04.484733 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:04.484733 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:04.485533 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:04.485533 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:04.485533 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:04.485533 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:04.485533 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:10:04.485533 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:10:04.485533 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:10:04.485533 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:10:04.485533 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:10:04.485533 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:10:04.485533 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:10:04.485533 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:10:04.485533 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 117/241] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:10:04 2022] GET /identity/v3/roles/reader => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:10:04.486793 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d053962c-06d5-440d-9e4a-43c58f760ac0 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:04.487006 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d053962c-06d5-440d-9e4a-43c58f760ac0 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:04.487342 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d053962c-06d5-440d-9e4a-43c58f760ac0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:04.505898 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-d053962c-06d5-440d-9e4a-43c58f760ac0 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:10:04.507209 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-d053962c-06d5-440d-9e4a-43c58f760ac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:10:04.565725 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7785b8c6-d224-474a-8a13-4b757001d89a None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:04.583891 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 118/242] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:10:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:10:04.590850 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d053962c-06d5-440d-9e4a-43c58f760ac0 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:04.596088 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:04.611111 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e7e3d4-b29e-4b17-9217-e316ceb63134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:04.612647 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e7e3d4-b29e-4b17-9217-e316ceb63134 None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:04.613337 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e7e3d4-b29e-4b17-9217-e316ceb63134 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:04.614337 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e7e3d4-b29e-4b17-9217-e316ceb63134 None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:04.615097 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e7e3d4-b29e-4b17-9217-e316ceb63134 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:04.616843 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e7e3d4-b29e-4b17-9217-e316ceb63134 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:04.648327 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 125/243] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 19 14:10:04 2022] POST /identity/v3/users => generated 293 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 14:10:04.649656 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e7e3d4-b29e-4b17-9217-e316ceb63134 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:04.827464 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e7e3d4-b29e-4b17-9217-e316ceb63134 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:04.851349 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d053962c-06d5-440d-9e4a-43c58f760ac0 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:04.864391 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:04.874011 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1262dd-bec5-47bb-b0a1-eb4847ef873e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:04.875475 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1262dd-bec5-47bb-b0a1-eb4847ef873e None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:04.875959 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1262dd-bec5-47bb-b0a1-eb4847ef873e None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:04.876481 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1262dd-bec5-47bb-b0a1-eb4847ef873e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:04.892087 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1262dd-bec5-47bb-b0a1-eb4847ef873e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:10:04.902714 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-9e1262dd-bec5-47bb-b0a1-eb4847ef873e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:10:04.902714 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:04.902714 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:04.902714 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:04.902714 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:04.902714 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:04.902714 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:04.902714 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:04.902714 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:04.902714 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:04.902714 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:04.902714 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:04.902714 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:10:04.902714 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:10:04.902714 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:10:04.902714 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:10:04.903635 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:10:04.903635 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:10:04.903635 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:10:04.903635 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:10:04.903635 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:04.903635 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:04.903635 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:04.903635 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:04.903635 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:04.903635 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:04.903635 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:04.903635 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:10:04.903635 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:04.903635 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:04.903635 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:04.903635 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:04.904446 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:04.904446 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:04.904446 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:04.904446 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:04.904446 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:10:04.904446 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:10:04.904446 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:10:04.904446 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:10:04.904446 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:10:04.904446 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:10:04.904446 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:10:04.904446 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:10:04.907205 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 126/244] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:10:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:10:04.910329 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce55cbb-a3a0-4871-8609-823df442c734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:04.911596 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce55cbb-a3a0-4871-8609-823df442c734 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:04.912094 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce55cbb-a3a0-4871-8609-823df442c734 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:04.912206 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce55cbb-a3a0-4871-8609-823df442c734 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:04.913279 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1262dd-bec5-47bb-b0a1-eb4847ef873e None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:04.913426 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce55cbb-a3a0-4871-8609-823df442c734 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:04.918152 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce55cbb-a3a0-4871-8609-823df442c734 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:04.920011 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce55cbb-a3a0-4871-8609-823df442c734 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:04.930421 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 119/245] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:10:04 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:04.936499 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce55cbb-a3a0-4871-8609-823df442c734 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:04.972034 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:05.038137 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1262dd-bec5-47bb-b0a1-eb4847ef873e None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:05.050528 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce55cbb-a3a0-4871-8609-823df442c734 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:05.078487 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbf8e54-89c5-480a-b413-94e999a8c188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:05.079564 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbf8e54-89c5-480a-b413-94e999a8c188 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:05.080125 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbf8e54-89c5-480a-b413-94e999a8c188 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:05.080606 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbf8e54-89c5-480a-b413-94e999a8c188 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:05.081577 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbf8e54-89c5-480a-b413-94e999a8c188 None admin] RBAC: Adding query filter params (name=member) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:05.083127 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbf8e54-89c5-480a-b413-94e999a8c188 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:05.085048 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbf8e54-89c5-480a-b413-94e999a8c188 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:05.089694 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff63bbc8-5bed-41e0-9710-f2955d561785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:05.090702 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff63bbc8-5bed-41e0-9710-f2955d561785 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:05.091061 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff63bbc8-5bed-41e0-9710-f2955d561785 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:05.091414 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff63bbc8-5bed-41e0-9710-f2955d561785 None admin] PATH_INFO: `/v3/users/8617ec97033d40e78f04e210afc8e531` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:05.096988 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 127/246] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:10:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:05.106655 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbf8e54-89c5-480a-b413-94e999a8c188 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:05.118604 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff63bbc8-5bed-41e0-9710-f2955d561785 None admin] RBAC: Authorizing `identity:get_user(user_id=8617ec97033d40e78f04e210afc8e531)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:05.120098 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff63bbc8-5bed-41e0-9710-f2955d561785 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:05.123283 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 120/247] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:10:04 2022] GET /identity/v3/users/8617ec97033d40e78f04e210afc8e531 => generated 297 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:05.129650 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff63bbc8-5bed-41e0-9710-f2955d561785 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:05.163003 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:05.253327 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbf8e54-89c5-480a-b413-94e999a8c188 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:05.261304 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff63bbc8-5bed-41e0-9710-f2955d561785 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:05.262188 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:05.295137 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01abf261-ce4e-4c6f-a773-e9128b5a1591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:05.296025 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01abf261-ce4e-4c6f-a773-e9128b5a1591 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:05.296309 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01abf261-ce4e-4c6f-a773-e9128b5a1591 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:05.296592 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01abf261-ce4e-4c6f-a773-e9128b5a1591 None admin] PATH_INFO: `/v3/users/349808e5d90e41e7961267c3ab752d9c` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:05.297809 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01abf261-ce4e-4c6f-a773-e9128b5a1591 None admin] RBAC: Authorizing `identity:get_user(user_id=349808e5d90e41e7961267c3ab752d9c)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:05.299323 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01abf261-ce4e-4c6f-a773-e9128b5a1591 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:05.305015 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 128/248] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:10:05 2022] GET /identity/v3/users/349808e5d90e41e7961267c3ab752d9c => generated 297 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:10:05.305015 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d084ba-43a1-4f8d-a635-a1f64e499c4e None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:05.305015 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d084ba-43a1-4f8d-a635-a1f64e499c4e None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:05.305015 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d084ba-43a1-4f8d-a635-a1f64e499c4e None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:05.305015 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 129/249] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:10:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:10:05.307695 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67863a5f-e86e-41aa-b6ef-cfa82d4c858a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:05.308548 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67863a5f-e86e-41aa-b6ef-cfa82d4c858a None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:05.309059 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67863a5f-e86e-41aa-b6ef-cfa82d4c858a None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:05.309478 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67863a5f-e86e-41aa-b6ef-cfa82d4c858a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:05.311199 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becdec68-1eb8-487c-b972-d372443c412d None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:05.311199 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becdec68-1eb8-487c-b972-d372443c412d None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:05.311199 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becdec68-1eb8-487c-b972-d372443c412d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:05.330103 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-becdec68-1eb8-487c-b972-d372443c412d None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:10:05.331769 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-becdec68-1eb8-487c-b972-d372443c412d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:10:05.406296 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67863a5f-e86e-41aa-b6ef-cfa82d4c858a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:05.408126 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67863a5f-e86e-41aa-b6ef-cfa82d4c858a None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:05.412276 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 130/250] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:10:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:10:05.413804 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becdec68-1eb8-487c-b972-d372443c412d None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:05.465876 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:05.532826 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 121/251] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:10:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 14:10:05.534566 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67863a5f-e86e-41aa-b6ef-cfa82d4c858a None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:05.583311 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becdec68-1eb8-487c-b972-d372443c412d None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:05.596120 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:05.662720 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04673403-88f0-4260-a461-99a27e9750fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:05.663498 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04673403-88f0-4260-a461-99a27e9750fd None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:05.663787 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04673403-88f0-4260-a461-99a27e9750fd None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:05.664065 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04673403-88f0-4260-a461-99a27e9750fd None admin] PATH_INFO: `/v3/projects/3c9e67fcd5b14da2a3775ef2b2cad6b2` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:05.668985 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04673403-88f0-4260-a461-99a27e9750fd None admin] RBAC: Authorizing `identity:get_project(project_id=3c9e67fcd5b14da2a3775ef2b2cad6b2)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:05.668985 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04673403-88f0-4260-a461-99a27e9750fd None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:05.668985 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 131/252] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:10:05 2022] GET /identity/v3/projects/3c9e67fcd5b14da2a3775ef2b2cad6b2 => generated 302 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:05.669548 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04673403-88f0-4260-a461-99a27e9750fd None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:05.680133 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67863a5f-e86e-41aa-b6ef-cfa82d4c858a None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:05.735279 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:05.750278 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bdb2fd-affa-49c8-9008-8d58b1c72ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:05.752225 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bdb2fd-affa-49c8-9008-8d58b1c72ce3 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:05.752513 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bdb2fd-affa-49c8-9008-8d58b1c72ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:05.752792 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bdb2fd-affa-49c8-9008-8d58b1c72ce3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:05.781433 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bdb2fd-affa-49c8-9008-8d58b1c72ce3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:10:05.795792 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-36bdb2fd-affa-49c8-9008-8d58b1c72ce3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:10:05.795792 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:05.795792 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:05.795792 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:05.795792 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:05.795792 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:05.795792 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:05.795792 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:05.795792 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:05.795792 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:05.795792 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:05.795792 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:05.795792 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:10:05.795792 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:10:05.795792 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:10:05.795792 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:10:05.796714 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:10:05.796714 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:10:05.796714 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:10:05.796714 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:10:05.796714 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:05.796714 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:05.796714 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:05.796714 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:05.796714 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:05.796714 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:05.796714 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:05.796714 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:10:05.796714 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:05.796714 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:05.796714 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:05.796714 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:05.797508 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:05.797508 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:05.797508 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:05.797508 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:05.797508 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:10:05.797508 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:10:05.797508 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:10:05.797508 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:10:05.797508 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:10:05.797508 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:10:05.797508 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:10:05.797508 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:10:05.799193 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 122/253] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:10:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:10:05.800113 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bdb2fd-affa-49c8-9008-8d58b1c72ce3 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:05.825268 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04673403-88f0-4260-a461-99a27e9750fd None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:05.877324 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f654ed-98f8-4e60-9cbe-b3d95a1767b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:05.878243 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f654ed-98f8-4e60-9cbe-b3d95a1767b3 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:05.878532 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f654ed-98f8-4e60-9cbe-b3d95a1767b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:05.878806 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f654ed-98f8-4e60-9cbe-b3d95a1767b3 None admin] PATH_INFO: `/v3/projects/3c9e67fcd5b14da2a3775ef2b2cad6b2` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:05.880563 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f654ed-98f8-4e60-9cbe-b3d95a1767b3 None admin] RBAC: Authorizing `identity:get_project(project_id=3c9e67fcd5b14da2a3775ef2b2cad6b2)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:05.882057 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f654ed-98f8-4e60-9cbe-b3d95a1767b3 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:05.883779 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 132/254] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:10:05 2022] GET /identity/v3/projects/3c9e67fcd5b14da2a3775ef2b2cad6b2 => generated 302 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:05.884962 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b3c293-fdc5-4613-83d1-17ad3bface43 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:05.885245 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b3c293-fdc5-4613-83d1-17ad3bface43 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:05.885519 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b3c293-fdc5-4613-83d1-17ad3bface43 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:05.891916 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 133/255] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:10:05 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:10:05.892690 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b3c293-fdc5-4613-83d1-17ad3bface43 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:05.938396 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:05.942427 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bdb2fd-affa-49c8-9008-8d58b1c72ce3 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:05.991545 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a42f61-30fe-49f6-a6ee-38fd733f5c0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:05.992966 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a42f61-30fe-49f6-a6ee-38fd733f5c0b None admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:05.993662 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a42f61-30fe-49f6-a6ee-38fd733f5c0b None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:05.994516 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a42f61-30fe-49f6-a6ee-38fd733f5c0b None admin] PATH_INFO: `/v3/projects/3c9e67fcd5b14da2a3775ef2b2cad6b2/users/349808e5d90e41e7961267c3ab752d9c/roles/7d0e162805b54c019ce6c954cbef3818` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:05.997718 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a42f61-30fe-49f6-a6ee-38fd733f5c0b None admin] RBAC: Authorizing `identity:create_grant(project_id=3c9e67fcd5b14da2a3775ef2b2cad6b2, user_id=349808e5d90e41e7961267c3ab752d9c, role_id=7d0e162805b54c019ce6c954cbef3818)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:06.000264 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a42f61-30fe-49f6-a6ee-38fd733f5c0b None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:06.003855 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b3c293-fdc5-4613-83d1-17ad3bface43 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:06.019390 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 123/256] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 14:10:05 2022] PUT /identity/v3/projects/3c9e67fcd5b14da2a3775ef2b2cad6b2/users/349808e5d90e41e7961267c3ab752d9c/roles/7d0e162805b54c019ce6c954cbef3818 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:10:06.021710 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a42f61-30fe-49f6-a6ee-38fd733f5c0b None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:06.044335 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fa148e-c830-4545-9d06-00309e0dd317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:06.045157 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fa148e-c830-4545-9d06-00309e0dd317 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:06.045300 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fa148e-c830-4545-9d06-00309e0dd317 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:06.045577 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fa148e-c830-4545-9d06-00309e0dd317 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:06.046571 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fa148e-c830-4545-9d06-00309e0dd317 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:06.046766 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fa148e-c830-4545-9d06-00309e0dd317 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:06.048316 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fa148e-c830-4545-9d06-00309e0dd317 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:06.063969 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 134/257] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:10:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:06.063969 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2aef48-7829-4cd2-9c02-07048b585b51 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:06.063969 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2aef48-7829-4cd2-9c02-07048b585b51 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:06.063969 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2aef48-7829-4cd2-9c02-07048b585b51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:06.082882 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-8d2aef48-7829-4cd2-9c02-07048b585b51 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:10:06.084207 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-8d2aef48-7829-4cd2-9c02-07048b585b51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:10:06.122685 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:06.183442 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a42f61-30fe-49f6-a6ee-38fd733f5c0b None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:06.199125 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 135/258] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:10:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:10:06.204684 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2aef48-7829-4cd2-9c02-07048b585b51 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:06.236591 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:06.259856 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320c673d-dad5-4a93-9ed2-69cbd0b30bb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:06.262327 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320c673d-dad5-4a93-9ed2-69cbd0b30bb1 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:06.263546 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320c673d-dad5-4a93-9ed2-69cbd0b30bb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:06.264063 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320c673d-dad5-4a93-9ed2-69cbd0b30bb1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:06.265149 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320c673d-dad5-4a93-9ed2-69cbd0b30bb1 None admin] RBAC: Adding query filter params (role.id=b0ae35f3e09648d0bbac778d23fa3463, scope.project.id=3c9e67fcd5b14da2a3775ef2b2cad6b2, user.id=8617ec97033d40e78f04e210afc8e531) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:06.265669 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320c673d-dad5-4a93-9ed2-69cbd0b30bb1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:06.267607 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320c673d-dad5-4a93-9ed2-69cbd0b30bb1 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:06.280678 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 124/259] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:10:06 2022] GET /identity/v3/role_assignments?user.id=8617ec97033d40e78f04e210afc8e531&scope.project.id=3c9e67fcd5b14da2a3775ef2b2cad6b2&role.id=b0ae35f3e09648d0bbac778d23fa3463 => generated 259 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:06.284956 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320c673d-dad5-4a93-9ed2-69cbd0b30bb1 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:06.396927 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2aef48-7829-4cd2-9c02-07048b585b51 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:06.460635 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d7f36b-b238-45cb-9b4d-ceb3184706cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:06.461901 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d7f36b-b238-45cb-9b4d-ceb3184706cd None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:06.462916 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d7f36b-b238-45cb-9b4d-ceb3184706cd None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:06.463579 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d7f36b-b238-45cb-9b4d-ceb3184706cd None admin] PATH_INFO: `/v3/users/6e59e9f7740a4dc28ef53b32621e82df` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:06.465410 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d7f36b-b238-45cb-9b4d-ceb3184706cd None admin] RBAC: Authorizing `identity:get_user(user_id=6e59e9f7740a4dc28ef53b32621e82df)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:06.467223 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d7f36b-b238-45cb-9b4d-ceb3184706cd None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:06.469265 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 136/260] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:10:06 2022] GET /identity/v3/users/6e59e9f7740a4dc28ef53b32621e82df => generated 283 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:06.476166 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d7f36b-b238-45cb-9b4d-ceb3184706cd None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:06.494137 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320c673d-dad5-4a93-9ed2-69cbd0b30bb1 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:06.505746 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:06.563913 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb22f7d-e6e7-4246-ae02-3d68e8f2cf6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:06.564754 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb22f7d-e6e7-4246-ae02-3d68e8f2cf6b None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:06.565040 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb22f7d-e6e7-4246-ae02-3d68e8f2cf6b None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:06.565316 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb22f7d-e6e7-4246-ae02-3d68e8f2cf6b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:06.590987 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb22f7d-e6e7-4246-ae02-3d68e8f2cf6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:10:06.607779 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-5bb22f7d-e6e7-4246-ae02-3d68e8f2cf6b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:10:06.607779 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:06.607779 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:06.607779 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:06.607779 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:06.607779 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:06.607779 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:06.607779 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:06.607779 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:06.607779 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:06.607779 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:06.607779 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:06.607779 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:10:06.607779 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:10:06.607779 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:10:06.607779 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:10:06.608740 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:10:06.608740 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:10:06.608740 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:10:06.608740 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:10:06.608740 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:06.608740 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:06.608740 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:06.608740 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:06.608740 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:06.608740 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:06.608740 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:06.608740 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:10:06.608740 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:06.608740 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:06.608740 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:06.608740 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:06.609647 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:06.609647 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:06.609647 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:06.609647 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:06.609647 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:10:06.609647 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:10:06.609647 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:10:06.609647 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:10:06.609647 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:10:06.609647 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:10:06.609647 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:10:06.609647 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:10:06.609647 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 125/261] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 19 14:10:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:10:06.616633 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb22f7d-e6e7-4246-ae02-3d68e8f2cf6b None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:06.659408 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:06.664551 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d7f36b-b238-45cb-9b4d-ceb3184706cd None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:06.752046 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f0b92f-8af2-40c4-98a0-d60d02396a87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:06.752628 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f0b92f-8af2-40c4-98a0-d60d02396a87 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:06.752915 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f0b92f-8af2-40c4-98a0-d60d02396a87 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:06.753193 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f0b92f-8af2-40c4-98a0-d60d02396a87 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:06.813553 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb22f7d-e6e7-4246-ae02-3d68e8f2cf6b None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:06.851730 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30773b7c-d100-48d1-ae7f-c49ac520ad64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:06.852410 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30773b7c-d100-48d1-ae7f-c49ac520ad64 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:06.852700 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30773b7c-d100-48d1-ae7f-c49ac520ad64 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:06.852983 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30773b7c-d100-48d1-ae7f-c49ac520ad64 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:06.853739 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30773b7c-d100-48d1-ae7f-c49ac520ad64 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:06.854081 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30773b7c-d100-48d1-ae7f-c49ac520ad64 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:06.855593 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30773b7c-d100-48d1-ae7f-c49ac520ad64 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:06.872002 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 126/262] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:10:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:06.877276 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30773b7c-d100-48d1-ae7f-c49ac520ad64 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:06.883851 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f0b92f-8af2-40c4-98a0-d60d02396a87 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:06.888838 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f0b92f-8af2-40c4-98a0-d60d02396a87 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:06.954846 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:07.040701 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30773b7c-d100-48d1-ae7f-c49ac520ad64 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:07.077274 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecc4c7f-4d19-4464-a066-900ac047c974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:07.078552 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecc4c7f-4d19-4464-a066-900ac047c974 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:07.079037 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecc4c7f-4d19-4464-a066-900ac047c974 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:07.079512 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecc4c7f-4d19-4464-a066-900ac047c974 None admin] PATH_INFO: `/v3/users/6e59e9f7740a4dc28ef53b32621e82df` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:07.080977 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecc4c7f-4d19-4464-a066-900ac047c974 None admin] RBAC: Authorizing `identity:get_user(user_id=6e59e9f7740a4dc28ef53b32621e82df)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:07.083258 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecc4c7f-4d19-4464-a066-900ac047c974 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:07.085343 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 127/263] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:10:06 2022] GET /identity/v3/users/6e59e9f7740a4dc28ef53b32621e82df => generated 283 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:07.090389 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:07.091230 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecc4c7f-4d19-4464-a066-900ac047c974 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:07.097041 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 137/264] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:10:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 621 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:10:07.118907 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f0b92f-8af2-40c4-98a0-d60d02396a87 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:07.216183 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:07.220621 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecc4c7f-4d19-4464-a066-900ac047c974 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:07.258997 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a9de97-e6f4-43f3-8e14-4d08248f2b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:07.259899 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a9de97-e6f4-43f3-8e14-4d08248f2b55 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:07.260359 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a9de97-e6f4-43f3-8e14-4d08248f2b55 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:07.260777 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a9de97-e6f4-43f3-8e14-4d08248f2b55 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:07.336689 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a9de97-e6f4-43f3-8e14-4d08248f2b55 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:07.338447 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a9de97-e6f4-43f3-8e14-4d08248f2b55 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:07.377640 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f0b92f-8af2-40c4-98a0-d60d02396a87 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:07.442767 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:07.447579 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 128/265] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:10:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:10:07.452645 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a9de97-e6f4-43f3-8e14-4d08248f2b55 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:07.459433 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb543c3-52f6-49c3-bba1-b047dfb98f3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:07.460508 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb543c3-52f6-49c3-bba1-b047dfb98f3e None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:07.464174 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb543c3-52f6-49c3-bba1-b047dfb98f3e None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:07.464611 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb543c3-52f6-49c3-bba1-b047dfb98f3e None admin] PATH_INFO: `/v3/projects/3c9e67fcd5b14da2a3775ef2b2cad6b2` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:07.466092 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb543c3-52f6-49c3-bba1-b047dfb98f3e None admin] RBAC: Authorizing `identity:get_project(project_id=3c9e67fcd5b14da2a3775ef2b2cad6b2)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:07.467699 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb543c3-52f6-49c3-bba1-b047dfb98f3e None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:07.478213 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 138/266] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:10:07 2022] GET /identity/v3/projects/3c9e67fcd5b14da2a3775ef2b2cad6b2 => generated 302 bytes in 364 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:10:07.491993 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb543c3-52f6-49c3-bba1-b047dfb98f3e None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:07.583879 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a9de97-e6f4-43f3-8e14-4d08248f2b55 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:07.637030 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd214a21-c66e-4135-8a3c-f0bcc3d07657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:07.637923 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd214a21-c66e-4135-8a3c-f0bcc3d07657 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:07.638101 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:07.638199 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd214a21-c66e-4135-8a3c-f0bcc3d07657 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:07.638461 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd214a21-c66e-4135-8a3c-f0bcc3d07657 None admin] PATH_INFO: `/v3/projects/3c9e67fcd5b14da2a3775ef2b2cad6b2` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:07.646740 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd214a21-c66e-4135-8a3c-f0bcc3d07657 None admin] RBAC: Authorizing `identity:get_project(project_id=3c9e67fcd5b14da2a3775ef2b2cad6b2)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:07.648207 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd214a21-c66e-4135-8a3c-f0bcc3d07657 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:07.652539 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 129/267] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:10:07 2022] GET /identity/v3/projects/3c9e67fcd5b14da2a3775ef2b2cad6b2 => generated 302 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:10:07.655598 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd214a21-c66e-4135-8a3c-f0bcc3d07657 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:07.701736 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb543c3-52f6-49c3-bba1-b047dfb98f3e None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:07.778454 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7ee2a5-3292-431c-b01a-5222e4a75e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:07.779124 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7ee2a5-3292-431c-b01a-5222e4a75e68 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:07.779405 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7ee2a5-3292-431c-b01a-5222e4a75e68 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:07.779681 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7ee2a5-3292-431c-b01a-5222e4a75e68 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:07.780527 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7ee2a5-3292-431c-b01a-5222e4a75e68 None admin] RBAC: Adding query filter params (role.id=4946cf0ff7164307b36efeee7c787146, scope.project.id=3c9e67fcd5b14da2a3775ef2b2cad6b2, user.id=6e59e9f7740a4dc28ef53b32621e82df) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:07.780846 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7ee2a5-3292-431c-b01a-5222e4a75e68 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:07.782354 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7ee2a5-3292-431c-b01a-5222e4a75e68 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:07.790201 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:07.794715 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 139/268] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:10:07 2022] GET /identity/v3/role_assignments?user.id=6e59e9f7740a4dc28ef53b32621e82df&scope.project.id=3c9e67fcd5b14da2a3775ef2b2cad6b2&role.id=4946cf0ff7164307b36efeee7c787146 => generated 608 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:10:07.799810 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237edfd3-2ff1-4c5c-b37a-630b50ec58a6 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:07.800082 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237edfd3-2ff1-4c5c-b37a-630b50ec58a6 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:07.800361 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237edfd3-2ff1-4c5c-b37a-630b50ec58a6 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:07.805986 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 140/269] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:10:07 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:10:07.821562 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19576732-d280-4591-b871-14b0c158e507 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:07.821914 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19576732-d280-4591-b871-14b0c158e507 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:07.822157 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19576732-d280-4591-b871-14b0c158e507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:07.827566 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd214a21-c66e-4135-8a3c-f0bcc3d07657 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:07.862358 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-19576732-d280-4591-b871-14b0c158e507 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:10:07.864136 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-19576732-d280-4591-b871-14b0c158e507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:10:07.868461 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad643da-9216-49f1-9856-807735a24fcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:07.869137 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad643da-9216-49f1-9856-807735a24fcf None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:07.869418 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad643da-9216-49f1-9856-807735a24fcf None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:07.869689 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad643da-9216-49f1-9856-807735a24fcf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:07.870574 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad643da-9216-49f1-9856-807735a24fcf None admin] RBAC: Adding query filter params (role.id=23d9dd31632340bdba9cf3093c095dbb, scope.project.id=3c9e67fcd5b14da2a3775ef2b2cad6b2, user.id=6e59e9f7740a4dc28ef53b32621e82df) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:07.870889 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad643da-9216-49f1-9856-807735a24fcf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:07.872401 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad643da-9216-49f1-9856-807735a24fcf None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:07.881017 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 130/270] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:10:07 2022] GET /identity/v3/role_assignments?user.id=6e59e9f7740a4dc28ef53b32621e82df&scope.project.id=3c9e67fcd5b14da2a3775ef2b2cad6b2&role.id=23d9dd31632340bdba9cf3093c095dbb => generated 608 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:10:08.003722 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 141/271] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:10:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:10:08.043489 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad643da-9216-49f1-9856-807735a24fcf None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:08.106138 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662451b1-4e68-473d-b11c-01df88d2f28f None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:08.106490 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662451b1-4e68-473d-b11c-01df88d2f28f None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:08.106738 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662451b1-4e68-473d-b11c-01df88d2f28f None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:08.108355 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 142/272] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:10:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:10:08.124729 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b57126f-d1c9-4371-a383-3be01c19d03e None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:08.125046 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b57126f-d1c9-4371-a383-3be01c19d03e None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:08.125290 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b57126f-d1c9-4371-a383-3be01c19d03e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:08.152838 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-6b57126f-d1c9-4371-a383-3be01c19d03e None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:10:08.154469 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-6b57126f-d1c9-4371-a383-3be01c19d03e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:10:08.217340 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad643da-9216-49f1-9856-807735a24fcf None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:08.228186 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:08.241556 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 143/273] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:10:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:10:08.257249 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:08.262621 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1295a70-339e-40f9-9a9f-4150caa4b9ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:08.263438 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1295a70-339e-40f9-9a9f-4150caa4b9ce None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:08.263723 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1295a70-339e-40f9-9a9f-4150caa4b9ce None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:08.264002 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1295a70-339e-40f9-9a9f-4150caa4b9ce None admin] PATH_INFO: `/v3/roles/member` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:08.275110 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b57126f-d1c9-4371-a383-3be01c19d03e None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:08.276582 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1295a70-339e-40f9-9a9f-4150caa4b9ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:10:08.287261 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-f1295a70-339e-40f9-9a9f-4150caa4b9ce None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:10:08.287261 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:08.287261 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:08.287261 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:08.287261 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:08.287261 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:08.287261 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:08.287261 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:08.287261 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:08.287261 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:08.287261 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:08.287261 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:08.287261 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:10:08.287261 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:10:08.287261 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:10:08.287261 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:10:08.288381 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:10:08.288381 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:10:08.288381 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:10:08.288381 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:10:08.288381 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:08.288381 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:08.288381 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:08.288381 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:08.288381 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:08.288381 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:08.288381 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:08.288381 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:10:08.288381 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:08.288381 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:08.288381 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:08.288381 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:08.289380 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:08.289380 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:08.289380 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:08.289380 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:08.289380 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:10:08.289380 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:10:08.289380 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:10:08.289380 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:10:08.289380 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:10:08.289380 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:10:08.289380 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:10:08.289380 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:10:08.292536 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 131/274] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:10:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:10:08.298142 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1295a70-339e-40f9-9a9f-4150caa4b9ce None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:08.432983 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b57126f-d1c9-4371-a383-3be01c19d03e None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:08.442253 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1295a70-339e-40f9-9a9f-4150caa4b9ce None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:08.474694 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c084b391-0b50-4b10-a8a0-f8a621273591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:08.475784 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c084b391-0b50-4b10-a8a0-f8a621273591 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:08.476205 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c084b391-0b50-4b10-a8a0-f8a621273591 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:08.476619 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c084b391-0b50-4b10-a8a0-f8a621273591 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:08.477954 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c084b391-0b50-4b10-a8a0-f8a621273591 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:08.479564 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c084b391-0b50-4b10-a8a0-f8a621273591 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:08.481322 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 144/275] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:10:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:10:08.489866 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d955f0-bf65-4206-86ad-4a7de6dac56d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:08.489866 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d955f0-bf65-4206-86ad-4a7de6dac56d None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:08.489866 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d955f0-bf65-4206-86ad-4a7de6dac56d None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:08.489866 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d955f0-bf65-4206-86ad-4a7de6dac56d None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:08.490287 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d955f0-bf65-4206-86ad-4a7de6dac56d None admin] RBAC: Adding query filter params (name=member) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:08.490287 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d955f0-bf65-4206-86ad-4a7de6dac56d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:08.490287 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d955f0-bf65-4206-86ad-4a7de6dac56d None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:08.491839 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c084b391-0b50-4b10-a8a0-f8a621273591 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:08.500685 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 132/276] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:10:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:08.507237 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d955f0-bf65-4206-86ad-4a7de6dac56d None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:08.560332 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:08.565597 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:08.605166 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c084b391-0b50-4b10-a8a0-f8a621273591 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:08.612726 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d955f0-bf65-4206-86ad-4a7de6dac56d None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:08.642443 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256d7246-38fb-4d90-8728-eb4121257fb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:08.643454 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256d7246-38fb-4d90-8728-eb4121257fb1 None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:08.643901 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256d7246-38fb-4d90-8728-eb4121257fb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:08.644396 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256d7246-38fb-4d90-8728-eb4121257fb1 None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:08.645595 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256d7246-38fb-4d90-8728-eb4121257fb1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:08.647277 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256d7246-38fb-4d90-8728-eb4121257fb1 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:08.648925 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8ee721-9bd8-44d3-b5ca-50d7fdec1058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:08.650002 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8ee721-9bd8-44d3-b5ca-50d7fdec1058 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:08.650429 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8ee721-9bd8-44d3-b5ca-50d7fdec1058 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:08.650847 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8ee721-9bd8-44d3-b5ca-50d7fdec1058 None admin] PATH_INFO: `/v3/users/16bc56af20bf4ae0bc00157276596735` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:08.666428 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8ee721-9bd8-44d3-b5ca-50d7fdec1058 None admin] RBAC: Authorizing `identity:get_user(user_id=16bc56af20bf4ae0bc00157276596735)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:08.668165 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8ee721-9bd8-44d3-b5ca-50d7fdec1058 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:08.670157 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 133/277] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:10:08 2022] GET /identity/v3/users/16bc56af20bf4ae0bc00157276596735 => generated 293 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:10:08.672526 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 145/278] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 19 14:10:08 2022] POST /identity/v3/users => generated 293 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:10:08.679683 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8ee721-9bd8-44d3-b5ca-50d7fdec1058 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:08.798491 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8ee721-9bd8-44d3-b5ca-50d7fdec1058 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:08.815655 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:08.841113 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90eafce6-a502-4271-b050-2e4d1af134b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:08.842132 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eafce6-a502-4271-b050-2e4d1af134b2 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:08.842599 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eafce6-a502-4271-b050-2e4d1af134b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:08.843079 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eafce6-a502-4271-b050-2e4d1af134b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:08.844109 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90eafce6-a502-4271-b050-2e4d1af134b2 None admin] RBAC: Adding query filter params (role.id=7d0e162805b54c019ce6c954cbef3818, user.id=16bc56af20bf4ae0bc00157276596735, scope.system=all) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:08.844608 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90eafce6-a502-4271-b050-2e4d1af134b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:08.846293 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90eafce6-a502-4271-b050-2e4d1af134b2 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:08.858006 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 134/279] 10.222.0.57 () {56 vars in 1308 bytes} [Thu May 19 14:10:08 2022] GET /identity/v3/role_assignments?user.id=16bc56af20bf4ae0bc00157276596735&scope.system=all&role.id=7d0e162805b54c019ce6c954cbef3818 => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:09.438620 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671d3586-5469-49b6-8b35-fd92b8d3d8b5 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:09.439627 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671d3586-5469-49b6-8b35-fd92b8d3d8b5 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:09.440117 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671d3586-5469-49b6-8b35-fd92b8d3d8b5 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:09.442530 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 146/280] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:10:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:10:09.448178 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1573f4e6-ebb2-4cf6-b7a8-f1bb57eab52f None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:09.449063 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1573f4e6-ebb2-4cf6-b7a8-f1bb57eab52f None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:09.449542 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1573f4e6-ebb2-4cf6-b7a8-f1bb57eab52f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:09.468015 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-1573f4e6-ebb2-4cf6-b7a8-f1bb57eab52f None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:10:09.469528 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-1573f4e6-ebb2-4cf6-b7a8-f1bb57eab52f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:10:09.565862 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 135/281] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:10:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:10:09.590425 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671d3586-5469-49b6-8b35-fd92b8d3d8b5 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:09.609219 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ba9a37-75c2-448f-9990-f6cf9b1319c1 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:09.609826 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ba9a37-75c2-448f-9990-f6cf9b1319c1 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:09.610342 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ba9a37-75c2-448f-9990-f6cf9b1319c1 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:09.611932 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 136/282] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:10:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:10:09.620406 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e529d69d-606f-4479-8d82-d40f7d429342 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:09.621028 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e529d69d-606f-4479-8d82-d40f7d429342 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:09.621504 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e529d69d-606f-4479-8d82-d40f7d429342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:09.643572 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-e529d69d-606f-4479-8d82-d40f7d429342 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:10:09.646228 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-e529d69d-606f-4479-8d82-d40f7d429342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:10:09.772240 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 137/283] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:10:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:10:09.776175 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671d3586-5469-49b6-8b35-fd92b8d3d8b5 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:09.794857 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e529d69d-606f-4479-8d82-d40f7d429342 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:09.818240 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:09.851850 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc8affd-11ea-49ce-896d-0103a598908c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:09.853052 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc8affd-11ea-49ce-896d-0103a598908c None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:09.853536 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc8affd-11ea-49ce-896d-0103a598908c None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:09.854035 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc8affd-11ea-49ce-896d-0103a598908c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:09.856242 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:09.878372 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc8affd-11ea-49ce-896d-0103a598908c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:10:09.888125 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-7bc8affd-11ea-49ce-896d-0103a598908c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:10:09.888125 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:09.888125 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:09.888125 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:09.888125 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:09.888125 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:09.888125 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:09.888125 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:09.888125 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:09.888125 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:09.888125 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:09.888125 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:09.888125 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:10:09.888125 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:10:09.888125 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:10:09.888125 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:10:09.889051 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:10:09.889051 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:10:09.889051 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:10:09.889051 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:10:09.889051 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:09.889051 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:09.889051 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:09.889051 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:09.889051 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:09.889051 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:09.889051 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:09.889051 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:10:09.889051 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:09.889051 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:09.889051 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:09.889051 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:09.889868 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:09.889868 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:09.889868 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:09.889868 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:09.889868 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:10:09.889868 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:10:09.889868 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:10:09.889868 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:10:09.889868 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:10:09.889868 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:10:09.889868 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:10:09.889868 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:10:09.897637 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 147/284] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:10:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:10:09.906944 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc8affd-11ea-49ce-896d-0103a598908c None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:10.011840 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e529d69d-606f-4479-8d82-d40f7d429342 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:10.043891 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc8affd-11ea-49ce-896d-0103a598908c None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:10.081121 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f0ae45-7983-4042-a564-d2f91d0a6912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:10.082484 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f0ae45-7983-4042-a564-d2f91d0a6912 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:10.083038 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f0ae45-7983-4042-a564-d2f91d0a6912 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:10.083515 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f0ae45-7983-4042-a564-d2f91d0a6912 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:10.084457 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f0ae45-7983-4042-a564-d2f91d0a6912 None admin] RBAC: Adding query filter params (name=member) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:10.084985 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f0ae45-7983-4042-a564-d2f91d0a6912 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:10.086601 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f0ae45-7983-4042-a564-d2f91d0a6912 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:10.096042 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 148/285] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:10:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:10:10.101781 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f0ae45-7983-4042-a564-d2f91d0a6912 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:10.158684 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365cab16-2d85-4086-ac4b-1ced464040d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:10.159525 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365cab16-2d85-4086-ac4b-1ced464040d0 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:10.159814 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365cab16-2d85-4086-ac4b-1ced464040d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:10.160196 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365cab16-2d85-4086-ac4b-1ced464040d0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:10.168076 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:10.206992 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365cab16-2d85-4086-ac4b-1ced464040d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:10:10.215638 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-365cab16-2d85-4086-ac4b-1ced464040d0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:10:10.215638 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:10.215638 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:10.215638 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:10.215638 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:10.215638 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:10.215638 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:10.215638 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:10.215638 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:10.215638 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:10.215638 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:10.215638 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:10.215638 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:10:10.215638 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:10:10.215638 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:10:10.215638 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:10:10.216550 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:10:10.216550 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:10:10.216550 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:10:10.216550 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:10:10.216550 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:10.216550 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:10.216550 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:10.216550 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:10.216550 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:10.216550 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:10.216550 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:10.216550 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:10:10.216550 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:10.216550 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:10.216550 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:10.216550 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:10.217350 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:10.217350 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:10.217350 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:10.217350 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:10.217350 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:10:10.217350 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:10:10.217350 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:10:10.217350 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:10:10.217350 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:10:10.217350 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:10:10.217350 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:10:10.217350 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:10:10.226158 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 138/286] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:10:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 432 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:10:10.227201 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365cab16-2d85-4086-ac4b-1ced464040d0 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:10.234754 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f0ae45-7983-4042-a564-d2f91d0a6912 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:10.275267 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c20f368-ec67-4796-b6b6-c9cb980d8adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:10.276744 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c20f368-ec67-4796-b6b6-c9cb980d8adf None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:10.277233 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c20f368-ec67-4796-b6b6-c9cb980d8adf None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:10.277707 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c20f368-ec67-4796-b6b6-c9cb980d8adf None admin] PATH_INFO: `/v3/users/349808e5d90e41e7961267c3ab752d9c` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:10.279468 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c20f368-ec67-4796-b6b6-c9cb980d8adf None admin] RBAC: Authorizing `identity:get_user(user_id=349808e5d90e41e7961267c3ab752d9c)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:10.281636 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c20f368-ec67-4796-b6b6-c9cb980d8adf None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:10.284325 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 149/287] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:10:10 2022] GET /identity/v3/users/349808e5d90e41e7961267c3ab752d9c => generated 297 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:10.289790 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c20f368-ec67-4796-b6b6-c9cb980d8adf None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:10.363216 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:10.372455 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365cab16-2d85-4086-ac4b-1ced464040d0 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:10.432595 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e41c8c5-bf1d-427f-951f-edb21e132161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:10.433271 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e41c8c5-bf1d-427f-951f-edb21e132161 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:10.433553 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e41c8c5-bf1d-427f-951f-edb21e132161 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:10.434100 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e41c8c5-bf1d-427f-951f-edb21e132161 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:10.434863 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e41c8c5-bf1d-427f-951f-edb21e132161 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:10.435173 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e41c8c5-bf1d-427f-951f-edb21e132161 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:10.436573 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e41c8c5-bf1d-427f-951f-edb21e132161 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:10.453469 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 139/288] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:10:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:10.460365 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e41c8c5-bf1d-427f-951f-edb21e132161 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:10.489831 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c20f368-ec67-4796-b6b6-c9cb980d8adf None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:10.501288 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:10.533160 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da6ac90-a627-4935-948d-78dd38b99121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:10.534221 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da6ac90-a627-4935-948d-78dd38b99121 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:10.534732 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da6ac90-a627-4935-948d-78dd38b99121 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:10.535740 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da6ac90-a627-4935-948d-78dd38b99121 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:10.613656 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da6ac90-a627-4935-948d-78dd38b99121 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:10.615852 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da6ac90-a627-4935-948d-78dd38b99121 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:10.664981 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e41c8c5-bf1d-427f-951f-edb21e132161 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:10.729588 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 150/289] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:10:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:10:10.740085 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da6ac90-a627-4935-948d-78dd38b99121 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:10.749394 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:10.756154 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5783fef-521c-4774-99a6-c5f014b6327c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:10.757884 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5783fef-521c-4774-99a6-c5f014b6327c None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:10.758173 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5783fef-521c-4774-99a6-c5f014b6327c None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:10.758450 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5783fef-521c-4774-99a6-c5f014b6327c None admin] PATH_INFO: `/v3/users/8617ec97033d40e78f04e210afc8e531` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:10.760766 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5783fef-521c-4774-99a6-c5f014b6327c None admin] RBAC: Authorizing `identity:get_user(user_id=8617ec97033d40e78f04e210afc8e531)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:10.762254 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5783fef-521c-4774-99a6-c5f014b6327c None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:10.764651 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 140/290] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:10:10 2022] GET /identity/v3/users/8617ec97033d40e78f04e210afc8e531 => generated 297 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:10.769212 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5783fef-521c-4774-99a6-c5f014b6327c None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:10.824131 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:10.919970 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da6ac90-a627-4935-948d-78dd38b99121 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:10.965161 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5783fef-521c-4774-99a6-c5f014b6327c None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:10.986113 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebc33d3-9db9-4294-ad92-c29883cad6f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:10.986989 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebc33d3-9db9-4294-ad92-c29883cad6f6 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:10.987268 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebc33d3-9db9-4294-ad92-c29883cad6f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:10.987543 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebc33d3-9db9-4294-ad92-c29883cad6f6 None admin] PATH_INFO: `/v3/projects/3c9e67fcd5b14da2a3775ef2b2cad6b2` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:10.989146 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebc33d3-9db9-4294-ad92-c29883cad6f6 None admin] RBAC: Authorizing `identity:get_project(project_id=3c9e67fcd5b14da2a3775ef2b2cad6b2)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:10.990893 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebc33d3-9db9-4294-ad92-c29883cad6f6 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:10.993013 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 151/291] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:10:10 2022] GET /identity/v3/projects/3c9e67fcd5b14da2a3775ef2b2cad6b2 => generated 302 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:10.998452 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebc33d3-9db9-4294-ad92-c29883cad6f6 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:11.038363 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6551934-26b1-40f5-9863-219fc7dcc4c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:11.039244 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6551934-26b1-40f5-9863-219fc7dcc4c8 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:11.039527 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6551934-26b1-40f5-9863-219fc7dcc4c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:11.040452 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6551934-26b1-40f5-9863-219fc7dcc4c8 None admin] PATH_INFO: `/v3/projects/3c9e67fcd5b14da2a3775ef2b2cad6b2` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:11.042680 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6551934-26b1-40f5-9863-219fc7dcc4c8 None admin] RBAC: Authorizing `identity:get_project(project_id=3c9e67fcd5b14da2a3775ef2b2cad6b2)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:11.045318 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6551934-26b1-40f5-9863-219fc7dcc4c8 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:11.047122 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 141/292] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:10:10 2022] GET /identity/v3/projects/3c9e67fcd5b14da2a3775ef2b2cad6b2 => generated 302 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:10:11.053298 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6551934-26b1-40f5-9863-219fc7dcc4c8 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:11.121877 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:11.153095 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebc33d3-9db9-4294-ad92-c29883cad6f6 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:11.196273 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63a4fa7-d1f1-4fa8-9afc-3e6c00328d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:11.196851 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63a4fa7-d1f1-4fa8-9afc-3e6c00328d3d None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:11.197136 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63a4fa7-d1f1-4fa8-9afc-3e6c00328d3d None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:11.197410 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63a4fa7-d1f1-4fa8-9afc-3e6c00328d3d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:11.198457 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63a4fa7-d1f1-4fa8-9afc-3e6c00328d3d None admin] RBAC: Adding query filter params (role.id=7d0e162805b54c019ce6c954cbef3818, scope.project.id=3c9e67fcd5b14da2a3775ef2b2cad6b2, user.id=349808e5d90e41e7961267c3ab752d9c) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:11.209880 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63a4fa7-d1f1-4fa8-9afc-3e6c00328d3d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:11.209880 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63a4fa7-d1f1-4fa8-9afc-3e6c00328d3d None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:11.216094 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 152/293] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:10:10 2022] GET /identity/v3/role_assignments?user.id=349808e5d90e41e7961267c3ab752d9c&scope.project.id=3c9e67fcd5b14da2a3775ef2b2cad6b2&role.id=7d0e162805b54c019ce6c954cbef3818 => generated 608 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:11.302681 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:11.311394 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6551934-26b1-40f5-9863-219fc7dcc4c8 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:11.362382 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badb67cd-a13f-4f0f-8a8f-79fca9f8af73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:11.363914 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badb67cd-a13f-4f0f-8a8f-79fca9f8af73 None admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:11.364200 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badb67cd-a13f-4f0f-8a8f-79fca9f8af73 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:11.364479 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badb67cd-a13f-4f0f-8a8f-79fca9f8af73 None admin] PATH_INFO: `/v3/projects/3c9e67fcd5b14da2a3775ef2b2cad6b2/users/8617ec97033d40e78f04e210afc8e531/roles/b0ae35f3e09648d0bbac778d23fa3463` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:11.371538 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badb67cd-a13f-4f0f-8a8f-79fca9f8af73 None admin] RBAC: Authorizing `identity:create_grant(project_id=3c9e67fcd5b14da2a3775ef2b2cad6b2, user_id=8617ec97033d40e78f04e210afc8e531, role_id=b0ae35f3e09648d0bbac778d23fa3463)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:11.373022 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badb67cd-a13f-4f0f-8a8f-79fca9f8af73 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:11.403184 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 142/294] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 14:10:11 2022] PUT /identity/v3/projects/3c9e67fcd5b14da2a3775ef2b2cad6b2/users/8617ec97033d40e78f04e210afc8e531/roles/b0ae35f3e09648d0bbac778d23fa3463 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:10:11.449546 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b861f461-746a-40d3-99f8-89c9368d280f None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:11.449546 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b861f461-746a-40d3-99f8-89c9368d280f None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:11.449546 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b861f461-746a-40d3-99f8-89c9368d280f None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:11.450410 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 153/295] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:10:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:10:11.458598 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a4dfb5-a165-4d6b-878d-dce250c15a32 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:11.458598 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a4dfb5-a165-4d6b-878d-dce250c15a32 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:11.458598 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a4dfb5-a165-4d6b-878d-dce250c15a32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:11.476663 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-b5a4dfb5-a165-4d6b-878d-dce250c15a32 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:10:11.478312 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-b5a4dfb5-a165-4d6b-878d-dce250c15a32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:10:11.535735 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72136499-0219-4f21-9ba3-98ff84e16817 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:11.536718 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72136499-0219-4f21-9ba3-98ff84e16817 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:11.537199 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72136499-0219-4f21-9ba3-98ff84e16817 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:11.539225 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 154/296] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:10:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:10:11.545821 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dbae3f-8be1-4dd0-9981-0a663e941799 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:11.546537 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dbae3f-8be1-4dd0-9981-0a663e941799 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:11.547011 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dbae3f-8be1-4dd0-9981-0a663e941799 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:11.565720 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 143/297] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:10:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:10:11.571834 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-b0dbae3f-8be1-4dd0-9981-0a663e941799 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:10:11.573200 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-b0dbae3f-8be1-4dd0-9981-0a663e941799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:10:11.593390 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a4dfb5-a165-4d6b-878d-dce250c15a32 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:11.680619 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 155/298] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:10:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:10:11.707418 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dbae3f-8be1-4dd0-9981-0a663e941799 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:11.712642 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:11.731048 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a4dfb5-a165-4d6b-878d-dce250c15a32 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:11.768695 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4c197e-fbf1-46a3-ab47-c0adb5dc337a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:11.769497 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4c197e-fbf1-46a3-ab47-c0adb5dc337a None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:11.769921 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4c197e-fbf1-46a3-ab47-c0adb5dc337a None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:11.770422 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4c197e-fbf1-46a3-ab47-c0adb5dc337a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:11.783934 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4c197e-fbf1-46a3-ab47-c0adb5dc337a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:10:11.786736 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:11.794267 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-9c4c197e-fbf1-46a3-ab47-c0adb5dc337a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:10:11.794267 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:11.794267 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:11.794267 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:11.794267 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:11.794267 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:11.794267 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:11.794267 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:11.794267 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:11.794267 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:11.794267 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:11.794267 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:11.794267 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:10:11.794267 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:10:11.794267 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:10:11.794267 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:10:11.795141 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:10:11.795141 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:10:11.795141 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:10:11.795141 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:10:11.795141 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:11.795141 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:11.795141 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:11.795141 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:11.795141 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:11.795141 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:11.795141 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:11.795141 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:10:11.795141 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:11.795141 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:11.795141 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:11.795141 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:11.795964 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:11.795964 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:11.795964 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:11.795964 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:11.795964 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:10:11.795964 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:10:11.795964 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:10:11.795964 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:10:11.795964 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:10:11.795964 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:10:11.795964 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:10:11.795964 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:10:11.795964 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 144/299] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:10:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:10:11.802010 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1143138-bb7e-4de0-9ccf-c7dfc629e1bf None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:11.802010 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1143138-bb7e-4de0-9ccf-c7dfc629e1bf None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:11.802010 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1143138-bb7e-4de0-9ccf-c7dfc629e1bf None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:11.803391 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 145/300] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:10:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:10:11.804379 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1143138-bb7e-4de0-9ccf-c7dfc629e1bf None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:11.825867 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dbae3f-8be1-4dd0-9981-0a663e941799 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:11.867556 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea218961-9472-439b-8132-2787079f226e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:11.868416 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea218961-9472-439b-8132-2787079f226e None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:11.868661 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea218961-9472-439b-8132-2787079f226e None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:11.868937 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea218961-9472-439b-8132-2787079f226e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:11.880029 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea218961-9472-439b-8132-2787079f226e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:10:11.887994 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-ea218961-9472-439b-8132-2787079f226e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:10:11.887994 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:11.887994 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:11.887994 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:11.887994 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:11.887994 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:11.887994 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:11.887994 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:11.887994 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:11.887994 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:11.887994 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:11.887994 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:11.887994 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:10:11.887994 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:10:11.887994 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:10:11.887994 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:10:11.888930 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:10:11.888930 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:10:11.888930 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:10:11.888930 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:10:11.888930 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:11.888930 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:11.888930 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:11.888930 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:11.888930 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:11.888930 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:11.888930 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:11.888930 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:10:11.888930 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:11.888930 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:11.888930 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:11.888930 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:11.889717 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:11.889717 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:11.889717 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:11.889717 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:11.889717 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:10:11.889717 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:10:11.889717 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:10:11.889717 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:10:11.889717 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:10:11.889717 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:10:11.889717 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:10:11.889717 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:10:11.889717 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 156/301] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:10:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:10:11.890824 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3144869e-8454-4a6a-b392-e9ae205bb8de None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:11.890824 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3144869e-8454-4a6a-b392-e9ae205bb8de None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:11.891462 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3144869e-8454-4a6a-b392-e9ae205bb8de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:11.907892 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-3144869e-8454-4a6a-b392-e9ae205bb8de None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:10:11.909083 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-3144869e-8454-4a6a-b392-e9ae205bb8de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:10:11.919678 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1143138-bb7e-4de0-9ccf-c7dfc629e1bf None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:11.957140 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1d5317-e5f0-4ea6-92e2-9bbc5e138cb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:11.958168 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1d5317-e5f0-4ea6-92e2-9bbc5e138cb2 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:11.958639 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1d5317-e5f0-4ea6-92e2-9bbc5e138cb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:11.959046 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1d5317-e5f0-4ea6-92e2-9bbc5e138cb2 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:11.959914 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1d5317-e5f0-4ea6-92e2-9bbc5e138cb2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:11.960371 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1d5317-e5f0-4ea6-92e2-9bbc5e138cb2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:11.961754 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1d5317-e5f0-4ea6-92e2-9bbc5e138cb2 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:11.976511 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 146/302] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:10:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:11.976511 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1d5317-e5f0-4ea6-92e2-9bbc5e138cb2 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:11.985156 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:11.993861 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 157/303] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:10:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:10:11.993861 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3144869e-8454-4a6a-b392-e9ae205bb8de None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:12.064549 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:12.083712 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1d5317-e5f0-4ea6-92e2-9bbc5e138cb2 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:12.114716 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3144869e-8454-4a6a-b392-e9ae205bb8de None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:12.122079 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551bdeb1-6748-42a3-b634-1b16992704d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:12.122079 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551bdeb1-6748-42a3-b634-1b16992704d1 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:12.122079 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551bdeb1-6748-42a3-b634-1b16992704d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:12.122390 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551bdeb1-6748-42a3-b634-1b16992704d1 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:12.123087 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551bdeb1-6748-42a3-b634-1b16992704d1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:12.123391 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551bdeb1-6748-42a3-b634-1b16992704d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:12.124845 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551bdeb1-6748-42a3-b634-1b16992704d1 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:12.134153 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 147/304] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:10:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:12.139566 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551bdeb1-6748-42a3-b634-1b16992704d1 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:12.153402 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0031621-48f4-45dc-8d93-4e5bf778d862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:12.154314 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0031621-48f4-45dc-8d93-4e5bf778d862 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:12.154603 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0031621-48f4-45dc-8d93-4e5bf778d862 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:12.154884 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0031621-48f4-45dc-8d93-4e5bf778d862 None admin] PATH_INFO: `/v3/users/5c2a73860fad4d30b031a49cf2784b32` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:12.169648 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0031621-48f4-45dc-8d93-4e5bf778d862 None admin] RBAC: Authorizing `identity:get_user(user_id=5c2a73860fad4d30b031a49cf2784b32)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:12.171139 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0031621-48f4-45dc-8d93-4e5bf778d862 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:12.177890 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 158/305] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:10:11 2022] GET /identity/v3/users/5c2a73860fad4d30b031a49cf2784b32 => generated 293 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:12.177890 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0031621-48f4-45dc-8d93-4e5bf778d862 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:12.217720 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:12.259030 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551bdeb1-6748-42a3-b634-1b16992704d1 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:12.286183 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0031621-48f4-45dc-8d93-4e5bf778d862 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:12.296526 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520a25aa-a158-40e0-b542-d2831827ea97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:12.297773 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520a25aa-a158-40e0-b542-d2831827ea97 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:12.298209 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520a25aa-a158-40e0-b542-d2831827ea97 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:12.298488 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520a25aa-a158-40e0-b542-d2831827ea97 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:12.299875 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520a25aa-a158-40e0-b542-d2831827ea97 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:12.302105 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520a25aa-a158-40e0-b542-d2831827ea97 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:12.302763 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:12.308880 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 148/306] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:10:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:12.308880 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520a25aa-a158-40e0-b542-d2831827ea97 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:12.321389 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c59b181-8990-42c9-a25d-731cfb0c3d4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:12.322299 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c59b181-8990-42c9-a25d-731cfb0c3d4c None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:12.322583 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c59b181-8990-42c9-a25d-731cfb0c3d4c None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:12.322860 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c59b181-8990-42c9-a25d-731cfb0c3d4c None admin] PATH_INFO: `/v3/users/16bc56af20bf4ae0bc00157276596735` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:12.324053 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c59b181-8990-42c9-a25d-731cfb0c3d4c None admin] RBAC: Authorizing `identity:get_user(user_id=16bc56af20bf4ae0bc00157276596735)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:12.325498 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c59b181-8990-42c9-a25d-731cfb0c3d4c None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:12.331532 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 159/307] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:10:12 2022] GET /identity/v3/users/16bc56af20bf4ae0bc00157276596735 => generated 293 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:12.331532 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c59b181-8990-42c9-a25d-731cfb0c3d4c None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:12.420166 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520a25aa-a158-40e0-b542-d2831827ea97 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:12.439205 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c59b181-8990-42c9-a25d-731cfb0c3d4c None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:12.457667 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b4f2a4-f917-4fc2-94d4-42e82ce1ee56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:12.458514 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b4f2a4-f917-4fc2-94d4-42e82ce1ee56 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:12.458810 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b4f2a4-f917-4fc2-94d4-42e82ce1ee56 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:12.459369 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b4f2a4-f917-4fc2-94d4-42e82ce1ee56 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:12.460204 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b4f2a4-f917-4fc2-94d4-42e82ce1ee56 None admin] RBAC: Adding query filter params (role.id=b0ae35f3e09648d0bbac778d23fa3463, user.id=5c2a73860fad4d30b031a49cf2784b32, scope.system=all) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:12.460526 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b4f2a4-f917-4fc2-94d4-42e82ce1ee56 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:12.462406 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b4f2a4-f917-4fc2-94d4-42e82ce1ee56 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:12.466076 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:12.474282 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 149/308] 10.222.0.57 () {56 vars in 1308 bytes} [Thu May 19 14:10:12 2022] GET /identity/v3/role_assignments?user.id=5c2a73860fad4d30b031a49cf2784b32&scope.system=all&role.id=b0ae35f3e09648d0bbac778d23fa3463 => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:12.475023 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b4f2a4-f917-4fc2-94d4-42e82ce1ee56 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:12.477115 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635b16b1-757b-4397-8952-22646357a8a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:12.478067 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635b16b1-757b-4397-8952-22646357a8a4 None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:12.478484 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635b16b1-757b-4397-8952-22646357a8a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:12.478938 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635b16b1-757b-4397-8952-22646357a8a4 None admin] PATH_INFO: `/v3/groups` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:12.479782 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635b16b1-757b-4397-8952-22646357a8a4 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:12.481474 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635b16b1-757b-4397-8952-22646357a8a4 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:12.502122 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 160/309] 10.222.0.57 () {60 vars in 1134 bytes} [Thu May 19 14:10:12 2022] POST /identity/v3/groups => generated 229 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:10:12.602568 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b4f2a4-f917-4fc2-94d4-42e82ce1ee56 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:12.645507 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b92db45-7560-498f-94a5-00ae6376d8b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:12.646522 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b92db45-7560-498f-94a5-00ae6376d8b1 None admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:12.646806 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b92db45-7560-498f-94a5-00ae6376d8b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:12.647084 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b92db45-7560-498f-94a5-00ae6376d8b1 None admin] PATH_INFO: `/v3/system/users/16bc56af20bf4ae0bc00157276596735/roles/7d0e162805b54c019ce6c954cbef3818` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:12.648743 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b92db45-7560-498f-94a5-00ae6376d8b1 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=16bc56af20bf4ae0bc00157276596735, role_id=7d0e162805b54c019ce6c954cbef3818)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:12.650198 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b92db45-7560-498f-94a5-00ae6376d8b1 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:12.663339 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 150/310] 10.222.0.57 () {58 vars in 1334 bytes} [Thu May 19 14:10:12 2022] PUT /identity/v3/system/users/16bc56af20bf4ae0bc00157276596735/roles/7d0e162805b54c019ce6c954cbef3818 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:10:13.835363 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57b00ae-02eb-4353-8474-d21778c74ec9 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:13.836370 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57b00ae-02eb-4353-8474-d21778c74ec9 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:13.836864 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57b00ae-02eb-4353-8474-d21778c74ec9 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:13.838527 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 161/311] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:10:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:10:13.843993 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e6307b-a413-446c-a3b7-704ed6746459 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:13.844851 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e6307b-a413-446c-a3b7-704ed6746459 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:13.845336 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e6307b-a413-446c-a3b7-704ed6746459 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:13.863970 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-92e6307b-a413-446c-a3b7-704ed6746459 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:10:13.865931 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-92e6307b-a413-446c-a3b7-704ed6746459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:10:13.927528 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:13.937644 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 151/312] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:10:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:10:13.961809 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57b00ae-02eb-4353-8474-d21778c74ec9 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:14.012781 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:14.072714 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57b00ae-02eb-4353-8474-d21778c74ec9 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:14.110965 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d87f311-1502-4095-b2c5-442ca351a3a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:14.111769 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d87f311-1502-4095-b2c5-442ca351a3a5 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:14.112056 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d87f311-1502-4095-b2c5-442ca351a3a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:14.112330 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d87f311-1502-4095-b2c5-442ca351a3a5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:14.130333 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d87f311-1502-4095-b2c5-442ca351a3a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:10:14.138220 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-5d87f311-1502-4095-b2c5-442ca351a3a5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:10:14.138220 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:14.138220 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:14.138220 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:14.138220 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:14.138220 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:14.138220 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:14.138220 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:14.138220 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:14.138220 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:14.138220 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:14.138220 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:14.138220 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:10:14.138220 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:10:14.138220 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:10:14.138220 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:10:14.139119 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:10:14.139119 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:10:14.139119 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:10:14.139119 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:10:14.139119 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:14.139119 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:14.139119 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:14.139119 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:14.139119 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:14.139119 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:14.139119 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:14.139119 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:10:14.139119 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:14.139119 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:14.139119 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:14.139119 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:14.140340 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:14.140340 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:14.140340 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:14.140340 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:14.140340 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:10:14.140340 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:10:14.140340 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:10:14.140340 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:10:14.140340 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:10:14.140340 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:10:14.140340 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:10:14.140340 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:10:14.140340 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 162/313] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:10:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:10:14.145407 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e6307b-a413-446c-a3b7-704ed6746459 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:14.252692 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e6307b-a413-446c-a3b7-704ed6746459 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:14.289567 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1442dd-fb3b-4352-b4a4-64306467a391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:14.290372 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1442dd-fb3b-4352-b4a4-64306467a391 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:14.290789 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1442dd-fb3b-4352-b4a4-64306467a391 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:14.291062 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1442dd-fb3b-4352-b4a4-64306467a391 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:14.291813 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1442dd-fb3b-4352-b4a4-64306467a391 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:14.292109 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1442dd-fb3b-4352-b4a4-64306467a391 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:14.293494 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1442dd-fb3b-4352-b4a4-64306467a391 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:14.303452 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 152/314] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:10:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:14.308612 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d87f311-1502-4095-b2c5-442ca351a3a5 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:14.418537 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d87f311-1502-4095-b2c5-442ca351a3a5 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:14.420957 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:14.455332 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5934c7c3-f7e0-4ec7-a1f8-f5ce7e06c248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:14.456210 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5934c7c3-f7e0-4ec7-a1f8-f5ce7e06c248 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:14.456493 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5934c7c3-f7e0-4ec7-a1f8-f5ce7e06c248 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:14.457289 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5934c7c3-f7e0-4ec7-a1f8-f5ce7e06c248 None admin] PATH_INFO: `/v3/users/8617ec97033d40e78f04e210afc8e531` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:14.458501 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5934c7c3-f7e0-4ec7-a1f8-f5ce7e06c248 None admin] RBAC: Authorizing `identity:get_user(user_id=8617ec97033d40e78f04e210afc8e531)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:14.459955 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5934c7c3-f7e0-4ec7-a1f8-f5ce7e06c248 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:14.461906 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 163/315] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:10:14 2022] GET /identity/v3/users/8617ec97033d40e78f04e210afc8e531 => generated 297 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:14.467305 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1442dd-fb3b-4352-b4a4-64306467a391 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:14.527876 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:14.575408 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1442dd-fb3b-4352-b4a4-64306467a391 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:14.612733 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cb4c78-a5ae-42bd-9816-9d98056480cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:14.613508 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cb4c78-a5ae-42bd-9816-9d98056480cb None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:14.613789 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cb4c78-a5ae-42bd-9816-9d98056480cb None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:14.614103 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cb4c78-a5ae-42bd-9816-9d98056480cb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:14.689618 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cb4c78-a5ae-42bd-9816-9d98056480cb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:14.691614 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cb4c78-a5ae-42bd-9816-9d98056480cb None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:14.752263 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:14.801242 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 153/316] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:10:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:10:14.801574 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf00c4d-56a0-46b3-941c-5fff13db4fc8 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:14.802041 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf00c4d-56a0-46b3-941c-5fff13db4fc8 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:14.802540 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf00c4d-56a0-46b3-941c-5fff13db4fc8 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:14.808481 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cb4c78-a5ae-42bd-9816-9d98056480cb None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:14.808829 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 164/317] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:10:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:10:14.810937 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766d438c-c62e-41f1-bfd6-f9caf0959587 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:14.811351 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766d438c-c62e-41f1-bfd6-f9caf0959587 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:14.811737 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766d438c-c62e-41f1-bfd6-f9caf0959587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:14.830413 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-766d438c-c62e-41f1-bfd6-f9caf0959587 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:10:14.831896 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-766d438c-c62e-41f1-bfd6-f9caf0959587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:10:14.920711 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 165/318] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:10:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:10:14.922150 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045476ff-904a-4f19-8d0a-ee92df69be9f None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:14.922446 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045476ff-904a-4f19-8d0a-ee92df69be9f None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:14.922734 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045476ff-904a-4f19-8d0a-ee92df69be9f None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:14.924025 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 166/319] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:10:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:10:14.929998 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c6818d-8ff9-4960-8229-8ce3f17a589d None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:14.930473 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c6818d-8ff9-4960-8229-8ce3f17a589d None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:14.930920 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c6818d-8ff9-4960-8229-8ce3f17a589d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:14.942844 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cb4c78-a5ae-42bd-9816-9d98056480cb None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:14.948493 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-16c6818d-8ff9-4960-8229-8ce3f17a589d None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:10:14.950025 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-16c6818d-8ff9-4960-8229-8ce3f17a589d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:10:14.984413 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57eba07-824d-4940-96d7-285d0b925332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:14.985269 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57eba07-824d-4940-96d7-285d0b925332 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:14.985616 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57eba07-824d-4940-96d7-285d0b925332 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:14.985831 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57eba07-824d-4940-96d7-285d0b925332 None admin] PATH_INFO: `/v3/projects/3c9e67fcd5b14da2a3775ef2b2cad6b2` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:14.987046 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57eba07-824d-4940-96d7-285d0b925332 None admin] RBAC: Authorizing `identity:get_project(project_id=3c9e67fcd5b14da2a3775ef2b2cad6b2)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:14.989168 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57eba07-824d-4940-96d7-285d0b925332 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:14.991178 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 154/320] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:10:14 2022] GET /identity/v3/projects/3c9e67fcd5b14da2a3775ef2b2cad6b2 => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:14.996324 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57eba07-824d-4940-96d7-285d0b925332 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:15.021857 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:15.023118 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 167/321] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:10:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:10:15.023807 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c6818d-8ff9-4960-8229-8ce3f17a589d None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:15.063443 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:15.102738 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57eba07-824d-4940-96d7-285d0b925332 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:15.133620 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c6818d-8ff9-4960-8229-8ce3f17a589d None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:15.140935 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32ce69a-a1c0-4d4b-8acd-3af4c4318246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:15.141729 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32ce69a-a1c0-4d4b-8acd-3af4c4318246 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:15.142034 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32ce69a-a1c0-4d4b-8acd-3af4c4318246 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:15.142302 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32ce69a-a1c0-4d4b-8acd-3af4c4318246 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:15.153404 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32ce69a-a1c0-4d4b-8acd-3af4c4318246 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:10:15.161952 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-c32ce69a-a1c0-4d4b-8acd-3af4c4318246 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:10:15.161952 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:15.161952 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:15.161952 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:15.161952 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:15.161952 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:15.161952 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:15.161952 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:15.161952 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:15.161952 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:15.161952 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:15.161952 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:15.161952 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:10:15.161952 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:10:15.161952 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:10:15.161952 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:10:15.162835 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:10:15.162835 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:10:15.162835 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:10:15.162835 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:10:15.162835 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:15.162835 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:15.162835 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:15.162835 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:15.162835 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:15.162835 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:15.162835 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:15.162835 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:10:15.162835 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:15.162835 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:15.162835 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:15.162835 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:15.168889 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:15.168889 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:15.168889 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:15.168889 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:15.168889 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:10:15.168889 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:10:15.168889 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:10:15.168889 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:10:15.168889 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:10:15.168889 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:10:15.168889 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:10:15.168889 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:10:15.168889 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 155/322] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:10:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:10:15.168889 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee57e7b-2eda-4d4f-9b17-aab3a751f9bf None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:15.169602 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee57e7b-2eda-4d4f-9b17-aab3a751f9bf None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:15.169602 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee57e7b-2eda-4d4f-9b17-aab3a751f9bf None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:15.169602 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 156/323] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:10:15 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:10:15.169602 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee57e7b-2eda-4d4f-9b17-aab3a751f9bf None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:15.174917 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a571ff1-d3b9-4eda-93b5-26ee40c1a292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:15.175683 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a571ff1-d3b9-4eda-93b5-26ee40c1a292 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:15.175971 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a571ff1-d3b9-4eda-93b5-26ee40c1a292 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:15.176249 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a571ff1-d3b9-4eda-93b5-26ee40c1a292 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:15.177096 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a571ff1-d3b9-4eda-93b5-26ee40c1a292 None admin] RBAC: Adding query filter params (role.id=b0ae35f3e09648d0bbac778d23fa3463, scope.project.id=3c9e67fcd5b14da2a3775ef2b2cad6b2, user.id=8617ec97033d40e78f04e210afc8e531) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:15.177405 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a571ff1-d3b9-4eda-93b5-26ee40c1a292 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:15.178897 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a571ff1-d3b9-4eda-93b5-26ee40c1a292 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:15.192497 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 168/324] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:10:15 2022] GET /identity/v3/role_assignments?user.id=8617ec97033d40e78f04e210afc8e531&scope.project.id=3c9e67fcd5b14da2a3775ef2b2cad6b2&role.id=b0ae35f3e09648d0bbac778d23fa3463 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:15.192497 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a571ff1-d3b9-4eda-93b5-26ee40c1a292 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:15.263882 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:15.284186 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee57e7b-2eda-4d4f-9b17-aab3a751f9bf None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:15.310347 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:15.318034 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a571ff1-d3b9-4eda-93b5-26ee40c1a292 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:15.324075 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c12e17c-946a-47a9-9992-595d85fcc9d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:15.324702 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c12e17c-946a-47a9-9992-595d85fcc9d8 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:15.324991 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c12e17c-946a-47a9-9992-595d85fcc9d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:15.325269 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c12e17c-946a-47a9-9992-595d85fcc9d8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:15.326685 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c12e17c-946a-47a9-9992-595d85fcc9d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:15.328372 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c12e17c-946a-47a9-9992-595d85fcc9d8 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:15.330311 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 157/325] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:10:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:15.331537 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444f3739-e677-4d0d-90b0-78b71daa43a9 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:15.332364 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444f3739-e677-4d0d-90b0-78b71daa43a9 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:15.332643 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444f3739-e677-4d0d-90b0-78b71daa43a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:15.352539 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-444f3739-e677-4d0d-90b0-78b71daa43a9 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:10:15.354041 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-444f3739-e677-4d0d-90b0-78b71daa43a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:10:15.361947 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca14a5f-4afe-4411-a335-4007d88fc92d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:15.362631 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca14a5f-4afe-4411-a335-4007d88fc92d None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:15.362911 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca14a5f-4afe-4411-a335-4007d88fc92d None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:15.363354 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca14a5f-4afe-4411-a335-4007d88fc92d None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:15.364122 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca14a5f-4afe-4411-a335-4007d88fc92d None admin] RBAC: Adding query filter params (name=reader) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:15.364432 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca14a5f-4afe-4411-a335-4007d88fc92d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:15.366493 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca14a5f-4afe-4411-a335-4007d88fc92d None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:15.376523 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 169/326] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:10:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:15.377351 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca14a5f-4afe-4411-a335-4007d88fc92d None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:15.443043 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 158/327] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:10:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:10:15.443984 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444f3739-e677-4d0d-90b0-78b71daa43a9 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:15.492063 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca14a5f-4afe-4411-a335-4007d88fc92d None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:15.525148 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfbbec0-00e6-444c-bf1a-ad86c94dc10a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:15.525325 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:15.525743 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfbbec0-00e6-444c-bf1a-ad86c94dc10a None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:15.526133 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfbbec0-00e6-444c-bf1a-ad86c94dc10a None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:15.526350 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfbbec0-00e6-444c-bf1a-ad86c94dc10a None admin] PATH_INFO: `/v3/groups` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:15.527170 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfbbec0-00e6-444c-bf1a-ad86c94dc10a None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:15.528735 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfbbec0-00e6-444c-bf1a-ad86c94dc10a None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:15.542248 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 170/328] 10.222.0.57 () {60 vars in 1134 bytes} [Thu May 19 14:10:15 2022] POST /identity/v3/groups => generated 226 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:10:15.543066 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfbbec0-00e6-444c-bf1a-ad86c94dc10a None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:15.554448 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444f3739-e677-4d0d-90b0-78b71daa43a9 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:15.564671 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:15.595904 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4b9f07-8e56-4363-a86d-2b9ebb1d88ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:15.597198 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4b9f07-8e56-4363-a86d-2b9ebb1d88ed None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:15.598713 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4b9f07-8e56-4363-a86d-2b9ebb1d88ed None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:15.599094 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4b9f07-8e56-4363-a86d-2b9ebb1d88ed None admin] PATH_INFO: `/v3/users/5c2a73860fad4d30b031a49cf2784b32` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:15.600556 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4b9f07-8e56-4363-a86d-2b9ebb1d88ed None admin] RBAC: Authorizing `identity:get_user(user_id=5c2a73860fad4d30b031a49cf2784b32)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:15.602083 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4b9f07-8e56-4363-a86d-2b9ebb1d88ed None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:15.608954 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 159/329] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:10:15 2022] GET /identity/v3/users/5c2a73860fad4d30b031a49cf2784b32 => generated 293 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:15.609880 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4b9f07-8e56-4363-a86d-2b9ebb1d88ed None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:15.662424 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfbbec0-00e6-444c-bf1a-ad86c94dc10a None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:15.699909 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ac2c45-bada-4ab4-9735-9cfb4f0c0aeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:15.700878 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ac2c45-bada-4ab4-9735-9cfb4f0c0aeb None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:15.701174 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ac2c45-bada-4ab4-9735-9cfb4f0c0aeb None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:15.701457 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ac2c45-bada-4ab4-9735-9cfb4f0c0aeb None admin] PATH_INFO: `/v3/roles/member` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:15.714349 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ac2c45-bada-4ab4-9735-9cfb4f0c0aeb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:10:15.723556 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-94ac2c45-bada-4ab4-9735-9cfb4f0c0aeb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:10:15.723556 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:15.723556 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:15.723556 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:15.723556 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:15.723556 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:15.723556 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:15.723556 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:15.723556 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:15.723556 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:15.723556 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:15.723556 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:15.723556 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:10:15.723556 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:10:15.723556 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:10:15.723556 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:10:15.724701 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:10:15.724701 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:10:15.724701 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:10:15.724701 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:10:15.724701 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:15.724701 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:15.724701 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:15.724701 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:15.724701 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:15.724701 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:15.724701 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:15.724701 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:10:15.724701 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:15.724701 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:15.724701 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:15.724701 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:15.725497 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:15.725497 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:15.725497 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:15.725497 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:15.725497 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:10:15.725497 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:10:15.725497 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:10:15.725497 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:10:15.725497 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:10:15.725497 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:10:15.725497 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:10:15.725497 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:10:15.725497 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 171/330] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:10:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:10:15.730250 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4b9f07-8e56-4363-a86d-2b9ebb1d88ed None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:15.730530 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ac2c45-bada-4ab4-9735-9cfb4f0c0aeb None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:15.770573 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf57943e-9912-4adc-8bdf-c1b27dfe132e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:15.771443 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf57943e-9912-4adc-8bdf-c1b27dfe132e None admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:15.772464 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf57943e-9912-4adc-8bdf-c1b27dfe132e None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:15.772464 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf57943e-9912-4adc-8bdf-c1b27dfe132e None admin] PATH_INFO: `/v3/system/users/5c2a73860fad4d30b031a49cf2784b32/roles/b0ae35f3e09648d0bbac778d23fa3463` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:15.773881 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf57943e-9912-4adc-8bdf-c1b27dfe132e None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=5c2a73860fad4d30b031a49cf2784b32, role_id=b0ae35f3e09648d0bbac778d23fa3463)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:15.774924 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf57943e-9912-4adc-8bdf-c1b27dfe132e None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:15.784523 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 160/331] 10.222.0.57 () {58 vars in 1334 bytes} [Thu May 19 14:10:15 2022] PUT /identity/v3/system/users/5c2a73860fad4d30b031a49cf2784b32/roles/b0ae35f3e09648d0bbac778d23fa3463 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:10:15.833027 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:15.861081 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ac2c45-bada-4ab4-9735-9cfb4f0c0aeb None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:15.902995 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7580fd04-1fb0-4ef1-a335-ce4816645d9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:15.903681 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7580fd04-1fb0-4ef1-a335-ce4816645d9f None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:15.903970 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7580fd04-1fb0-4ef1-a335-ce4816645d9f None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:15.904249 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7580fd04-1fb0-4ef1-a335-ce4816645d9f None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:15.905015 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7580fd04-1fb0-4ef1-a335-ce4816645d9f None admin] RBAC: Adding query filter params (name=member) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:15.905322 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7580fd04-1fb0-4ef1-a335-ce4816645d9f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:15.906795 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7580fd04-1fb0-4ef1-a335-ce4816645d9f None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:15.916283 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 172/332] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:10:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:15.922094 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf57943e-9912-4adc-8bdf-c1b27dfe132e None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:15.939759 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:16.040615 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf57943e-9912-4adc-8bdf-c1b27dfe132e None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:16.077741 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fe04ae-66e0-41dc-a054-4d91b7029f6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:16.078649 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fe04ae-66e0-41dc-a054-4d91b7029f6b None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:16.078933 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fe04ae-66e0-41dc-a054-4d91b7029f6b None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:16.079209 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fe04ae-66e0-41dc-a054-4d91b7029f6b None admin] PATH_INFO: `/v3/users/16bc56af20bf4ae0bc00157276596735` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:16.080424 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fe04ae-66e0-41dc-a054-4d91b7029f6b None admin] RBAC: Authorizing `identity:get_user(user_id=16bc56af20bf4ae0bc00157276596735)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:16.082097 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fe04ae-66e0-41dc-a054-4d91b7029f6b None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:16.084200 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 161/333] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:10:15 2022] GET /identity/v3/users/16bc56af20bf4ae0bc00157276596735 => generated 293 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:16.089753 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7580fd04-1fb0-4ef1-a335-ce4816645d9f None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:16.198112 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7580fd04-1fb0-4ef1-a335-ce4816645d9f None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:16.237172 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cd7c0c-5fdd-40d0-904e-76853da39dbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:16.237864 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cd7c0c-5fdd-40d0-904e-76853da39dbc None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:16.238785 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cd7c0c-5fdd-40d0-904e-76853da39dbc None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:16.239065 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cd7c0c-5fdd-40d0-904e-76853da39dbc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:16.239976 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cd7c0c-5fdd-40d0-904e-76853da39dbc None admin] RBAC: Adding query filter params (role.id=7d0e162805b54c019ce6c954cbef3818, user.id=16bc56af20bf4ae0bc00157276596735, scope.system=all) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:16.240587 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cd7c0c-5fdd-40d0-904e-76853da39dbc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:16.242095 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cd7c0c-5fdd-40d0-904e-76853da39dbc None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:16.250357 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:16.258587 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 173/334] 10.222.0.57 () {56 vars in 1308 bytes} [Thu May 19 14:10:16 2022] GET /identity/v3/role_assignments?user.id=16bc56af20bf4ae0bc00157276596735&scope.system=all&role.id=7d0e162805b54c019ce6c954cbef3818 => generated 510 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:10:18.293973 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0b1dfb-ef54-4a19-9da5-fcb1bda63b94 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:18.294522 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0b1dfb-ef54-4a19-9da5-fcb1bda63b94 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:18.294522 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0b1dfb-ef54-4a19-9da5-fcb1bda63b94 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:18.296194 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 162/335] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:10:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:10:18.301910 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9aa1d8-f6d7-4c1d-8ce8-191a392e2ae5 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:18.302803 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9aa1d8-f6d7-4c1d-8ce8-191a392e2ae5 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:18.303278 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9aa1d8-f6d7-4c1d-8ce8-191a392e2ae5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:18.326842 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-af9aa1d8-f6d7-4c1d-8ce8-191a392e2ae5 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:10:18.328371 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-af9aa1d8-f6d7-4c1d-8ce8-191a392e2ae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:10:18.399228 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec7db38-0981-4055-8e56-5e09a2b85ce6 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:18.399574 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec7db38-0981-4055-8e56-5e09a2b85ce6 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:18.399792 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec7db38-0981-4055-8e56-5e09a2b85ce6 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:18.401408 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 163/336] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:10:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:10:18.409870 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac780fec-528e-42d0-93d7-7ecff19d369e None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:18.410482 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac780fec-528e-42d0-93d7-7ecff19d369e None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:18.410972 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac780fec-528e-42d0-93d7-7ecff19d369e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:18.417867 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 174/337] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:10:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:10:18.447009 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-ac780fec-528e-42d0-93d7-7ecff19d369e None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:10:18.448645 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-ac780fec-528e-42d0-93d7-7ecff19d369e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:10:18.495089 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9aa1d8-f6d7-4c1d-8ce8-191a392e2ae5 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:18.554717 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 164/338] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:10:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:10:18.590823 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac780fec-528e-42d0-93d7-7ecff19d369e None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:18.617880 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:18.714152 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9aa1d8-f6d7-4c1d-8ce8-191a392e2ae5 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:18.717795 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac780fec-528e-42d0-93d7-7ecff19d369e None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:18.764419 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:18.768254 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd0fd2b-dc39-4c1c-b03f-31cb39596dd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:18.769323 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd0fd2b-dc39-4c1c-b03f-31cb39596dd9 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:18.769773 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd0fd2b-dc39-4c1c-b03f-31cb39596dd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:18.770236 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd0fd2b-dc39-4c1c-b03f-31cb39596dd9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:18.775105 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11538701-2eb4-400f-83eb-0ca04bc53e28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:18.776362 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11538701-2eb4-400f-83eb-0ca04bc53e28 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:18.776648 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11538701-2eb4-400f-83eb-0ca04bc53e28 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:18.776925 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11538701-2eb4-400f-83eb-0ca04bc53e28 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:18.783578 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd0fd2b-dc39-4c1c-b03f-31cb39596dd9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:10:18.792512 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-2cd0fd2b-dc39-4c1c-b03f-31cb39596dd9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:10:18.792512 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:18.792512 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:18.792512 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:18.792512 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:18.792512 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:18.792512 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:18.792512 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:18.792512 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:18.792512 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:18.792512 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:18.792512 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:18.792512 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:10:18.792512 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:10:18.792512 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:10:18.792512 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:10:18.793461 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:10:18.793461 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:10:18.793461 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:10:18.793461 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:10:18.793461 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:18.793461 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:18.793461 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:18.793461 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:18.793461 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:18.793461 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:18.793461 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:18.793461 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:10:18.793461 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:18.793461 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:18.793461 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:18.793461 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:18.794351 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:18.794351 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:18.794351 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:18.794351 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:18.794351 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:10:18.794351 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:10:18.794351 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:10:18.794351 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:10:18.794351 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:10:18.794351 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:10:18.794351 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:10:18.794351 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:10:18.797008 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 175/339] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 19 14:10:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:10:18.797008 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b985e7-4752-4123-aa21-24e52ed49698 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:18.797182 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b985e7-4752-4123-aa21-24e52ed49698 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:18.797374 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b985e7-4752-4123-aa21-24e52ed49698 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:18.798895 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 176/340] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:10:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:10:18.803676 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11538701-2eb4-400f-83eb-0ca04bc53e28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:10:18.807320 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe37627-5567-486b-b685-1ce90adbec74 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:18.807618 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe37627-5567-486b-b685-1ce90adbec74 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:18.807897 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe37627-5567-486b-b685-1ce90adbec74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:18.811518 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-11538701-2eb4-400f-83eb-0ca04bc53e28 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:10:18.811518 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:18.811518 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:18.811518 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:18.811518 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:18.811518 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:18.811518 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:18.811518 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:18.811518 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:18.811518 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:18.811518 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:18.811518 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:18.811518 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:10:18.811518 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:10:18.811518 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:10:18.811518 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:10:18.813564 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:10:18.813564 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:10:18.813564 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:10:18.813564 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:10:18.813564 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:18.813564 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:18.813564 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:18.813564 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:18.813564 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:18.813564 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:18.813564 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:18.813564 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:10:18.813564 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:18.813564 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:18.813564 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:18.813564 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:18.815558 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:18.815558 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:18.815558 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:18.815558 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:18.815558 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:10:18.815558 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:10:18.815558 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:10:18.815558 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:10:18.815558 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:10:18.815558 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:10:18.815558 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:10:18.815558 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:10:18.817273 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 165/341] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:10:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:10:18.825874 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11538701-2eb4-400f-83eb-0ca04bc53e28 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:18.826119 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-4fe37627-5567-486b-b685-1ce90adbec74 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:10:18.826647 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-4fe37627-5567-486b-b685-1ce90adbec74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:10:18.914870 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 177/342] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:10:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:10:18.915740 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe37627-5567-486b-b685-1ce90adbec74 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:18.939742 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:19.000778 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11538701-2eb4-400f-83eb-0ca04bc53e28 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:19.042507 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfab122c-b20a-4897-968d-1ec202227c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:19.043193 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfab122c-b20a-4897-968d-1ec202227c9d None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:19.043477 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfab122c-b20a-4897-968d-1ec202227c9d None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:19.043754 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfab122c-b20a-4897-968d-1ec202227c9d None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:19.046059 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfab122c-b20a-4897-968d-1ec202227c9d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:19.046385 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfab122c-b20a-4897-968d-1ec202227c9d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:19.047908 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfab122c-b20a-4897-968d-1ec202227c9d None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:19.058423 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 166/343] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:10:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:19.059578 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfab122c-b20a-4897-968d-1ec202227c9d None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:19.130460 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe37627-5567-486b-b685-1ce90adbec74 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:19.150513 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:19.173056 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72c7885-d33e-4541-b2ba-d54083543e84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:19.173912 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72c7885-d33e-4541-b2ba-d54083543e84 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:19.174292 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72c7885-d33e-4541-b2ba-d54083543e84 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:19.174664 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72c7885-d33e-4541-b2ba-d54083543e84 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:19.175581 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72c7885-d33e-4541-b2ba-d54083543e84 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:19.176002 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72c7885-d33e-4541-b2ba-d54083543e84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:19.177499 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72c7885-d33e-4541-b2ba-d54083543e84 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:19.179188 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfab122c-b20a-4897-968d-1ec202227c9d None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:19.187903 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 178/344] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:10:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:19.188711 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72c7885-d33e-4541-b2ba-d54083543e84 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:19.219677 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea70581-a18b-4b61-8b46-4df20810d7ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:19.220617 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea70581-a18b-4b61-8b46-4df20810d7ed None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:19.220962 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea70581-a18b-4b61-8b46-4df20810d7ed None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:19.221251 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea70581-a18b-4b61-8b46-4df20810d7ed None admin] PATH_INFO: `/v3/roles/member` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:19.233314 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea70581-a18b-4b61-8b46-4df20810d7ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:10:19.242279 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:19.243790 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-dea70581-a18b-4b61-8b46-4df20810d7ed None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:10:19.243790 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:19.243790 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:19.243790 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:19.243790 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:19.243790 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:19.243790 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:19.243790 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:19.243790 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:19.243790 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:19.243790 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:19.243790 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:19.243790 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:10:19.243790 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:10:19.243790 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:10:19.243790 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:10:19.250479 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:10:19.250479 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:10:19.250479 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:10:19.250479 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:10:19.250479 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:19.250479 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:19.250479 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:19.250479 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:19.250479 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:19.250479 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:19.250479 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:19.250479 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:10:19.250479 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:19.250479 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:19.250479 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:19.250479 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:19.251360 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:19.251360 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:19.251360 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:19.251360 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:19.251360 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:10:19.251360 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:10:19.251360 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:10:19.251360 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:10:19.251360 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:10:19.251360 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:10:19.251360 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:10:19.251360 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:10:19.251360 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 167/345] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:10:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:10:19.252130 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badb1ec9-0566-4f2f-9707-8d7ea984122e None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:19.252207 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badb1ec9-0566-4f2f-9707-8d7ea984122e None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:19.252486 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badb1ec9-0566-4f2f-9707-8d7ea984122e None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:19.257640 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 168/346] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:10:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:10:19.257640 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badb1ec9-0566-4f2f-9707-8d7ea984122e None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:19.302774 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72c7885-d33e-4541-b2ba-d54083543e84 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:19.342901 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a52900b-6c60-42f3-9cd6-bbae4ace348b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:19.343808 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a52900b-6c60-42f3-9cd6-bbae4ace348b None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:19.344246 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a52900b-6c60-42f3-9cd6-bbae4ace348b None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:19.344669 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a52900b-6c60-42f3-9cd6-bbae4ace348b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:19.377287 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badb1ec9-0566-4f2f-9707-8d7ea984122e None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:19.401318 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:19.419305 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f057156-64ff-49f4-9d43-e3ac18445d48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:19.420622 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f057156-64ff-49f4-9d43-e3ac18445d48 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:19.421108 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f057156-64ff-49f4-9d43-e3ac18445d48 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:19.421524 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f057156-64ff-49f4-9d43-e3ac18445d48 None admin] PATH_INFO: `/v3/users/5c2a73860fad4d30b031a49cf2784b32` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:19.423055 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f057156-64ff-49f4-9d43-e3ac18445d48 None admin] RBAC: Authorizing `identity:get_user(user_id=5c2a73860fad4d30b031a49cf2784b32)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:19.424652 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f057156-64ff-49f4-9d43-e3ac18445d48 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:19.424783 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a52900b-6c60-42f3-9cd6-bbae4ace348b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:19.426333 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a52900b-6c60-42f3-9cd6-bbae4ace348b None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:19.427150 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 169/347] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:10:19 2022] GET /identity/v3/users/5c2a73860fad4d30b031a49cf2784b32 => generated 293 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:19.428331 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f057156-64ff-49f4-9d43-e3ac18445d48 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:19.518133 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:19.546462 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 179/348] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:10:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:10:19.547874 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499ac60c-45f8-4050-8060-03151e50b747 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:19.548159 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499ac60c-45f8-4050-8060-03151e50b747 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:19.548436 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499ac60c-45f8-4050-8060-03151e50b747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:19.568400 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f057156-64ff-49f4-9d43-e3ac18445d48 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:19.572662 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-499ac60c-45f8-4050-8060-03151e50b747 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:10:19.574249 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-499ac60c-45f8-4050-8060-03151e50b747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:10:19.617760 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6309e025-cb7a-44d3-9b97-af8522f4e235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:19.618765 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6309e025-cb7a-44d3-9b97-af8522f4e235 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:19.619229 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6309e025-cb7a-44d3-9b97-af8522f4e235 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:19.621422 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6309e025-cb7a-44d3-9b97-af8522f4e235 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:19.622471 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6309e025-cb7a-44d3-9b97-af8522f4e235 None admin] RBAC: Adding query filter params (name=member) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:19.622996 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6309e025-cb7a-44d3-9b97-af8522f4e235 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:19.624562 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6309e025-cb7a-44d3-9b97-af8522f4e235 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:19.634863 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 170/349] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:10:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:19.636001 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6309e025-cb7a-44d3-9b97-af8522f4e235 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:19.666894 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 180/350] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:10:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:10:19.668170 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499ac60c-45f8-4050-8060-03151e50b747 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:19.683154 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:19.773519 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6309e025-cb7a-44d3-9b97-af8522f4e235 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:19.797708 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499ac60c-45f8-4050-8060-03151e50b747 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:19.799547 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:19.812305 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39cc3d8-20db-447c-b34c-3fc4490c8bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:19.813293 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39cc3d8-20db-447c-b34c-3fc4490c8bf2 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:19.813771 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39cc3d8-20db-447c-b34c-3fc4490c8bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:19.814277 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39cc3d8-20db-447c-b34c-3fc4490c8bf2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:19.815320 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39cc3d8-20db-447c-b34c-3fc4490c8bf2 None admin] RBAC: Adding query filter params (role.id=b0ae35f3e09648d0bbac778d23fa3463, user.id=5c2a73860fad4d30b031a49cf2784b32, scope.system=all) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:19.815839 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39cc3d8-20db-447c-b34c-3fc4490c8bf2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:19.818486 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39cc3d8-20db-447c-b34c-3fc4490c8bf2 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:19.833277 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 171/351] 10.222.0.57 () {56 vars in 1308 bytes} [Thu May 19 14:10:19 2022] GET /identity/v3/role_assignments?user.id=5c2a73860fad4d30b031a49cf2784b32&scope.system=all&role.id=b0ae35f3e09648d0bbac778d23fa3463 => generated 510 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:19.834459 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39cc3d8-20db-447c-b34c-3fc4490c8bf2 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:19.839039 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6f667f-1d0c-4c08-989f-c31467d36a23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:19.839266 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6f667f-1d0c-4c08-989f-c31467d36a23 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:19.839475 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6f667f-1d0c-4c08-989f-c31467d36a23 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:19.839754 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6f667f-1d0c-4c08-989f-c31467d36a23 None admin] PATH_INFO: `/v3/projects/ed85b374a73a4c94a310482b7ba566f3` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:19.841049 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6f667f-1d0c-4c08-989f-c31467d36a23 None admin] RBAC: Authorizing `identity:get_project(project_id=ed85b374a73a4c94a310482b7ba566f3)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:19.842589 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6f667f-1d0c-4c08-989f-c31467d36a23 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:19.844411 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 181/352] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:10:19 2022] GET /identity/v3/projects/ed85b374a73a4c94a310482b7ba566f3 => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:19.845243 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6f667f-1d0c-4c08-989f-c31467d36a23 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:19.939071 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:19.952250 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39cc3d8-20db-447c-b34c-3fc4490c8bf2 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:19.967207 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6f667f-1d0c-4c08-989f-c31467d36a23 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:19.991911 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec50a891-331d-4b40-8ab2-e8b68e143063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:19.992937 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec50a891-331d-4b40-8ab2-e8b68e143063 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:19.993433 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec50a891-331d-4b40-8ab2-e8b68e143063 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:19.993954 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec50a891-331d-4b40-8ab2-e8b68e143063 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:20.009432 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f62fe7-09d8-44ed-b90e-960c5f3d6991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:20.010298 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f62fe7-09d8-44ed-b90e-960c5f3d6991 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:20.010583 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f62fe7-09d8-44ed-b90e-960c5f3d6991 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:20.010869 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f62fe7-09d8-44ed-b90e-960c5f3d6991 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:20.023048 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f62fe7-09d8-44ed-b90e-960c5f3d6991 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:10:20.031620 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-69f62fe7-09d8-44ed-b90e-960c5f3d6991 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:10:20.031620 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:20.031620 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:20.031620 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:20.031620 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:20.031620 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:20.031620 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:20.031620 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:20.031620 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:20.031620 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:20.031620 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:20.031620 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:20.031620 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:10:20.031620 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:10:20.031620 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:10:20.031620 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:10:20.032541 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:10:20.032541 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:10:20.032541 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:10:20.032541 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:10:20.032541 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:20.032541 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:20.032541 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:20.032541 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:20.032541 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:20.032541 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:20.032541 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:20.032541 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:10:20.032541 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:20.032541 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:20.032541 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:20.032541 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:20.033330 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:20.033330 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:20.033330 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:20.033330 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:20.033330 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:10:20.033330 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:10:20.033330 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:10:20.033330 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:10:20.033330 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:10:20.033330 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:10:20.033330 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:10:20.033330 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:10:20.033330 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 182/353] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:10:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:10:20.034014 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f62fe7-09d8-44ed-b90e-960c5f3d6991 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:20.056065 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:20.077501 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec50a891-331d-4b40-8ab2-e8b68e143063 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:20.079370 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec50a891-331d-4b40-8ab2-e8b68e143063 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:20.174104 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f62fe7-09d8-44ed-b90e-960c5f3d6991 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:20.200188 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 172/354] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:10:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:10:20.201447 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec50a891-331d-4b40-8ab2-e8b68e143063 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:20.218931 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7212766-5bc1-4639-932b-105bc9f85142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:20.219801 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7212766-5bc1-4639-932b-105bc9f85142 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:20.220083 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7212766-5bc1-4639-932b-105bc9f85142 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:20.220356 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7212766-5bc1-4639-932b-105bc9f85142 None admin] PATH_INFO: `/v3/groups/53f2efd4d9254ad4a528fceb5a783b5c` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:20.229173 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7212766-5bc1-4639-932b-105bc9f85142 None admin] RBAC: Authorizing `identity:get_group(group_id=53f2efd4d9254ad4a528fceb5a783b5c)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:20.233866 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7212766-5bc1-4639-932b-105bc9f85142 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:20.233866 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 183/355] 10.222.0.57 () {56 vars in 1180 bytes} [Thu May 19 14:10:20 2022] GET /identity/v3/groups/53f2efd4d9254ad4a528fceb5a783b5c => generated 226 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:20.233866 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7212766-5bc1-4639-932b-105bc9f85142 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:20.235752 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:20.335378 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:20.350226 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7212766-5bc1-4639-932b-105bc9f85142 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:20.358093 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec50a891-331d-4b40-8ab2-e8b68e143063 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:20.399033 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d2662f-b5f5-4953-bdcd-465dff9da2d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:20.399878 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d2662f-b5f5-4953-bdcd-465dff9da2d1 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:20.400166 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d2662f-b5f5-4953-bdcd-465dff9da2d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:20.400445 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d2662f-b5f5-4953-bdcd-465dff9da2d1 None admin] PATH_INFO: `/v3/projects/ed85b374a73a4c94a310482b7ba566f3` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:20.401652 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d2662f-b5f5-4953-bdcd-465dff9da2d1 None admin] RBAC: Authorizing `identity:get_project(project_id=ed85b374a73a4c94a310482b7ba566f3)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:20.403163 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d2662f-b5f5-4953-bdcd-465dff9da2d1 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:20.404874 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 184/356] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:10:20 2022] GET /identity/v3/projects/ed85b374a73a4c94a310482b7ba566f3 => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:20.405636 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d2662f-b5f5-4953-bdcd-465dff9da2d1 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:20.406266 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f47daa2-6561-4deb-b9f3-beb8684019fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:20.407527 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f47daa2-6561-4deb-b9f3-beb8684019fe None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:20.408055 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f47daa2-6561-4deb-b9f3-beb8684019fe None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:20.409433 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f47daa2-6561-4deb-b9f3-beb8684019fe None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:20.411273 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f47daa2-6561-4deb-b9f3-beb8684019fe None admin] RBAC: Adding query filter params (name=member) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:20.411971 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f47daa2-6561-4deb-b9f3-beb8684019fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:20.413652 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f47daa2-6561-4deb-b9f3-beb8684019fe None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:20.423365 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 173/357] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:10:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:20.425458 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd76f33c-74eb-40ac-91b5-a22959d0d7eb None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:20.425834 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd76f33c-74eb-40ac-91b5-a22959d0d7eb None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:20.426212 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd76f33c-74eb-40ac-91b5-a22959d0d7eb None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:20.431836 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 174/358] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:10:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:10:20.433110 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd76f33c-74eb-40ac-91b5-a22959d0d7eb None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:20.497711 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:20.541402 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d2662f-b5f5-4953-bdcd-465dff9da2d1 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:20.579056 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd76f33c-74eb-40ac-91b5-a22959d0d7eb None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:20.589626 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bf5787-9174-4d10-8270-27ceff5e7881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:20.590925 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bf5787-9174-4d10-8270-27ceff5e7881 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:20.591365 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bf5787-9174-4d10-8270-27ceff5e7881 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:20.592090 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bf5787-9174-4d10-8270-27ceff5e7881 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:20.593398 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bf5787-9174-4d10-8270-27ceff5e7881 None admin] RBAC: Adding query filter params (group.id=53f2efd4d9254ad4a528fceb5a783b5c, role.id=23d9dd31632340bdba9cf3093c095dbb, scope.project.id=ed85b374a73a4c94a310482b7ba566f3) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:20.593925 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bf5787-9174-4d10-8270-27ceff5e7881 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:20.596131 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bf5787-9174-4d10-8270-27ceff5e7881 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:20.602882 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 185/359] 10.222.0.57 () {56 vars in 1376 bytes} [Thu May 19 14:10:20 2022] GET /identity/v3/role_assignments?group.id=53f2efd4d9254ad4a528fceb5a783b5c&scope.project.id=ed85b374a73a4c94a310482b7ba566f3&role.id=23d9dd31632340bdba9cf3093c095dbb => generated 260 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:20.603689 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bf5787-9174-4d10-8270-27ceff5e7881 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:20.634508 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21742c3-0dae-49a7-8501-0a64721b03fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:20.635513 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21742c3-0dae-49a7-8501-0a64721b03fa None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:20.635813 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21742c3-0dae-49a7-8501-0a64721b03fa None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:20.636079 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21742c3-0dae-49a7-8501-0a64721b03fa None admin] PATH_INFO: `/v3/groups/53f2efd4d9254ad4a528fceb5a783b5c` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:20.637890 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21742c3-0dae-49a7-8501-0a64721b03fa None admin] RBAC: Authorizing `identity:get_group(group_id=53f2efd4d9254ad4a528fceb5a783b5c)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:20.639480 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21742c3-0dae-49a7-8501-0a64721b03fa None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:20.642641 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 175/360] 10.222.0.57 () {56 vars in 1180 bytes} [Thu May 19 14:10:20 2022] GET /identity/v3/groups/53f2efd4d9254ad4a528fceb5a783b5c => generated 226 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:20.643960 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7296ef9f-5a9c-45f8-a436-820bac54351e None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:20.644809 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7296ef9f-5a9c-45f8-a436-820bac54351e None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:20.645242 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7296ef9f-5a9c-45f8-a436-820bac54351e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:20.665597 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-7296ef9f-5a9c-45f8-a436-820bac54351e None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:10:20.667083 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-7296ef9f-5a9c-45f8-a436-820bac54351e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:10:20.678381 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:20.754399 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bf5787-9174-4d10-8270-27ceff5e7881 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:20.757280 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 176/361] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:10:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:10:20.758353 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7296ef9f-5a9c-45f8-a436-820bac54351e None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:20.799026 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f1073d-b3cf-402d-94ea-37184262a360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:20.799708 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f1073d-b3cf-402d-94ea-37184262a360 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:20.799996 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f1073d-b3cf-402d-94ea-37184262a360 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:20.800330 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f1073d-b3cf-402d-94ea-37184262a360 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:20.804552 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:20.892428 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f1073d-b3cf-402d-94ea-37184262a360 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:20.893935 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f1073d-b3cf-402d-94ea-37184262a360 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:20.894854 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7296ef9f-5a9c-45f8-a436-820bac54351e None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:20.932704 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83042df4-2b6c-45da-bab6-7685dfdb5db6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:20.933743 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83042df4-2b6c-45da-bab6-7685dfdb5db6 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:20.934246 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83042df4-2b6c-45da-bab6-7685dfdb5db6 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:20.934741 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83042df4-2b6c-45da-bab6-7685dfdb5db6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:20.935881 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83042df4-2b6c-45da-bab6-7685dfdb5db6 None admin] RBAC: Adding query filter params (group.id=53f2efd4d9254ad4a528fceb5a783b5c, role.id=7d0e162805b54c019ce6c954cbef3818, scope.project.id=ed85b374a73a4c94a310482b7ba566f3) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:20.936384 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83042df4-2b6c-45da-bab6-7685dfdb5db6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:20.938190 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83042df4-2b6c-45da-bab6-7685dfdb5db6 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:20.945421 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 177/362] 10.222.0.57 () {56 vars in 1376 bytes} [Thu May 19 14:10:20 2022] GET /identity/v3/role_assignments?group.id=53f2efd4d9254ad4a528fceb5a783b5c&scope.project.id=ed85b374a73a4c94a310482b7ba566f3&role.id=7d0e162805b54c019ce6c954cbef3818 => generated 260 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:20.946768 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83042df4-2b6c-45da-bab6-7685dfdb5db6 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:20.966287 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:21.005820 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 186/363] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:10:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:10:21.011263 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f1073d-b3cf-402d-94ea-37184262a360 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:21.051494 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:21.071793 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83042df4-2b6c-45da-bab6-7685dfdb5db6 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:21.117582 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2df3e4-93d4-4fec-9bee-3ec2676e6d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:21.118791 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2df3e4-93d4-4fec-9bee-3ec2676e6d58 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:21.119274 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2df3e4-93d4-4fec-9bee-3ec2676e6d58 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:21.119798 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2df3e4-93d4-4fec-9bee-3ec2676e6d58 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:21.133636 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2df3e4-93d4-4fec-9bee-3ec2676e6d58 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:10:21.140009 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f1073d-b3cf-402d-94ea-37184262a360 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:21.144365 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-2a2df3e4-93d4-4fec-9bee-3ec2676e6d58 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:10:21.144365 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:21.144365 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:21.144365 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:21.144365 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:21.144365 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:21.144365 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:21.144365 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:21.144365 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:21.144365 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:21.144365 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:21.144365 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:21.144365 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:10:21.144365 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:10:21.144365 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:10:21.144365 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:10:21.145486 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:10:21.145486 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:10:21.145486 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:10:21.145486 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:10:21.145486 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:21.145486 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:21.145486 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:21.145486 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:21.145486 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:21.145486 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:21.145486 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:21.145486 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:10:21.145486 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:21.145486 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:21.145486 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:21.145486 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:21.146336 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:21.146336 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:21.146336 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:21.146336 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:21.146336 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:10:21.146336 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:10:21.146336 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:10:21.146336 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:10:21.146336 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:10:21.146336 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:10:21.146336 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:10:21.146336 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:10:21.149037 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 178/364] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 19 14:10:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:10:21.155794 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba13dba-3ae3-4f42-b4c6-6daf1dbd7bb6 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:21.156511 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba13dba-3ae3-4f42-b4c6-6daf1dbd7bb6 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:21.156945 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba13dba-3ae3-4f42-b4c6-6daf1dbd7bb6 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:21.158671 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 179/365] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:10:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:10:21.159419 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba13dba-3ae3-4f42-b4c6-6daf1dbd7bb6 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:21.180196 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac9e542-7caa-44e7-8226-4fbfd62274e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:21.181139 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac9e542-7caa-44e7-8226-4fbfd62274e5 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:21.181455 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac9e542-7caa-44e7-8226-4fbfd62274e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:21.181735 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac9e542-7caa-44e7-8226-4fbfd62274e5 None admin] PATH_INFO: `/v3/projects/3c9e67fcd5b14da2a3775ef2b2cad6b2` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:21.183112 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac9e542-7caa-44e7-8226-4fbfd62274e5 None admin] RBAC: Authorizing `identity:get_project(project_id=3c9e67fcd5b14da2a3775ef2b2cad6b2)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:21.184609 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac9e542-7caa-44e7-8226-4fbfd62274e5 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:21.188065 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 187/366] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:10:21 2022] GET /identity/v3/projects/3c9e67fcd5b14da2a3775ef2b2cad6b2 => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:10:21.189456 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8e8621-8784-44be-99bc-ba9a2c49d3bf None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:21.189747 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8e8621-8784-44be-99bc-ba9a2c49d3bf None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:21.194203 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8e8621-8784-44be-99bc-ba9a2c49d3bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:21.214144 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-cc8e8621-8784-44be-99bc-ba9a2c49d3bf None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:10:21.215387 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-cc8e8621-8784-44be-99bc-ba9a2c49d3bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:10:21.241105 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:21.298573 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba13dba-3ae3-4f42-b4c6-6daf1dbd7bb6 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:21.329198 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 188/367] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:10:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:10:21.330591 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8e8621-8784-44be-99bc-ba9a2c49d3bf None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:21.342766 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c6b066-bbf6-4359-aaa2-723ba1e27ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:21.343911 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c6b066-bbf6-4359-aaa2-723ba1e27ebd None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:21.344201 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c6b066-bbf6-4359-aaa2-723ba1e27ebd None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:21.344481 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c6b066-bbf6-4359-aaa2-723ba1e27ebd None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:21.345248 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c6b066-bbf6-4359-aaa2-723ba1e27ebd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:21.345557 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c6b066-bbf6-4359-aaa2-723ba1e27ebd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:21.347784 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c6b066-bbf6-4359-aaa2-723ba1e27ebd None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:21.359233 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 180/368] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:10:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:21.364989 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c6b066-bbf6-4359-aaa2-723ba1e27ebd None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:21.372926 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:21.467137 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8e8621-8784-44be-99bc-ba9a2c49d3bf None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:21.489225 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c6b066-bbf6-4359-aaa2-723ba1e27ebd None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:21.512578 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825fdbf2-3024-4f45-a87a-4986f444faed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:21.514010 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825fdbf2-3024-4f45-a87a-4986f444faed None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:21.514293 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825fdbf2-3024-4f45-a87a-4986f444faed None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:21.514563 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825fdbf2-3024-4f45-a87a-4986f444faed None admin] PATH_INFO: `/v3/groups/53f2efd4d9254ad4a528fceb5a783b5c` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:21.515983 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825fdbf2-3024-4f45-a87a-4986f444faed None admin] RBAC: Authorizing `identity:get_group(group_id=53f2efd4d9254ad4a528fceb5a783b5c)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:21.518171 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825fdbf2-3024-4f45-a87a-4986f444faed None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:21.521049 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 189/369] 10.222.0.57 () {56 vars in 1180 bytes} [Thu May 19 14:10:21 2022] GET /identity/v3/groups/53f2efd4d9254ad4a528fceb5a783b5c => generated 226 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:21.521990 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825fdbf2-3024-4f45-a87a-4986f444faed None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:21.527018 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:21.539197 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c76552-ffa9-48d3-b0f9-e3ea2a4a5abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:21.540108 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c76552-ffa9-48d3-b0f9-e3ea2a4a5abe None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:21.540397 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c76552-ffa9-48d3-b0f9-e3ea2a4a5abe None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:21.540675 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c76552-ffa9-48d3-b0f9-e3ea2a4a5abe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:21.553343 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c76552-ffa9-48d3-b0f9-e3ea2a4a5abe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:10:21.561869 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-10c76552-ffa9-48d3-b0f9-e3ea2a4a5abe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:10:21.561869 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:21.561869 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:21.561869 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:21.561869 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:21.561869 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:21.561869 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:21.561869 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:21.561869 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:21.561869 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:21.561869 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:21.561869 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:21.561869 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:10:21.561869 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:10:21.561869 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:10:21.561869 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:10:21.564047 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:10:21.564047 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:10:21.564047 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:10:21.564047 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:10:21.564047 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:21.564047 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:21.564047 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:21.564047 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:21.564047 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:21.564047 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:21.564047 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:21.564047 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:10:21.564047 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:21.564047 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:21.564047 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:21.564047 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:21.566399 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:21.566399 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:21.566399 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:21.566399 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:21.566399 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:10:21.566399 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:10:21.566399 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:10:21.566399 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:10:21.566399 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:10:21.566399 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:10:21.566399 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:10:21.566399 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:10:21.566399 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 181/370] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:10:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:10:21.566399 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c76552-ffa9-48d3-b0f9-e3ea2a4a5abe None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:21.653442 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:21.656831 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825fdbf2-3024-4f45-a87a-4986f444faed None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:21.687900 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c76552-ffa9-48d3-b0f9-e3ea2a4a5abe None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:21.705355 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88cc831-7412-4367-b806-513b5963aa46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:21.706058 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88cc831-7412-4367-b806-513b5963aa46 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:21.706344 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88cc831-7412-4367-b806-513b5963aa46 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:21.709862 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88cc831-7412-4367-b806-513b5963aa46 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:21.730410 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56419b4b-18c8-4b36-a9d0-3978639934c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:21.730998 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56419b4b-18c8-4b36-a9d0-3978639934c3 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:21.731282 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56419b4b-18c8-4b36-a9d0-3978639934c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:21.731558 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56419b4b-18c8-4b36-a9d0-3978639934c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:21.732392 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56419b4b-18c8-4b36-a9d0-3978639934c3 None admin] RBAC: Adding query filter params (group.id=53f2efd4d9254ad4a528fceb5a783b5c, role.id=7d0e162805b54c019ce6c954cbef3818, scope.project.id=3c9e67fcd5b14da2a3775ef2b2cad6b2) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:21.733179 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56419b4b-18c8-4b36-a9d0-3978639934c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:21.734984 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56419b4b-18c8-4b36-a9d0-3978639934c3 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:21.745044 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 182/371] 10.222.0.57 () {56 vars in 1376 bytes} [Thu May 19 14:10:21 2022] GET /identity/v3/role_assignments?group.id=53f2efd4d9254ad4a528fceb5a783b5c&scope.project.id=3c9e67fcd5b14da2a3775ef2b2cad6b2&role.id=7d0e162805b54c019ce6c954cbef3818 => generated 260 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:21.746236 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56419b4b-18c8-4b36-a9d0-3978639934c3 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:21.812579 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:21.842747 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88cc831-7412-4367-b806-513b5963aa46 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:21.844268 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88cc831-7412-4367-b806-513b5963aa46 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:21.880615 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56419b4b-18c8-4b36-a9d0-3978639934c3 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:21.942582 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c807d71b-112d-4ffb-9e3e-beb7ad06e639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:21.943275 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c807d71b-112d-4ffb-9e3e-beb7ad06e639 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:21.943556 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c807d71b-112d-4ffb-9e3e-beb7ad06e639 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:21.943829 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c807d71b-112d-4ffb-9e3e-beb7ad06e639 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:21.944601 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c807d71b-112d-4ffb-9e3e-beb7ad06e639 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:21.944910 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c807d71b-112d-4ffb-9e3e-beb7ad06e639 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:21.946344 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c807d71b-112d-4ffb-9e3e-beb7ad06e639 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:21.967290 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 183/372] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:10:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:21.973349 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c807d71b-112d-4ffb-9e3e-beb7ad06e639 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:22.059629 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:22.064402 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 190/373] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:10:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 543 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:10:22.070979 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88cc831-7412-4367-b806-513b5963aa46 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:22.192614 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c807d71b-112d-4ffb-9e3e-beb7ad06e639 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:22.219033 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:22.259365 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88cc831-7412-4367-b806-513b5963aa46 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:22.274934 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0c7ff8-ded7-4add-972e-11ef14dc9bc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:22.275593 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0c7ff8-ded7-4add-972e-11ef14dc9bc4 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:22.275880 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0c7ff8-ded7-4add-972e-11ef14dc9bc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:22.276158 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0c7ff8-ded7-4add-972e-11ef14dc9bc4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:22.347684 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e30311-28dd-49c3-ab42-967e8b67e0da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:22.348488 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e30311-28dd-49c3-ab42-967e8b67e0da None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:22.348772 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e30311-28dd-49c3-ab42-967e8b67e0da None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:22.349053 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e30311-28dd-49c3-ab42-967e8b67e0da None admin] PATH_INFO: `/v3/projects/3c9e67fcd5b14da2a3775ef2b2cad6b2` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:22.350300 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e30311-28dd-49c3-ab42-967e8b67e0da None admin] RBAC: Authorizing `identity:get_project(project_id=3c9e67fcd5b14da2a3775ef2b2cad6b2)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:22.351828 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e30311-28dd-49c3-ab42-967e8b67e0da None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:22.353940 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 191/374] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:10:22 2022] GET /identity/v3/projects/3c9e67fcd5b14da2a3775ef2b2cad6b2 => generated 302 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:22.357344 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0c7ff8-ded7-4add-972e-11ef14dc9bc4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:22.359141 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0c7ff8-ded7-4add-972e-11ef14dc9bc4 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:22.359963 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e30311-28dd-49c3-ab42-967e8b67e0da None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:22.497979 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:22.503662 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 184/375] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:10:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 530 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:10:22.510322 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0c7ff8-ded7-4add-972e-11ef14dc9bc4 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:22.534196 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:22.558732 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e30311-28dd-49c3-ab42-967e8b67e0da None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:22.626034 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb34009a-6cb3-49f2-9c76-f877a2efeb5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:22.626915 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb34009a-6cb3-49f2-9c76-f877a2efeb5e None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:22.627199 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb34009a-6cb3-49f2-9c76-f877a2efeb5e None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:22.627473 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb34009a-6cb3-49f2-9c76-f877a2efeb5e None admin] PATH_INFO: `/v3/groups/53f2efd4d9254ad4a528fceb5a783b5c` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:22.629194 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb34009a-6cb3-49f2-9c76-f877a2efeb5e None admin] RBAC: Authorizing `identity:get_group(group_id=53f2efd4d9254ad4a528fceb5a783b5c)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:22.630570 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb34009a-6cb3-49f2-9c76-f877a2efeb5e None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:22.634816 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 192/376] 10.222.0.57 () {56 vars in 1180 bytes} [Thu May 19 14:10:22 2022] GET /identity/v3/groups/53f2efd4d9254ad4a528fceb5a783b5c => generated 226 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:22.640928 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb34009a-6cb3-49f2-9c76-f877a2efeb5e None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:22.690152 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0c7ff8-ded7-4add-972e-11ef14dc9bc4 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:22.753686 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ccfb7b-82b3-48c3-a6ae-64417e07c4f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:22.754563 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ccfb7b-82b3-48c3-a6ae-64417e07c4f0 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:22.754848 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ccfb7b-82b3-48c3-a6ae-64417e07c4f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:22.755122 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ccfb7b-82b3-48c3-a6ae-64417e07c4f0 None admin] PATH_INFO: `/v3/projects/dc6295ed5b1f4894a42c16df5aa39db4` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:22.757305 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ccfb7b-82b3-48c3-a6ae-64417e07c4f0 None admin] RBAC: Authorizing `identity:get_project(project_id=dc6295ed5b1f4894a42c16df5aa39db4)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:22.758787 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ccfb7b-82b3-48c3-a6ae-64417e07c4f0 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:22.760518 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 185/377] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:10:22 2022] GET /identity/v3/projects/dc6295ed5b1f4894a42c16df5aa39db4 => generated 344 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:22.766641 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ccfb7b-82b3-48c3-a6ae-64417e07c4f0 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:22.830179 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb34009a-6cb3-49f2-9c76-f877a2efeb5e None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:22.907400 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3e1f1a-d4f8-4095-b552-cb014ca7e0c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:22.907984 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3e1f1a-d4f8-4095-b552-cb014ca7e0c9 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:22.908268 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3e1f1a-d4f8-4095-b552-cb014ca7e0c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:22.908543 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3e1f1a-d4f8-4095-b552-cb014ca7e0c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:22.909382 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3e1f1a-d4f8-4095-b552-cb014ca7e0c9 None admin] RBAC: Adding query filter params (group.id=53f2efd4d9254ad4a528fceb5a783b5c, role.id=23d9dd31632340bdba9cf3093c095dbb, scope.project.id=3c9e67fcd5b14da2a3775ef2b2cad6b2) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:22.911025 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3e1f1a-d4f8-4095-b552-cb014ca7e0c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:22.911938 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3e1f1a-d4f8-4095-b552-cb014ca7e0c9 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:22.918194 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:22.926181 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:22.926947 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 193/378] 10.222.0.57 () {56 vars in 1376 bytes} [Thu May 19 14:10:22 2022] GET /identity/v3/role_assignments?group.id=53f2efd4d9254ad4a528fceb5a783b5c&scope.project.id=3c9e67fcd5b14da2a3775ef2b2cad6b2&role.id=23d9dd31632340bdba9cf3093c095dbb => generated 260 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:22.945942 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ccfb7b-82b3-48c3-a6ae-64417e07c4f0 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:22.988217 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25f1ecd-84a6-4eb1-b0f8-bdf28b398423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:22.989047 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25f1ecd-84a6-4eb1-b0f8-bdf28b398423 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:22.989332 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25f1ecd-84a6-4eb1-b0f8-bdf28b398423 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:22.989610 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25f1ecd-84a6-4eb1-b0f8-bdf28b398423 None admin] PATH_INFO: `/v3/groups/94ff5f55c6904ba1bb185d07d5d8607e` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:22.996755 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25f1ecd-84a6-4eb1-b0f8-bdf28b398423 None admin] RBAC: Authorizing `identity:get_group(group_id=94ff5f55c6904ba1bb185d07d5d8607e)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:22.998257 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25f1ecd-84a6-4eb1-b0f8-bdf28b398423 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:23.000042 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 186/379] 10.222.0.57 () {56 vars in 1180 bytes} [Thu May 19 14:10:22 2022] GET /identity/v3/groups/94ff5f55c6904ba1bb185d07d5d8607e => generated 229 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:23.006361 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3e1f1a-d4f8-4095-b552-cb014ca7e0c9 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:23.145953 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3e1f1a-d4f8-4095-b552-cb014ca7e0c9 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:23.195915 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa58f21-fb63-45ad-8b25-e61d2d631c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:23.196528 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa58f21-fb63-45ad-8b25-e61d2d631c1a None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:23.196813 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa58f21-fb63-45ad-8b25-e61d2d631c1a None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:23.197090 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa58f21-fb63-45ad-8b25-e61d2d631c1a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:23.197981 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa58f21-fb63-45ad-8b25-e61d2d631c1a None admin] RBAC: Adding query filter params (group.id=94ff5f55c6904ba1bb185d07d5d8607e, role.id=4946cf0ff7164307b36efeee7c787146, scope.project.id=dc6295ed5b1f4894a42c16df5aa39db4) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:23.198299 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa58f21-fb63-45ad-8b25-e61d2d631c1a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:23.199806 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa58f21-fb63-45ad-8b25-e61d2d631c1a None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:23.211870 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 194/380] 10.222.0.57 () {56 vars in 1376 bytes} [Thu May 19 14:10:23 2022] GET /identity/v3/role_assignments?group.id=94ff5f55c6904ba1bb185d07d5d8607e&scope.project.id=dc6295ed5b1f4894a42c16df5aa39db4&role.id=4946cf0ff7164307b36efeee7c787146 => generated 260 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:23.351876 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3248281-400b-4e33-a4d0-7e26dd9363e5 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:23.352165 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3248281-400b-4e33-a4d0-7e26dd9363e5 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:23.352675 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3248281-400b-4e33-a4d0-7e26dd9363e5 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:23.354628 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 187/381] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:10:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:10:23.360710 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c068646e-3e9b-4baa-9b08-e1a506564143 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:23.361052 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c068646e-3e9b-4baa-9b08-e1a506564143 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:23.361275 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c068646e-3e9b-4baa-9b08-e1a506564143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:23.382105 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-c068646e-3e9b-4baa-9b08-e1a506564143 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:10:23.383697 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-c068646e-3e9b-4baa-9b08-e1a506564143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:10:23.460990 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:23.469524 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 195/382] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:10:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:10:23.510658 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3248281-400b-4e33-a4d0-7e26dd9363e5 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:23.628862 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3248281-400b-4e33-a4d0-7e26dd9363e5 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:23.672057 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8917c55-948c-43ac-a560-cc39f670d1e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:23.673116 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8917c55-948c-43ac-a560-cc39f670d1e6 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:23.673582 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8917c55-948c-43ac-a560-cc39f670d1e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:23.674028 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8917c55-948c-43ac-a560-cc39f670d1e6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:23.685981 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8917c55-948c-43ac-a560-cc39f670d1e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:10:23.694948 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-a8917c55-948c-43ac-a560-cc39f670d1e6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:10:23.694948 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:23.694948 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:23.694948 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:23.694948 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:23.694948 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:23.694948 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:23.694948 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:23.694948 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:23.694948 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:23.694948 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:23.694948 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:23.694948 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:10:23.694948 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:10:23.694948 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:10:23.694948 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:10:23.696042 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:10:23.696042 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:10:23.696042 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:10:23.696042 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:10:23.696042 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:23.696042 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:23.696042 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:23.696042 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:23.696042 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:23.696042 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:23.696042 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:23.696042 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:10:23.696042 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:23.696042 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:23.696042 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:23.696042 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:23.697057 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:23.697057 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:23.697057 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:23.697057 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:23.697057 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:10:23.697057 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:10:23.697057 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:10:23.697057 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:10:23.697057 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:10:23.697057 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:10:23.697057 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:10:23.697057 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:10:23.699567 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 188/383] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 19 14:10:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:10:23.704966 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c068646e-3e9b-4baa-9b08-e1a506564143 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:23.821894 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c068646e-3e9b-4baa-9b08-e1a506564143 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:23.862510 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b585af4-f324-4d16-8d29-3081bf2fc67e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:23.864209 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b585af4-f324-4d16-8d29-3081bf2fc67e None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:23.864678 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b585af4-f324-4d16-8d29-3081bf2fc67e None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:23.865417 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b585af4-f324-4d16-8d29-3081bf2fc67e None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:23.866425 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b585af4-f324-4d16-8d29-3081bf2fc67e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:23.866937 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b585af4-f324-4d16-8d29-3081bf2fc67e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:23.868579 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b585af4-f324-4d16-8d29-3081bf2fc67e None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:23.880942 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 196/384] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:10:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:23.886572 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8917c55-948c-43ac-a560-cc39f670d1e6 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:23.900684 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:24.024919 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8917c55-948c-43ac-a560-cc39f670d1e6 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:24.069545 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9ac570-66f3-49f2-bfbc-71a25680999f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:24.070456 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9ac570-66f3-49f2-bfbc-71a25680999f None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:24.070741 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9ac570-66f3-49f2-bfbc-71a25680999f None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:24.071018 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9ac570-66f3-49f2-bfbc-71a25680999f None admin] PATH_INFO: `/v3/groups/53f2efd4d9254ad4a528fceb5a783b5c` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:24.072321 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9ac570-66f3-49f2-bfbc-71a25680999f None admin] RBAC: Authorizing `identity:get_group(group_id=53f2efd4d9254ad4a528fceb5a783b5c)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:24.074101 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9ac570-66f3-49f2-bfbc-71a25680999f None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:24.075258 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27969d1b-b58d-47f0-8fe0-2e8f97e9afab None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:24.075913 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 189/385] 10.222.0.57 () {56 vars in 1180 bytes} [Thu May 19 14:10:23 2022] GET /identity/v3/groups/53f2efd4d9254ad4a528fceb5a783b5c => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:24.076230 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27969d1b-b58d-47f0-8fe0-2e8f97e9afab None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:24.076836 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27969d1b-b58d-47f0-8fe0-2e8f97e9afab None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:24.080721 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 197/386] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:10:24 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:10:24.082180 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9ac570-66f3-49f2-bfbc-71a25680999f None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:24.087013 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022e2cf1-03e4-43fb-8afa-e20821c90948 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:24.087538 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022e2cf1-03e4-43fb-8afa-e20821c90948 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:24.088039 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022e2cf1-03e4-43fb-8afa-e20821c90948 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:24.114348 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-022e2cf1-03e4-43fb-8afa-e20821c90948 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:10:24.116064 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-022e2cf1-03e4-43fb-8afa-e20821c90948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:10:24.171247 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:24.197358 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9ac570-66f3-49f2-bfbc-71a25680999f None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:24.232696 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:24.241279 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85c6075-05ef-49ca-9c8c-cfda50d0c57a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:24.242183 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85c6075-05ef-49ca-9c8c-cfda50d0c57a None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:24.242469 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85c6075-05ef-49ca-9c8c-cfda50d0c57a None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:24.242744 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85c6075-05ef-49ca-9c8c-cfda50d0c57a None admin] PATH_INFO: `/v3/projects/ed85b374a73a4c94a310482b7ba566f3` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:24.244650 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85c6075-05ef-49ca-9c8c-cfda50d0c57a None admin] RBAC: Authorizing `identity:get_project(project_id=ed85b374a73a4c94a310482b7ba566f3)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:24.246147 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85c6075-05ef-49ca-9c8c-cfda50d0c57a None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:24.247874 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 190/387] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:10:24 2022] GET /identity/v3/projects/ed85b374a73a4c94a310482b7ba566f3 => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:10:24.255508 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85c6075-05ef-49ca-9c8c-cfda50d0c57a None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:24.263967 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 198/388] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:10:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:10:24.308883 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022e2cf1-03e4-43fb-8afa-e20821c90948 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:24.428142 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022e2cf1-03e4-43fb-8afa-e20821c90948 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:24.480265 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85c6075-05ef-49ca-9c8c-cfda50d0c57a None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:24.506398 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da09b9f-980d-40d3-bcbf-965456a0b5ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:24.507602 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da09b9f-980d-40d3-bcbf-965456a0b5ed None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:24.508087 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da09b9f-980d-40d3-bcbf-965456a0b5ed None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:24.508595 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da09b9f-980d-40d3-bcbf-965456a0b5ed None admin] PATH_INFO: `/v3/roles/member` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:24.533031 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da09b9f-980d-40d3-bcbf-965456a0b5ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:10:24.542837 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-3da09b9f-980d-40d3-bcbf-965456a0b5ed None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:10:24.542837 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:24.542837 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:24.542837 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:24.542837 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:24.542837 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:24.542837 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:24.542837 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:24.542837 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:24.542837 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:24.542837 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:24.542837 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:24.542837 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:10:24.542837 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:10:24.542837 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:10:24.542837 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:10:24.543738 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:10:24.543738 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:10:24.543738 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:10:24.543738 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:10:24.543738 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:24.543738 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:24.543738 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:24.543738 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:24.543738 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:24.543738 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:24.543738 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:24.543738 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:10:24.543738 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:24.543738 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:24.543738 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:24.543738 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:24.545346 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:24.545346 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:24.545346 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:24.545346 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:24.545346 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:10:24.545346 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:10:24.545346 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:10:24.545346 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:10:24.545346 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:10:24.545346 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:10:24.545346 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:10:24.545346 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:10:24.553065 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 199/389] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:10:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:10:24.562791 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da09b9f-980d-40d3-bcbf-965456a0b5ed None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:24.565173 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2df8f8-fd65-40fe-9bcf-ce71bf2b8947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:24.567189 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2df8f8-fd65-40fe-9bcf-ce71bf2b8947 None admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:24.567477 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2df8f8-fd65-40fe-9bcf-ce71bf2b8947 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:24.567755 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2df8f8-fd65-40fe-9bcf-ce71bf2b8947 None admin] PATH_INFO: `/v3/projects/ed85b374a73a4c94a310482b7ba566f3/groups/53f2efd4d9254ad4a528fceb5a783b5c/roles/23d9dd31632340bdba9cf3093c095dbb` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:24.572554 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2df8f8-fd65-40fe-9bcf-ce71bf2b8947 None admin] RBAC: Authorizing `identity:create_grant(project_id=ed85b374a73a4c94a310482b7ba566f3, group_id=53f2efd4d9254ad4a528fceb5a783b5c, role_id=23d9dd31632340bdba9cf3093c095dbb)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:24.574105 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2df8f8-fd65-40fe-9bcf-ce71bf2b8947 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:24.593215 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 191/390] 10.222.0.57 () {58 vars in 1442 bytes} [Thu May 19 14:10:24 2022] PUT /identity/v3/projects/ed85b374a73a4c94a310482b7ba566f3/groups/53f2efd4d9254ad4a528fceb5a783b5c/roles/23d9dd31632340bdba9cf3093c095dbb => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:10:24.640909 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:24.686759 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2606508d-2dcb-4b06-b852-2c0e5de8a05a None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:24.687456 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2606508d-2dcb-4b06-b852-2c0e5de8a05a None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:24.687928 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2606508d-2dcb-4b06-b852-2c0e5de8a05a None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:24.689972 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 192/391] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:10:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:10:24.696127 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34db0cf-7f5d-416a-9c61-d32d045c5658 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:24.696786 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34db0cf-7f5d-416a-9c61-d32d045c5658 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:24.697312 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34db0cf-7f5d-416a-9c61-d32d045c5658 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:24.709335 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da09b9f-980d-40d3-bcbf-965456a0b5ed None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:24.723038 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:24.725444 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-c34db0cf-7f5d-416a-9c61-d32d045c5658 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:10:24.727274 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-c34db0cf-7f5d-416a-9c61-d32d045c5658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:10:24.760179 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409471ea-401c-4730-95e5-892844c2dfb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:24.761059 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409471ea-401c-4730-95e5-892844c2dfb7 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:24.761167 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409471ea-401c-4730-95e5-892844c2dfb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:24.762407 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409471ea-401c-4730-95e5-892844c2dfb7 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:24.762407 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409471ea-401c-4730-95e5-892844c2dfb7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:24.762607 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409471ea-401c-4730-95e5-892844c2dfb7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:24.763918 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409471ea-401c-4730-95e5-892844c2dfb7 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:24.781473 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 200/392] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:10:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:24.798949 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409471ea-401c-4730-95e5-892844c2dfb7 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:24.824710 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 193/393] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:10:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:10:24.850745 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34db0cf-7f5d-416a-9c61-d32d045c5658 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:24.955913 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409471ea-401c-4730-95e5-892844c2dfb7 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:24.967991 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:25.015951 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecd16a0-c4e4-4e34-85b7-57f0f389cdcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:25.017048 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecd16a0-c4e4-4e34-85b7-57f0f389cdcf None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:25.017338 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecd16a0-c4e4-4e34-85b7-57f0f389cdcf None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:25.017608 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecd16a0-c4e4-4e34-85b7-57f0f389cdcf None admin] PATH_INFO: `/v3/groups/53f2efd4d9254ad4a528fceb5a783b5c` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:25.020984 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecd16a0-c4e4-4e34-85b7-57f0f389cdcf None admin] RBAC: Authorizing `identity:get_group(group_id=53f2efd4d9254ad4a528fceb5a783b5c)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:25.022499 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecd16a0-c4e4-4e34-85b7-57f0f389cdcf None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:25.025613 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 201/394] 10.222.0.57 () {56 vars in 1180 bytes} [Thu May 19 14:10:24 2022] GET /identity/v3/groups/53f2efd4d9254ad4a528fceb5a783b5c => generated 226 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:25.034944 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecd16a0-c4e4-4e34-85b7-57f0f389cdcf None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:25.036740 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34db0cf-7f5d-416a-9c61-d32d045c5658 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:25.070668 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:25.078246 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd52f20a-94b5-4998-910a-c8f3dcbe1c68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:25.079051 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd52f20a-94b5-4998-910a-c8f3dcbe1c68 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:25.079333 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd52f20a-94b5-4998-910a-c8f3dcbe1c68 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:25.079600 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd52f20a-94b5-4998-910a-c8f3dcbe1c68 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:25.093927 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd52f20a-94b5-4998-910a-c8f3dcbe1c68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:10:25.104030 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-dd52f20a-94b5-4998-910a-c8f3dcbe1c68 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:10:25.104030 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:25.104030 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:25.104030 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:25.104030 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:25.104030 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:25.104030 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:25.104030 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:25.104030 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:25.104030 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:25.104030 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:25.104030 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:25.104030 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:10:25.104030 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:10:25.104030 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:10:25.104030 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:10:25.104969 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:10:25.104969 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:10:25.104969 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:10:25.104969 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:10:25.104969 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:25.104969 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:25.104969 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:25.104969 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:25.104969 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:25.104969 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:25.104969 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:25.104969 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:10:25.104969 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:25.104969 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:25.104969 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:25.104969 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:25.105768 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:25.105768 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:25.105768 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:25.105768 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:25.105768 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:10:25.105768 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:10:25.105768 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:10:25.105768 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:10:25.105768 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:10:25.105768 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:10:25.105768 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:10:25.105768 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:10:25.105768 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 194/395] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:10:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:10:25.124606 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd52f20a-94b5-4998-910a-c8f3dcbe1c68 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:25.250824 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd52f20a-94b5-4998-910a-c8f3dcbe1c68 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:25.299566 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4429ff-16af-417b-aedc-012244706552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:25.300890 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4429ff-16af-417b-aedc-012244706552 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:25.301172 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4429ff-16af-417b-aedc-012244706552 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:25.301535 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4429ff-16af-417b-aedc-012244706552 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:25.302304 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4429ff-16af-417b-aedc-012244706552 None admin] RBAC: Adding query filter params (name=member) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:25.302614 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4429ff-16af-417b-aedc-012244706552 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:25.303980 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4429ff-16af-417b-aedc-012244706552 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:25.313261 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecd16a0-c4e4-4e34-85b7-57f0f389cdcf None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:25.323608 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 195/396] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:10:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:10:25.324555 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4429ff-16af-417b-aedc-012244706552 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:25.385625 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444d7fa1-0435-453a-ac35-ce792c00d5d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:25.387166 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444d7fa1-0435-453a-ac35-ce792c00d5d3 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:25.387664 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444d7fa1-0435-453a-ac35-ce792c00d5d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:25.388141 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444d7fa1-0435-453a-ac35-ce792c00d5d3 None admin] PATH_INFO: `/v3/projects/ed85b374a73a4c94a310482b7ba566f3` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:25.389868 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444d7fa1-0435-453a-ac35-ce792c00d5d3 None admin] RBAC: Authorizing `identity:get_project(project_id=ed85b374a73a4c94a310482b7ba566f3)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:25.391599 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444d7fa1-0435-453a-ac35-ce792c00d5d3 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:25.393923 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 202/397] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:10:25 2022] GET /identity/v3/projects/ed85b374a73a4c94a310482b7ba566f3 => generated 298 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:10:25.402438 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444d7fa1-0435-453a-ac35-ce792c00d5d3 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:25.449431 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:25.498045 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:25.548003 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4429ff-16af-417b-aedc-012244706552 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:25.595840 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444d7fa1-0435-453a-ac35-ce792c00d5d3 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:25.621022 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61131bda-6ff7-49f7-b8e9-1595dfd21513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:25.621921 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61131bda-6ff7-49f7-b8e9-1595dfd21513 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:25.622208 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61131bda-6ff7-49f7-b8e9-1595dfd21513 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:25.622481 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61131bda-6ff7-49f7-b8e9-1595dfd21513 None admin] PATH_INFO: `/v3/groups/53f2efd4d9254ad4a528fceb5a783b5c` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:25.623896 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61131bda-6ff7-49f7-b8e9-1595dfd21513 None admin] RBAC: Authorizing `identity:get_group(group_id=53f2efd4d9254ad4a528fceb5a783b5c)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:25.625366 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61131bda-6ff7-49f7-b8e9-1595dfd21513 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:25.627047 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 196/398] 10.222.0.57 () {56 vars in 1180 bytes} [Thu May 19 14:10:25 2022] GET /identity/v3/groups/53f2efd4d9254ad4a528fceb5a783b5c => generated 226 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:25.634511 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61131bda-6ff7-49f7-b8e9-1595dfd21513 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:25.676215 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb25fad-07cd-458f-818a-59dcc9536452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:25.677114 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb25fad-07cd-458f-818a-59dcc9536452 None admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:25.677397 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb25fad-07cd-458f-818a-59dcc9536452 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:25.677673 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb25fad-07cd-458f-818a-59dcc9536452 None admin] PATH_INFO: `/v3/projects/ed85b374a73a4c94a310482b7ba566f3/groups/53f2efd4d9254ad4a528fceb5a783b5c/roles/7d0e162805b54c019ce6c954cbef3818` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:25.683596 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb25fad-07cd-458f-818a-59dcc9536452 None admin] RBAC: Authorizing `identity:create_grant(project_id=ed85b374a73a4c94a310482b7ba566f3, group_id=53f2efd4d9254ad4a528fceb5a783b5c, role_id=7d0e162805b54c019ce6c954cbef3818)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:25.685060 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb25fad-07cd-458f-818a-59dcc9536452 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:25.709749 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 203/399] 10.222.0.57 () {58 vars in 1442 bytes} [Thu May 19 14:10:25 2022] PUT /identity/v3/projects/ed85b374a73a4c94a310482b7ba566f3/groups/53f2efd4d9254ad4a528fceb5a783b5c/roles/7d0e162805b54c019ce6c954cbef3818 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:10:25.806299 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61131bda-6ff7-49f7-b8e9-1595dfd21513 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:25.842013 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:25.855939 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7000530-8945-4d86-88fb-82b832b5db70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:25.856802 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7000530-8945-4d86-88fb-82b832b5db70 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:25.857084 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7000530-8945-4d86-88fb-82b832b5db70 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:25.857364 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7000530-8945-4d86-88fb-82b832b5db70 None admin] PATH_INFO: `/v3/projects/3c9e67fcd5b14da2a3775ef2b2cad6b2` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:25.863347 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7000530-8945-4d86-88fb-82b832b5db70 None admin] RBAC: Authorizing `identity:get_project(project_id=3c9e67fcd5b14da2a3775ef2b2cad6b2)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:25.865151 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7000530-8945-4d86-88fb-82b832b5db70 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:25.867183 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 197/400] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:10:25 2022] GET /identity/v3/projects/3c9e67fcd5b14da2a3775ef2b2cad6b2 => generated 302 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:25.873085 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb25fad-07cd-458f-818a-59dcc9536452 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:26.005035 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb25fad-07cd-458f-818a-59dcc9536452 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:26.014227 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:26.058709 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caf3eaa-b2a6-4417-9c41-9e12e1c62e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:26.059523 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caf3eaa-b2a6-4417-9c41-9e12e1c62e5c None admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:26.059810 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caf3eaa-b2a6-4417-9c41-9e12e1c62e5c None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:26.060086 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caf3eaa-b2a6-4417-9c41-9e12e1c62e5c None admin] PATH_INFO: `/v3/projects/3c9e67fcd5b14da2a3775ef2b2cad6b2/groups/53f2efd4d9254ad4a528fceb5a783b5c/roles/7d0e162805b54c019ce6c954cbef3818` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:26.062156 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1caf3eaa-b2a6-4417-9c41-9e12e1c62e5c None admin] RBAC: Authorizing `identity:create_grant(project_id=3c9e67fcd5b14da2a3775ef2b2cad6b2, group_id=53f2efd4d9254ad4a528fceb5a783b5c, role_id=7d0e162805b54c019ce6c954cbef3818)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:26.063724 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1caf3eaa-b2a6-4417-9c41-9e12e1c62e5c None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:26.091135 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 204/401] 10.222.0.57 () {58 vars in 1442 bytes} [Thu May 19 14:10:25 2022] PUT /identity/v3/projects/3c9e67fcd5b14da2a3775ef2b2cad6b2/groups/53f2efd4d9254ad4a528fceb5a783b5c/roles/7d0e162805b54c019ce6c954cbef3818 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:10:26.208842 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37103be-7dfb-4fb5-ba3d-9e8d91eec0c4 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:26.209263 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37103be-7dfb-4fb5-ba3d-9e8d91eec0c4 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:26.209553 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37103be-7dfb-4fb5-ba3d-9e8d91eec0c4 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:26.211488 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 198/402] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:10:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:10:26.217182 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a25dcbe-a9ee-425d-85bb-9d11ccdb1207 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:26.217510 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a25dcbe-a9ee-425d-85bb-9d11ccdb1207 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:26.217738 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a25dcbe-a9ee-425d-85bb-9d11ccdb1207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:26.238923 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-8a25dcbe-a9ee-425d-85bb-9d11ccdb1207 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:10:26.240545 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-8a25dcbe-a9ee-425d-85bb-9d11ccdb1207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:10:26.336875 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 205/403] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:10:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:10:26.377411 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37103be-7dfb-4fb5-ba3d-9e8d91eec0c4 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:26.520033 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37103be-7dfb-4fb5-ba3d-9e8d91eec0c4 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:26.555486 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d622532-adf4-4362-848e-5607cbab0682 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:26.555774 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d622532-adf4-4362-848e-5607cbab0682 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:26.556054 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d622532-adf4-4362-848e-5607cbab0682 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:26.557419 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 206/404] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:10:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:10:26.558819 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43642754-4cf5-4bd2-8794-eeac38491648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:26.559834 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43642754-4cf5-4bd2-8794-eeac38491648 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:26.560292 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43642754-4cf5-4bd2-8794-eeac38491648 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:26.560738 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43642754-4cf5-4bd2-8794-eeac38491648 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:26.563368 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3570f64f-2a2c-4de8-8e42-e6a9318b28cc None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:26.563982 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3570f64f-2a2c-4de8-8e42-e6a9318b28cc None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:26.564482 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3570f64f-2a2c-4de8-8e42-e6a9318b28cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:26.572717 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43642754-4cf5-4bd2-8794-eeac38491648 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:10:26.581132 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-43642754-4cf5-4bd2-8794-eeac38491648 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:10:26.581132 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:26.581132 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:26.581132 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:26.581132 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:26.581132 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:26.581132 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:26.581132 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:26.581132 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:26.581132 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:26.581132 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:26.581132 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:26.581132 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:10:26.581132 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:10:26.581132 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:10:26.581132 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:10:26.582871 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:10:26.582871 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:10:26.582871 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:10:26.582871 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:10:26.582871 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:26.582871 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:26.582871 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:26.582871 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:26.582871 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:26.582871 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:26.582871 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:26.582871 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:10:26.582871 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:26.582871 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:26.582871 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:26.582871 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:26.583860 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:26.583860 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:26.583860 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:26.583860 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:26.583860 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:10:26.583860 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:10:26.583860 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:10:26.583860 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:10:26.583860 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:10:26.583860 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:10:26.583860 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:10:26.583860 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:10:26.586281 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 199/405] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 19 14:10:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:10:26.597962 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43642754-4cf5-4bd2-8794-eeac38491648 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:26.599972 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-3570f64f-2a2c-4de8-8e42-e6a9318b28cc None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:10:26.601181 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-3570f64f-2a2c-4de8-8e42-e6a9318b28cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:10:26.694915 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:26.719882 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 207/406] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:10:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:10:26.754761 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43642754-4cf5-4bd2-8794-eeac38491648 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:26.755649 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3570f64f-2a2c-4de8-8e42-e6a9318b28cc None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:26.790394 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33e5b2e-cc65-4cb3-a194-fe46a91c9ccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:26.791431 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33e5b2e-cc65-4cb3-a194-fe46a91c9ccb None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:26.792022 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33e5b2e-cc65-4cb3-a194-fe46a91c9ccb None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:26.792476 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33e5b2e-cc65-4cb3-a194-fe46a91c9ccb None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:26.793360 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33e5b2e-cc65-4cb3-a194-fe46a91c9ccb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:26.793810 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33e5b2e-cc65-4cb3-a194-fe46a91c9ccb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:26.795367 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33e5b2e-cc65-4cb3-a194-fe46a91c9ccb None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:26.804898 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 200/407] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:10:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:26.812222 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33e5b2e-cc65-4cb3-a194-fe46a91c9ccb None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:26.820206 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:26.917930 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3570f64f-2a2c-4de8-8e42-e6a9318b28cc None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:26.920384 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33e5b2e-cc65-4cb3-a194-fe46a91c9ccb None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:26.933862 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:26.961871 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f932283-2db5-4c13-ae6d-8cf73557ce6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:26.961871 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f932283-2db5-4c13-ae6d-8cf73557ce6d None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:26.961871 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f932283-2db5-4c13-ae6d-8cf73557ce6d None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:26.961871 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f932283-2db5-4c13-ae6d-8cf73557ce6d None admin] PATH_INFO: `/v3/groups/53f2efd4d9254ad4a528fceb5a783b5c` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:26.962342 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f932283-2db5-4c13-ae6d-8cf73557ce6d None admin] RBAC: Authorizing `identity:get_group(group_id=53f2efd4d9254ad4a528fceb5a783b5c)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:26.962634 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f932283-2db5-4c13-ae6d-8cf73557ce6d None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:26.964613 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 201/408] 10.222.0.57 () {56 vars in 1180 bytes} [Thu May 19 14:10:26 2022] GET /identity/v3/groups/53f2efd4d9254ad4a528fceb5a783b5c => generated 226 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:10:26.970412 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cbb512-ab49-417f-bb2f-526c94f23e98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:26.971422 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cbb512-ab49-417f-bb2f-526c94f23e98 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:26.971924 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cbb512-ab49-417f-bb2f-526c94f23e98 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:26.972519 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cbb512-ab49-417f-bb2f-526c94f23e98 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:26.974452 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f932283-2db5-4c13-ae6d-8cf73557ce6d None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:26.984920 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cbb512-ab49-417f-bb2f-526c94f23e98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:10:26.993396 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-18cbb512-ab49-417f-bb2f-526c94f23e98 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:10:26.993396 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:26.993396 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:26.993396 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:26.993396 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:26.993396 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:26.993396 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:26.993396 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:26.993396 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:26.993396 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:26.993396 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:26.993396 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:26.993396 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:10:26.993396 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:10:26.993396 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:10:26.993396 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:10:26.994533 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:10:26.994533 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:10:26.994533 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:10:26.994533 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:10:26.994533 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:26.994533 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:26.994533 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:26.994533 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:26.994533 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:26.994533 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:26.994533 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:26.994533 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:10:26.994533 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:26.994533 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:26.994533 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:26.994533 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:26.995404 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:26.995404 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:26.995404 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:26.995404 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:26.995404 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:10:26.995404 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:10:26.995404 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:10:26.995404 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:10:26.995404 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:10:26.995404 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:10:26.995404 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:10:26.995404 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:10:27.002018 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 208/409] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:10:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:10:27.004339 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cbb512-ab49-417f-bb2f-526c94f23e98 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:27.107394 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:27.131952 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cbb512-ab49-417f-bb2f-526c94f23e98 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:27.164001 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f932283-2db5-4c13-ae6d-8cf73557ce6d None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:27.164859 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76767b9-1893-4004-825f-26ab1385f3f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:27.165795 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76767b9-1893-4004-825f-26ab1385f3f2 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:27.166386 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76767b9-1893-4004-825f-26ab1385f3f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:27.166816 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76767b9-1893-4004-825f-26ab1385f3f2 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:27.167734 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76767b9-1893-4004-825f-26ab1385f3f2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:27.168253 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76767b9-1893-4004-825f-26ab1385f3f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:27.169801 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76767b9-1893-4004-825f-26ab1385f3f2 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:27.179191 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 209/410] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:10:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:10:27.184969 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76767b9-1893-4004-825f-26ab1385f3f2 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:27.221864 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55381b87-702c-44a3-b357-85866a2741ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:27.221864 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55381b87-702c-44a3-b357-85866a2741ac None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:27.221864 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55381b87-702c-44a3-b357-85866a2741ac None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:27.221864 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55381b87-702c-44a3-b357-85866a2741ac None admin] PATH_INFO: `/v3/projects/3c9e67fcd5b14da2a3775ef2b2cad6b2` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:27.222300 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55381b87-702c-44a3-b357-85866a2741ac None admin] RBAC: Authorizing `identity:get_project(project_id=3c9e67fcd5b14da2a3775ef2b2cad6b2)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:27.222300 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55381b87-702c-44a3-b357-85866a2741ac None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:27.222676 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 202/411] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:10:26 2022] GET /identity/v3/projects/3c9e67fcd5b14da2a3775ef2b2cad6b2 => generated 302 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:27.229686 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55381b87-702c-44a3-b357-85866a2741ac None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:27.294254 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:27.352953 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55381b87-702c-44a3-b357-85866a2741ac None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:27.395519 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a06537-2f2d-4cc3-9440-49590d41accb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:27.396438 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a06537-2f2d-4cc3-9440-49590d41accb None admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:27.396720 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a06537-2f2d-4cc3-9440-49590d41accb None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:27.396996 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a06537-2f2d-4cc3-9440-49590d41accb None admin] PATH_INFO: `/v3/projects/3c9e67fcd5b14da2a3775ef2b2cad6b2/groups/53f2efd4d9254ad4a528fceb5a783b5c/roles/23d9dd31632340bdba9cf3093c095dbb` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:27.399447 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a06537-2f2d-4cc3-9440-49590d41accb None admin] RBAC: Authorizing `identity:create_grant(project_id=3c9e67fcd5b14da2a3775ef2b2cad6b2, group_id=53f2efd4d9254ad4a528fceb5a783b5c, role_id=23d9dd31632340bdba9cf3093c095dbb)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:27.401582 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a06537-2f2d-4cc3-9440-49590d41accb None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:27.417602 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 203/412] 10.222.0.57 () {58 vars in 1442 bytes} [Thu May 19 14:10:27 2022] PUT /identity/v3/projects/3c9e67fcd5b14da2a3775ef2b2cad6b2/groups/53f2efd4d9254ad4a528fceb5a783b5c/roles/23d9dd31632340bdba9cf3093c095dbb => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 14:10:27.449947 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76767b9-1893-4004-825f-26ab1385f3f2 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:27.491100 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77d41ce-bbf1-4904-b2e2-c73c858c7f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:27.491970 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77d41ce-bbf1-4904-b2e2-c73c858c7f92 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:27.492259 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77d41ce-bbf1-4904-b2e2-c73c858c7f92 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:27.492535 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77d41ce-bbf1-4904-b2e2-c73c858c7f92 None admin] PATH_INFO: `/v3/groups/94ff5f55c6904ba1bb185d07d5d8607e` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:27.493736 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77d41ce-bbf1-4904-b2e2-c73c858c7f92 None admin] RBAC: Authorizing `identity:get_group(group_id=94ff5f55c6904ba1bb185d07d5d8607e)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:27.495334 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77d41ce-bbf1-4904-b2e2-c73c858c7f92 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:27.497283 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 210/413] 10.222.0.57 () {56 vars in 1180 bytes} [Thu May 19 14:10:27 2022] GET /identity/v3/groups/94ff5f55c6904ba1bb185d07d5d8607e => generated 229 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:10:27.502807 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a06537-2f2d-4cc3-9440-49590d41accb None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:27.626988 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a06537-2f2d-4cc3-9440-49590d41accb None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:27.628770 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178b7ad7-5b23-4e08-974a-e22eff705eed None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:27.629060 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178b7ad7-5b23-4e08-974a-e22eff705eed None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:27.629342 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178b7ad7-5b23-4e08-974a-e22eff705eed None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:27.630737 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 211/414] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:10:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:10:27.636934 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5617a89-10d4-487d-9418-e3e6d22ce8a1 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:27.637308 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5617a89-10d4-487d-9418-e3e6d22ce8a1 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:27.637598 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5617a89-10d4-487d-9418-e3e6d22ce8a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:27.662219 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:27.668242 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-f5617a89-10d4-487d-9418-e3e6d22ce8a1 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:10:27.669477 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-f5617a89-10d4-487d-9418-e3e6d22ce8a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:10:27.696721 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:27.701404 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd8e8f7-3f31-4272-8d7c-eae2db2d050a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:27.702952 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd8e8f7-3f31-4272-8d7c-eae2db2d050a None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:27.703385 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd8e8f7-3f31-4272-8d7c-eae2db2d050a None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:27.704264 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd8e8f7-3f31-4272-8d7c-eae2db2d050a None admin] PATH_INFO: `/v3/projects/dc6295ed5b1f4894a42c16df5aa39db4` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:27.706635 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd8e8f7-3f31-4272-8d7c-eae2db2d050a None admin] RBAC: Authorizing `identity:get_project(project_id=dc6295ed5b1f4894a42c16df5aa39db4)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:27.708675 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd8e8f7-3f31-4272-8d7c-eae2db2d050a None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:27.711218 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 204/415] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:10:27 2022] GET /identity/v3/projects/dc6295ed5b1f4894a42c16df5aa39db4 => generated 344 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:27.718763 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd8e8f7-3f31-4272-8d7c-eae2db2d050a None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:27.810251 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 212/416] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:10:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:10:27.837020 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5617a89-10d4-487d-9418-e3e6d22ce8a1 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:27.853691 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd8e8f7-3f31-4272-8d7c-eae2db2d050a None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:27.895991 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0079f0-c1c7-4290-acd4-6ecdab1b223d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:27.897201 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0079f0-c1c7-4290-acd4-6ecdab1b223d None admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:27.897661 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0079f0-c1c7-4290-acd4-6ecdab1b223d None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:27.898150 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0079f0-c1c7-4290-acd4-6ecdab1b223d None admin] PATH_INFO: `/v3/projects/dc6295ed5b1f4894a42c16df5aa39db4/groups/94ff5f55c6904ba1bb185d07d5d8607e/roles/4946cf0ff7164307b36efeee7c787146` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:27.903675 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0079f0-c1c7-4290-acd4-6ecdab1b223d None admin] RBAC: Authorizing `identity:create_grant(project_id=dc6295ed5b1f4894a42c16df5aa39db4, group_id=94ff5f55c6904ba1bb185d07d5d8607e, role_id=4946cf0ff7164307b36efeee7c787146)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:27.905464 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0079f0-c1c7-4290-acd4-6ecdab1b223d None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:27.919960 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 205/417] 10.222.0.57 () {58 vars in 1442 bytes} [Thu May 19 14:10:27 2022] PUT /identity/v3/projects/dc6295ed5b1f4894a42c16df5aa39db4/groups/94ff5f55c6904ba1bb185d07d5d8607e/roles/4946cf0ff7164307b36efeee7c787146 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 14:10:28.003704 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5617a89-10d4-487d-9418-e3e6d22ce8a1 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:28.041520 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73380f7f-ce53-48e3-80f9-7e3a5c556b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:28.042679 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73380f7f-ce53-48e3-80f9-7e3a5c556b44 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:28.043198 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73380f7f-ce53-48e3-80f9-7e3a5c556b44 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:28.043658 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73380f7f-ce53-48e3-80f9-7e3a5c556b44 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:28.056572 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73380f7f-ce53-48e3-80f9-7e3a5c556b44 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:10:28.065261 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-73380f7f-ce53-48e3-80f9-7e3a5c556b44 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:10:28.065261 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:28.065261 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:28.065261 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:28.065261 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:28.065261 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:28.065261 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:28.065261 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:28.065261 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:28.065261 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:28.065261 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:28.065261 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:28.065261 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:10:28.065261 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:10:28.065261 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:10:28.065261 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:10:28.066819 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:10:28.066819 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:10:28.066819 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:10:28.066819 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:10:28.066819 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:28.066819 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:28.066819 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:28.066819 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:28.066819 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:28.066819 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:28.066819 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:28.066819 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:10:28.066819 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:28.066819 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:28.066819 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:28.066819 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:28.067712 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:28.067712 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:28.067712 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:28.067712 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:28.067712 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:10:28.067712 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:10:28.067712 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:10:28.067712 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:10:28.067712 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:10:28.067712 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:10:28.067712 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:10:28.067712 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:10:28.073942 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 213/418] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 19 14:10:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:10:28.076487 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0079f0-c1c7-4290-acd4-6ecdab1b223d None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:28.122243 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:28.192826 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0079f0-c1c7-4290-acd4-6ecdab1b223d None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:28.230468 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dbab06-37b2-4b53-8e51-858f3989ab05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:28.231148 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dbab06-37b2-4b53-8e51-858f3989ab05 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:28.231431 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dbab06-37b2-4b53-8e51-858f3989ab05 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:28.231709 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dbab06-37b2-4b53-8e51-858f3989ab05 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:28.232461 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dbab06-37b2-4b53-8e51-858f3989ab05 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:28.232772 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dbab06-37b2-4b53-8e51-858f3989ab05 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:28.234227 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dbab06-37b2-4b53-8e51-858f3989ab05 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:28.247241 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 206/419] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:10:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:28.250182 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73380f7f-ce53-48e3-80f9-7e3a5c556b44 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:28.254555 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:28.366371 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73380f7f-ce53-48e3-80f9-7e3a5c556b44 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:28.411026 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb4f379-6114-4a33-bcda-97238e9e7b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:28.411686 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb4f379-6114-4a33-bcda-97238e9e7b24 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:28.411972 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb4f379-6114-4a33-bcda-97238e9e7b24 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:28.412247 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb4f379-6114-4a33-bcda-97238e9e7b24 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:28.510282 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb4f379-6114-4a33-bcda-97238e9e7b24 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:28.511728 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb4f379-6114-4a33-bcda-97238e9e7b24 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:28.514999 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:28.645280 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dbab06-37b2-4b53-8e51-858f3989ab05 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:28.646156 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 214/420] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:10:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:10:28.748999 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08209e63-7209-4f87-99e4-9699c2d2fd6e None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:28.749316 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08209e63-7209-4f87-99e4-9699c2d2fd6e None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:28.749568 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08209e63-7209-4f87-99e4-9699c2d2fd6e None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:28.751220 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 215/421] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:10:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:10:28.757251 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aac98a9-892d-4581-a753-e69ff93ba3ea None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:28.757573 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aac98a9-892d-4581-a753-e69ff93ba3ea None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:28.757824 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aac98a9-892d-4581-a753-e69ff93ba3ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:28.788513 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-1aac98a9-892d-4581-a753-e69ff93ba3ea None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:10:28.789636 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-1aac98a9-892d-4581-a753-e69ff93ba3ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:10:28.791066 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:28.795757 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dbab06-37b2-4b53-8e51-858f3989ab05 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:28.844096 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442e0d37-a9cf-44fb-9c77-949640487ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:28.845143 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442e0d37-a9cf-44fb-9c77-949640487ea4 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:28.845440 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442e0d37-a9cf-44fb-9c77-949640487ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:28.845752 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442e0d37-a9cf-44fb-9c77-949640487ea4 None admin] PATH_INFO: `/v3/projects/ed85b374a73a4c94a310482b7ba566f3` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:28.849272 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442e0d37-a9cf-44fb-9c77-949640487ea4 None admin] RBAC: Authorizing `identity:get_project(project_id=ed85b374a73a4c94a310482b7ba566f3)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:28.849572 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442e0d37-a9cf-44fb-9c77-949640487ea4 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:28.851475 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 207/422] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:10:28 2022] GET /identity/v3/projects/ed85b374a73a4c94a310482b7ba566f3 => generated 298 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:10:28.855857 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdb28ad-40ca-457e-9f0a-a494d636984a None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:28.855857 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdb28ad-40ca-457e-9f0a-a494d636984a None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:28.855857 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdb28ad-40ca-457e-9f0a-a494d636984a None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:28.856373 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 208/423] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:10:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:10:28.858728 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdb28ad-40ca-457e-9f0a-a494d636984a None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:28.907012 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 216/424] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:10:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:10:28.907012 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580b3daf-ecf6-45e4-98b6-7a254aec0e20 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:28.907012 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580b3daf-ecf6-45e4-98b6-7a254aec0e20 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:28.907012 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580b3daf-ecf6-45e4-98b6-7a254aec0e20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:28.933824 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-580b3daf-ecf6-45e4-98b6-7a254aec0e20 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:10:28.935050 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-580b3daf-ecf6-45e4-98b6-7a254aec0e20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:10:28.970231 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:28.997322 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdb28ad-40ca-457e-9f0a-a494d636984a None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:29.017636 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 217/425] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:10:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:10:29.019792 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580b3daf-ecf6-45e4-98b6-7a254aec0e20 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:29.058496 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d66a624-5547-467e-a1f0-7087cf8096d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:29.059293 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d66a624-5547-467e-a1f0-7087cf8096d1 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:29.059577 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d66a624-5547-467e-a1f0-7087cf8096d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:29.059851 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d66a624-5547-467e-a1f0-7087cf8096d1 None admin] PATH_INFO: `/v3/groups/53f2efd4d9254ad4a528fceb5a783b5c` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:29.061427 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d66a624-5547-467e-a1f0-7087cf8096d1 None admin] RBAC: Authorizing `identity:get_group(group_id=53f2efd4d9254ad4a528fceb5a783b5c)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:29.063222 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d66a624-5547-467e-a1f0-7087cf8096d1 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:29.064830 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 209/426] 10.222.0.57 () {56 vars in 1180 bytes} [Thu May 19 14:10:28 2022] GET /identity/v3/groups/53f2efd4d9254ad4a528fceb5a783b5c => generated 226 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:29.070225 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d66a624-5547-467e-a1f0-7087cf8096d1 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:29.129707 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:29.153736 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580b3daf-ecf6-45e4-98b6-7a254aec0e20 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:29.199649 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17252799-bd6b-423b-a9d6-3935b0f06080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:29.200708 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17252799-bd6b-423b-a9d6-3935b0f06080 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:29.201005 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17252799-bd6b-423b-a9d6-3935b0f06080 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:29.201284 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17252799-bd6b-423b-a9d6-3935b0f06080 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:29.209608 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d66a624-5547-467e-a1f0-7087cf8096d1 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:29.215145 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17252799-bd6b-423b-a9d6-3935b0f06080 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:10:29.226272 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-17252799-bd6b-423b-a9d6-3935b0f06080 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:10:29.226272 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:29.226272 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:29.226272 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:29.226272 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:29.226272 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:29.226272 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:29.226272 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:29.226272 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:29.226272 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:29.226272 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:29.226272 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:29.226272 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:10:29.226272 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:10:29.226272 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:10:29.226272 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:10:29.227186 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:10:29.227186 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:10:29.227186 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:10:29.227186 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:10:29.227186 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:29.227186 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:29.227186 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:29.227186 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:29.227186 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:29.227186 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:29.227186 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:29.227186 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:10:29.227186 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:29.227186 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:29.227186 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:29.227186 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:29.227988 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:29.227988 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:29.227988 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:29.227988 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:29.227988 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:10:29.227988 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:10:29.227988 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:10:29.227988 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:10:29.227988 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:10:29.227988 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:10:29.227988 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:10:29.227988 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:10:29.229917 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 218/427] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:10:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:10:29.230934 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17252799-bd6b-423b-a9d6-3935b0f06080 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:29.255456 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f51962-f101-49f8-96c6-4e72aa65a1eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:29.256257 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f51962-f101-49f8-96c6-4e72aa65a1eb None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:29.256541 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f51962-f101-49f8-96c6-4e72aa65a1eb None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:29.256817 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f51962-f101-49f8-96c6-4e72aa65a1eb None admin] PATH_INFO: `/v3/roles/member` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:29.268327 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f51962-f101-49f8-96c6-4e72aa65a1eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:10:29.269227 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:29.278711 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-a9f51962-f101-49f8-96c6-4e72aa65a1eb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:10:29.278711 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:29.278711 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:29.278711 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:29.278711 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:29.278711 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:29.278711 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:29.278711 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:29.278711 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:29.278711 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:29.278711 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:29.278711 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:29.278711 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:10:29.278711 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:10:29.278711 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:10:29.278711 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:10:29.279643 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:10:29.279643 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:10:29.279643 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:10:29.279643 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:10:29.279643 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:29.279643 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:29.279643 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:29.279643 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:29.279643 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:29.279643 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:29.279643 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:29.279643 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:10:29.279643 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:29.279643 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:29.279643 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:29.279643 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:29.280457 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:29.280457 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:29.280457 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:29.280457 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:29.280457 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:10:29.280457 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:10:29.280457 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:10:29.280457 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:10:29.280457 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:10:29.280457 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:10:29.280457 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:10:29.280457 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:10:29.288520 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 210/428] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:10:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:10:29.289614 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f51962-f101-49f8-96c6-4e72aa65a1eb None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:29.366136 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17252799-bd6b-423b-a9d6-3935b0f06080 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:29.404386 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ebaef2-a9d9-4e63-8184-a65ae025287b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:29.405491 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ebaef2-a9d9-4e63-8184-a65ae025287b None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:29.406083 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ebaef2-a9d9-4e63-8184-a65ae025287b None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:29.406563 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ebaef2-a9d9-4e63-8184-a65ae025287b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:29.407023 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:29.408894 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ebaef2-a9d9-4e63-8184-a65ae025287b None admin] RBAC: Adding query filter params (group.id=53f2efd4d9254ad4a528fceb5a783b5c, role.id=23d9dd31632340bdba9cf3093c095dbb, scope.project.id=ed85b374a73a4c94a310482b7ba566f3) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:29.409456 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ebaef2-a9d9-4e63-8184-a65ae025287b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:29.412743 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ebaef2-a9d9-4e63-8184-a65ae025287b None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:29.414651 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f51962-f101-49f8-96c6-4e72aa65a1eb None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:29.425929 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 219/429] 10.222.0.57 () {56 vars in 1376 bytes} [Thu May 19 14:10:29 2022] GET /identity/v3/role_assignments?group.id=53f2efd4d9254ad4a528fceb5a783b5c&scope.project.id=ed85b374a73a4c94a310482b7ba566f3&role.id=23d9dd31632340bdba9cf3093c095dbb => generated 611 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:29.427139 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ebaef2-a9d9-4e63-8184-a65ae025287b None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:29.485299 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7543ad-1870-4e7b-8657-9babdd7dae81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:29.486402 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7543ad-1870-4e7b-8657-9babdd7dae81 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:29.486996 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7543ad-1870-4e7b-8657-9babdd7dae81 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:29.487742 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7543ad-1870-4e7b-8657-9babdd7dae81 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:29.488723 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7543ad-1870-4e7b-8657-9babdd7dae81 None admin] RBAC: Adding query filter params (name=member) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:29.489232 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7543ad-1870-4e7b-8657-9babdd7dae81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:29.491327 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7543ad-1870-4e7b-8657-9babdd7dae81 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:29.519922 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 211/430] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:10:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:29.527232 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7543ad-1870-4e7b-8657-9babdd7dae81 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:29.647384 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:29.663994 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ebaef2-a9d9-4e63-8184-a65ae025287b None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:29.704149 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c12b7f-8614-422c-beca-e831035634d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:29.704831 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c12b7f-8614-422c-beca-e831035634d3 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:29.705120 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c12b7f-8614-422c-beca-e831035634d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:29.705399 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c12b7f-8614-422c-beca-e831035634d3 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:29.706796 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c12b7f-8614-422c-beca-e831035634d3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:29.707197 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c12b7f-8614-422c-beca-e831035634d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:29.707326 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7543ad-1870-4e7b-8657-9babdd7dae81 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:29.708647 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c12b7f-8614-422c-beca-e831035634d3 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:29.721865 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 220/431] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:10:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:29.724581 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c12b7f-8614-422c-beca-e831035634d3 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:29.755786 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5756d1-689c-48dc-9287-90fd0201b8d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:29.757377 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5756d1-689c-48dc-9287-90fd0201b8d7 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:29.757901 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5756d1-689c-48dc-9287-90fd0201b8d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:29.758667 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5756d1-689c-48dc-9287-90fd0201b8d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:29.793769 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:29.847080 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5756d1-689c-48dc-9287-90fd0201b8d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:29.848384 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c12b7f-8614-422c-beca-e831035634d3 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:29.849199 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5756d1-689c-48dc-9287-90fd0201b8d7 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:29.886809 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1858d3-9e24-4b82-bbaf-edd0fc5699d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:29.887587 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1858d3-9e24-4b82-bbaf-edd0fc5699d2 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:29.888043 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1858d3-9e24-4b82-bbaf-edd0fc5699d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:29.888269 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1858d3-9e24-4b82-bbaf-edd0fc5699d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:29.910043 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:29.963740 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1858d3-9e24-4b82-bbaf-edd0fc5699d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:29.965489 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1858d3-9e24-4b82-bbaf-edd0fc5699d2 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:29.973401 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 212/432] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:10:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:10:29.975102 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5756d1-689c-48dc-9287-90fd0201b8d7 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:30.033979 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:30.085887 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 221/433] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:10:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:10:30.091473 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5756d1-689c-48dc-9287-90fd0201b8d7 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:30.094198 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1858d3-9e24-4b82-bbaf-edd0fc5699d2 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:30.138487 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f22da22-2c90-4099-bc26-9ea5ac2d80aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:30.139716 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f22da22-2c90-4099-bc26-9ea5ac2d80aa None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:30.140202 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f22da22-2c90-4099-bc26-9ea5ac2d80aa None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:30.140678 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f22da22-2c90-4099-bc26-9ea5ac2d80aa None admin] PATH_INFO: `/v3/projects/ed85b374a73a4c94a310482b7ba566f3` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:30.142107 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f22da22-2c90-4099-bc26-9ea5ac2d80aa None admin] RBAC: Authorizing `identity:get_project(project_id=ed85b374a73a4c94a310482b7ba566f3)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:30.144455 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f22da22-2c90-4099-bc26-9ea5ac2d80aa None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:30.150897 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 213/434] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:10:29 2022] GET /identity/v3/projects/ed85b374a73a4c94a310482b7ba566f3 => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:10:30.152506 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f22da22-2c90-4099-bc26-9ea5ac2d80aa None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:30.154483 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:30.210122 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1858d3-9e24-4b82-bbaf-edd0fc5699d2 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:30.247389 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7da8f17-c202-4e8c-ac28-1b2916d365d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:30.248255 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7da8f17-c202-4e8c-ac28-1b2916d365d4 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:30.248542 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7da8f17-c202-4e8c-ac28-1b2916d365d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:30.248820 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7da8f17-c202-4e8c-ac28-1b2916d365d4 None admin] PATH_INFO: `/v3/projects/3c9e67fcd5b14da2a3775ef2b2cad6b2` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:30.250058 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7da8f17-c202-4e8c-ac28-1b2916d365d4 None admin] RBAC: Authorizing `identity:get_project(project_id=3c9e67fcd5b14da2a3775ef2b2cad6b2)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:30.251528 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7da8f17-c202-4e8c-ac28-1b2916d365d4 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:30.253699 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 222/435] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:10:30 2022] GET /identity/v3/projects/3c9e67fcd5b14da2a3775ef2b2cad6b2 => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:30.259495 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7da8f17-c202-4e8c-ac28-1b2916d365d4 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:30.273024 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f22da22-2c90-4099-bc26-9ea5ac2d80aa None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:30.303575 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:30.316383 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13982aa-57fe-46d9-a0a7-5a8f19a5859b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:30.317195 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13982aa-57fe-46d9-a0a7-5a8f19a5859b None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:30.317479 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13982aa-57fe-46d9-a0a7-5a8f19a5859b None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:30.317760 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13982aa-57fe-46d9-a0a7-5a8f19a5859b None admin] PATH_INFO: `/v3/groups/53f2efd4d9254ad4a528fceb5a783b5c` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:30.319504 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13982aa-57fe-46d9-a0a7-5a8f19a5859b None admin] RBAC: Authorizing `identity:get_group(group_id=53f2efd4d9254ad4a528fceb5a783b5c)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:30.321068 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13982aa-57fe-46d9-a0a7-5a8f19a5859b None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:30.323143 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 214/436] 10.222.0.57 () {56 vars in 1180 bytes} [Thu May 19 14:10:30 2022] GET /identity/v3/groups/53f2efd4d9254ad4a528fceb5a783b5c => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:10:30.328731 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92386b1f-c684-4551-9d5a-3daff62b47e9 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:30.329028 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92386b1f-c684-4551-9d5a-3daff62b47e9 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:30.329312 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92386b1f-c684-4551-9d5a-3daff62b47e9 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:30.330741 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 215/437] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:10:30 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:10:30.331953 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92386b1f-c684-4551-9d5a-3daff62b47e9 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:30.373536 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7da8f17-c202-4e8c-ac28-1b2916d365d4 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:30.391892 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:30.412406 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff315daa-82d7-461c-b61f-7116db7f693f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:30.413269 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff315daa-82d7-461c-b61f-7116db7f693f None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:30.413557 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff315daa-82d7-461c-b61f-7116db7f693f None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:30.413835 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff315daa-82d7-461c-b61f-7116db7f693f None admin] PATH_INFO: `/v3/groups/53f2efd4d9254ad4a528fceb5a783b5c` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:30.415063 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff315daa-82d7-461c-b61f-7116db7f693f None admin] RBAC: Authorizing `identity:get_group(group_id=53f2efd4d9254ad4a528fceb5a783b5c)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:30.416516 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff315daa-82d7-461c-b61f-7116db7f693f None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:30.418534 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 223/438] 10.222.0.57 () {56 vars in 1180 bytes} [Thu May 19 14:10:30 2022] GET /identity/v3/groups/53f2efd4d9254ad4a528fceb5a783b5c => generated 226 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:30.419819 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1b69c9-3c9a-4b3a-8bd3-ced9d6c736b1 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:30.420115 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1b69c9-3c9a-4b3a-8bd3-ced9d6c736b1 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:30.420397 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1b69c9-3c9a-4b3a-8bd3-ced9d6c736b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:30.437689 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-ee1b69c9-3c9a-4b3a-8bd3-ced9d6c736b1 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:10:30.438967 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-ee1b69c9-3c9a-4b3a-8bd3-ced9d6c736b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:10:30.445630 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92386b1f-c684-4551-9d5a-3daff62b47e9 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:30.488360 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbbbed7-0fb4-4d19-b144-a05adb28cdef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:30.488946 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbbbed7-0fb4-4d19-b144-a05adb28cdef None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:30.489233 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbbbed7-0fb4-4d19-b144-a05adb28cdef None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:30.489515 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbbbed7-0fb4-4d19-b144-a05adb28cdef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:30.490657 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbbbed7-0fb4-4d19-b144-a05adb28cdef None admin] RBAC: Adding query filter params (group.id=53f2efd4d9254ad4a528fceb5a783b5c, role.id=7d0e162805b54c019ce6c954cbef3818, scope.project.id=ed85b374a73a4c94a310482b7ba566f3) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:30.490975 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbbbed7-0fb4-4d19-b144-a05adb28cdef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:30.492412 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbbbed7-0fb4-4d19-b144-a05adb28cdef None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:30.500071 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 216/439] 10.222.0.57 () {56 vars in 1376 bytes} [Thu May 19 14:10:30 2022] GET /identity/v3/role_assignments?group.id=53f2efd4d9254ad4a528fceb5a783b5c&scope.project.id=ed85b374a73a4c94a310482b7ba566f3&role.id=7d0e162805b54c019ce6c954cbef3818 => generated 611 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:30.500784 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbbbed7-0fb4-4d19-b144-a05adb28cdef None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:30.524690 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 224/440] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:10:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:10:30.548449 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1b69c9-3c9a-4b3a-8bd3-ced9d6c736b1 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:30.582914 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:30.648980 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbbbed7-0fb4-4d19-b144-a05adb28cdef None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:30.685191 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a287460-174e-4302-bff0-305dfda7abe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:30.685892 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a287460-174e-4302-bff0-305dfda7abe8 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:30.686174 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a287460-174e-4302-bff0-305dfda7abe8 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:30.686455 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a287460-174e-4302-bff0-305dfda7abe8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:30.687299 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a287460-174e-4302-bff0-305dfda7abe8 None admin] RBAC: Adding query filter params (group.id=53f2efd4d9254ad4a528fceb5a783b5c, role.id=7d0e162805b54c019ce6c954cbef3818, scope.project.id=3c9e67fcd5b14da2a3775ef2b2cad6b2) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:30.687618 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a287460-174e-4302-bff0-305dfda7abe8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:30.689070 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a287460-174e-4302-bff0-305dfda7abe8 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:30.705872 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 217/441] 10.222.0.57 () {56 vars in 1376 bytes} [Thu May 19 14:10:30 2022] GET /identity/v3/role_assignments?group.id=53f2efd4d9254ad4a528fceb5a783b5c&scope.project.id=3c9e67fcd5b14da2a3775ef2b2cad6b2&role.id=7d0e162805b54c019ce6c954cbef3818 => generated 611 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:30.705872 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0ba90d-77a6-474b-bd30-7fae467a6f5b None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:30.705872 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0ba90d-77a6-474b-bd30-7fae467a6f5b None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:30.705872 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0ba90d-77a6-474b-bd30-7fae467a6f5b None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:30.705872 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 218/442] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:10:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:10:30.705872 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3440aa-9ef2-4c4f-ba74-8d469e65326e None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:30.706998 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3440aa-9ef2-4c4f-ba74-8d469e65326e None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:30.706998 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3440aa-9ef2-4c4f-ba74-8d469e65326e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:30.711113 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1b69c9-3c9a-4b3a-8bd3-ced9d6c736b1 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:30.727214 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-3d3440aa-9ef2-4c4f-ba74-8d469e65326e None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:10:30.729273 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-3d3440aa-9ef2-4c4f-ba74-8d469e65326e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:10:30.740924 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:30.770945 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102049d3-3905-4056-9fb3-6574f1cd49e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:30.772529 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102049d3-3905-4056-9fb3-6574f1cd49e9 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:30.772828 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102049d3-3905-4056-9fb3-6574f1cd49e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:30.773113 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102049d3-3905-4056-9fb3-6574f1cd49e9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:30.788009 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102049d3-3905-4056-9fb3-6574f1cd49e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:10:30.797110 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-102049d3-3905-4056-9fb3-6574f1cd49e9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:10:30.797110 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:30.797110 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:30.797110 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:30.797110 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:30.797110 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:30.797110 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:30.797110 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:30.797110 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:30.797110 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:30.797110 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:30.797110 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:30.797110 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:10:30.797110 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:10:30.797110 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:10:30.797110 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:10:30.798186 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:10:30.798186 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:10:30.798186 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:10:30.798186 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:10:30.798186 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:30.798186 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:30.798186 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:30.798186 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:30.798186 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:30.798186 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:30.798186 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:30.798186 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:10:30.798186 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:30.798186 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:30.798186 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:30.798186 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:30.798991 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:30.798991 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:30.798991 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:30.798991 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:30.798991 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:10:30.798991 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:10:30.798991 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:10:30.798991 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:10:30.798991 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:10:30.798991 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:10:30.798991 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:10:30.798991 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:10:30.804097 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 225/443] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 19 14:10:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:10:30.804987 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102049d3-3905-4056-9fb3-6574f1cd49e9 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:30.829982 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 219/444] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:10:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:10:30.856748 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3440aa-9ef2-4c4f-ba74-8d469e65326e None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:30.924307 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:30.944328 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102049d3-3905-4056-9fb3-6574f1cd49e9 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:30.986247 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dec45e-fd6a-48db-a815-6f1582fbe27f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:30.986919 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dec45e-fd6a-48db-a815-6f1582fbe27f None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:30.987198 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dec45e-fd6a-48db-a815-6f1582fbe27f None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:30.988006 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dec45e-fd6a-48db-a815-6f1582fbe27f None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:30.989329 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dec45e-fd6a-48db-a815-6f1582fbe27f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:30.989638 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dec45e-fd6a-48db-a815-6f1582fbe27f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:30.991381 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dec45e-fd6a-48db-a815-6f1582fbe27f None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:30.992029 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3440aa-9ef2-4c4f-ba74-8d469e65326e None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:31.000585 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 226/445] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:10:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:31.007139 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dec45e-fd6a-48db-a815-6f1582fbe27f None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:31.036735 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c243b3c-0a65-45d7-ac8c-f33a8efbe1e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:31.037810 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c243b3c-0a65-45d7-ac8c-f33a8efbe1e6 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:31.038125 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c243b3c-0a65-45d7-ac8c-f33a8efbe1e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:31.038404 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c243b3c-0a65-45d7-ac8c-f33a8efbe1e6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:31.051254 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c243b3c-0a65-45d7-ac8c-f33a8efbe1e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:10:31.057205 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:31.059691 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-0c243b3c-0a65-45d7-ac8c-f33a8efbe1e6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:10:31.059691 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:31.059691 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:31.059691 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:31.059691 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:31.059691 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:31.059691 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:31.059691 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:31.059691 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:31.059691 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:31.059691 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:31.059691 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:31.059691 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:10:31.059691 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:10:31.059691 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:10:31.059691 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:10:31.060595 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:10:31.060595 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:10:31.060595 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:10:31.060595 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:10:31.060595 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:31.060595 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:31.060595 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:31.060595 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:31.060595 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:31.060595 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:31.060595 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:31.060595 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:10:31.060595 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:31.060595 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:31.060595 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:31.060595 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:31.061398 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:31.061398 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:31.061398 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:31.061398 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:31.061398 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:10:31.061398 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:10:31.061398 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:10:31.061398 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:10:31.061398 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:10:31.061398 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:10:31.061398 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:10:31.061398 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:10:31.061398 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 220/446] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:10:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:10:31.067581 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c243b3c-0a65-45d7-ac8c-f33a8efbe1e6 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:31.128983 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dec45e-fd6a-48db-a815-6f1582fbe27f None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:31.169665 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eba957-8955-450d-bb8d-cf8988b14779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:31.170457 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eba957-8955-450d-bb8d-cf8988b14779 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:31.170814 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eba957-8955-450d-bb8d-cf8988b14779 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:31.171104 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eba957-8955-450d-bb8d-cf8988b14779 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:31.194177 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c243b3c-0a65-45d7-ac8c-f33a8efbe1e6 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:31.207986 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:31.236765 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3631cb16-a656-43d6-8007-ca3e8526820c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:31.238171 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3631cb16-a656-43d6-8007-ca3e8526820c None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:31.238471 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3631cb16-a656-43d6-8007-ca3e8526820c None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:31.238932 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3631cb16-a656-43d6-8007-ca3e8526820c None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:31.239689 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3631cb16-a656-43d6-8007-ca3e8526820c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:31.240264 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3631cb16-a656-43d6-8007-ca3e8526820c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:31.241711 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3631cb16-a656-43d6-8007-ca3e8526820c None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:31.254659 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 221/447] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:10:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:10:31.258980 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3eba957-8955-450d-bb8d-cf8988b14779 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:31.260336 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3631cb16-a656-43d6-8007-ca3e8526820c None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:31.260648 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3eba957-8955-450d-bb8d-cf8988b14779 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:31.305669 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:31.377366 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 227/448] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:10:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 14:10:31.380158 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3631cb16-a656-43d6-8007-ca3e8526820c None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:31.383146 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eba957-8955-450d-bb8d-cf8988b14779 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:31.414286 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae88f1a-5cf9-4f12-9eeb-1947928bdb38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:31.414931 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae88f1a-5cf9-4f12-9eeb-1947928bdb38 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:31.415217 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae88f1a-5cf9-4f12-9eeb-1947928bdb38 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:31.415499 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae88f1a-5cf9-4f12-9eeb-1947928bdb38 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:31.455795 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:31.480390 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eba957-8955-450d-bb8d-cf8988b14779 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:31.485435 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae88f1a-5cf9-4f12-9eeb-1947928bdb38 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:31.486939 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae88f1a-5cf9-4f12-9eeb-1947928bdb38 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:31.513956 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947be9d6-3696-45d2-91a4-8daf0aa00862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:31.514742 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947be9d6-3696-45d2-91a4-8daf0aa00862 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:31.515026 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947be9d6-3696-45d2-91a4-8daf0aa00862 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:31.515301 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947be9d6-3696-45d2-91a4-8daf0aa00862 None admin] PATH_INFO: `/v3/projects/3c9e67fcd5b14da2a3775ef2b2cad6b2` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:31.516469 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947be9d6-3696-45d2-91a4-8daf0aa00862 None admin] RBAC: Authorizing `identity:get_project(project_id=3c9e67fcd5b14da2a3775ef2b2cad6b2)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:31.518006 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947be9d6-3696-45d2-91a4-8daf0aa00862 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:31.519748 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 228/449] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:10:31 2022] GET /identity/v3/projects/3c9e67fcd5b14da2a3775ef2b2cad6b2 => generated 302 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:10:31.525087 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947be9d6-3696-45d2-91a4-8daf0aa00862 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:31.533472 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:31.588380 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 222/450] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:10:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 14:10:31.594183 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae88f1a-5cf9-4f12-9eeb-1947928bdb38 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:31.627215 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947be9d6-3696-45d2-91a4-8daf0aa00862 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:31.662450 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa4af22-6a95-4d54-bd3e-b4d537e176ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:31.663299 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa4af22-6a95-4d54-bd3e-b4d537e176ff None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:31.663579 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa4af22-6a95-4d54-bd3e-b4d537e176ff None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:31.663860 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa4af22-6a95-4d54-bd3e-b4d537e176ff None admin] PATH_INFO: `/v3/groups/53f2efd4d9254ad4a528fceb5a783b5c` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:31.664703 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:31.665017 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa4af22-6a95-4d54-bd3e-b4d537e176ff None admin] RBAC: Authorizing `identity:get_group(group_id=53f2efd4d9254ad4a528fceb5a783b5c)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:31.666725 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa4af22-6a95-4d54-bd3e-b4d537e176ff None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:31.668410 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 229/451] 10.222.0.57 () {56 vars in 1180 bytes} [Thu May 19 14:10:31 2022] GET /identity/v3/groups/53f2efd4d9254ad4a528fceb5a783b5c => generated 226 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:10:31.674015 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa4af22-6a95-4d54-bd3e-b4d537e176ff None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:31.695223 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae88f1a-5cf9-4f12-9eeb-1947928bdb38 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:31.730464 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d97c90-4701-4903-a6e3-73a1e7efea18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:31.731322 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d97c90-4701-4903-a6e3-73a1e7efea18 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:31.731603 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d97c90-4701-4903-a6e3-73a1e7efea18 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:31.731875 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d97c90-4701-4903-a6e3-73a1e7efea18 None admin] PATH_INFO: `/v3/projects/dc6295ed5b1f4894a42c16df5aa39db4` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:31.733056 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d97c90-4701-4903-a6e3-73a1e7efea18 None admin] RBAC: Authorizing `identity:get_project(project_id=dc6295ed5b1f4894a42c16df5aa39db4)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:31.734535 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d97c90-4701-4903-a6e3-73a1e7efea18 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:31.736207 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 223/452] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:10:31 2022] GET /identity/v3/projects/dc6295ed5b1f4894a42c16df5aa39db4 => generated 344 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:10:31.742457 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d97c90-4701-4903-a6e3-73a1e7efea18 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:31.750647 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:31.776543 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa4af22-6a95-4d54-bd3e-b4d537e176ff None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:31.810849 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69fb82a-7725-4818-83ff-015a19871f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:31.811501 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69fb82a-7725-4818-83ff-015a19871f60 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:31.811791 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69fb82a-7725-4818-83ff-015a19871f60 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:31.812076 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69fb82a-7725-4818-83ff-015a19871f60 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:31.812910 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69fb82a-7725-4818-83ff-015a19871f60 None admin] RBAC: Adding query filter params (group.id=53f2efd4d9254ad4a528fceb5a783b5c, role.id=23d9dd31632340bdba9cf3093c095dbb, scope.project.id=3c9e67fcd5b14da2a3775ef2b2cad6b2) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:31.813220 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69fb82a-7725-4818-83ff-015a19871f60 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:31.814701 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69fb82a-7725-4818-83ff-015a19871f60 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:31.821365 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 230/453] 10.222.0.57 () {56 vars in 1376 bytes} [Thu May 19 14:10:31 2022] GET /identity/v3/role_assignments?group.id=53f2efd4d9254ad4a528fceb5a783b5c&scope.project.id=3c9e67fcd5b14da2a3775ef2b2cad6b2&role.id=23d9dd31632340bdba9cf3093c095dbb => generated 611 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:10:31.843179 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d97c90-4701-4903-a6e3-73a1e7efea18 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:31.881336 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bd857b-67c2-4f29-bbbb-26fbf724df58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:31.882252 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bd857b-67c2-4f29-bbbb-26fbf724df58 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:31.882541 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bd857b-67c2-4f29-bbbb-26fbf724df58 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:31.882821 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bd857b-67c2-4f29-bbbb-26fbf724df58 None admin] PATH_INFO: `/v3/groups/94ff5f55c6904ba1bb185d07d5d8607e` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:31.884113 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bd857b-67c2-4f29-bbbb-26fbf724df58 None admin] RBAC: Authorizing `identity:get_group(group_id=94ff5f55c6904ba1bb185d07d5d8607e)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:31.885610 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bd857b-67c2-4f29-bbbb-26fbf724df58 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:31.887378 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 224/454] 10.222.0.57 () {56 vars in 1180 bytes} [Thu May 19 14:10:31 2022] GET /identity/v3/groups/94ff5f55c6904ba1bb185d07d5d8607e => generated 229 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:10:31.893418 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69fb82a-7725-4818-83ff-015a19871f60 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:31.998642 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69fb82a-7725-4818-83ff-015a19871f60 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:32.032637 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba15899-d7a3-4a03-be40-a3976d35286c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:32.033222 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba15899-d7a3-4a03-be40-a3976d35286c None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:32.033509 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba15899-d7a3-4a03-be40-a3976d35286c None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:32.033788 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba15899-d7a3-4a03-be40-a3976d35286c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:32.035356 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba15899-d7a3-4a03-be40-a3976d35286c None admin] RBAC: Adding query filter params (group.id=94ff5f55c6904ba1bb185d07d5d8607e, role.id=4946cf0ff7164307b36efeee7c787146, scope.project.id=dc6295ed5b1f4894a42c16df5aa39db4) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:32.035754 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba15899-d7a3-4a03-be40-a3976d35286c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:32.037205 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba15899-d7a3-4a03-be40-a3976d35286c None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:32.043253 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 231/455] 10.222.0.57 () {56 vars in 1376 bytes} [Thu May 19 14:10:31 2022] GET /identity/v3/role_assignments?group.id=94ff5f55c6904ba1bb185d07d5d8607e&scope.project.id=dc6295ed5b1f4894a42c16df5aa39db4&role.id=4946cf0ff7164307b36efeee7c787146 => generated 611 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:34.720571 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8570ab03-e37d-4efd-9834-6159c5be5df9 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:34.721493 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8570ab03-e37d-4efd-9834-6159c5be5df9 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:34.722124 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8570ab03-e37d-4efd-9834-6159c5be5df9 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:34.723870 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 225/456] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:10:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:10:34.729565 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ec479e-3b07-407d-ba4b-10e6d69c1ae3 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:34.730424 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ec479e-3b07-407d-ba4b-10e6d69c1ae3 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:34.730878 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ec479e-3b07-407d-ba4b-10e6d69c1ae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:34.736807 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:34.751739 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-14ec479e-3b07-407d-ba4b-10e6d69c1ae3 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:10:34.753555 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-14ec479e-3b07-407d-ba4b-10e6d69c1ae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:10:34.848319 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 232/457] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:10:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:10:34.889201 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8570ab03-e37d-4efd-9834-6159c5be5df9 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:34.907697 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:34.960645 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1701959d-f63e-4956-a609-da3f1c888a32 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:34.960645 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1701959d-f63e-4956-a609-da3f1c888a32 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:34.960645 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1701959d-f63e-4956-a609-da3f1c888a32 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:34.961200 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 233/458] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:10:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:10:34.967676 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737663b9-08d6-4cf7-aca9-4277e1e41292 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:34.968143 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737663b9-08d6-4cf7-aca9-4277e1e41292 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:34.968568 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737663b9-08d6-4cf7-aca9-4277e1e41292 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:35.010952 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-737663b9-08d6-4cf7-aca9-4277e1e41292 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:10:35.012374 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-737663b9-08d6-4cf7-aca9-4277e1e41292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:10:35.029929 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8570ab03-e37d-4efd-9834-6159c5be5df9 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:35.086776 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4b1ab1-6cb0-440a-972f-90ae0f269d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:35.087584 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4b1ab1-6cb0-440a-972f-90ae0f269d06 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:35.087871 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4b1ab1-6cb0-440a-972f-90ae0f269d06 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:35.088151 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4b1ab1-6cb0-440a-972f-90ae0f269d06 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:35.109913 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4b1ab1-6cb0-440a-972f-90ae0f269d06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:35.111501 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4b1ab1-6cb0-440a-972f-90ae0f269d06 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:35.131597 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-ec4b1ab1-6cb0-440a-972f-90ae0f269d06 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:10:35.131597 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:35.131597 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:35.131597 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:35.131597 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:35.131597 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:35.131597 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:35.131597 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:35.131597 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:35.131597 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:35.131597 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:35.131597 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:35.131597 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:10:35.131597 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:10:35.131597 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:10:35.132714 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:10:35.132714 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:35.132714 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:35.132714 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:35.132714 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:35.132714 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:35.132714 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:35.132714 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:35.132714 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:10:35.132714 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:35.132714 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:35.132714 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:35.132714 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:35.132714 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:35.132714 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:35.132714 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:35.132714 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:35.133770 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:10:35.133770 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:10:35.133770 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:10:35.133770 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:10:35.135248 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 226/459] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:10:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:10:35.136684 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcefb2c4-67af-4787-8ba8-2f734e355929 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:35.137194 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcefb2c4-67af-4787-8ba8-2f734e355929 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:35.137683 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcefb2c4-67af-4787-8ba8-2f734e355929 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:35.141656 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 234/461] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:10:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:10:35.142739 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737663b9-08d6-4cf7-aca9-4277e1e41292 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:35.143575 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 227/461] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:10:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:10:35.145461 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f992b229-acfc-4c26-9534-04ae287194d8 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:35.145999 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f992b229-acfc-4c26-9534-04ae287194d8 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:35.146294 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f992b229-acfc-4c26-9534-04ae287194d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:35.166497 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-f992b229-acfc-4c26-9534-04ae287194d8 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:10:35.168232 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-f992b229-acfc-4c26-9534-04ae287194d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:10:35.240894 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:35.242752 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:35.266256 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 228/462] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:10:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:10:35.266856 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f992b229-acfc-4c26-9534-04ae287194d8 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:35.302113 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737663b9-08d6-4cf7-aca9-4277e1e41292 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:35.346417 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f74c3f-55eb-48df-b89c-d1370025551e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:35.347507 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f74c3f-55eb-48df-b89c-d1370025551e None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:35.348042 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f74c3f-55eb-48df-b89c-d1370025551e None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:35.348539 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f74c3f-55eb-48df-b89c-d1370025551e None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:35.349468 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f74c3f-55eb-48df-b89c-d1370025551e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:35.350050 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f74c3f-55eb-48df-b89c-d1370025551e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:35.351625 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f74c3f-55eb-48df-b89c-d1370025551e None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:35.365445 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 235/463] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:10:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:35.366612 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f74c3f-55eb-48df-b89c-d1370025551e None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:35.405422 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f992b229-acfc-4c26-9534-04ae287194d8 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:35.449881 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba778ce-6ce6-42ae-a947-8c8f0f745602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:35.451215 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba778ce-6ce6-42ae-a947-8c8f0f745602 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:35.451845 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba778ce-6ce6-42ae-a947-8c8f0f745602 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:35.452328 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba778ce-6ce6-42ae-a947-8c8f0f745602 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:35.466425 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba778ce-6ce6-42ae-a947-8c8f0f745602 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:35.468342 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba778ce-6ce6-42ae-a947-8c8f0f745602 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:35.495301 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-2ba778ce-6ce6-42ae-a947-8c8f0f745602 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:10:35.495301 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:35.495301 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:35.495301 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:35.495301 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:35.495301 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:35.495301 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:35.495301 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:35.495301 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:35.495301 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:35.495301 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:35.495301 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:35.495301 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:10:35.495301 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:10:35.495301 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:10:35.496245 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:10:35.496245 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:35.496245 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:35.496245 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:35.496245 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:35.496245 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:35.496245 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:35.496245 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:35.496245 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:10:35.496245 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:35.496245 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:35.496245 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:35.496245 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:35.496245 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:35.496245 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:35.496245 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:35.496245 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:35.497086 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:10:35.497086 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:10:35.497086 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:10:35.497086 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:10:35.501969 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 229/464] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:10:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:10:35.502488 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f74c3f-55eb-48df-b89c-d1370025551e None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:35.503083 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba778ce-6ce6-42ae-a947-8c8f0f745602 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:35.529773 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:35.543251 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191bf901-9ebb-4641-b21c-e6e896d05012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:35.544392 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191bf901-9ebb-4641-b21c-e6e896d05012 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:35.544879 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191bf901-9ebb-4641-b21c-e6e896d05012 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:35.545355 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191bf901-9ebb-4641-b21c-e6e896d05012 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:35.556180 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:35.558027 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191bf901-9ebb-4641-b21c-e6e896d05012 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:35.559769 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191bf901-9ebb-4641-b21c-e6e896d05012 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:35.579653 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-191bf901-9ebb-4641-b21c-e6e896d05012 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:10:35.579653 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:35.579653 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:35.579653 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:35.579653 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:35.579653 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:35.579653 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:35.579653 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:35.579653 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:35.579653 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:35.579653 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:35.579653 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:35.579653 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:10:35.579653 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:10:35.579653 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:10:35.580558 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:10:35.580558 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:35.580558 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:35.580558 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:35.580558 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:35.580558 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:35.580558 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:35.580558 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:35.580558 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:10:35.580558 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:35.580558 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:35.580558 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:35.580558 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:35.580558 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:35.580558 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:35.580558 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:35.580558 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:35.581460 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:10:35.581460 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:10:35.581460 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:10:35.581460 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:10:35.588192 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 236/465] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:10:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:10:35.589384 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191bf901-9ebb-4641-b21c-e6e896d05012 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:35.621325 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba778ce-6ce6-42ae-a947-8c8f0f745602 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:35.660087 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479c8dcc-b9ae-4359-9247-6ddf0e0de45c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:35.660819 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479c8dcc-b9ae-4359-9247-6ddf0e0de45c None admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:35.661103 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479c8dcc-b9ae-4359-9247-6ddf0e0de45c None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:35.661377 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479c8dcc-b9ae-4359-9247-6ddf0e0de45c None admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:35.662220 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479c8dcc-b9ae-4359-9247-6ddf0e0de45c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:35.663796 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479c8dcc-b9ae-4359-9247-6ddf0e0de45c None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:35.687106 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 230/466] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 19 14:10:35 2022] POST /identity/v3/users => generated 246 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:10:35.688020 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479c8dcc-b9ae-4359-9247-6ddf0e0de45c None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:35.702698 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191bf901-9ebb-4641-b21c-e6e896d05012 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:35.743760 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b6014b-7d07-4bdd-a41c-be45734c05de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:35.745307 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b6014b-7d07-4bdd-a41c-be45734c05de None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:35.745813 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b6014b-7d07-4bdd-a41c-be45734c05de None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:35.746337 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b6014b-7d07-4bdd-a41c-be45734c05de None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:35.747334 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b6014b-7d07-4bdd-a41c-be45734c05de None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:35.747878 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b6014b-7d07-4bdd-a41c-be45734c05de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:35.749622 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b6014b-7d07-4bdd-a41c-be45734c05de None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:35.763858 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 237/467] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:10:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:35.765796 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febed4d7-b8f0-4a58-8410-6faf0a6acf08 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:35.766319 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febed4d7-b8f0-4a58-8410-6faf0a6acf08 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:35.766790 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febed4d7-b8f0-4a58-8410-6faf0a6acf08 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:35.768875 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 238/468] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:10:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:10:35.769798 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febed4d7-b8f0-4a58-8410-6faf0a6acf08 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:35.808116 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:35.812120 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:35.812729 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479c8dcc-b9ae-4359-9247-6ddf0e0de45c None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:35.857789 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36e9492-d694-4a7d-a48f-2c74a8c154a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:35.858568 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36e9492-d694-4a7d-a48f-2c74a8c154a5 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:35.858857 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36e9492-d694-4a7d-a48f-2c74a8c154a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:35.859160 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36e9492-d694-4a7d-a48f-2c74a8c154a5 None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:35.860551 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36e9492-d694-4a7d-a48f-2c74a8c154a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:35.860551 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36e9492-d694-4a7d-a48f-2c74a8c154a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:35.861947 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36e9492-d694-4a7d-a48f-2c74a8c154a5 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:35.886115 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 231/469] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:10:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:35.887505 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71be6f3f-3e98-4bdf-9e75-f603d218a002 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:35.887761 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71be6f3f-3e98-4bdf-9e75-f603d218a002 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:35.888043 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71be6f3f-3e98-4bdf-9e75-f603d218a002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:35.903165 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febed4d7-b8f0-4a58-8410-6faf0a6acf08 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:35.906709 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-71be6f3f-3e98-4bdf-9e75-f603d218a002 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:10:35.907573 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-71be6f3f-3e98-4bdf-9e75-f603d218a002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:10:35.951282 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4954a0-8c24-434b-8d7d-a9395aeaf9b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:35.951940 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4954a0-8c24-434b-8d7d-a9395aeaf9b1 None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:35.952220 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4954a0-8c24-434b-8d7d-a9395aeaf9b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:35.952493 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4954a0-8c24-434b-8d7d-a9395aeaf9b1 None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:35.953219 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4954a0-8c24-434b-8d7d-a9395aeaf9b1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:35.959035 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4954a0-8c24-434b-8d7d-a9395aeaf9b1 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:35.985107 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 239/470] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 19 14:10:35 2022] POST /identity/v3/users => generated 248 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:10:35.986384 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4954a0-8c24-434b-8d7d-a9395aeaf9b1 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:36.005101 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 232/471] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:10:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:10:36.041704 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71be6f3f-3e98-4bdf-9e75-f603d218a002 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:36.127401 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:36.137592 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4954a0-8c24-434b-8d7d-a9395aeaf9b1 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:36.192894 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:36.209092 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71be6f3f-3e98-4bdf-9e75-f603d218a002 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:36.210679 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41f2a6f-88bd-4f12-a9f7-75dda9bf1a68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:36.211368 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41f2a6f-88bd-4f12-a9f7-75dda9bf1a68 None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:36.211657 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41f2a6f-88bd-4f12-a9f7-75dda9bf1a68 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:36.212010 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41f2a6f-88bd-4f12-a9f7-75dda9bf1a68 None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:36.212650 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41f2a6f-88bd-4f12-a9f7-75dda9bf1a68 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:36.214154 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41f2a6f-88bd-4f12-a9f7-75dda9bf1a68 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:36.245860 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 240/472] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 19 14:10:35 2022] POST /identity/v3/users => generated 248 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:10:36.248454 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1dcd80-8d58-466d-8634-a947047d9578 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:36.248753 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1dcd80-8d58-466d-8634-a947047d9578 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:36.249039 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1dcd80-8d58-466d-8634-a947047d9578 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:36.251748 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 241/473] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:10:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:10:36.254139 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa2e606-998a-421b-8b65-4dea529f96c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:36.255224 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa2e606-998a-421b-8b65-4dea529f96c4 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:36.255651 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa2e606-998a-421b-8b65-4dea529f96c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:36.256120 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa2e606-998a-421b-8b65-4dea529f96c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:36.257071 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26f996a-5fbf-4085-9b1b-0b0ea45aefeb None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:36.257366 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26f996a-5fbf-4085-9b1b-0b0ea45aefeb None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:36.257703 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26f996a-5fbf-4085-9b1b-0b0ea45aefeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:36.269802 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa2e606-998a-421b-8b65-4dea529f96c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:36.271460 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa2e606-998a-421b-8b65-4dea529f96c4 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:36.277469 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-e26f996a-5fbf-4085-9b1b-0b0ea45aefeb None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:10:36.278745 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-e26f996a-5fbf-4085-9b1b-0b0ea45aefeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:10:36.292628 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-3aa2e606-998a-421b-8b65-4dea529f96c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:10:36.292628 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:36.292628 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:36.292628 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:36.292628 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:36.292628 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:36.292628 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:36.292628 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:36.292628 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:36.292628 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:36.292628 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:36.292628 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:36.292628 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:10:36.292628 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:10:36.292628 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:10:36.293576 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:10:36.293576 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:36.293576 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:36.293576 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:36.293576 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:36.293576 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:36.293576 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:36.293576 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:36.293576 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:10:36.293576 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:36.293576 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:36.293576 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:36.293576 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:36.293576 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:36.293576 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:36.293576 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:36.293576 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:36.294462 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:10:36.294462 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:10:36.294462 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:10:36.294462 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:10:36.294462 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 233/474] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:10:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:10:36.302426 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa2e606-998a-421b-8b65-4dea529f96c4 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:36.382776 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 242/475] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:10:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:10:36.409995 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26f996a-5fbf-4085-9b1b-0b0ea45aefeb None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:36.420542 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa2e606-998a-421b-8b65-4dea529f96c4 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:36.462518 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31af7cf-cf82-4ce6-aa4c-44fab633e07e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:36.463294 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31af7cf-cf82-4ce6-aa4c-44fab633e07e None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:36.463471 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31af7cf-cf82-4ce6-aa4c-44fab633e07e None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:36.463738 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31af7cf-cf82-4ce6-aa4c-44fab633e07e None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:36.464472 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31af7cf-cf82-4ce6-aa4c-44fab633e07e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:36.464853 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31af7cf-cf82-4ce6-aa4c-44fab633e07e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:36.466209 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31af7cf-cf82-4ce6-aa4c-44fab633e07e None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:36.478395 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:36.482322 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 234/476] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:10:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:10:36.490914 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31af7cf-cf82-4ce6-aa4c-44fab633e07e None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:36.509812 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:36.547438 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26f996a-5fbf-4085-9b1b-0b0ea45aefeb None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:36.608050 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb2d674-64b9-43dd-85d8-9bf79b2781c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:36.608896 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb2d674-64b9-43dd-85d8-9bf79b2781c9 None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:36.609301 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb2d674-64b9-43dd-85d8-9bf79b2781c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:36.609587 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb2d674-64b9-43dd-85d8-9bf79b2781c9 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:36.610371 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb2d674-64b9-43dd-85d8-9bf79b2781c9 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:36.612846 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb2d674-64b9-43dd-85d8-9bf79b2781c9 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:36.622170 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.sql.core [None req-dbb2d674-64b9-43dd-85d8-9bf79b2781c9 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 19 14:10:36.622170 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 19 14:10:36.622170 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [parameters: {'id': '97d4edbb34a349dfa10bd73083965ef3', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 19 14:10:36.622170 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80203) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 19 14:10:36.635702 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31af7cf-cf82-4ce6-aa4c-44fab633e07e None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:36.646409 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-dbb2d674-64b9-43dd-85d8-9bf79b2781c9 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 19 14:10:36.646409 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:36.646409 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 14:10:36.646409 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 14:10:36.646409 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 14:10:36.646409 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 14:10:36.646409 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 14:10:36.646409 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application result = self._query(query) May 19 14:10:36.646409 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 14:10:36.646409 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application conn.query(q) May 19 14:10:36.646409 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 14:10:36.646409 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 14:10:36.646409 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 14:10:36.646409 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application result.read() May 19 14:10:36.646409 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 14:10:36.647580 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 14:10:36.647580 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 14:10:36.647580 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 14:10:36.647580 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 14:10:36.647580 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 14:10:36.647580 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 14:10:36.647580 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 14:10:36.647580 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 19 14:10:36.647580 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:10:36.647580 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 19 14:10:36.647580 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:10:36.647580 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:36.647580 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 19 14:10:36.647580 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 19 14:10:36.647580 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role May 19 14:10:36.647580 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return ref.to_dict() May 19 14:10:36.647580 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 14:10:36.647580 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application next(self.gen) May 19 14:10:36.648959 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 19 14:10:36.648959 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application yield resource May 19 14:10:36.648959 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 14:10:36.648959 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application next(self.gen) May 19 14:10:36.648959 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 19 14:10:36.648959 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self.session.rollback() May 19 14:10:36.648959 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 19 14:10:36.648959 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self.force_reraise() May 19 14:10:36.648959 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 19 14:10:36.648959 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise self.value May 19 14:10:36.648959 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 19 14:10:36.648959 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 19 14:10:36.648959 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 19 14:10:36.648959 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application session.commit() May 19 14:10:36.648959 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 19 14:10:36.648959 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 19 14:10:36.648959 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 19 14:10:36.651446 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._prepare_impl() May 19 14:10:36.651446 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 19 14:10:36.651446 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self.session.flush() May 19 14:10:36.651446 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 19 14:10:36.651446 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._flush(objects) May 19 14:10:36.651446 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 19 14:10:36.651446 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 19 14:10:36.651446 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 19 14:10:36.651446 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application compat.raise_( May 19 14:10:36.651446 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 14:10:36.651446 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception May 19 14:10:36.651446 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 19 14:10:36.651446 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application flush_context.execute() May 19 14:10:36.651446 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 19 14:10:36.651446 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rec.execute(self) May 19 14:10:36.651446 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 19 14:10:36.651446 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 19 14:10:36.653100 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 19 14:10:36.653100 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application _emit_insert_statements( May 19 14:10:36.653100 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 19 14:10:36.653100 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application c = connection._execute_20( May 19 14:10:36.653100 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 19 14:10:36.653100 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 19 14:10:36.653100 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 19 14:10:36.653100 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 19 14:10:36.653100 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 19 14:10:36.653100 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ret = self._execute_context( May 19 14:10:36.653100 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 19 14:10:36.653100 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 19 14:10:36.653100 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 19 14:10:36.653100 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 19 14:10:36.653100 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 14:10:36.653100 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception May 19 14:10:36.654066 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 14:10:36.654066 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 14:10:36.654066 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 14:10:36.654066 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 14:10:36.654066 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 14:10:36.654066 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application result = self._query(query) May 19 14:10:36.654066 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 14:10:36.654066 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application conn.query(q) May 19 14:10:36.654066 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 14:10:36.654066 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 14:10:36.654066 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 14:10:36.654066 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application result.read() May 19 14:10:36.654066 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 14:10:36.654066 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 14:10:36.654066 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 14:10:36.654066 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 14:10:36.655609 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 14:10:36.655609 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 14:10:36.655609 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 14:10:36.655609 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 14:10:36.655609 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 19 14:10:36.655609 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 19 14:10:36.655609 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [parameters: {'id': '97d4edbb34a349dfa10bd73083965ef3', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 19 14:10:36.655609 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 19 14:10:36.655609 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:10:36.655609 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:10:36.655609 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:10:36.655609 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:36.655609 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:36.655609 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:36.655609 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:36.655609 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:36.662596 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:36.662596 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:36.662596 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:36.662596 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:36.662596 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:36.662596 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:36.662596 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post May 19 14:10:36.662596 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( May 19 14:10:36.662596 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:36.662596 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:36.662596 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role May 19 14:10:36.662596 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) May 19 14:10:36.662596 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 19 14:10:36.662596 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 19 14:10:36.662596 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 19 14:10:36.662596 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:10:36.664322 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 243/477] 10.222.0.57 () {60 vars in 1131 bytes} [Thu May 19 14:10:36 2022] POST /identity/v3/roles => generated 144 bytes in 240 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) May 19 14:10:36.664322 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb2d674-64b9-43dd-85d8-9bf79b2781c9 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:36.695645 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3303d7da-6a64-4328-b851-4e0ea2614d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:36.696711 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3303d7da-6a64-4328-b851-4e0ea2614d0f None admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:36.697002 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3303d7da-6a64-4328-b851-4e0ea2614d0f None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:36.697688 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3303d7da-6a64-4328-b851-4e0ea2614d0f None admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:36.698781 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3303d7da-6a64-4328-b851-4e0ea2614d0f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:36.701077 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3303d7da-6a64-4328-b851-4e0ea2614d0f None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:36.734135 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 235/478] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 19 14:10:36 2022] POST /identity/v3/users => generated 249 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:10:36.798073 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb2d674-64b9-43dd-85d8-9bf79b2781c9 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:36.841591 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea624de-e83d-47bd-9067-6202cde7d467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:36.842451 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea624de-e83d-47bd-9067-6202cde7d467 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:36.842913 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea624de-e83d-47bd-9067-6202cde7d467 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:36.843203 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea624de-e83d-47bd-9067-6202cde7d467 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:36.855157 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:36.859759 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea624de-e83d-47bd-9067-6202cde7d467 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:10:36.870588 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-5ea624de-e83d-47bd-9067-6202cde7d467 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:10:36.870588 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:36.870588 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:36.870588 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:36.870588 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:36.870588 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:36.870588 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:36.870588 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:36.870588 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:36.870588 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:36.870588 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:36.870588 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:36.870588 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:10:36.870588 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:10:36.870588 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:10:36.870588 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:10:36.871562 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:10:36.871562 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:10:36.871562 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:10:36.871562 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:10:36.871562 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:36.871562 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:36.871562 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:36.871562 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:36.871562 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:36.871562 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:36.871562 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:36.871562 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:10:36.871562 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:36.871562 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:36.871562 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:36.871562 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:36.872416 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:36.872416 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:36.872416 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:36.872416 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:36.872416 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:10:36.872416 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:10:36.872416 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:10:36.872416 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:10:36.872416 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:10:36.872416 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:10:36.872416 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:10:36.872416 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:10:36.875348 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 244/479] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 19 14:10:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:10:36.881585 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3303d7da-6a64-4328-b851-4e0ea2614d0f None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:36.945814 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:36.994918 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3303d7da-6a64-4328-b851-4e0ea2614d0f None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:37.031986 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518de6b4-c043-4e08-b996-30cc714f080b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:37.033043 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518de6b4-c043-4e08-b996-30cc714f080b None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:37.033529 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518de6b4-c043-4e08-b996-30cc714f080b None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:37.034199 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518de6b4-c043-4e08-b996-30cc714f080b None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:37.035229 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518de6b4-c043-4e08-b996-30cc714f080b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:37.035740 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518de6b4-c043-4e08-b996-30cc714f080b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:37.037311 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518de6b4-c043-4e08-b996-30cc714f080b None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:37.046899 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 236/480] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:10:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:10:38.481823 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3e6bec-9dd4-473f-b551-bf60f8ef2851 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:38.482387 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3e6bec-9dd4-473f-b551-bf60f8ef2851 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:38.482682 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3e6bec-9dd4-473f-b551-bf60f8ef2851 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:38.484447 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 245/481] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:10:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:10:38.490195 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c559a8e-dbbd-4f94-ad10-76da8e2866ed None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:38.490656 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c559a8e-dbbd-4f94-ad10-76da8e2866ed None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:38.490753 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c559a8e-dbbd-4f94-ad10-76da8e2866ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:38.515132 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-1c559a8e-dbbd-4f94-ad10-76da8e2866ed None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:10:38.516391 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-1c559a8e-dbbd-4f94-ad10-76da8e2866ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:10:38.610126 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 237/482] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:10:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:10:38.645627 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3e6bec-9dd4-473f-b551-bf60f8ef2851 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:38.794614 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3e6bec-9dd4-473f-b551-bf60f8ef2851 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:38.846804 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b700ce33-88d9-4edb-b0f3-2a8d530c23e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:38.847574 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b700ce33-88d9-4edb-b0f3-2a8d530c23e8 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:38.847858 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b700ce33-88d9-4edb-b0f3-2a8d530c23e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:38.848132 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b700ce33-88d9-4edb-b0f3-2a8d530c23e8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:38.863163 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b700ce33-88d9-4edb-b0f3-2a8d530c23e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:10:38.871204 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-b700ce33-88d9-4edb-b0f3-2a8d530c23e8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:10:38.871204 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:38.871204 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:38.871204 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:38.871204 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:38.871204 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:38.871204 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:38.871204 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:38.871204 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:38.871204 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:38.871204 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:38.871204 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:38.871204 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:10:38.871204 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:10:38.871204 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:10:38.871204 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:10:38.872104 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:10:38.872104 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:10:38.872104 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:10:38.872104 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:10:38.872104 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:38.872104 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:38.872104 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:38.872104 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:38.872104 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:38.872104 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:38.872104 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:38.872104 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:10:38.872104 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:38.872104 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:38.872104 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:38.872104 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:38.872956 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:38.872956 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:38.872956 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:38.872956 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:38.872956 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:10:38.872956 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:10:38.872956 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:10:38.872956 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:10:38.872956 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:10:38.872956 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:10:38.872956 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:10:38.872956 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:10:38.875915 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 246/483] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 14:10:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:10:38.878919 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c559a8e-dbbd-4f94-ad10-76da8e2866ed None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:38.905904 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9acab8e-f05c-4b32-9a70-5f1deb0487f1 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:38.906180 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9acab8e-f05c-4b32-9a70-5f1deb0487f1 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:38.906467 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9acab8e-f05c-4b32-9a70-5f1deb0487f1 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:38.908083 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 247/484] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:10:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:10:38.913600 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31814639-a035-444f-bcae-f040f696e311 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:38.913918 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31814639-a035-444f-bcae-f040f696e311 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:38.914308 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31814639-a035-444f-bcae-f040f696e311 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:38.946829 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-31814639-a035-444f-bcae-f040f696e311 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:10:38.948036 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-31814639-a035-444f-bcae-f040f696e311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:10:38.950689 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:39.001202 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c559a8e-dbbd-4f94-ad10-76da8e2866ed None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:39.045344 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe355598-5c96-492c-b129-629064ab394b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:39.046432 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe355598-5c96-492c-b129-629064ab394b None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:39.048828 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe355598-5c96-492c-b129-629064ab394b None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:39.055062 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe355598-5c96-492c-b129-629064ab394b None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:39.056233 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe355598-5c96-492c-b129-629064ab394b None admin] RBAC: Adding query filter params (name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:39.056763 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe355598-5c96-492c-b129-629064ab394b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:39.058424 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe355598-5c96-492c-b129-629064ab394b None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:39.077320 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 238/485] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 19 14:10:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:39.084414 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe355598-5c96-492c-b129-629064ab394b None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:39.090297 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:39.093557 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 248/486] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:10:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:10:39.139057 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31814639-a035-444f-bcae-f040f696e311 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:39.283211 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe355598-5c96-492c-b129-629064ab394b None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:39.310977 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:39.349815 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31814639-a035-444f-bcae-f040f696e311 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:39.362795 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82ca1dd-1ec1-4378-9ee2-2b2d01f758e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:39.365135 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82ca1dd-1ec1-4378-9ee2-2b2d01f758e7 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:39.365425 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82ca1dd-1ec1-4378-9ee2-2b2d01f758e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:39.365707 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82ca1dd-1ec1-4378-9ee2-2b2d01f758e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:39.398041 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82ca1dd-1ec1-4378-9ee2-2b2d01f758e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:39.400098 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82ca1dd-1ec1-4378-9ee2-2b2d01f758e7 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:39.428893 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-c82ca1dd-1ec1-4378-9ee2-2b2d01f758e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:10:39.428893 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:39.428893 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:39.428893 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:39.428893 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:39.428893 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:39.428893 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:39.428893 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:39.428893 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:39.428893 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:39.428893 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:39.428893 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:39.428893 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:10:39.428893 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:10:39.428893 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:10:39.429792 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:10:39.429792 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:39.429792 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:39.429792 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:39.429792 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:39.429792 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:39.429792 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:39.429792 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:39.429792 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:10:39.429792 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:39.429792 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:39.429792 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:39.429792 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:39.429792 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:39.429792 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:39.429792 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:39.429792 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:39.430677 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:10:39.430677 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:10:39.430677 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:10:39.430677 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:10:39.430913 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 239/487] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:10:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:10:39.432747 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53d3978-ccc8-42d6-b25c-11391494bf16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:39.433727 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53d3978-ccc8-42d6-b25c-11391494bf16 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:39.434170 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53d3978-ccc8-42d6-b25c-11391494bf16 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:39.434575 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53d3978-ccc8-42d6-b25c-11391494bf16 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:39.441675 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14be7621-bd14-4e5e-aecf-4e06185118d3 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:39.442465 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14be7621-bd14-4e5e-aecf-4e06185118d3 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:39.443557 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14be7621-bd14-4e5e-aecf-4e06185118d3 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:39.445282 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 240/488] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:10:39 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:10:39.447266 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14be7621-bd14-4e5e-aecf-4e06185118d3 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:39.456052 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53d3978-ccc8-42d6-b25c-11391494bf16 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:10:39.467039 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-d53d3978-ccc8-42d6-b25c-11391494bf16 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:10:39.467039 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:39.467039 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:39.467039 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:39.467039 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:39.467039 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:39.467039 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:39.467039 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:39.467039 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:39.467039 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:39.467039 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:39.467039 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:39.467039 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:10:39.467039 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:10:39.467039 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:10:39.467039 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:10:39.467954 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:10:39.467954 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:10:39.467954 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:10:39.467954 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:10:39.467954 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:39.467954 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:39.467954 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:39.467954 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:39.467954 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:39.467954 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:39.467954 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:39.467954 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:10:39.467954 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:39.467954 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:39.467954 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:39.467954 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:39.468838 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:39.468838 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:39.468838 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:39.468838 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:39.468838 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:10:39.468838 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:10:39.468838 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:10:39.468838 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:10:39.468838 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:10:39.468838 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:10:39.468838 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:10:39.468838 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:10:39.472433 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 249/489] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 14:10:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:10:39.474026 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62224464-8c6c-489e-95c9-39bd88a30c1d None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:39.474535 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62224464-8c6c-489e-95c9-39bd88a30c1d None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:39.475014 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62224464-8c6c-489e-95c9-39bd88a30c1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:39.503950 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-62224464-8c6c-489e-95c9-39bd88a30c1d None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:10:39.506249 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-62224464-8c6c-489e-95c9-39bd88a30c1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:10:39.590670 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14be7621-bd14-4e5e-aecf-4e06185118d3 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:39.635760 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a95a894-1109-4178-ab15-6a24a868843c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:39.636471 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a95a894-1109-4178-ab15-6a24a868843c None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:39.636754 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a95a894-1109-4178-ab15-6a24a868843c None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:39.637032 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a95a894-1109-4178-ab15-6a24a868843c None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:39.637762 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a95a894-1109-4178-ab15-6a24a868843c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:39.638095 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a95a894-1109-4178-ab15-6a24a868843c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:39.640104 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a95a894-1109-4178-ab15-6a24a868843c None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:39.662265 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:39.667416 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 250/490] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:10:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:10:39.671122 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62224464-8c6c-489e-95c9-39bd88a30c1d None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:39.681881 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 241/491] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:10:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:39.691180 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a95a894-1109-4178-ab15-6a24a868843c None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:39.756266 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:39.807632 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62224464-8c6c-489e-95c9-39bd88a30c1d None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:39.842685 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a95a894-1109-4178-ab15-6a24a868843c None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:39.846216 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a689f3-9f37-4255-a072-206c05cce89a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:39.846958 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a689f3-9f37-4255-a072-206c05cce89a None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:39.847242 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a689f3-9f37-4255-a072-206c05cce89a None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:39.847518 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a689f3-9f37-4255-a072-206c05cce89a None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:39.848270 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a689f3-9f37-4255-a072-206c05cce89a None admin] RBAC: Adding query filter params (name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:39.848575 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a689f3-9f37-4255-a072-206c05cce89a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:39.849986 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a689f3-9f37-4255-a072-206c05cce89a None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:39.860899 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 251/492] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 19 14:10:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:39.861612 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a689f3-9f37-4255-a072-206c05cce89a None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:39.885016 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c1c6c7-1042-4fba-ac77-dae8935d0ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:39.886217 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c1c6c7-1042-4fba-ac77-dae8935d0ce4 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:39.886727 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c1c6c7-1042-4fba-ac77-dae8935d0ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:39.887337 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c1c6c7-1042-4fba-ac77-dae8935d0ce4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:39.895569 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c1c6c7-1042-4fba-ac77-dae8935d0ce4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:39.897519 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c1c6c7-1042-4fba-ac77-dae8935d0ce4 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:39.905263 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:39.915449 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-81c1c6c7-1042-4fba-ac77-dae8935d0ce4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 14:10:39.915449 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:39.915449 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:39.915449 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:39.915449 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:39.915449 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:39.915449 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:39.915449 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:39.915449 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:39.915449 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:39.915449 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:39.915449 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:39.915449 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:10:39.915449 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:10:39.915449 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:10:39.915449 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:10:39.916555 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:39.916555 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:39.916555 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:10:39.916555 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:10:39.916555 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:10:39.916555 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:10:39.916555 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:39.916555 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:39.916555 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:39.916555 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:39.916555 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:39.916555 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:10:39.916555 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:39.916555 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:39.916555 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:39.916555 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:39.916555 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:39.916555 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:39.917762 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:39.917762 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:39.917762 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:10:39.917762 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:10:39.917762 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:10:39.917762 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:10:39.917762 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:10:39.917762 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:10:39.917762 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 14:10:39.917762 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:10:39.917762 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 242/493] 10.222.0.57 () {56 vars in 1093 bytes} [Thu May 19 14:10:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:10:39.918388 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87934d8c-7f88-4fbb-8245-df090ccee129 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:39.918388 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87934d8c-7f88-4fbb-8245-df090ccee129 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:39.918700 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87934d8c-7f88-4fbb-8245-df090ccee129 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:39.919814 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 243/494] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:10:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:10:39.920502 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87934d8c-7f88-4fbb-8245-df090ccee129 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:39.968654 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a689f3-9f37-4255-a072-206c05cce89a None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:40.004458 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0b8c17-d99b-42e0-a744-2d466f6482d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:40.005254 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0b8c17-d99b-42e0-a744-2d466f6482d0 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:40.005541 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0b8c17-d99b-42e0-a744-2d466f6482d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:40.005820 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0b8c17-d99b-42e0-a744-2d466f6482d0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:40.006373 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:40.016253 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0b8c17-d99b-42e0-a744-2d466f6482d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:10:40.024246 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-8a0b8c17-d99b-42e0-a744-2d466f6482d0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:10:40.024246 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:40.024246 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:40.024246 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:40.024246 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:40.024246 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:40.024246 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:40.024246 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:40.024246 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:40.024246 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:40.024246 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:40.024246 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:40.024246 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:10:40.024246 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:10:40.024246 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:10:40.024246 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:10:40.025158 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:10:40.025158 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:10:40.025158 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:10:40.025158 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:10:40.025158 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:40.025158 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:40.025158 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:40.025158 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:40.025158 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:40.025158 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:40.025158 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:40.025158 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:10:40.025158 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:40.025158 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:40.025158 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:40.025158 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:40.026291 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:40.026291 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:40.026291 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:40.026291 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:40.026291 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:10:40.026291 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:10:40.026291 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:10:40.026291 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:10:40.026291 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:10:40.026291 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:10:40.026291 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:10:40.026291 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:10:40.027079 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87934d8c-7f88-4fbb-8245-df090ccee129 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:40.027150 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 252/495] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 14:10:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:10:40.027150 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705757d1-ad4b-474c-b9fb-fb055096502e None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:40.027275 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705757d1-ad4b-474c-b9fb-fb055096502e None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:40.027544 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705757d1-ad4b-474c-b9fb-fb055096502e None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:40.028796 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 253/496] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:10:40 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:10:40.029488 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705757d1-ad4b-474c-b9fb-fb055096502e None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:40.063800 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab303682-dcbb-4758-9dea-a2c031aa0230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:40.064591 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab303682-dcbb-4758-9dea-a2c031aa0230 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:40.064882 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab303682-dcbb-4758-9dea-a2c031aa0230 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:40.065796 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab303682-dcbb-4758-9dea-a2c031aa0230 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:40.076128 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab303682-dcbb-4758-9dea-a2c031aa0230 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:40.077591 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab303682-dcbb-4758-9dea-a2c031aa0230 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:40.094975 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-ab303682-dcbb-4758-9dea-a2c031aa0230 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:10:40.094975 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:40.094975 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:40.094975 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:40.094975 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:40.094975 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:40.094975 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:40.094975 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:40.094975 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:40.094975 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:40.094975 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:40.094975 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:40.094975 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:10:40.094975 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:10:40.094975 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:10:40.095885 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:10:40.095885 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:40.095885 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:40.095885 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:40.095885 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:40.095885 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:40.095885 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:40.095885 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:40.095885 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:10:40.095885 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:40.095885 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:40.095885 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:40.095885 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:40.095885 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:40.095885 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:40.095885 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:40.095885 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:40.097016 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:10:40.097016 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:10:40.097016 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:10:40.097016 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:10:40.097016 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 244/497] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:10:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:10:40.097279 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a70a92-d9e6-439c-9598-e1a201464664 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:40.097613 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a70a92-d9e6-439c-9598-e1a201464664 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:40.097814 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a70a92-d9e6-439c-9598-e1a201464664 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:40.115331 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-83a70a92-d9e6-439c-9598-e1a201464664 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:10:40.116552 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-83a70a92-d9e6-439c-9598-e1a201464664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:10:40.141542 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705757d1-ad4b-474c-b9fb-fb055096502e None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:40.143109 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:40.177038 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4153a9c9-9aab-429a-ae12-1d6dc570ca9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:40.177710 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4153a9c9-9aab-429a-ae12-1d6dc570ca9d None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:40.178042 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4153a9c9-9aab-429a-ae12-1d6dc570ca9d None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:40.178326 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4153a9c9-9aab-429a-ae12-1d6dc570ca9d None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:40.179162 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4153a9c9-9aab-429a-ae12-1d6dc570ca9d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:40.179468 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4153a9c9-9aab-429a-ae12-1d6dc570ca9d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:40.180921 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4153a9c9-9aab-429a-ae12-1d6dc570ca9d None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:40.185663 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 245/498] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:10:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:10:40.186976 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb781a85-0af3-44b1-a751-ff431bac7379 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:40.187269 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb781a85-0af3-44b1-a751-ff431bac7379 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:40.187552 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb781a85-0af3-44b1-a751-ff431bac7379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:40.204436 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-fb781a85-0af3-44b1-a751-ff431bac7379 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:10:40.205769 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-fb781a85-0af3-44b1-a751-ff431bac7379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:10:40.222978 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 254/499] 10.222.0.57 () {56 vars in 1133 bytes} [Thu May 19 14:10:40 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:40.223678 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4153a9c9-9aab-429a-ae12-1d6dc570ca9d None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:40.267716 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:40.276347 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 246/500] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:10:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:10:40.277155 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb781a85-0af3-44b1-a751-ff431bac7379 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:40.330383 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4153a9c9-9aab-429a-ae12-1d6dc570ca9d None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:40.365458 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220e1b7f-381f-42a5-bfdc-98fbfefd81cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:40.366090 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220e1b7f-381f-42a5-bfdc-98fbfefd81cb None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:40.366375 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220e1b7f-381f-42a5-bfdc-98fbfefd81cb None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:40.366651 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220e1b7f-381f-42a5-bfdc-98fbfefd81cb None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:40.367389 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220e1b7f-381f-42a5-bfdc-98fbfefd81cb None admin] RBAC: Adding query filter params (name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:40.367693 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220e1b7f-381f-42a5-bfdc-98fbfefd81cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:40.369076 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220e1b7f-381f-42a5-bfdc-98fbfefd81cb None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:40.376929 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 255/501] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 19 14:10:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:40.377710 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220e1b7f-381f-42a5-bfdc-98fbfefd81cb None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:40.378811 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb781a85-0af3-44b1-a751-ff431bac7379 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:40.400892 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:40.413442 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c1b290-97b7-405d-9d82-6ba9004a7c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:40.414145 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c1b290-97b7-405d-9d82-6ba9004a7c8d None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:40.414426 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c1b290-97b7-405d-9d82-6ba9004a7c8d None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:40.414700 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c1b290-97b7-405d-9d82-6ba9004a7c8d None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:40.415417 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c1b290-97b7-405d-9d82-6ba9004a7c8d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:40.415722 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c1b290-97b7-405d-9d82-6ba9004a7c8d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:40.417115 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c1b290-97b7-405d-9d82-6ba9004a7c8d None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:40.428899 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 247/502] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:10:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:40.429695 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c1b290-97b7-405d-9d82-6ba9004a7c8d None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:40.480973 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220e1b7f-381f-42a5-bfdc-98fbfefd81cb None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:40.485133 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:40.516193 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e80e9d-3063-4e5c-97dc-75db80bf7400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:40.516983 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e80e9d-3063-4e5c-97dc-75db80bf7400 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:40.517272 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e80e9d-3063-4e5c-97dc-75db80bf7400 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:40.517580 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e80e9d-3063-4e5c-97dc-75db80bf7400 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:40.527992 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e80e9d-3063-4e5c-97dc-75db80bf7400 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:40.529461 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e80e9d-3063-4e5c-97dc-75db80bf7400 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:40.532365 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c1b290-97b7-405d-9d82-6ba9004a7c8d None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:40.546782 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-a8e80e9d-3063-4e5c-97dc-75db80bf7400 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:10:40.546782 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:40.546782 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:40.546782 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:40.546782 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:40.546782 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:40.546782 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:40.546782 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:40.546782 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:40.546782 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:40.546782 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:40.546782 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:40.546782 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:10:40.546782 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:10:40.546782 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:10:40.547676 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:10:40.547676 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:40.547676 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:40.547676 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:40.547676 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:40.547676 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:40.547676 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:40.547676 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:40.547676 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:10:40.547676 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:40.547676 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:40.547676 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:40.547676 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:40.547676 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:40.547676 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:40.547676 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:40.547676 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:40.548517 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:10:40.548517 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:10:40.548517 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:10:40.548517 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:10:40.548517 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 256/503] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:10:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:10:40.548797 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e80e9d-3063-4e5c-97dc-75db80bf7400 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:40.571580 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337f7599-7d1a-4945-a8d7-3634da47a8af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:40.572230 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337f7599-7d1a-4945-a8d7-3634da47a8af None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:40.572509 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337f7599-7d1a-4945-a8d7-3634da47a8af None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:40.572783 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337f7599-7d1a-4945-a8d7-3634da47a8af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:40.625396 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:40.642736 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337f7599-7d1a-4945-a8d7-3634da47a8af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:40.645087 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337f7599-7d1a-4945-a8d7-3634da47a8af None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:40.651023 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e80e9d-3063-4e5c-97dc-75db80bf7400 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:40.685226 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775335e0-5b65-4b80-8cba-3d1c3a5d84a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:40.686057 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775335e0-5b65-4b80-8cba-3d1c3a5d84a2 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:40.686344 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775335e0-5b65-4b80-8cba-3d1c3a5d84a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:40.686616 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775335e0-5b65-4b80-8cba-3d1c3a5d84a2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:40.697716 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775335e0-5b65-4b80-8cba-3d1c3a5d84a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:10:40.698344 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:40.708155 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-775335e0-5b65-4b80-8cba-3d1c3a5d84a2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:10:40.708155 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:40.708155 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:40.708155 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:40.708155 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:40.708155 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:40.708155 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:40.708155 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:40.708155 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:40.708155 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:40.708155 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:40.708155 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:40.708155 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:10:40.708155 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:10:40.708155 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:10:40.708155 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:10:40.709057 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:10:40.709057 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:10:40.709057 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:10:40.709057 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:10:40.709057 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:40.709057 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:40.709057 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:40.709057 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:40.709057 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:40.709057 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:40.709057 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:40.709057 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:10:40.709057 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:40.709057 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:40.709057 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:40.709057 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:40.709868 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:40.709868 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:40.709868 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:40.709868 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:40.709868 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:10:40.709868 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:10:40.709868 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:10:40.709868 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:10:40.709868 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:10:40.709868 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:10:40.709868 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:10:40.709868 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:10:40.710673 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 257/504] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 14:10:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:10:40.710824 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775335e0-5b65-4b80-8cba-3d1c3a5d84a2 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:40.750629 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 248/505] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:10:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:10:40.750629 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337f7599-7d1a-4945-a8d7-3634da47a8af None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:40.816607 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775335e0-5b65-4b80-8cba-3d1c3a5d84a2 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:40.846920 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337f7599-7d1a-4945-a8d7-3634da47a8af None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:40.850091 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1891248a-0bdf-4f90-99cb-b3039a43e87c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:40.850880 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1891248a-0bdf-4f90-99cb-b3039a43e87c None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:40.851166 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1891248a-0bdf-4f90-99cb-b3039a43e87c None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:40.851445 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1891248a-0bdf-4f90-99cb-b3039a43e87c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:40.853362 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:40.862881 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1891248a-0bdf-4f90-99cb-b3039a43e87c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:40.864464 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1891248a-0bdf-4f90-99cb-b3039a43e87c None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:40.881189 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d103103-a734-4047-a39a-6e64eac73496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:40.881945 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d103103-a734-4047-a39a-6e64eac73496 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:40.882229 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d103103-a734-4047-a39a-6e64eac73496 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:40.882318 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-1891248a-0bdf-4f90-99cb-b3039a43e87c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:10:40.882318 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:40.882318 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:40.882318 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:40.882318 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:40.882318 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:40.882318 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:40.882318 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:40.882318 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:40.882318 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:40.882318 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:40.882318 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:40.882318 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:10:40.882318 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:10:40.882318 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:10:40.883297 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:10:40.883297 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:40.883297 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:40.883297 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:40.883297 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:40.883297 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:40.883297 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:40.883297 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:40.883297 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:10:40.883297 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:40.883297 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:40.883297 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:40.883297 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:40.883297 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:40.883297 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:40.883297 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:40.883297 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:40.884394 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:10:40.884394 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:10:40.884394 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:10:40.884394 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:10:40.884956 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d103103-a734-4047-a39a-6e64eac73496 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:40.885024 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 258/506] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:10:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:10:40.885024 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1891248a-0bdf-4f90-99cb-b3039a43e87c None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:40.891584 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d103103-a734-4047-a39a-6e64eac73496 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:40.893048 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d103103-a734-4047-a39a-6e64eac73496 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:40.907922 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-6d103103-a734-4047-a39a-6e64eac73496 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 19 14:10:40.907922 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:40.907922 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:40.907922 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:40.907922 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:40.907922 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:40.907922 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:40.907922 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:40.907922 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:40.907922 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:40.907922 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:40.907922 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:40.907922 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:10:40.907922 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:10:40.907922 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:10:40.907922 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:10:40.909577 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:40.909577 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:40.909577 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:10:40.909577 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:10:40.909577 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:10:40.909577 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:10:40.909577 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:40.909577 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:40.909577 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:40.909577 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:40.909577 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:40.909577 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:10:40.909577 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:40.909577 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:40.909577 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:40.909577 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:40.909577 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:40.909577 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:40.911301 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:40.911301 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:40.911301 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:10:40.911301 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:10:40.911301 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:10:40.911301 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:10:40.911301 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:10:40.911301 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:10:40.911301 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 19 14:10:40.911301 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:10:40.911301 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 249/507] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:10:40 2022] GET /identity/v3/users/glance => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:10:40.911301 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d103103-a734-4047-a39a-6e64eac73496 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:40.927025 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:40.987983 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1891248a-0bdf-4f90-99cb-b3039a43e87c None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:41.016918 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d103103-a734-4047-a39a-6e64eac73496 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:41.022523 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0415e2-0fa3-4bba-98e0-374c34dcf619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:41.023189 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0415e2-0fa3-4bba-98e0-374c34dcf619 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:41.023478 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0415e2-0fa3-4bba-98e0-374c34dcf619 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:41.023758 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0415e2-0fa3-4bba-98e0-374c34dcf619 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:41.024476 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0415e2-0fa3-4bba-98e0-374c34dcf619 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:41.024787 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0415e2-0fa3-4bba-98e0-374c34dcf619 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:41.026228 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0415e2-0fa3-4bba-98e0-374c34dcf619 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:41.038019 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 259/508] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:10:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:41.038832 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0415e2-0fa3-4bba-98e0-374c34dcf619 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:41.052428 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb882f00-931a-442f-8ebb-a39f0cb5779a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:41.053097 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb882f00-931a-442f-8ebb-a39f0cb5779a None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:41.053377 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb882f00-931a-442f-8ebb-a39f0cb5779a None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:41.053647 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb882f00-931a-442f-8ebb-a39f0cb5779a None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:41.054413 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb882f00-931a-442f-8ebb-a39f0cb5779a None admin] RBAC: Adding query filter params (name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:41.054713 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb882f00-931a-442f-8ebb-a39f0cb5779a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:41.056077 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb882f00-931a-442f-8ebb-a39f0cb5779a None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:41.064534 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 250/509] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 19 14:10:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:41.066170 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb882f00-931a-442f-8ebb-a39f0cb5779a None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:41.087373 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:41.143762 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0415e2-0fa3-4bba-98e0-374c34dcf619 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:41.153654 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:41.175987 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb882f00-931a-442f-8ebb-a39f0cb5779a None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:41.177761 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d59b74-7c79-4900-8ecd-09a875ab3c4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:41.178585 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d59b74-7c79-4900-8ecd-09a875ab3c4a None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:41.178873 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d59b74-7c79-4900-8ecd-09a875ab3c4a None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:41.179152 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d59b74-7c79-4900-8ecd-09a875ab3c4a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:41.189259 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d59b74-7c79-4900-8ecd-09a875ab3c4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:41.191268 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d59b74-7c79-4900-8ecd-09a875ab3c4a None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:41.209453 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-97d59b74-7c79-4900-8ecd-09a875ab3c4a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:10:41.209453 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:41.209453 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:41.209453 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:41.209453 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:41.209453 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:41.209453 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:41.209453 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:41.209453 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:41.209453 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:41.209453 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:41.209453 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:41.209453 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:10:41.209453 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:10:41.209453 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:10:41.210375 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:10:41.210375 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:41.210375 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:41.210375 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:41.210375 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:41.210375 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:41.210375 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:41.210375 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:41.210375 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:10:41.210375 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:41.210375 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:41.210375 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:41.210375 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:41.210375 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:41.210375 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:41.210375 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:41.210375 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:41.211940 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:10:41.211940 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:10:41.211940 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:10:41.211940 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:10:41.211940 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 260/510] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:10:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:10:41.211940 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d59b74-7c79-4900-8ecd-09a875ab3c4a None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:41.212277 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c6559d-8d36-41e9-99cf-4fa771dd6d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:41.212277 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c6559d-8d36-41e9-99cf-4fa771dd6d67 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:41.212573 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c6559d-8d36-41e9-99cf-4fa771dd6d67 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:41.212796 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c6559d-8d36-41e9-99cf-4fa771dd6d67 None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:41.213507 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c6559d-8d36-41e9-99cf-4fa771dd6d67 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:41.213882 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c6559d-8d36-41e9-99cf-4fa771dd6d67 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:41.215286 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c6559d-8d36-41e9-99cf-4fa771dd6d67 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:41.227394 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 251/511] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:10:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:41.228178 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c6559d-8d36-41e9-99cf-4fa771dd6d67 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:41.307354 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:41.311353 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d59b74-7c79-4900-8ecd-09a875ab3c4a None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:41.326723 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c6559d-8d36-41e9-99cf-4fa771dd6d67 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:41.347625 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bef8cc9-25dc-4292-813a-d22d3bbe69db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:41.348301 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bef8cc9-25dc-4292-813a-d22d3bbe69db None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:41.348587 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bef8cc9-25dc-4292-813a-d22d3bbe69db None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:41.348865 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bef8cc9-25dc-4292-813a-d22d3bbe69db None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:41.349704 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bef8cc9-25dc-4292-813a-d22d3bbe69db None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:41.350041 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bef8cc9-25dc-4292-813a-d22d3bbe69db None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:41.351501 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bef8cc9-25dc-4292-813a-d22d3bbe69db None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:41.360973 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e499071-3bf1-4098-ae70-4f422c66af29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:41.361648 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e499071-3bf1-4098-ae70-4f422c66af29 None admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:41.362485 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e499071-3bf1-4098-ae70-4f422c66af29 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:41.362766 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e499071-3bf1-4098-ae70-4f422c66af29 None admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:41.363482 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e499071-3bf1-4098-ae70-4f422c66af29 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:41.365210 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e499071-3bf1-4098-ae70-4f422c66af29 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:41.369616 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 261/512] 10.222.0.57 () {56 vars in 1137 bytes} [Thu May 19 14:10:41 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:41.370476 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bef8cc9-25dc-4292-813a-d22d3bbe69db None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:41.375705 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:41.390284 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 252/513] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 19 14:10:41 2022] POST /identity/v3/users => generated 247 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:10:41.391084 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e499071-3bf1-4098-ae70-4f422c66af29 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:41.495285 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bef8cc9-25dc-4292-813a-d22d3bbe69db None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:41.511282 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e499071-3bf1-4098-ae70-4f422c66af29 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:41.533286 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3448b6-8e42-4a26-aa29-89d7dc9215da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:41.534815 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3448b6-8e42-4a26-aa29-89d7dc9215da None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:41.535100 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3448b6-8e42-4a26-aa29-89d7dc9215da None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:41.535376 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3448b6-8e42-4a26-aa29-89d7dc9215da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:41.547179 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3448b6-8e42-4a26-aa29-89d7dc9215da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:41.548672 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3448b6-8e42-4a26-aa29-89d7dc9215da None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:41.554429 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a889e6c-a72e-4902-925f-8c367e4d2db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:41.555111 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a889e6c-a72e-4902-925f-8c367e4d2db9 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:41.555394 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a889e6c-a72e-4902-925f-8c367e4d2db9 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:41.555673 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a889e6c-a72e-4902-925f-8c367e4d2db9 None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:41.556394 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a889e6c-a72e-4902-925f-8c367e4d2db9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:41.556699 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a889e6c-a72e-4902-925f-8c367e4d2db9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:41.559398 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a889e6c-a72e-4902-925f-8c367e4d2db9 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:41.570388 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-7e3448b6-8e42-4a26-aa29-89d7dc9215da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:10:41.570388 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:41.570388 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:41.570388 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:41.570388 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:41.570388 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:41.570388 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:41.570388 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:41.570388 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:41.570388 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:41.570388 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:41.570388 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:41.570388 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:10:41.570388 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:10:41.570388 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:10:41.571304 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:10:41.571304 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:41.571304 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:41.571304 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:41.571304 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:41.571304 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:41.571304 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:41.571304 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:41.571304 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:10:41.571304 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:41.571304 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:41.571304 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:41.571304 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:41.571304 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:41.571304 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:41.571304 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:41.571304 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:41.573230 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:10:41.573230 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:10:41.573230 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:10:41.573230 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:10:41.573230 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 262/514] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:10:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:10:41.573230 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3448b6-8e42-4a26-aa29-89d7dc9215da None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:41.573554 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 253/515] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:10:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:41.574255 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a889e6c-a72e-4902-925f-8c367e4d2db9 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:41.587895 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:41.643485 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:41.692344 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3448b6-8e42-4a26-aa29-89d7dc9215da None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:41.695264 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a889e6c-a72e-4902-925f-8c367e4d2db9 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:41.728707 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4796e3-5b7f-4081-8a20-2c9583935d55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:41.729503 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4796e3-5b7f-4081-8a20-2c9583935d55 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:41.729787 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4796e3-5b7f-4081-8a20-2c9583935d55 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:41.730091 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4796e3-5b7f-4081-8a20-2c9583935d55 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:41.731507 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141c182b-8a97-462a-81e6-5d67ef1db99e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:41.732160 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141c182b-8a97-462a-81e6-5d67ef1db99e None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:41.732445 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141c182b-8a97-462a-81e6-5d67ef1db99e None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:41.733173 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141c182b-8a97-462a-81e6-5d67ef1db99e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:41.738019 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4796e3-5b7f-4081-8a20-2c9583935d55 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:41.739731 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4796e3-5b7f-4081-8a20-2c9583935d55 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:41.755882 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-da4796e3-5b7f-4081-8a20-2c9583935d55 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 14:10:41.755882 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:41.755882 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:41.755882 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:41.755882 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:41.755882 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:41.755882 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:41.755882 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:41.755882 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:41.755882 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:41.755882 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:41.755882 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:41.755882 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:10:41.755882 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:10:41.755882 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:10:41.755882 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:10:41.756798 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:41.756798 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:41.756798 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:10:41.756798 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:10:41.756798 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:10:41.756798 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:10:41.756798 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:41.756798 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:41.756798 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:41.756798 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:41.756798 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:41.756798 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:10:41.756798 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:41.756798 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:41.756798 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:41.756798 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:41.756798 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:41.756798 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:41.757698 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:41.757698 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:41.757698 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:10:41.757698 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:10:41.757698 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:10:41.757698 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:10:41.757698 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:10:41.757698 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:10:41.757698 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 14:10:41.757698 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:10:41.757698 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 263/516] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:10:41 2022] GET /identity/v3/users/cinder => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:10:41.759347 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4796e3-5b7f-4081-8a20-2c9583935d55 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:41.811581 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141c182b-8a97-462a-81e6-5d67ef1db99e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:41.813716 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141c182b-8a97-462a-81e6-5d67ef1db99e None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:41.829645 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:41.868440 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4796e3-5b7f-4081-8a20-2c9583935d55 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:41.873470 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:41.909712 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a31eeb9-2dfc-49d2-81f8-e22a6110d4ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:41.910415 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a31eeb9-2dfc-49d2-81f8-e22a6110d4ff None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:41.910703 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a31eeb9-2dfc-49d2-81f8-e22a6110d4ff None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:41.910978 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a31eeb9-2dfc-49d2-81f8-e22a6110d4ff None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:41.911702 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a31eeb9-2dfc-49d2-81f8-e22a6110d4ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:41.912011 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a31eeb9-2dfc-49d2-81f8-e22a6110d4ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:41.913410 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a31eeb9-2dfc-49d2-81f8-e22a6110d4ff None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:41.925778 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 264/517] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:10:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:41.926681 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a31eeb9-2dfc-49d2-81f8-e22a6110d4ff None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:41.938010 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 254/518] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:10:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:10:41.938777 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141c182b-8a97-462a-81e6-5d67ef1db99e None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:42.037736 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a31eeb9-2dfc-49d2-81f8-e22a6110d4ff None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:42.051412 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141c182b-8a97-462a-81e6-5d67ef1db99e None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:42.069461 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:42.081423 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cd9169-990d-4138-b242-ea0bd842f08c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:42.082249 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cd9169-990d-4138-b242-ea0bd842f08c None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:42.082536 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cd9169-990d-4138-b242-ea0bd842f08c None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:42.082826 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cd9169-990d-4138-b242-ea0bd842f08c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:42.089968 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cd9169-990d-4138-b242-ea0bd842f08c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:42.090533 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cde7bd0-c955-4e8c-b257-e11a74cecd47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:42.091411 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cde7bd0-c955-4e8c-b257-e11a74cecd47 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:42.091515 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cd9169-990d-4138-b242-ea0bd842f08c None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:42.092135 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cde7bd0-c955-4e8c-b257-e11a74cecd47 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:42.092588 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cde7bd0-c955-4e8c-b257-e11a74cecd47 None admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:42.093603 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cde7bd0-c955-4e8c-b257-e11a74cecd47 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:42.094122 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cde7bd0-c955-4e8c-b257-e11a74cecd47 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:42.095716 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cde7bd0-c955-4e8c-b257-e11a74cecd47 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:42.107323 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-38cd9169-990d-4138-b242-ea0bd842f08c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:10:42.107323 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:42.107323 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:42.107323 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:42.107323 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:42.107323 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:42.107323 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:42.107323 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:42.107323 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:42.107323 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:42.107323 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:42.107323 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:42.107323 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:10:42.107323 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:10:42.107323 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:10:42.108222 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:10:42.108222 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:42.108222 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:42.108222 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:42.108222 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:42.108222 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:42.108222 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:42.108222 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:42.108222 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:10:42.108222 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:42.108222 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:42.108222 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:42.108222 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:42.108222 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:42.108222 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:42.108222 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:42.108222 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:42.109072 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:10:42.109072 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:10:42.109072 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:10:42.109072 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:10:42.109072 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:10:42.109072 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:10:42.109072 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:10:42.109072 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:10:42.113862 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 265/519] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:10:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:10:42.114868 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cd9169-990d-4138-b242-ea0bd842f08c None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:42.138164 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 255/520] 10.222.0.57 () {56 vars in 1137 bytes} [Thu May 19 14:10:41 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:42.139324 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cde7bd0-c955-4e8c-b257-e11a74cecd47 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:42.154071 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:42.229075 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cd9169-990d-4138-b242-ea0bd842f08c None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:42.253216 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cde7bd0-c955-4e8c-b257-e11a74cecd47 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:42.265722 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea3589d-c804-4605-8983-7cc1ac5bd09d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:42.266557 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea3589d-c804-4605-8983-7cc1ac5bd09d None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:42.266844 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea3589d-c804-4605-8983-7cc1ac5bd09d None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:42.267122 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea3589d-c804-4605-8983-7cc1ac5bd09d None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:42.275289 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea3589d-c804-4605-8983-7cc1ac5bd09d None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:42.276781 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea3589d-c804-4605-8983-7cc1ac5bd09d None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:42.291174 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abfcd05-32c5-43c7-9354-ae33cabce36d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:42.291888 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abfcd05-32c5-43c7-9354-ae33cabce36d None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:42.292176 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abfcd05-32c5-43c7-9354-ae33cabce36d None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:42.292453 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abfcd05-32c5-43c7-9354-ae33cabce36d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:42.292545 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-7ea3589d-c804-4605-8983-7cc1ac5bd09d None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 19 14:10:42.292545 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:42.292545 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:42.292545 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:42.292545 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:42.292545 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:42.292545 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:42.292545 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:42.292545 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:42.292545 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:42.292545 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:42.292545 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:42.292545 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:10:42.292545 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:10:42.292545 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:10:42.292545 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:10:42.293385 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:42.293385 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:42.293385 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:10:42.293385 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:10:42.293385 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:10:42.293385 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:10:42.293385 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:42.293385 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:42.293385 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:42.293385 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:42.293385 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:42.293385 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:10:42.293385 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:42.293385 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:42.293385 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:42.293385 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:42.293385 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:42.293385 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:42.294823 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:42.294823 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:42.294823 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:10:42.294823 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:10:42.294823 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:10:42.294823 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:10:42.294823 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:10:42.294823 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:10:42.294823 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 19 14:10:42.294823 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:10:42.294823 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 266/521] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 14:10:42 2022] GET /identity/v3/users/neutron => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:10:42.294823 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea3589d-c804-4605-8983-7cc1ac5bd09d None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:42.310168 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abfcd05-32c5-43c7-9354-ae33cabce36d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:42.311924 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abfcd05-32c5-43c7-9354-ae33cabce36d None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:42.330061 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-3abfcd05-32c5-43c7-9354-ae33cabce36d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:10:42.330061 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:42.330061 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:42.330061 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:42.330061 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:42.330061 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:42.330061 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:42.330061 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:42.330061 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:42.330061 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:42.330061 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:42.330061 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:42.330061 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:10:42.330061 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:10:42.330061 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:10:42.331708 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:10:42.331708 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:42.331708 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:42.331708 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:42.331708 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:42.331708 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:42.331708 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:42.331708 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:42.331708 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:10:42.331708 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:42.331708 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:42.331708 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:42.331708 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:42.331708 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:42.331708 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:42.331708 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:42.331708 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:42.332744 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:10:42.332744 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:10:42.332744 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:10:42.332744 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:10:42.334270 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 256/522] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:10:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:10:42.335177 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abfcd05-32c5-43c7-9354-ae33cabce36d None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:42.350150 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:42.401533 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:42.409344 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea3589d-c804-4605-8983-7cc1ac5bd09d None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:42.443167 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abfcd05-32c5-43c7-9354-ae33cabce36d None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:42.447601 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96129fb-6f67-4643-8314-58a40dafeadb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:42.448251 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96129fb-6f67-4643-8314-58a40dafeadb None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:42.448535 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96129fb-6f67-4643-8314-58a40dafeadb None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:42.448810 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96129fb-6f67-4643-8314-58a40dafeadb None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:42.449588 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96129fb-6f67-4643-8314-58a40dafeadb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:42.449920 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96129fb-6f67-4643-8314-58a40dafeadb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:42.451368 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96129fb-6f67-4643-8314-58a40dafeadb None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:42.469895 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 267/523] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:10:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:42.470611 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96129fb-6f67-4643-8314-58a40dafeadb None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:42.483529 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd39f2c3-5ce9-44e2-81de-beffdc91e587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:42.484197 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd39f2c3-5ce9-44e2-81de-beffdc91e587 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:42.484469 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd39f2c3-5ce9-44e2-81de-beffdc91e587 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:42.484744 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd39f2c3-5ce9-44e2-81de-beffdc91e587 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:42.556454 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd39f2c3-5ce9-44e2-81de-beffdc91e587 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:42.557948 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd39f2c3-5ce9-44e2-81de-beffdc91e587 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:42.582186 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96129fb-6f67-4643-8314-58a40dafeadb None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:42.593874 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:42.623876 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:42.629379 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b03909-abd6-4fb7-8b17-ea66a9959bb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:42.630551 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b03909-abd6-4fb7-8b17-ea66a9959bb3 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:42.630837 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b03909-abd6-4fb7-8b17-ea66a9959bb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:42.631113 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b03909-abd6-4fb7-8b17-ea66a9959bb3 None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:42.631958 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b03909-abd6-4fb7-8b17-ea66a9959bb3 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:42.632270 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b03909-abd6-4fb7-8b17-ea66a9959bb3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:42.633732 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b03909-abd6-4fb7-8b17-ea66a9959bb3 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:42.657675 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 268/524] 10.222.0.57 () {56 vars in 1139 bytes} [Thu May 19 14:10:42 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:42.657675 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b03909-abd6-4fb7-8b17-ea66a9959bb3 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:42.681368 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 257/525] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:10:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:10:42.681368 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd39f2c3-5ce9-44e2-81de-beffdc91e587 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:42.786908 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b03909-abd6-4fb7-8b17-ea66a9959bb3 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:42.797642 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd39f2c3-5ce9-44e2-81de-beffdc91e587 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:42.840893 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f062b595-2c32-4b6e-90a7-ee14d7e00870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:42.841476 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f062b595-2c32-4b6e-90a7-ee14d7e00870 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:42.841762 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f062b595-2c32-4b6e-90a7-ee14d7e00870 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:42.842074 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f062b595-2c32-4b6e-90a7-ee14d7e00870 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:42.842807 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f062b595-2c32-4b6e-90a7-ee14d7e00870 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:42.843121 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f062b595-2c32-4b6e-90a7-ee14d7e00870 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:42.844532 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f062b595-2c32-4b6e-90a7-ee14d7e00870 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:42.852435 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844e4210-9638-430e-8152-49d43115fef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:42.853498 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844e4210-9638-430e-8152-49d43115fef4 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:42.853689 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844e4210-9638-430e-8152-49d43115fef4 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:42.854019 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844e4210-9638-430e-8152-49d43115fef4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:42.855068 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844e4210-9638-430e-8152-49d43115fef4 None admin] RBAC: Adding query filter params (role.id=9c7d4d848dd549adb9f3465d4201f26d, scope.project.id=c6f995ffa0a94b1584f25dbe31e20524, user.id=8c87f361a92d4834854ca07f652be2ff) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:42.855396 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844e4210-9638-430e-8152-49d43115fef4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:42.857221 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 269/526] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:10:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:42.857425 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844e4210-9638-430e-8152-49d43115fef4 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:42.857924 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f062b595-2c32-4b6e-90a7-ee14d7e00870 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:42.869032 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 258/527] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:10:42 2022] GET /identity/v3/role_assignments?user.id=8c87f361a92d4834854ca07f652be2ff&scope.project.id=c6f995ffa0a94b1584f25dbe31e20524&role.id=9c7d4d848dd549adb9f3465d4201f26d => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:42.870692 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844e4210-9638-430e-8152-49d43115fef4 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:42.896125 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:42.920343 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:43.012023 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f062b595-2c32-4b6e-90a7-ee14d7e00870 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:43.021943 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844e4210-9638-430e-8152-49d43115fef4 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:43.056789 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7811e8b-811c-4839-b160-d73dda35f2d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:43.057834 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7811e8b-811c-4839-b160-d73dda35f2d0 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:43.058431 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7811e8b-811c-4839-b160-d73dda35f2d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:43.058726 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7811e8b-811c-4839-b160-d73dda35f2d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:43.064412 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8fd486-2cf5-44ce-b0db-d9a8e4df80db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:43.066275 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8fd486-2cf5-44ce-b0db-d9a8e4df80db None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:43.066796 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8fd486-2cf5-44ce-b0db-d9a8e4df80db None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:43.067269 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8fd486-2cf5-44ce-b0db-d9a8e4df80db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:43.079468 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8fd486-2cf5-44ce-b0db-d9a8e4df80db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:43.081386 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8fd486-2cf5-44ce-b0db-d9a8e4df80db None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:43.100839 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-ee8fd486-2cf5-44ce-b0db-d9a8e4df80db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:10:43.100839 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:43.100839 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:43.100839 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:43.100839 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:43.100839 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:43.100839 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:43.100839 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:43.100839 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:43.100839 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:43.100839 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:43.100839 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:43.100839 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:10:43.100839 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:10:43.100839 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:10:43.101889 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:10:43.101889 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:43.101889 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:43.101889 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:43.101889 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:43.101889 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:43.101889 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:43.101889 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:43.101889 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:10:43.101889 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:43.101889 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:43.101889 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:43.101889 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:43.101889 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:43.101889 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:43.101889 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:43.101889 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:43.102782 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:10:43.102782 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:10:43.102782 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:10:43.102782 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:10:43.104713 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 259/528] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:10:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:10:43.105772 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8fd486-2cf5-44ce-b0db-d9a8e4df80db None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:43.155086 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7811e8b-811c-4839-b160-d73dda35f2d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:43.156860 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7811e8b-811c-4839-b160-d73dda35f2d0 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:43.189590 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:43.196128 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:43.218916 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8fd486-2cf5-44ce-b0db-d9a8e4df80db None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:43.254892 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01b474c-23da-4485-a3da-07edec7b9f4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:43.255929 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01b474c-23da-4485-a3da-07edec7b9f4a None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:43.256376 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01b474c-23da-4485-a3da-07edec7b9f4a None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:43.256789 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01b474c-23da-4485-a3da-07edec7b9f4a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:43.264146 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01b474c-23da-4485-a3da-07edec7b9f4a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:43.265789 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01b474c-23da-4485-a3da-07edec7b9f4a None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:43.280959 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 270/529] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:10:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:10:43.281669 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-d01b474c-23da-4485-a3da-07edec7b9f4a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:10:43.281669 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:43.281669 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:43.281669 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:43.281669 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:43.281669 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:43.281669 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:43.281669 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:43.281669 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:43.281669 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:43.281669 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:43.281669 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:43.281669 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:10:43.281669 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:10:43.281669 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:10:43.282682 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:10:43.282682 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:43.282682 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:43.282682 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:43.282682 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:43.282682 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:43.282682 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:43.282682 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:43.282682 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:10:43.282682 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:43.282682 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:43.282682 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:43.282682 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:43.282682 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:43.282682 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:43.282682 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:43.282682 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:43.284258 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:10:43.284258 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:10:43.284258 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:10:43.284258 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:10:43.284258 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:10:43.284258 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:10:43.284258 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:10:43.284258 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:10:43.284906 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7811e8b-811c-4839-b160-d73dda35f2d0 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:43.286311 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 260/530] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:10:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:10:43.287308 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01b474c-23da-4485-a3da-07edec7b9f4a None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:43.409753 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7811e8b-811c-4839-b160-d73dda35f2d0 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:43.410988 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01b474c-23da-4485-a3da-07edec7b9f4a None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:43.438353 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:43.448136 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e47a9f-9ae4-4205-b05c-0883e3b64a76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:43.449083 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e47a9f-9ae4-4205-b05c-0883e3b64a76 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:43.449541 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e47a9f-9ae4-4205-b05c-0883e3b64a76 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:43.450010 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e47a9f-9ae4-4205-b05c-0883e3b64a76 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:43.450900 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e47a9f-9ae4-4205-b05c-0883e3b64a76 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:43.451378 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e47a9f-9ae4-4205-b05c-0883e3b64a76 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:43.453255 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e47a9f-9ae4-4205-b05c-0883e3b64a76 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:43.455612 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b46fabe-e558-4eca-9065-f66e8628b842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:43.456417 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b46fabe-e558-4eca-9065-f66e8628b842 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:43.456702 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b46fabe-e558-4eca-9065-f66e8628b842 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:43.456962 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b46fabe-e558-4eca-9065-f66e8628b842 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:43.467449 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 271/531] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:10:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:43.468508 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:43.468769 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e47a9f-9ae4-4205-b05c-0883e3b64a76 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:43.470225 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b46fabe-e558-4eca-9065-f66e8628b842 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:43.472940 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b46fabe-e558-4eca-9065-f66e8628b842 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:43.494342 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-0b46fabe-e558-4eca-9065-f66e8628b842 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:10:43.494342 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:43.494342 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:43.494342 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:43.494342 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:43.494342 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:43.494342 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:43.494342 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:43.494342 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:43.494342 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:43.494342 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:43.494342 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:43.494342 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:10:43.494342 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:10:43.494342 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:10:43.495259 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:10:43.495259 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:43.495259 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:43.495259 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:43.495259 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:43.495259 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:43.495259 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:43.495259 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:43.495259 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:10:43.495259 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:43.495259 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:43.495259 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:43.495259 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:43.495259 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:43.495259 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:43.495259 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:43.495259 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:43.496099 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:10:43.496099 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:10:43.496099 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:10:43.496099 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:10:43.496099 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 261/532] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:10:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:10:43.496364 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b46fabe-e558-4eca-9065-f66e8628b842 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:43.583813 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e47a9f-9ae4-4205-b05c-0883e3b64a76 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:43.606627 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b46fabe-e558-4eca-9065-f66e8628b842 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:43.620863 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831e8998-a6c5-41dd-9897-73cbd5cb5a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:43.621870 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831e8998-a6c5-41dd-9897-73cbd5cb5a77 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:43.622371 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831e8998-a6c5-41dd-9897-73cbd5cb5a77 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:43.622861 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831e8998-a6c5-41dd-9897-73cbd5cb5a77 None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:43.623836 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831e8998-a6c5-41dd-9897-73cbd5cb5a77 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:43.624336 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831e8998-a6c5-41dd-9897-73cbd5cb5a77 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:43.625998 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831e8998-a6c5-41dd-9897-73cbd5cb5a77 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:43.639071 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 272/533] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:10:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:43.640432 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831e8998-a6c5-41dd-9897-73cbd5cb5a77 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:43.642593 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e1ef5c-e377-44ba-94bb-6f77ce4528c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:43.643933 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e1ef5c-e377-44ba-94bb-6f77ce4528c2 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:43.644488 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e1ef5c-e377-44ba-94bb-6f77ce4528c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:43.644778 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e1ef5c-e377-44ba-94bb-6f77ce4528c2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:43.652182 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e1ef5c-e377-44ba-94bb-6f77ce4528c2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:43.653663 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e1ef5c-e377-44ba-94bb-6f77ce4528c2 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:43.669654 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-f5e1ef5c-e377-44ba-94bb-6f77ce4528c2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:10:43.669654 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:43.669654 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:43.669654 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:43.669654 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:43.669654 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:43.669654 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:43.669654 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:43.669654 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:43.669654 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:43.669654 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:43.669654 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:43.669654 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:10:43.669654 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:10:43.669654 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:10:43.670542 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:10:43.670542 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:43.670542 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:43.670542 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:43.670542 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:43.670542 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:43.670542 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:43.670542 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:43.670542 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:10:43.670542 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:43.670542 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:43.670542 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:43.670542 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:43.670542 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:43.670542 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:43.670542 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:43.670542 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:43.671375 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:10:43.671375 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:10:43.671375 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:10:43.671375 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:10:43.671375 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:10:43.671375 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:10:43.671375 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:10:43.671375 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:10:43.671375 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 262/534] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:10:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:10:43.672316 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e1ef5c-e377-44ba-94bb-6f77ce4528c2 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:43.695985 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:43.727947 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:43.755568 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831e8998-a6c5-41dd-9897-73cbd5cb5a77 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:43.785250 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e1ef5c-e377-44ba-94bb-6f77ce4528c2 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:43.793835 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa91591-a0a0-4564-8aaa-6b53eaf9ca2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:43.794551 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa91591-a0a0-4564-8aaa-6b53eaf9ca2f None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:43.794842 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa91591-a0a0-4564-8aaa-6b53eaf9ca2f None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:43.795129 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa91591-a0a0-4564-8aaa-6b53eaf9ca2f None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:43.795855 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa91591-a0a0-4564-8aaa-6b53eaf9ca2f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:43.796169 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa91591-a0a0-4564-8aaa-6b53eaf9ca2f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:43.797562 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa91591-a0a0-4564-8aaa-6b53eaf9ca2f None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:43.813941 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 273/535] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:10:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:43.813941 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa91591-a0a0-4564-8aaa-6b53eaf9ca2f None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:43.827849 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39da2e84-9219-4830-a1c8-52e01e650407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:43.828725 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39da2e84-9219-4830-a1c8-52e01e650407 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:43.828911 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39da2e84-9219-4830-a1c8-52e01e650407 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:43.829193 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39da2e84-9219-4830-a1c8-52e01e650407 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:43.830122 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39da2e84-9219-4830-a1c8-52e01e650407 None admin] RBAC: Adding query filter params (role.id=9c7d4d848dd549adb9f3465d4201f26d, scope.project.id=c6f995ffa0a94b1584f25dbe31e20524, user.id=0b91de04b5014f379b758fd83435b263) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:43.830372 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39da2e84-9219-4830-a1c8-52e01e650407 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:43.832691 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39da2e84-9219-4830-a1c8-52e01e650407 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:43.845881 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 263/536] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:10:43 2022] GET /identity/v3/role_assignments?user.id=0b91de04b5014f379b758fd83435b263&scope.project.id=c6f995ffa0a94b1584f25dbe31e20524&role.id=9c7d4d848dd549adb9f3465d4201f26d => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:43.845881 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d9f964-8c35-4418-999b-fbc0a4f0134d None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:43.845881 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d9f964-8c35-4418-999b-fbc0a4f0134d None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:43.845881 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d9f964-8c35-4418-999b-fbc0a4f0134d None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:43.845881 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 264/537] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:10:43 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:10:43.845881 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d9f964-8c35-4418-999b-fbc0a4f0134d None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:43.940736 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa91591-a0a0-4564-8aaa-6b53eaf9ca2f None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:43.949117 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:43.984474 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09935e4c-fa15-4efd-99b0-4d95398c3ea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:43.985303 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09935e4c-fa15-4efd-99b0-4d95398c3ea2 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:43.985684 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09935e4c-fa15-4efd-99b0-4d95398c3ea2 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:43.986107 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09935e4c-fa15-4efd-99b0-4d95398c3ea2 None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:43.987709 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09935e4c-fa15-4efd-99b0-4d95398c3ea2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:43.988113 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09935e4c-fa15-4efd-99b0-4d95398c3ea2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:43.990170 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09935e4c-fa15-4efd-99b0-4d95398c3ea2 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:43.998806 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d9f964-8c35-4418-999b-fbc0a4f0134d None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:44.006603 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 274/538] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:10:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:44.008143 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d0762a-3631-4beb-8cea-02e04b91b037 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:44.008533 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d0762a-3631-4beb-8cea-02e04b91b037 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:44.008907 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d0762a-3631-4beb-8cea-02e04b91b037 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:44.018647 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:44.027382 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-02d0762a-3631-4beb-8cea-02e04b91b037 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:10:44.028766 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-02d0762a-3631-4beb-8cea-02e04b91b037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:10:44.048838 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3738bcfd-9fbe-4ffb-b4d8-64673073c540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:44.050638 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3738bcfd-9fbe-4ffb-b4d8-64673073c540 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:44.051209 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3738bcfd-9fbe-4ffb-b4d8-64673073c540 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:44.051496 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3738bcfd-9fbe-4ffb-b4d8-64673073c540 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:44.060659 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3738bcfd-9fbe-4ffb-b4d8-64673073c540 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:44.062563 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3738bcfd-9fbe-4ffb-b4d8-64673073c540 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:44.080609 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-3738bcfd-9fbe-4ffb-b4d8-64673073c540 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:10:44.080609 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:44.080609 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:44.080609 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:44.080609 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:44.080609 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:44.080609 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:44.080609 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:44.080609 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:44.080609 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:44.080609 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:44.080609 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:44.080609 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:10:44.080609 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:10:44.080609 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:10:44.081646 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:10:44.081646 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:44.081646 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:44.081646 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:44.081646 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:44.081646 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:44.081646 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:44.081646 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:44.081646 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:10:44.081646 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:44.081646 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:44.081646 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:44.081646 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:44.081646 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:44.081646 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:44.081646 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:44.081646 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:44.082584 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:10:44.082584 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:10:44.082584 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:10:44.082584 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:10:44.082584 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:10:44.082584 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:10:44.082584 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:10:44.082584 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:10:44.082584 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 265/539] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:10:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:10:44.083053 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3738bcfd-9fbe-4ffb-b4d8-64673073c540 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:44.146013 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 275/540] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:10:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:10:44.146013 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d0762a-3631-4beb-8cea-02e04b91b037 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:44.228822 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3738bcfd-9fbe-4ffb-b4d8-64673073c540 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:44.256583 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:44.262790 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d0762a-3631-4beb-8cea-02e04b91b037 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:44.271785 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c350d9-93f0-4aab-9be3-17d0ad719dbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:44.272737 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c350d9-93f0-4aab-9be3-17d0ad719dbe None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:44.273674 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c350d9-93f0-4aab-9be3-17d0ad719dbe None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:44.274247 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c350d9-93f0-4aab-9be3-17d0ad719dbe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:44.275789 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c350d9-93f0-4aab-9be3-17d0ad719dbe None admin] RBAC: Adding query filter params (role.id=9c7d4d848dd549adb9f3465d4201f26d, scope.project.id=c6f995ffa0a94b1584f25dbe31e20524, user.id=fefdec9076444495ad679120510fe271) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:44.276425 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c350d9-93f0-4aab-9be3-17d0ad719dbe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:44.278260 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c350d9-93f0-4aab-9be3-17d0ad719dbe None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:44.286821 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 266/541] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:10:44 2022] GET /identity/v3/role_assignments?user.id=fefdec9076444495ad679120510fe271&scope.project.id=c6f995ffa0a94b1584f25dbe31e20524&role.id=9c7d4d848dd549adb9f3465d4201f26d => generated 259 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:44.288190 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c350d9-93f0-4aab-9be3-17d0ad719dbe None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:44.304890 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1307a7-dba7-46de-84bd-66e84205ae2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:44.305564 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1307a7-dba7-46de-84bd-66e84205ae2a None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:44.305888 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1307a7-dba7-46de-84bd-66e84205ae2a None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:44.306570 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1307a7-dba7-46de-84bd-66e84205ae2a None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:44.307392 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1307a7-dba7-46de-84bd-66e84205ae2a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:44.307991 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1307a7-dba7-46de-84bd-66e84205ae2a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:44.309774 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1307a7-dba7-46de-84bd-66e84205ae2a None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:44.327629 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 276/542] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:10:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:44.337416 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1307a7-dba7-46de-84bd-66e84205ae2a None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:44.363908 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:44.476976 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1307a7-dba7-46de-84bd-66e84205ae2a None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:44.513302 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ca2be8-f521-429d-b30c-198d79ca8925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:44.514358 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ca2be8-f521-429d-b30c-198d79ca8925 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:44.514871 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ca2be8-f521-429d-b30c-198d79ca8925 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:44.515343 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ca2be8-f521-429d-b30c-198d79ca8925 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:44.516418 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ca2be8-f521-429d-b30c-198d79ca8925 None admin] RBAC: Adding query filter params (role.id=9c7d4d848dd549adb9f3465d4201f26d, scope.project.id=c6f995ffa0a94b1584f25dbe31e20524, user.id=56ce23600bfe42e6b98eeebfe7470560) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:44.516975 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ca2be8-f521-429d-b30c-198d79ca8925 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:44.519506 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ca2be8-f521-429d-b30c-198d79ca8925 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:44.526116 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 277/543] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:10:44 2022] GET /identity/v3/role_assignments?user.id=56ce23600bfe42e6b98eeebfe7470560&scope.project.id=c6f995ffa0a94b1584f25dbe31e20524&role.id=9c7d4d848dd549adb9f3465d4201f26d => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:44.544205 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c350d9-93f0-4aab-9be3-17d0ad719dbe None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:44.581393 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb4e22f-ef28-4d22-9aab-745e12ea4d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:44.582663 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb4e22f-ef28-4d22-9aab-745e12ea4d7f None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:44.583173 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb4e22f-ef28-4d22-9aab-745e12ea4d7f None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:44.583680 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb4e22f-ef28-4d22-9aab-745e12ea4d7f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:44.596784 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb4e22f-ef28-4d22-9aab-745e12ea4d7f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:10:44.605091 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-bcb4e22f-ef28-4d22-9aab-745e12ea4d7f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:10:44.605091 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:44.605091 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:44.605091 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:44.605091 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:44.605091 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:44.605091 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:44.605091 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:44.605091 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:44.605091 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:44.605091 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:44.605091 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:44.605091 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:10:44.605091 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:10:44.605091 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:10:44.605091 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:10:44.606004 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:10:44.606004 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:10:44.606004 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:10:44.606004 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:10:44.606004 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:44.606004 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:44.606004 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:44.606004 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:44.606004 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:44.606004 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:44.606004 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:44.606004 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:10:44.606004 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:44.606004 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:44.606004 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:44.606004 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:44.606793 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:44.606793 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:44.606793 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:44.606793 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:44.606793 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:10:44.606793 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:10:44.606793 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:10:44.606793 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:10:44.606793 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:10:44.606793 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:10:44.606793 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:10:44.606793 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:10:44.609574 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 267/544] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 14:10:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:10:44.616116 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ca2be8-f521-429d-b30c-198d79ca8925 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:44.646242 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:44.731340 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ca2be8-f521-429d-b30c-198d79ca8925 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:44.770883 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7474dbb-fc52-48e3-9889-94022a0bcb2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:44.771952 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7474dbb-fc52-48e3-9889-94022a0bcb2a None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:44.772459 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7474dbb-fc52-48e3-9889-94022a0bcb2a None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:44.772940 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7474dbb-fc52-48e3-9889-94022a0bcb2a None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:44.773927 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7474dbb-fc52-48e3-9889-94022a0bcb2a None admin] RBAC: Adding query filter params (name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:44.774438 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7474dbb-fc52-48e3-9889-94022a0bcb2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:44.776060 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7474dbb-fc52-48e3-9889-94022a0bcb2a None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:44.790225 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 278/545] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 19 14:10:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:44.791977 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb4e22f-ef28-4d22-9aab-745e12ea4d7f None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:44.926728 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:44.930755 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb4e22f-ef28-4d22-9aab-745e12ea4d7f None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:44.967045 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b59c4a-6277-4204-93c9-64deed82c12a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:44.967855 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b59c4a-6277-4204-93c9-64deed82c12a None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:44.968152 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b59c4a-6277-4204-93c9-64deed82c12a None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:44.968406 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b59c4a-6277-4204-93c9-64deed82c12a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:44.979392 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b59c4a-6277-4204-93c9-64deed82c12a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:44.980842 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b59c4a-6277-4204-93c9-64deed82c12a None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:44.998950 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-13b59c4a-6277-4204-93c9-64deed82c12a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:10:44.998950 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:44.998950 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:44.998950 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:44.998950 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:44.998950 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:44.998950 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:44.998950 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:44.998950 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:44.998950 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:44.998950 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:44.998950 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:44.998950 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:10:44.998950 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:10:44.998950 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:10:44.999829 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:10:44.999829 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:44.999829 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:44.999829 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:44.999829 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:44.999829 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:44.999829 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:44.999829 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:44.999829 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:10:44.999829 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:44.999829 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:44.999829 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:44.999829 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:44.999829 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:44.999829 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:44.999829 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:44.999829 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:45.000685 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:10:45.000685 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:10:45.000685 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:10:45.000685 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:10:45.000685 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 268/546] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:10:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:10:45.006231 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7474dbb-fc52-48e3-9889-94022a0bcb2a None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:45.102757 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:45.128746 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7474dbb-fc52-48e3-9889-94022a0bcb2a None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:45.186754 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbff410-099c-4174-81c2-722ba254707e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:45.187424 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbff410-099c-4174-81c2-722ba254707e None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:45.187711 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbff410-099c-4174-81c2-722ba254707e None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:45.187989 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbff410-099c-4174-81c2-722ba254707e None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:45.188711 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbff410-099c-4174-81c2-722ba254707e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:45.189017 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbff410-099c-4174-81c2-722ba254707e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:45.191244 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbff410-099c-4174-81c2-722ba254707e None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:45.215597 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 279/547] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:10:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:45.220479 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b59c4a-6277-4204-93c9-64deed82c12a None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:45.365789 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b59c4a-6277-4204-93c9-64deed82c12a None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:45.403391 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ae17ed-4a29-48ed-9619-9c5fdb808968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:45.404190 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ae17ed-4a29-48ed-9619-9c5fdb808968 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:45.404468 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ae17ed-4a29-48ed-9619-9c5fdb808968 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:45.404739 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ae17ed-4a29-48ed-9619-9c5fdb808968 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:45.412955 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ae17ed-4a29-48ed-9619-9c5fdb808968 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:45.414433 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ae17ed-4a29-48ed-9619-9c5fdb808968 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:45.429573 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-66ae17ed-4a29-48ed-9619-9c5fdb808968 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 14:10:45.429573 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:45.429573 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:45.429573 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:45.429573 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:45.429573 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:45.429573 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:45.429573 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:45.429573 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:45.429573 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:45.429573 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:45.429573 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:45.429573 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:10:45.429573 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:10:45.429573 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:10:45.429573 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:10:45.431056 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:45.431056 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:45.431056 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:10:45.431056 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:10:45.431056 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:10:45.431056 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:10:45.431056 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:45.431056 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:45.431056 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:45.431056 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:45.431056 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:45.431056 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:10:45.431056 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:45.431056 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:45.431056 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:45.431056 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:45.431056 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:45.431056 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:45.432357 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:45.432357 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:45.432357 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:10:45.432357 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:10:45.432357 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:10:45.432357 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:10:45.432357 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:10:45.432357 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:10:45.432357 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 14:10:45.432357 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:10:45.432357 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 269/548] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:10:45 2022] GET /identity/v3/users/swift => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:10:45.436774 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbff410-099c-4174-81c2-722ba254707e None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:45.474420 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0676d675-ab90-4b3f-bcf8-87250a84dce2 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:45.475047 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0676d675-ab90-4b3f-bcf8-87250a84dce2 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:45.475545 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0676d675-ab90-4b3f-bcf8-87250a84dce2 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:45.477358 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 270/549] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:10:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:10:45.483316 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb0f7be-5164-497b-a1e1-9404e82dd373 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:45.483851 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb0f7be-5164-497b-a1e1-9404e82dd373 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:45.484325 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb0f7be-5164-497b-a1e1-9404e82dd373 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:45.501890 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-2eb0f7be-5164-497b-a1e1-9404e82dd373 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:10:45.504091 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-2eb0f7be-5164-497b-a1e1-9404e82dd373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:10:45.507453 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:45.580014 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 271/550] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:10:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:10:45.603513 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb0f7be-5164-497b-a1e1-9404e82dd373 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:45.697818 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbff410-099c-4174-81c2-722ba254707e None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:45.713887 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb0f7be-5164-497b-a1e1-9404e82dd373 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:45.746536 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff09a81f-4dcc-4a57-876d-d04f5fc079d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:45.747566 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff09a81f-4dcc-4a57-876d-d04f5fc079d3 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:45.748053 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff09a81f-4dcc-4a57-876d-d04f5fc079d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:45.748553 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff09a81f-4dcc-4a57-876d-d04f5fc079d3 None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:45.749600 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff09a81f-4dcc-4a57-876d-d04f5fc079d3 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:45.749960 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa9446d-683c-4f9a-a6b8-f6d49b7a9ea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:45.750477 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff09a81f-4dcc-4a57-876d-d04f5fc079d3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:45.750755 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa9446d-683c-4f9a-a6b8-f6d49b7a9ea6 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:45.751036 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa9446d-683c-4f9a-a6b8-f6d49b7a9ea6 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:45.751307 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa9446d-683c-4f9a-a6b8-f6d49b7a9ea6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:45.753435 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff09a81f-4dcc-4a57-876d-d04f5fc079d3 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:45.765978 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa9446d-683c-4f9a-a6b8-f6d49b7a9ea6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:10:45.777864 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-1aa9446d-683c-4f9a-a6b8-f6d49b7a9ea6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:10:45.777864 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:45.777864 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:45.777864 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:45.777864 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:45.777864 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:45.777864 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:45.777864 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:45.777864 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:45.777864 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:45.777864 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:45.777864 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:45.777864 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:10:45.777864 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:10:45.777864 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:10:45.777864 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:10:45.778884 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:10:45.778884 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:10:45.778884 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:10:45.778884 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:10:45.778884 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:45.778884 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:45.778884 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:45.778884 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:45.778884 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:45.778884 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:45.778884 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:45.778884 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:10:45.778884 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:45.778884 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:45.778884 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:45.778884 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:45.779805 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:45.779805 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:45.779805 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:45.779805 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:45.779805 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:10:45.779805 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:10:45.779805 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:10:45.779805 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:10:45.779805 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:10:45.779805 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:10:45.779805 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:10:45.779805 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:10:45.779805 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:45.779805 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 272/551] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 14:10:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:10:45.782133 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa9446d-683c-4f9a-a6b8-f6d49b7a9ea6 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:45.782628 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:45.793711 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 280/552] 10.222.0.57 () {56 vars in 1135 bytes} [Thu May 19 14:10:45 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:45.799306 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff09a81f-4dcc-4a57-876d-d04f5fc079d3 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:45.979863 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa9446d-683c-4f9a-a6b8-f6d49b7a9ea6 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:45.997280 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff09a81f-4dcc-4a57-876d-d04f5fc079d3 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:46.055865 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79aea87-0176-4ea0-a631-2bb69cb3b0ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:46.056895 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79aea87-0176-4ea0-a631-2bb69cb3b0ae None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:46.057423 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79aea87-0176-4ea0-a631-2bb69cb3b0ae None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:46.057962 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79aea87-0176-4ea0-a631-2bb69cb3b0ae None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:46.059013 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79aea87-0176-4ea0-a631-2bb69cb3b0ae None admin] RBAC: Adding query filter params (name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:46.061251 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79aea87-0176-4ea0-a631-2bb69cb3b0ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:46.062884 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79aea87-0176-4ea0-a631-2bb69cb3b0ae None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:46.073725 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57e97c7-c7ae-43dc-88e2-6b1b8d5b2e90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:46.074699 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57e97c7-c7ae-43dc-88e2-6b1b8d5b2e90 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:46.075183 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57e97c7-c7ae-43dc-88e2-6b1b8d5b2e90 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:46.075658 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57e97c7-c7ae-43dc-88e2-6b1b8d5b2e90 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:46.080158 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 273/553] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 19 14:10:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:46.085622 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79aea87-0176-4ea0-a631-2bb69cb3b0ae None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:46.162236 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:46.170278 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57e97c7-c7ae-43dc-88e2-6b1b8d5b2e90 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:46.172139 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57e97c7-c7ae-43dc-88e2-6b1b8d5b2e90 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:46.243243 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:46.325565 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79aea87-0176-4ea0-a631-2bb69cb3b0ae None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:46.360241 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 281/554] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:10:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 561 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 14:10:46.373079 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57e97c7-c7ae-43dc-88e2-6b1b8d5b2e90 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:46.393695 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4590f454-6394-4dbb-98cc-9d8f55a7009e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:46.394698 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4590f454-6394-4dbb-98cc-9d8f55a7009e None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:46.394973 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4590f454-6394-4dbb-98cc-9d8f55a7009e None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:46.395254 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4590f454-6394-4dbb-98cc-9d8f55a7009e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:46.415906 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4590f454-6394-4dbb-98cc-9d8f55a7009e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:46.417392 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4590f454-6394-4dbb-98cc-9d8f55a7009e None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:46.440282 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-4590f454-6394-4dbb-98cc-9d8f55a7009e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:10:46.440282 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:46.440282 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:46.440282 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:46.440282 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:46.440282 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:46.440282 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:46.440282 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:46.440282 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:46.440282 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:46.440282 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:46.440282 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:46.440282 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:10:46.440282 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:10:46.440282 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:10:46.441320 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:10:46.441320 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:46.441320 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:46.441320 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:46.441320 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:46.441320 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:46.441320 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:46.441320 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:46.441320 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:10:46.441320 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:46.441320 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:46.441320 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:46.441320 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:46.441320 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:46.441320 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:46.441320 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:46.441320 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:46.442216 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:10:46.442216 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:10:46.442216 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:10:46.442216 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:10:46.442216 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 274/555] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:10:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 357 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:10:46.446894 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4590f454-6394-4dbb-98cc-9d8f55a7009e None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:46.575290 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:46.578397 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57e97c7-c7ae-43dc-88e2-6b1b8d5b2e90 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:46.610966 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4590f454-6394-4dbb-98cc-9d8f55a7009e None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:46.632756 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:46.635109 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105dd6b3-6099-4678-aea4-e4608ba1006e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:46.636250 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105dd6b3-6099-4678-aea4-e4608ba1006e None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:46.636544 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105dd6b3-6099-4678-aea4-e4608ba1006e None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:46.636833 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105dd6b3-6099-4678-aea4-e4608ba1006e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:46.654218 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105dd6b3-6099-4678-aea4-e4608ba1006e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:46.656022 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105dd6b3-6099-4678-aea4-e4608ba1006e None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:46.663787 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7c2e9c-6922-4b9a-ab26-628195391fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:46.664800 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7c2e9c-6922-4b9a-ab26-628195391fb8 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:46.665338 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7c2e9c-6922-4b9a-ab26-628195391fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:46.666250 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7c2e9c-6922-4b9a-ab26-628195391fb8 None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:46.667002 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7c2e9c-6922-4b9a-ab26-628195391fb8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:46.667322 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7c2e9c-6922-4b9a-ab26-628195391fb8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:46.668762 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7c2e9c-6922-4b9a-ab26-628195391fb8 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:46.681188 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-105dd6b3-6099-4678-aea4-e4608ba1006e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:10:46.681188 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:46.681188 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:46.681188 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:46.681188 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:46.681188 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:46.681188 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:46.681188 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:46.681188 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:46.681188 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:46.681188 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:46.681188 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:46.681188 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:10:46.681188 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:10:46.681188 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:10:46.682585 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:10:46.682585 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:46.682585 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:46.682585 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:46.682585 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:46.682585 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:46.682585 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:46.682585 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:46.682585 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:10:46.682585 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:46.682585 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:46.682585 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:46.682585 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:46.682585 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:46.682585 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:46.682585 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:46.682585 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:46.683875 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:10:46.683875 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:10:46.683875 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:10:46.683875 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:10:46.690026 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 275/557] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:10:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:46.690026 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9c5cba-73f0-4469-8d0e-d4bb7d8aa588 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:46.690026 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9c5cba-73f0-4469-8d0e-d4bb7d8aa588 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:46.690026 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9c5cba-73f0-4469-8d0e-d4bb7d8aa588 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:46.690396 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 276/558] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:10:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:10:46.690708 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 282/558] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:10:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:10:46.690940 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9c5cba-73f0-4469-8d0e-d4bb7d8aa588 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:46.692486 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105dd6b3-6099-4678-aea4-e4608ba1006e None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:46.800897 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9c5cba-73f0-4469-8d0e-d4bb7d8aa588 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:46.803984 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105dd6b3-6099-4678-aea4-e4608ba1006e None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:46.838365 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb7f5b5-c36a-40b6-8d88-1040c0a9f474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:46.839082 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb7f5b5-c36a-40b6-8d88-1040c0a9f474 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:46.839369 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb7f5b5-c36a-40b6-8d88-1040c0a9f474 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:46.839646 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb7f5b5-c36a-40b6-8d88-1040c0a9f474 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:46.843931 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893ad1ad-649f-47ea-85bb-1b04939740b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:46.844825 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893ad1ad-649f-47ea-85bb-1b04939740b0 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:46.845364 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893ad1ad-649f-47ea-85bb-1b04939740b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:46.845877 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893ad1ad-649f-47ea-85bb-1b04939740b0 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:46.847637 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb7f5b5-c36a-40b6-8d88-1040c0a9f474 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:46.847754 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893ad1ad-649f-47ea-85bb-1b04939740b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:46.848268 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893ad1ad-649f-47ea-85bb-1b04939740b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:46.849078 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb7f5b5-c36a-40b6-8d88-1040c0a9f474 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:46.850100 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893ad1ad-649f-47ea-85bb-1b04939740b0 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:46.869435 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 283/559] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:10:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:10:46.869621 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-edb7f5b5-c36a-40b6-8d88-1040c0a9f474 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 14:10:46.869621 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:46.869621 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:46.869621 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:46.869621 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:46.869621 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:46.869621 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:46.869621 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:46.869621 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:46.869621 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:46.869621 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:46.869621 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:46.869621 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:10:46.869621 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:10:46.869621 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:10:46.869621 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:10:46.871768 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:46.871768 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:46.871768 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:10:46.871768 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:10:46.871768 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:10:46.871768 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:10:46.871768 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:46.871768 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:46.871768 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:46.871768 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:46.871768 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:46.871768 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:10:46.871768 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:46.871768 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:46.871768 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:46.871768 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:46.871768 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:46.871768 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:46.872716 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:46.872716 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:46.872716 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:10:46.872716 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:10:46.872716 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:10:46.872716 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:10:46.872716 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:10:46.872716 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:10:46.872716 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 14:10:46.872716 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:10:46.873264 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1afc4c-578b-4aee-9b2e-90a6c4d4086b None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:46.873264 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1afc4c-578b-4aee-9b2e-90a6c4d4086b None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:46.873264 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1afc4c-578b-4aee-9b2e-90a6c4d4086b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:46.873932 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 277/560] 10.222.0.57 () {56 vars in 1093 bytes} [Thu May 19 14:10:46 2022] GET /identity/v3/users/nova => generated 82 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:10:46.873932 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1914e43f-dbb8-4b6c-9db5-a05397149dca None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:46.873932 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1914e43f-dbb8-4b6c-9db5-a05397149dca None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:46.873932 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1914e43f-dbb8-4b6c-9db5-a05397149dca None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:46.873932 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 278/561] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:10:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:10:46.873932 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1914e43f-dbb8-4b6c-9db5-a05397149dca None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:46.889781 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-4d1afc4c-578b-4aee-9b2e-90a6c4d4086b None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:10:46.891013 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-4d1afc4c-578b-4aee-9b2e-90a6c4d4086b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:10:46.900183 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:46.917898 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:46.965053 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 284/562] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:10:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:10:46.966216 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1afc4c-578b-4aee-9b2e-90a6c4d4086b None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:47.009022 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1914e43f-dbb8-4b6c-9db5-a05397149dca None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:47.044646 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cbed42-d11b-42ed-87a7-176c5e344bee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:47.045444 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cbed42-d11b-42ed-87a7-176c5e344bee None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:47.045730 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cbed42-d11b-42ed-87a7-176c5e344bee None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:47.046041 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cbed42-d11b-42ed-87a7-176c5e344bee None admin] PATH_INFO: `/v3/projects/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:47.053357 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cbed42-d11b-42ed-87a7-176c5e344bee None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:47.054839 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cbed42-d11b-42ed-87a7-176c5e344bee None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:47.071302 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-79cbed42-d11b-42ed-87a7-176c5e344bee None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:10:47.071302 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:47.071302 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:47.071302 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:47.071302 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:47.071302 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:47.071302 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:47.071302 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:47.071302 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:47.071302 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:47.071302 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:47.071302 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:47.071302 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:10:47.071302 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:10:47.071302 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:10:47.072326 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:10:47.072326 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:47.072326 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:47.072326 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:47.072326 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:47.072326 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:47.072326 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:47.072326 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:47.072326 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:10:47.072326 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:47.072326 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:47.072326 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:47.072326 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:47.072326 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:47.072326 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:47.072326 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:47.072326 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:47.073283 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:10:47.073283 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:10:47.073283 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:10:47.073283 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:10:47.073283 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:10:47.073283 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:10:47.073283 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:10:47.073283 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:10:47.073283 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 279/563] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:10:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:10:47.074346 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaa731f-2ab0-4a9d-9e9b-9aca23273747 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:47.074346 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaa731f-2ab0-4a9d-9e9b-9aca23273747 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:47.074533 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaa731f-2ab0-4a9d-9e9b-9aca23273747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:47.087768 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1afc4c-578b-4aee-9b2e-90a6c4d4086b None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:47.092871 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-faaa731f-2ab0-4a9d-9e9b-9aca23273747 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:10:47.094053 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-faaa731f-2ab0-4a9d-9e9b-9aca23273747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:10:47.129041 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b877385-b5a1-4b1c-8704-1ef74da05566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:47.129712 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b877385-b5a1-4b1c-8704-1ef74da05566 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:47.130019 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b877385-b5a1-4b1c-8704-1ef74da05566 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:47.130288 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b877385-b5a1-4b1c-8704-1ef74da05566 None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:47.131124 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b877385-b5a1-4b1c-8704-1ef74da05566 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:47.131424 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b877385-b5a1-4b1c-8704-1ef74da05566 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:47.132860 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b877385-b5a1-4b1c-8704-1ef74da05566 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:47.147755 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:47.157773 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 285/564] 10.222.0.57 () {56 vars in 1133 bytes} [Thu May 19 14:10:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:47.159076 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b877385-b5a1-4b1c-8704-1ef74da05566 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:47.185868 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 280/565] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:10:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:10:47.185868 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaa731f-2ab0-4a9d-9e9b-9aca23273747 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:47.213536 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:47.283953 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b877385-b5a1-4b1c-8704-1ef74da05566 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:47.299073 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaa731f-2ab0-4a9d-9e9b-9aca23273747 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:47.318424 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ab8c57-e327-4fb0-96bc-0483c4df310c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:47.319315 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ab8c57-e327-4fb0-96bc-0483c4df310c None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:47.319668 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ab8c57-e327-4fb0-96bc-0483c4df310c None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:47.320020 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ab8c57-e327-4fb0-96bc-0483c4df310c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:47.331071 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ab8c57-e327-4fb0-96bc-0483c4df310c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:10:47.337531 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f449ab1-916b-4ed7-929e-d1cf217b8757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:47.338307 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f449ab1-916b-4ed7-929e-d1cf217b8757 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:47.338671 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f449ab1-916b-4ed7-929e-d1cf217b8757 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:47.339020 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f449ab1-916b-4ed7-929e-d1cf217b8757 None admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:47.339202 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-08ab8c57-e327-4fb0-96bc-0483c4df310c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:10:47.339202 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:47.339202 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:47.339202 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:47.339202 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:47.339202 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:47.339202 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:47.339202 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:47.339202 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:47.339202 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:47.339202 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:47.339202 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:47.339202 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:10:47.339202 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:10:47.339202 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:10:47.339202 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:10:47.340345 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:10:47.340345 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:10:47.340345 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:10:47.340345 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:10:47.340345 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:47.340345 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:47.340345 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:47.340345 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:47.340345 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:47.340345 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:47.340345 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:47.340345 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:10:47.340345 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:47.340345 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:47.340345 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:47.340345 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:47.341395 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:47.341395 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:47.341395 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:47.341395 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:47.341395 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:10:47.341395 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:10:47.341395 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:10:47.341395 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:10:47.341395 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:10:47.341395 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:10:47.341395 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:10:47.341395 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:10:47.347739 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f449ab1-916b-4ed7-929e-d1cf217b8757 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:47.347847 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 286/566] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 14:10:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:10:47.347940 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f449ab1-916b-4ed7-929e-d1cf217b8757 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:47.348030 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ab8c57-e327-4fb0-96bc-0483c4df310c None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:47.348143 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f449ab1-916b-4ed7-929e-d1cf217b8757 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:47.366236 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 281/567] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:10:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:47.367154 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f449ab1-916b-4ed7-929e-d1cf217b8757 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:47.406504 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:47.451338 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ab8c57-e327-4fb0-96bc-0483c4df310c None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:47.456581 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:47.473107 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f449ab1-916b-4ed7-929e-d1cf217b8757 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:47.484695 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3741750-0bb0-4cd9-98cb-36b7b7f6b76f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:47.485479 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3741750-0bb0-4cd9-98cb-36b7b7f6b76f None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:47.485787 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3741750-0bb0-4cd9-98cb-36b7b7f6b76f None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:47.486091 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3741750-0bb0-4cd9-98cb-36b7b7f6b76f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:47.497259 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3741750-0bb0-4cd9-98cb-36b7b7f6b76f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:47.498734 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3741750-0bb0-4cd9-98cb-36b7b7f6b76f None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:47.508639 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff34c5fd-3bd6-428a-9b49-019a6dea5200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:47.509431 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff34c5fd-3bd6-428a-9b49-019a6dea5200 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:47.509714 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff34c5fd-3bd6-428a-9b49-019a6dea5200 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:47.510032 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff34c5fd-3bd6-428a-9b49-019a6dea5200 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:47.515957 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-f3741750-0bb0-4cd9-98cb-36b7b7f6b76f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:10:47.515957 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:47.515957 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:47.515957 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:47.515957 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:47.515957 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:47.515957 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:47.515957 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:47.515957 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:47.515957 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:47.515957 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:47.515957 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:47.515957 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:10:47.515957 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:10:47.515957 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:10:47.517002 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:10:47.517002 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:47.517002 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:47.517002 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:47.517002 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:47.517002 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:47.517002 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:47.517002 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:47.517002 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:10:47.517002 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:47.517002 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:47.517002 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:47.517002 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:47.517002 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:47.517002 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:47.517002 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:47.517002 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:47.518219 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:10:47.518219 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:10:47.518219 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:10:47.518219 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:10:47.518219 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 287/568] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:10:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:10:47.518564 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3741750-0bb0-4cd9-98cb-36b7b7f6b76f None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:47.522771 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff34c5fd-3bd6-428a-9b49-019a6dea5200 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:10:47.530557 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-ff34c5fd-3bd6-428a-9b49-019a6dea5200 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:10:47.530557 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:47.530557 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:47.530557 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:47.530557 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:47.530557 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:47.530557 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:47.530557 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:47.530557 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:47.530557 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:47.530557 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:47.530557 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:47.530557 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:10:47.530557 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:10:47.530557 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:10:47.530557 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:10:47.531663 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:10:47.531663 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:10:47.531663 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:10:47.531663 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:10:47.531663 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:47.531663 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:47.531663 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:47.531663 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:47.531663 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:47.531663 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:47.531663 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:47.531663 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:10:47.531663 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:47.531663 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:47.531663 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:47.531663 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:47.532703 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:47.532703 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:47.532703 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:47.532703 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:47.532703 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:10:47.532703 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:10:47.532703 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:10:47.532703 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:10:47.532703 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:10:47.532703 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:10:47.532703 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:10:47.532703 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:10:47.532703 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 282/569] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 14:10:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:10:47.533597 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f2ed52-1f12-4eca-99dd-3e19683c7517 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:47.533597 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f2ed52-1f12-4eca-99dd-3e19683c7517 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:47.533597 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f2ed52-1f12-4eca-99dd-3e19683c7517 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:47.535160 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 283/570] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:10:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:10:47.535868 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f2ed52-1f12-4eca-99dd-3e19683c7517 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:47.625640 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3741750-0bb0-4cd9-98cb-36b7b7f6b76f None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:47.636932 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f2ed52-1f12-4eca-99dd-3e19683c7517 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:47.638864 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:47.661344 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c081619a-7cd8-4465-8656-da2a3d318b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:47.662047 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c081619a-7cd8-4465-8656-da2a3d318b96 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:47.662332 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c081619a-7cd8-4465-8656-da2a3d318b96 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:47.662607 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c081619a-7cd8-4465-8656-da2a3d318b96 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:47.663353 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c081619a-7cd8-4465-8656-da2a3d318b96 None admin] RBAC: Adding query filter params (name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:47.663659 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c081619a-7cd8-4465-8656-da2a3d318b96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:47.665048 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c081619a-7cd8-4465-8656-da2a3d318b96 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:47.671700 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765961ec-4af1-4245-8bec-392c46add27b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:47.672356 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765961ec-4af1-4245-8bec-392c46add27b None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:47.672641 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765961ec-4af1-4245-8bec-392c46add27b None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:47.672924 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765961ec-4af1-4245-8bec-392c46add27b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:47.673066 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 288/571] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 19 14:10:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:47.673748 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c081619a-7cd8-4465-8656-da2a3d318b96 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:47.673882 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765961ec-4af1-4245-8bec-392c46add27b None admin] RBAC: Adding query filter params (role.id=9c7d4d848dd549adb9f3465d4201f26d, scope.project.id=c6f995ffa0a94b1584f25dbe31e20524, user.id=09dd957f216a46dd8a3f400fcd2ba944) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:47.674850 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765961ec-4af1-4245-8bec-392c46add27b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:47.676181 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765961ec-4af1-4245-8bec-392c46add27b None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:47.693878 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 284/572] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:10:47 2022] GET /identity/v3/role_assignments?user.id=09dd957f216a46dd8a3f400fcd2ba944&scope.project.id=c6f995ffa0a94b1584f25dbe31e20524&role.id=9c7d4d848dd549adb9f3465d4201f26d => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:47.693878 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765961ec-4af1-4245-8bec-392c46add27b None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:47.705872 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:47.790460 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c081619a-7cd8-4465-8656-da2a3d318b96 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:47.803609 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765961ec-4af1-4245-8bec-392c46add27b None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:47.828289 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8479c6ed-3fe1-4c8f-8d3e-7dce3fdf1590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:47.828969 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8479c6ed-3fe1-4c8f-8d3e-7dce3fdf1590 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:47.829258 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8479c6ed-3fe1-4c8f-8d3e-7dce3fdf1590 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:47.829535 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8479c6ed-3fe1-4c8f-8d3e-7dce3fdf1590 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:47.830352 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8479c6ed-3fe1-4c8f-8d3e-7dce3fdf1590 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:47.830670 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8479c6ed-3fe1-4c8f-8d3e-7dce3fdf1590 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:47.832123 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8479c6ed-3fe1-4c8f-8d3e-7dce3fdf1590 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:47.842442 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c595f5-179f-4059-9302-4b77d09e7133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:47.843475 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c595f5-179f-4059-9302-4b77d09e7133 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:47.843931 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c595f5-179f-4059-9302-4b77d09e7133 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:47.844388 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c595f5-179f-4059-9302-4b77d09e7133 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:47.844833 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 289/573] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:10:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:47.845514 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c595f5-179f-4059-9302-4b77d09e7133 None admin] RBAC: Adding query filter params (name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:47.846572 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c595f5-179f-4059-9302-4b77d09e7133 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:47.846806 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0e2ab7-6e33-4911-ab97-55f921132498 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:47.847185 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0e2ab7-6e33-4911-ab97-55f921132498 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:47.847511 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0e2ab7-6e33-4911-ab97-55f921132498 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:47.848950 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c595f5-179f-4059-9302-4b77d09e7133 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:47.859369 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 285/574] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 19 14:10:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:47.860584 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c595f5-179f-4059-9302-4b77d09e7133 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:47.872016 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-7b0e2ab7-6e33-4911-ab97-55f921132498 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:10:47.873196 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-7b0e2ab7-6e33-4911-ab97-55f921132498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:10:47.902386 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:47.939516 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:47.957886 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 290/575] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:10:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:10:47.957886 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0e2ab7-6e33-4911-ab97-55f921132498 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:47.987668 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c595f5-179f-4059-9302-4b77d09e7133 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:48.023668 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56121c3-699f-4851-95e8-0354d5481ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:48.024466 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56121c3-699f-4851-95e8-0354d5481ea5 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:48.024751 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56121c3-699f-4851-95e8-0354d5481ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:48.025022 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56121c3-699f-4851-95e8-0354d5481ea5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:48.037297 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56121c3-699f-4851-95e8-0354d5481ea5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:48.038780 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56121c3-699f-4851-95e8-0354d5481ea5 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:48.055441 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0e2ab7-6e33-4911-ab97-55f921132498 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:48.057161 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-e56121c3-699f-4851-95e8-0354d5481ea5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:10:48.057161 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:48.057161 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:48.057161 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:48.057161 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:48.057161 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:48.057161 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:48.057161 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:48.057161 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:48.057161 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:48.057161 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:48.057161 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:48.057161 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:10:48.057161 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:10:48.057161 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:10:48.058220 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:10:48.058220 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:48.058220 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:48.058220 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:48.058220 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:48.058220 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:48.058220 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:48.058220 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:48.058220 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:10:48.058220 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:48.058220 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:48.058220 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:48.058220 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:48.058220 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:48.058220 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:48.058220 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:48.058220 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:48.059114 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:10:48.059114 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:10:48.059114 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:10:48.059114 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:10:48.059114 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 286/576] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:10:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:10:48.059393 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56121c3-699f-4851-95e8-0354d5481ea5 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:48.099675 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5201f3bc-5891-41b3-b1ee-ed873c229a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:48.100474 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5201f3bc-5891-41b3-b1ee-ed873c229a49 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:48.100764 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5201f3bc-5891-41b3-b1ee-ed873c229a49 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:48.101042 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5201f3bc-5891-41b3-b1ee-ed873c229a49 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:48.109899 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5201f3bc-5891-41b3-b1ee-ed873c229a49 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:48.111666 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5201f3bc-5891-41b3-b1ee-ed873c229a49 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:48.126587 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-5201f3bc-5891-41b3-b1ee-ed873c229a49 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:10:48.126587 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:48.126587 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:48.126587 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:48.126587 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:48.126587 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:48.126587 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:48.126587 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:48.126587 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:48.126587 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:48.126587 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:48.126587 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:48.126587 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:10:48.126587 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:10:48.126587 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:10:48.127807 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:10:48.127807 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:48.127807 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:48.127807 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:48.127807 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:48.127807 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:48.127807 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:48.127807 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:48.127807 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:10:48.127807 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:48.127807 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:48.127807 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:48.127807 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:48.127807 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:48.127807 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:48.127807 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:48.127807 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:48.129006 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:10:48.129006 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:10:48.129006 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:10:48.129006 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:10:48.129006 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:10:48.129006 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:10:48.129006 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:10:48.129006 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:10:48.130866 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 291/577] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:10:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:10:48.131835 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5201f3bc-5891-41b3-b1ee-ed873c229a49 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:48.162589 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:48.179043 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56121c3-699f-4851-95e8-0354d5481ea5 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:48.203551 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:48.217307 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c099e5-a177-42c6-be5d-77ae251b4796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:48.218233 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c099e5-a177-42c6-be5d-77ae251b4796 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:48.218515 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c099e5-a177-42c6-be5d-77ae251b4796 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:48.218788 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c099e5-a177-42c6-be5d-77ae251b4796 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:48.231972 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c099e5-a177-42c6-be5d-77ae251b4796 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:48.233431 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c099e5-a177-42c6-be5d-77ae251b4796 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:48.251402 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5201f3bc-5891-41b3-b1ee-ed873c229a49 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:48.254558 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-b7c099e5-a177-42c6-be5d-77ae251b4796 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:10:48.254558 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:48.254558 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:48.254558 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:48.254558 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:48.254558 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:48.254558 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:48.254558 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:48.254558 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:48.254558 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:48.254558 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:48.254558 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:48.254558 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:10:48.254558 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:10:48.254558 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:10:48.255456 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:10:48.255456 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:48.255456 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:48.255456 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:48.255456 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:48.255456 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:48.255456 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:48.255456 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:48.255456 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:10:48.255456 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:48.255456 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:48.255456 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:48.255456 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:48.255456 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:48.255456 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:48.255456 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:48.255456 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:48.256342 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:10:48.256342 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:10:48.256342 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:10:48.256342 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:10:48.256342 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 287/578] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:10:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:10:48.256342 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c099e5-a177-42c6-be5d-77ae251b4796 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:48.295299 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2bb308-e5c8-4cd6-8add-26be56117958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:48.296093 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2bb308-e5c8-4cd6-8add-26be56117958 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:48.296375 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2bb308-e5c8-4cd6-8add-26be56117958 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:48.296651 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2bb308-e5c8-4cd6-8add-26be56117958 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:48.307122 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2bb308-e5c8-4cd6-8add-26be56117958 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:10:48.314748 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-ff2bb308-e5c8-4cd6-8add-26be56117958 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:10:48.314748 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:48.314748 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:48.314748 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:48.314748 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:48.314748 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:48.314748 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:48.314748 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:48.314748 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:48.314748 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:48.314748 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:48.314748 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:48.314748 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:10:48.314748 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:10:48.314748 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:10:48.314748 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:10:48.315685 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:10:48.315685 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:10:48.315685 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:10:48.315685 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:10:48.315685 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:48.315685 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:48.315685 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:48.315685 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:48.315685 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:48.315685 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:48.315685 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:48.315685 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:10:48.315685 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:48.315685 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:48.315685 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:48.315685 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:48.316534 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:48.316534 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:48.316534 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:48.316534 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:48.316534 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:10:48.316534 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:10:48.316534 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:10:48.316534 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:10:48.316534 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:10:48.316534 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:10:48.316534 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:10:48.316534 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:10:48.316534 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 292/579] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 14:10:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:10:48.317223 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2bb308-e5c8-4cd6-8add-26be56117958 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:48.371528 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c099e5-a177-42c6-be5d-77ae251b4796 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:48.407226 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374978c0-a53e-42c4-b91d-fe088755672b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:48.407895 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374978c0-a53e-42c4-b91d-fe088755672b None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:48.408179 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374978c0-a53e-42c4-b91d-fe088755672b None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:48.408985 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374978c0-a53e-42c4-b91d-fe088755672b None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:48.409718 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374978c0-a53e-42c4-b91d-fe088755672b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:48.410058 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374978c0-a53e-42c4-b91d-fe088755672b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:48.410262 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:48.411448 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374978c0-a53e-42c4-b91d-fe088755672b None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:48.423680 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 288/580] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:10:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:48.424568 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374978c0-a53e-42c4-b91d-fe088755672b None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:48.433878 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2bb308-e5c8-4cd6-8add-26be56117958 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:48.461349 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:48.472463 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b513d575-0e99-4acd-a23c-e6951c19cc3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:48.473126 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b513d575-0e99-4acd-a23c-e6951c19cc3f None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:48.473413 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b513d575-0e99-4acd-a23c-e6951c19cc3f None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:48.473688 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b513d575-0e99-4acd-a23c-e6951c19cc3f None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:48.474515 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b513d575-0e99-4acd-a23c-e6951c19cc3f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:48.474828 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b513d575-0e99-4acd-a23c-e6951c19cc3f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:48.476263 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b513d575-0e99-4acd-a23c-e6951c19cc3f None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:48.488752 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 293/581] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:10:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:48.489762 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b513d575-0e99-4acd-a23c-e6951c19cc3f None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:48.538659 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374978c0-a53e-42c4-b91d-fe088755672b None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:48.574570 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d81806-9bc8-4ca9-837a-9deb44d8deb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:48.575351 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d81806-9bc8-4ca9-837a-9deb44d8deb9 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:48.575634 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d81806-9bc8-4ca9-837a-9deb44d8deb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:48.575908 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d81806-9bc8-4ca9-837a-9deb44d8deb9 None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:48.576711 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d81806-9bc8-4ca9-837a-9deb44d8deb9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:48.577019 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d81806-9bc8-4ca9-837a-9deb44d8deb9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:48.578668 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d81806-9bc8-4ca9-837a-9deb44d8deb9 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:48.593817 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b513d575-0e99-4acd-a23c-e6951c19cc3f None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:48.596689 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 289/582] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:10:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:48.597551 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d81806-9bc8-4ca9-837a-9deb44d8deb9 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:48.630191 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89ae0b2-2dd2-4415-b4df-46a19e4ff070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:48.630863 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89ae0b2-2dd2-4415-b4df-46a19e4ff070 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:48.631155 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89ae0b2-2dd2-4415-b4df-46a19e4ff070 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:48.631435 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89ae0b2-2dd2-4415-b4df-46a19e4ff070 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:48.632174 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89ae0b2-2dd2-4415-b4df-46a19e4ff070 None admin] RBAC: Adding query filter params (name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:48.632481 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89ae0b2-2dd2-4415-b4df-46a19e4ff070 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:48.633871 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89ae0b2-2dd2-4415-b4df-46a19e4ff070 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:48.649924 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 294/583] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 19 14:10:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:48.649924 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89ae0b2-2dd2-4415-b4df-46a19e4ff070 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:48.656729 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:48.699338 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:48.707095 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d81806-9bc8-4ca9-837a-9deb44d8deb9 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:48.742827 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907d5f23-3ff8-4ab7-80d4-76c036c8bc83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:48.743633 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907d5f23-3ff8-4ab7-80d4-76c036c8bc83 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:48.743912 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907d5f23-3ff8-4ab7-80d4-76c036c8bc83 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:48.744185 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907d5f23-3ff8-4ab7-80d4-76c036c8bc83 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:48.753119 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907d5f23-3ff8-4ab7-80d4-76c036c8bc83 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:48.754603 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907d5f23-3ff8-4ab7-80d4-76c036c8bc83 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:48.755790 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89ae0b2-2dd2-4415-b4df-46a19e4ff070 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:48.769165 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-907d5f23-3ff8-4ab7-80d4-76c036c8bc83 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 19 14:10:48.769165 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:48.769165 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:48.769165 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:48.769165 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:48.769165 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:48.769165 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:48.769165 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:48.769165 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:48.769165 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:48.769165 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:48.769165 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:48.769165 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:10:48.769165 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:10:48.769165 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:10:48.769165 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:10:48.770386 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:48.770386 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:48.770386 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:10:48.770386 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:10:48.770386 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:10:48.770386 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:10:48.770386 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:48.770386 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:48.770386 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:48.770386 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:48.770386 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:48.770386 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:10:48.770386 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:48.770386 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:48.770386 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:48.770386 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:48.770386 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:48.770386 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:48.771370 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:48.771370 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:48.771370 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:10:48.771370 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:10:48.771370 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:10:48.771370 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:10:48.771370 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:10:48.771370 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:10:48.771370 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 19 14:10:48.771370 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:10:48.771370 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 290/584] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:10:48 2022] GET /identity/v3/users/glance => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:10:48.771983 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907d5f23-3ff8-4ab7-80d4-76c036c8bc83 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:48.799229 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935e47fa-0059-4f3b-ba87-d61336fb768e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:48.800508 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935e47fa-0059-4f3b-ba87-d61336fb768e None admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:48.800957 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935e47fa-0059-4f3b-ba87-d61336fb768e None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:48.801476 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935e47fa-0059-4f3b-ba87-d61336fb768e None admin] PATH_INFO: `/v3/projects/c6f995ffa0a94b1584f25dbe31e20524/users/8c87f361a92d4834854ca07f652be2ff/roles/9c7d4d848dd549adb9f3465d4201f26d` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:48.818612 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935e47fa-0059-4f3b-ba87-d61336fb768e None admin] RBAC: Authorizing `identity:create_grant(project_id=c6f995ffa0a94b1584f25dbe31e20524, user_id=8c87f361a92d4834854ca07f652be2ff, role_id=9c7d4d848dd549adb9f3465d4201f26d)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:48.820371 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935e47fa-0059-4f3b-ba87-d61336fb768e None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:48.834150 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 295/585] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 14:10:48 2022] PUT /identity/v3/projects/c6f995ffa0a94b1584f25dbe31e20524/users/8c87f361a92d4834854ca07f652be2ff/roles/9c7d4d848dd549adb9f3465d4201f26d => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:10:48.835716 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935e47fa-0059-4f3b-ba87-d61336fb768e None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:48.912782 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907d5f23-3ff8-4ab7-80d4-76c036c8bc83 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:48.912987 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:48.962796 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e9725a-e008-4c7f-ab1e-fb14e3e78ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:48.964052 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e9725a-e008-4c7f-ab1e-fb14e3e78ce0 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:48.964340 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e9725a-e008-4c7f-ab1e-fb14e3e78ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:48.964751 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e9725a-e008-4c7f-ab1e-fb14e3e78ce0 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:48.966815 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935e47fa-0059-4f3b-ba87-d61336fb768e None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:48.975110 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e9725a-e008-4c7f-ab1e-fb14e3e78ce0 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:48.977196 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e9725a-e008-4c7f-ab1e-fb14e3e78ce0 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:48.996641 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-18e9725a-e008-4c7f-ab1e-fb14e3e78ce0 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 14:10:48.996641 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:48.996641 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:48.996641 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:48.996641 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:48.996641 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:48.996641 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:48.996641 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:48.996641 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:48.996641 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:48.996641 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:48.996641 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:48.996641 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:10:48.996641 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:10:48.996641 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:10:48.996641 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:10:48.997685 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:48.997685 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:48.997685 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:10:48.997685 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:10:48.997685 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:10:48.997685 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:10:48.997685 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:48.997685 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:48.997685 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:48.997685 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:48.997685 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:48.997685 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:10:48.997685 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:48.997685 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:48.997685 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:48.997685 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:48.997685 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:48.997685 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:48.998678 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:48.998678 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:48.998678 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:10:48.998678 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:10:48.998678 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:10:48.998678 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:10:48.998678 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:10:48.998678 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:10:48.998678 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 14:10:48.998678 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:10:48.998678 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 291/586] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:10:48 2022] GET /identity/v3/users/cinder => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:10:49.003881 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e9725a-e008-4c7f-ab1e-fb14e3e78ce0 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:49.017976 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421db9ad-eef7-4b00-a62c-321285f40043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:49.018844 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421db9ad-eef7-4b00-a62c-321285f40043 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:49.019181 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421db9ad-eef7-4b00-a62c-321285f40043 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:49.019460 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421db9ad-eef7-4b00-a62c-321285f40043 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:49.026409 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:49.032778 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421db9ad-eef7-4b00-a62c-321285f40043 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:49.034780 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421db9ad-eef7-4b00-a62c-321285f40043 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:49.054749 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-421db9ad-eef7-4b00-a62c-321285f40043 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:10:49.054749 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:49.054749 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:49.054749 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:49.054749 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:49.054749 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:49.054749 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:49.054749 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:49.054749 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:49.054749 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:49.054749 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:49.054749 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:49.054749 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:10:49.054749 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:10:49.054749 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:10:49.055717 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:10:49.055717 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:49.055717 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:49.055717 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:49.055717 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:49.055717 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:49.055717 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:49.055717 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:49.055717 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:10:49.055717 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:49.055717 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:49.055717 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:49.055717 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:49.055717 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:49.055717 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:49.055717 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:49.055717 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:49.056612 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:10:49.056612 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:10:49.056612 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:10:49.056612 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:10:49.056612 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 296/587] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:10:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:10:49.057080 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421db9ad-eef7-4b00-a62c-321285f40043 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:49.140440 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e9725a-e008-4c7f-ab1e-fb14e3e78ce0 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:49.191812 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5e86c3-c4e2-4d22-9e7a-8121cf238929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:49.192782 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5e86c3-c4e2-4d22-9e7a-8121cf238929 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:49.193252 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5e86c3-c4e2-4d22-9e7a-8121cf238929 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:49.193374 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421db9ad-eef7-4b00-a62c-321285f40043 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:49.193793 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5e86c3-c4e2-4d22-9e7a-8121cf238929 None admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:49.194904 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5e86c3-c4e2-4d22-9e7a-8121cf238929 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:49.195613 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5e86c3-c4e2-4d22-9e7a-8121cf238929 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:49.198260 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5e86c3-c4e2-4d22-9e7a-8121cf238929 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:49.205051 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:49.225090 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 292/588] 10.222.0.57 () {56 vars in 1137 bytes} [Thu May 19 14:10:48 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:49.227000 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5e86c3-c4e2-4d22-9e7a-8121cf238929 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:49.234553 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63716ba4-347c-4f58-af90-1d286a71b83c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:49.235360 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63716ba4-347c-4f58-af90-1d286a71b83c None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:49.235739 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63716ba4-347c-4f58-af90-1d286a71b83c None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:49.236108 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63716ba4-347c-4f58-af90-1d286a71b83c None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:49.237665 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63716ba4-347c-4f58-af90-1d286a71b83c None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:49.238197 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63716ba4-347c-4f58-af90-1d286a71b83c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:49.240113 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63716ba4-347c-4f58-af90-1d286a71b83c None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:49.264542 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 297/589] 10.222.0.57 () {56 vars in 1137 bytes} [Thu May 19 14:10:49 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:49.265558 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63716ba4-347c-4f58-af90-1d286a71b83c None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:49.313503 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:49.345140 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5e86c3-c4e2-4d22-9e7a-8121cf238929 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:49.382644 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0936788-ea47-4615-a99c-fdcb886a0c0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:49.383454 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0936788-ea47-4615-a99c-fdcb886a0c0b None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:49.383832 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0936788-ea47-4615-a99c-fdcb886a0c0b None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:49.384217 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0936788-ea47-4615-a99c-fdcb886a0c0b None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:49.385169 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0936788-ea47-4615-a99c-fdcb886a0c0b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:49.385511 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63716ba4-347c-4f58-af90-1d286a71b83c None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:49.385715 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0936788-ea47-4615-a99c-fdcb886a0c0b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:49.387138 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0936788-ea47-4615-a99c-fdcb886a0c0b None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:49.399824 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 293/590] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:10:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:49.401034 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0936788-ea47-4615-a99c-fdcb886a0c0b None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:49.426424 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6653c5e2-e381-42ab-b0ef-2236552170e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:49.427502 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6653c5e2-e381-42ab-b0ef-2236552170e0 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:49.427947 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6653c5e2-e381-42ab-b0ef-2236552170e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:49.428371 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6653c5e2-e381-42ab-b0ef-2236552170e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:49.447189 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6653c5e2-e381-42ab-b0ef-2236552170e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:49.449010 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6653c5e2-e381-42ab-b0ef-2236552170e0 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:49.469568 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-6653c5e2-e381-42ab-b0ef-2236552170e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:10:49.469568 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:49.469568 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:49.469568 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:49.469568 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:49.469568 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:49.469568 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:49.469568 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:49.469568 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:49.469568 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:49.469568 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:49.469568 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:49.469568 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:10:49.469568 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:10:49.469568 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:10:49.470688 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:10:49.470688 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:49.470688 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:49.470688 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:49.470688 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:49.470688 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:49.470688 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:49.470688 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:49.470688 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:10:49.470688 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:49.470688 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:49.470688 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:49.470688 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:49.470688 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:49.470688 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:49.470688 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:49.470688 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:49.472283 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:10:49.472283 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:10:49.472283 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:10:49.472283 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:10:49.474439 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 298/591] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:10:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:10:49.475643 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6653c5e2-e381-42ab-b0ef-2236552170e0 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:49.497396 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:49.511421 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0936788-ea47-4615-a99c-fdcb886a0c0b None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:49.541825 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:49.547688 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c6abc8-b379-4f8a-8d51-875e72362acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:49.548593 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c6abc8-b379-4f8a-8d51-875e72362acf None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:49.549081 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c6abc8-b379-4f8a-8d51-875e72362acf None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:49.550279 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c6abc8-b379-4f8a-8d51-875e72362acf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:49.562280 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c6abc8-b379-4f8a-8d51-875e72362acf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:49.563928 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c6abc8-b379-4f8a-8d51-875e72362acf None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:49.583408 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-39c6abc8-b379-4f8a-8d51-875e72362acf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:10:49.583408 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:49.583408 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:49.583408 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:49.583408 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:49.583408 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:49.583408 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:49.583408 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:49.583408 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:49.583408 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:49.583408 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:49.583408 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:49.583408 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:10:49.583408 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:10:49.583408 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:10:49.584473 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:10:49.584473 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:49.584473 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:49.584473 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:49.584473 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:49.584473 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:49.584473 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:49.584473 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:49.584473 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:10:49.584473 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:49.584473 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:49.584473 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:49.584473 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:49.584473 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:49.584473 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:49.584473 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:49.584473 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:49.585357 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:10:49.585357 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:10:49.585357 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:10:49.585357 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:10:49.590097 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 294/592] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:10:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:10:49.591023 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c6abc8-b379-4f8a-8d51-875e72362acf None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:49.594390 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6653c5e2-e381-42ab-b0ef-2236552170e0 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:49.633255 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3754ee29-ad33-4cc1-a6c6-d27989b323a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:49.634793 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3754ee29-ad33-4cc1-a6c6-d27989b323a5 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:49.635273 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3754ee29-ad33-4cc1-a6c6-d27989b323a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:49.635724 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3754ee29-ad33-4cc1-a6c6-d27989b323a5 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:49.644596 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3754ee29-ad33-4cc1-a6c6-d27989b323a5 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:49.646555 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3754ee29-ad33-4cc1-a6c6-d27989b323a5 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:49.663435 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-3754ee29-ad33-4cc1-a6c6-d27989b323a5 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 19 14:10:49.663435 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:49.663435 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:49.663435 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:49.663435 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:49.663435 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:49.663435 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:49.663435 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:49.663435 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:49.663435 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:49.663435 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:49.663435 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:49.663435 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:10:49.663435 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:10:49.663435 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:10:49.663435 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:10:49.664435 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:49.664435 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:49.664435 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:10:49.664435 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:10:49.664435 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:10:49.664435 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:10:49.664435 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:49.664435 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:49.664435 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:49.664435 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:49.664435 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:49.664435 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:10:49.664435 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:49.664435 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:49.664435 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:49.664435 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:49.664435 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:49.664435 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:49.665384 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:49.665384 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:49.665384 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:10:49.665384 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:10:49.665384 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:10:49.665384 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:10:49.665384 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:10:49.665384 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:10:49.665384 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 19 14:10:49.665384 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:10:49.668450 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 299/593] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 14:10:49 2022] GET /identity/v3/users/neutron => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:10:49.669636 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3754ee29-ad33-4cc1-a6c6-d27989b323a5 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:49.701128 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c6abc8-b379-4f8a-8d51-875e72362acf None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:49.737464 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbba4fe-0577-4556-9f2a-4b47382467a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:49.738177 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbba4fe-0577-4556-9f2a-4b47382467a1 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:49.738654 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbba4fe-0577-4556-9f2a-4b47382467a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:49.738997 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbba4fe-0577-4556-9f2a-4b47382467a1 None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:49.740455 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbba4fe-0577-4556-9f2a-4b47382467a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:49.741060 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbba4fe-0577-4556-9f2a-4b47382467a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:49.742536 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbba4fe-0577-4556-9f2a-4b47382467a1 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:49.757516 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:49.759244 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 295/594] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:10:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:49.760708 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbba4fe-0577-4556-9f2a-4b47382467a1 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:49.787704 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3754ee29-ad33-4cc1-a6c6-d27989b323a5 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:49.823686 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:49.828804 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d03282-6800-4923-bcca-cf2ea8291c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:49.829712 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d03282-6800-4923-bcca-cf2ea8291c59 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:49.830765 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d03282-6800-4923-bcca-cf2ea8291c59 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:49.831496 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d03282-6800-4923-bcca-cf2ea8291c59 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:49.833343 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d03282-6800-4923-bcca-cf2ea8291c59 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:49.833864 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d03282-6800-4923-bcca-cf2ea8291c59 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:49.835550 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d03282-6800-4923-bcca-cf2ea8291c59 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:49.849480 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 300/595] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:10:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:49.850716 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d03282-6800-4923-bcca-cf2ea8291c59 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:49.901779 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbba4fe-0577-4556-9f2a-4b47382467a1 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:49.947263 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97a752c-5805-484d-9450-4c8cb2261de5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:49.948007 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97a752c-5805-484d-9450-4c8cb2261de5 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:49.948296 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97a752c-5805-484d-9450-4c8cb2261de5 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:49.948573 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97a752c-5805-484d-9450-4c8cb2261de5 None admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:49.949424 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97a752c-5805-484d-9450-4c8cb2261de5 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:49.949735 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97a752c-5805-484d-9450-4c8cb2261de5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:49.951679 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97a752c-5805-484d-9450-4c8cb2261de5 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:49.964516 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d03282-6800-4923-bcca-cf2ea8291c59 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:49.973093 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 296/596] 10.222.0.57 () {56 vars in 1139 bytes} [Thu May 19 14:10:49 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:49.973964 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97a752c-5805-484d-9450-4c8cb2261de5 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:50.005264 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d3fd0a-7702-4cea-9323-a438d4963dab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:50.006430 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d3fd0a-7702-4cea-9323-a438d4963dab None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:50.006862 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d3fd0a-7702-4cea-9323-a438d4963dab None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:50.007284 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d3fd0a-7702-4cea-9323-a438d4963dab None admin] PATH_INFO: `/v3/projects/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:50.015245 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d3fd0a-7702-4cea-9323-a438d4963dab None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:50.016986 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d3fd0a-7702-4cea-9323-a438d4963dab None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:50.019651 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:50.034220 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-41d3fd0a-7702-4cea-9323-a438d4963dab None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:10:50.034220 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:50.034220 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:50.034220 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:50.034220 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:50.034220 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:50.034220 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:50.034220 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:50.034220 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:50.034220 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:50.034220 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:50.034220 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:50.034220 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:10:50.034220 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:10:50.034220 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:10:50.035203 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:10:50.035203 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:50.035203 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:50.035203 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:50.035203 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:50.035203 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:50.035203 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:50.035203 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:50.035203 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:10:50.035203 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:50.035203 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:50.035203 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:50.035203 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:50.035203 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:50.035203 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:50.035203 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:50.035203 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:50.036091 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:10:50.036091 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:10:50.036091 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:10:50.036091 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:10:50.036091 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:10:50.036091 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:10:50.036091 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:10:50.036091 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:10:50.036706 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 301/597] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:10:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:10:50.038317 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfc2b97-1eee-4ef8-bc86-ef48a4f5e947 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:50.038918 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfc2b97-1eee-4ef8-bc86-ef48a4f5e947 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:50.039220 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfc2b97-1eee-4ef8-bc86-ef48a4f5e947 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:50.046112 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 302/598] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:10:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:10:50.046112 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfc2b97-1eee-4ef8-bc86-ef48a4f5e947 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:50.083865 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97a752c-5805-484d-9450-4c8cb2261de5 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:50.085579 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:50.123762 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8e1a8e-ad8e-4325-a3dc-ede765bb343a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:50.124569 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8e1a8e-ad8e-4325-a3dc-ede765bb343a None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:50.124851 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8e1a8e-ad8e-4325-a3dc-ede765bb343a None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:50.125126 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8e1a8e-ad8e-4325-a3dc-ede765bb343a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:50.133116 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8e1a8e-ad8e-4325-a3dc-ede765bb343a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:50.134593 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8e1a8e-ad8e-4325-a3dc-ede765bb343a None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:50.149154 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-dd8e1a8e-ad8e-4325-a3dc-ede765bb343a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:10:50.149154 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:50.149154 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:50.149154 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:50.149154 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:50.149154 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:50.149154 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:50.149154 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:50.149154 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:50.149154 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:50.149154 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:50.149154 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:50.149154 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:10:50.149154 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:10:50.149154 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:10:50.150150 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:10:50.150150 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:50.150150 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:50.150150 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:50.150150 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:50.150150 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:50.150150 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:50.150150 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:50.150150 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:10:50.150150 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:50.150150 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:50.150150 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:50.150150 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:50.150150 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:50.150150 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:50.150150 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:50.150150 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:50.151046 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:10:50.151046 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:10:50.151046 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:10:50.151046 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:10:50.151046 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:10:50.151046 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:10:50.151046 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:10:50.151046 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:10:50.151046 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 297/599] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:10:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:10:50.151552 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8e1a8e-ad8e-4325-a3dc-ede765bb343a None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:50.170495 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfc2b97-1eee-4ef8-bc86-ef48a4f5e947 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:50.209113 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570a9316-51bb-4d6f-b421-3b23bb6ef8dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:50.209933 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570a9316-51bb-4d6f-b421-3b23bb6ef8dc None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:50.210225 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570a9316-51bb-4d6f-b421-3b23bb6ef8dc None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:50.210501 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570a9316-51bb-4d6f-b421-3b23bb6ef8dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:50.220740 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570a9316-51bb-4d6f-b421-3b23bb6ef8dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:50.222271 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570a9316-51bb-4d6f-b421-3b23bb6ef8dc None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:50.241482 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-570a9316-51bb-4d6f-b421-3b23bb6ef8dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:10:50.241482 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:50.241482 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:50.241482 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:50.241482 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:50.241482 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:50.241482 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:50.241482 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:50.241482 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:50.241482 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:50.241482 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:50.241482 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:50.241482 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:10:50.241482 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:10:50.241482 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:10:50.242466 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:10:50.242466 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:50.242466 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:50.242466 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:50.242466 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:50.242466 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:50.242466 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:50.242466 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:50.242466 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:10:50.242466 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:50.242466 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:50.242466 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:50.242466 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:50.242466 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:50.242466 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:50.242466 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:50.242466 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:50.243386 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:10:50.243386 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:10:50.243386 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:10:50.243386 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:10:50.243386 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 303/600] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:10:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:10:50.244388 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25116a5-9a80-4a36-a651-9e4d79b12fa4 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:50.244388 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25116a5-9a80-4a36-a651-9e4d79b12fa4 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:50.244567 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25116a5-9a80-4a36-a651-9e4d79b12fa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:50.261947 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-f25116a5-9a80-4a36-a651-9e4d79b12fa4 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:10:50.263152 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-f25116a5-9a80-4a36-a651-9e4d79b12fa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:10:50.270912 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8e1a8e-ad8e-4325-a3dc-ede765bb343a None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:50.306562 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41e065e-4c8f-4461-895e-54d2a665a5f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:50.307231 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41e065e-4c8f-4461-895e-54d2a665a5f8 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:50.307610 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41e065e-4c8f-4461-895e-54d2a665a5f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:50.307897 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41e065e-4c8f-4461-895e-54d2a665a5f8 None admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:50.308675 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41e065e-4c8f-4461-895e-54d2a665a5f8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:50.308985 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41e065e-4c8f-4461-895e-54d2a665a5f8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:50.310496 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41e065e-4c8f-4461-895e-54d2a665a5f8 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:50.317230 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:50.328916 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 298/601] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:10:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:50.328916 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41e065e-4c8f-4461-895e-54d2a665a5f8 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:50.345128 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 304/602] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:10:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:10:50.346226 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25116a5-9a80-4a36-a651-9e4d79b12fa4 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:50.357309 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:50.443681 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41e065e-4c8f-4461-895e-54d2a665a5f8 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:50.450637 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25116a5-9a80-4a36-a651-9e4d79b12fa4 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:50.481580 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabc4032-5857-4ec8-a74b-7f5523f112ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:50.482633 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabc4032-5857-4ec8-a74b-7f5523f112ce None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:50.483146 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabc4032-5857-4ec8-a74b-7f5523f112ce None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:50.483518 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabc4032-5857-4ec8-a74b-7f5523f112ce None admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:50.484304 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabc4032-5857-4ec8-a74b-7f5523f112ce None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:50.484427 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc6811d-f074-496c-a06e-da356089c8f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:50.484620 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabc4032-5857-4ec8-a74b-7f5523f112ce None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:50.485085 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc6811d-f074-496c-a06e-da356089c8f7 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:50.485369 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc6811d-f074-496c-a06e-da356089c8f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:50.485644 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc6811d-f074-496c-a06e-da356089c8f7 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:50.486056 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabc4032-5857-4ec8-a74b-7f5523f112ce None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:50.486885 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc6811d-f074-496c-a06e-da356089c8f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:50.487348 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc6811d-f074-496c-a06e-da356089c8f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:50.488929 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc6811d-f074-496c-a06e-da356089c8f7 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:50.505139 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 305/603] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:10:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:50.505139 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc6811d-f074-496c-a06e-da356089c8f7 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:50.505425 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 299/604] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:10:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:50.505948 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabc4032-5857-4ec8-a74b-7f5523f112ce None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:50.554392 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:50.592781 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:50.613341 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabc4032-5857-4ec8-a74b-7f5523f112ce None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:50.622020 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc6811d-f074-496c-a06e-da356089c8f7 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:50.651615 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e30ac1-f7b7-4960-ad22-f1a88850fc8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:50.652440 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e30ac1-f7b7-4960-ad22-f1a88850fc8b None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:50.652723 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e30ac1-f7b7-4960-ad22-f1a88850fc8b None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:50.652999 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e30ac1-f7b7-4960-ad22-f1a88850fc8b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:50.663493 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e30ac1-f7b7-4960-ad22-f1a88850fc8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:10:50.663806 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e80ee17-a424-4168-bedb-f8ee4684c325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:50.664923 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e80ee17-a424-4168-bedb-f8ee4684c325 None admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:50.665405 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e80ee17-a424-4168-bedb-f8ee4684c325 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:50.665931 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e80ee17-a424-4168-bedb-f8ee4684c325 None admin] PATH_INFO: `/v3/projects/c6f995ffa0a94b1584f25dbe31e20524/users/0b91de04b5014f379b758fd83435b263/roles/9c7d4d848dd549adb9f3465d4201f26d` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:50.671267 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-38e30ac1-f7b7-4960-ad22-f1a88850fc8b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:10:50.671267 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:50.671267 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:50.671267 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:50.671267 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:50.671267 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:50.671267 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:50.671267 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:50.671267 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:50.671267 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:50.671267 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:50.671267 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:50.671267 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:10:50.671267 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:10:50.671267 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:10:50.671267 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:10:50.672255 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:10:50.672255 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:10:50.672255 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:10:50.672255 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:10:50.672255 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:50.672255 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:50.672255 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:50.672255 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:50.672255 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:50.672255 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:50.672255 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:50.672255 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:10:50.672255 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:50.672255 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:50.672255 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:50.672255 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:50.678896 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:50.678896 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:50.678896 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:50.678896 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:50.678896 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:10:50.678896 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:10:50.678896 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:10:50.678896 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:10:50.678896 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:10:50.678896 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:10:50.678896 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:10:50.678896 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:10:50.678896 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 300/605] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 14:10:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:10:50.678896 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e30ac1-f7b7-4960-ad22-f1a88850fc8b None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:50.685953 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e80ee17-a424-4168-bedb-f8ee4684c325 None admin] RBAC: Authorizing `identity:create_grant(project_id=c6f995ffa0a94b1584f25dbe31e20524, user_id=0b91de04b5014f379b758fd83435b263, role_id=9c7d4d848dd549adb9f3465d4201f26d)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:50.687423 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e80ee17-a424-4168-bedb-f8ee4684c325 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:50.701653 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 306/606] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 14:10:50 2022] PUT /identity/v3/projects/c6f995ffa0a94b1584f25dbe31e20524/users/0b91de04b5014f379b758fd83435b263/roles/9c7d4d848dd549adb9f3465d4201f26d => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:10:50.702302 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e80ee17-a424-4168-bedb-f8ee4684c325 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:50.822084 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:50.834717 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e30ac1-f7b7-4960-ad22-f1a88850fc8b None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:50.838970 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e80ee17-a424-4168-bedb-f8ee4684c325 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:50.881604 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ce9b70-1c51-4a2d-abcc-fa7b61144051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:50.882584 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ce9b70-1c51-4a2d-abcc-fa7b61144051 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:50.882884 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ce9b70-1c51-4a2d-abcc-fa7b61144051 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:50.883235 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ce9b70-1c51-4a2d-abcc-fa7b61144051 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:50.888267 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54265306-0fc5-4bb2-9451-d99bbf3a2e89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:50.889157 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54265306-0fc5-4bb2-9451-d99bbf3a2e89 None admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:50.889442 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54265306-0fc5-4bb2-9451-d99bbf3a2e89 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:50.889719 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54265306-0fc5-4bb2-9451-d99bbf3a2e89 None admin] PATH_INFO: `/v3/projects/c6f995ffa0a94b1584f25dbe31e20524/users/fefdec9076444495ad679120510fe271/roles/9c7d4d848dd549adb9f3465d4201f26d` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:50.894407 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ce9b70-1c51-4a2d-abcc-fa7b61144051 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:50.895887 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ce9b70-1c51-4a2d-abcc-fa7b61144051 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:50.905763 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:50.910234 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54265306-0fc5-4bb2-9451-d99bbf3a2e89 None admin] RBAC: Authorizing `identity:create_grant(project_id=c6f995ffa0a94b1584f25dbe31e20524, user_id=fefdec9076444495ad679120510fe271, role_id=9c7d4d848dd549adb9f3465d4201f26d)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:50.911706 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54265306-0fc5-4bb2-9451-d99bbf3a2e89 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:50.914246 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-f0ce9b70-1c51-4a2d-abcc-fa7b61144051 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:10:50.914246 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:50.914246 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:50.914246 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:50.914246 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:50.914246 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:50.914246 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:50.914246 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:50.914246 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:50.914246 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:50.914246 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:50.914246 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:50.914246 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:10:50.914246 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:10:50.914246 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:10:50.915136 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:10:50.915136 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:50.915136 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:50.915136 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:50.915136 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:50.915136 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:50.915136 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:50.915136 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:50.915136 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:10:50.915136 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:50.915136 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:50.915136 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:50.915136 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:50.915136 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:50.915136 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:50.915136 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:50.915136 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:50.917512 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:10:50.917512 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:10:50.917512 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:10:50.917512 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:10:50.917512 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:10:50.917512 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:10:50.917512 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:10:50.917512 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:10:50.917512 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 307/607] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:10:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:10:50.917512 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ce9b70-1c51-4a2d-abcc-fa7b61144051 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:50.934842 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 301/608] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 14:10:50 2022] PUT /identity/v3/projects/c6f995ffa0a94b1584f25dbe31e20524/users/fefdec9076444495ad679120510fe271/roles/9c7d4d848dd549adb9f3465d4201f26d => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:10:50.935760 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54265306-0fc5-4bb2-9451-d99bbf3a2e89 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:51.047139 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ce9b70-1c51-4a2d-abcc-fa7b61144051 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:51.087712 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b129b6f7-ef55-4ed4-bb6a-c3f54673fe12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:51.088726 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b129b6f7-ef55-4ed4-bb6a-c3f54673fe12 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:51.089198 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b129b6f7-ef55-4ed4-bb6a-c3f54673fe12 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:51.089689 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b129b6f7-ef55-4ed4-bb6a-c3f54673fe12 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:51.091011 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b129b6f7-ef55-4ed4-bb6a-c3f54673fe12 None admin] RBAC: Adding query filter params (name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:51.091511 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b129b6f7-ef55-4ed4-bb6a-c3f54673fe12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:51.093132 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b129b6f7-ef55-4ed4-bb6a-c3f54673fe12 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:51.103728 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 308/609] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 19 14:10:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:51.109559 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b129b6f7-ef55-4ed4-bb6a-c3f54673fe12 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:51.121708 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54265306-0fc5-4bb2-9451-d99bbf3a2e89 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:51.146257 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:51.235664 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d9a296-f89c-4313-8da9-dca594672f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:51.236926 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d9a296-f89c-4313-8da9-dca594672f1a None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:51.237448 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d9a296-f89c-4313-8da9-dca594672f1a None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:51.237931 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d9a296-f89c-4313-8da9-dca594672f1a None admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:51.238939 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d9a296-f89c-4313-8da9-dca594672f1a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:51.239680 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d9a296-f89c-4313-8da9-dca594672f1a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:51.241298 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d9a296-f89c-4313-8da9-dca594672f1a None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:51.246544 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b129b6f7-ef55-4ed4-bb6a-c3f54673fe12 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:51.260117 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 302/610] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:10:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:51.261813 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a20ad2-af4b-425c-8ebf-2148cef9a15a None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:51.262350 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a20ad2-af4b-425c-8ebf-2148cef9a15a None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:51.262830 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a20ad2-af4b-425c-8ebf-2148cef9a15a None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:51.264611 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 303/611] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:10:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:10:51.270523 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60420567-5ce3-4a36-bf41-a47035b1980a None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:51.271159 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60420567-5ce3-4a36-bf41-a47035b1980a None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:51.271654 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60420567-5ce3-4a36-bf41-a47035b1980a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:51.288268 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f3f752-0953-4e5d-a276-f9c6eae96453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:51.289069 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f3f752-0953-4e5d-a276-f9c6eae96453 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:51.289353 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f3f752-0953-4e5d-a276-f9c6eae96453 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:51.289627 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f3f752-0953-4e5d-a276-f9c6eae96453 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:51.292164 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-60420567-5ce3-4a36-bf41-a47035b1980a None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:10:51.293936 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-60420567-5ce3-4a36-bf41-a47035b1980a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:10:51.306487 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f3f752-0953-4e5d-a276-f9c6eae96453 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:51.308361 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f3f752-0953-4e5d-a276-f9c6eae96453 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:51.327531 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-f2f3f752-0953-4e5d-a276-f9c6eae96453 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:10:51.327531 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:51.327531 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:51.327531 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:51.327531 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:51.327531 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:51.327531 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:51.327531 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:51.327531 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:51.327531 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:51.327531 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:51.327531 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:51.327531 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:10:51.327531 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:10:51.327531 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:10:51.328642 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:10:51.328642 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:51.328642 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:51.328642 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:51.328642 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:51.328642 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:51.328642 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:51.328642 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:51.328642 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:10:51.328642 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:51.328642 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:51.328642 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:51.328642 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:51.328642 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:51.328642 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:51.328642 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:51.328642 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:51.329579 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:10:51.329579 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:10:51.329579 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:10:51.329579 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:10:51.332297 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 309/612] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:10:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:10:51.333392 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f3f752-0953-4e5d-a276-f9c6eae96453 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:51.365489 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:51.404189 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 304/613] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:10:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:10:51.405423 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60420567-5ce3-4a36-bf41-a47035b1980a None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:51.411784 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:51.470611 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f3f752-0953-4e5d-a276-f9c6eae96453 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:51.516352 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ffdabb-be08-4fd9-8c98-055239ae3fe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:51.517276 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ffdabb-be08-4fd9-8c98-055239ae3fe1 None admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:51.517556 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ffdabb-be08-4fd9-8c98-055239ae3fe1 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:51.519234 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ffdabb-be08-4fd9-8c98-055239ae3fe1 None admin] PATH_INFO: `/v3/projects/c6f995ffa0a94b1584f25dbe31e20524/users/56ce23600bfe42e6b98eeebfe7470560/roles/9c7d4d848dd549adb9f3465d4201f26d` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:51.540644 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ffdabb-be08-4fd9-8c98-055239ae3fe1 None admin] RBAC: Authorizing `identity:create_grant(project_id=c6f995ffa0a94b1584f25dbe31e20524, user_id=56ce23600bfe42e6b98eeebfe7470560, role_id=9c7d4d848dd549adb9f3465d4201f26d)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:51.542776 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ffdabb-be08-4fd9-8c98-055239ae3fe1 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:51.560223 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 310/614] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 14:10:51 2022] PUT /identity/v3/projects/c6f995ffa0a94b1584f25dbe31e20524/users/56ce23600bfe42e6b98eeebfe7470560/roles/9c7d4d848dd549adb9f3465d4201f26d => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:10:51.561394 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ffdabb-be08-4fd9-8c98-055239ae3fe1 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:51.562535 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60420567-5ce3-4a36-bf41-a47035b1980a None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:51.633533 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d72c032-2888-4aec-a902-486faa1c83a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:51.634308 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d72c032-2888-4aec-a902-486faa1c83a7 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:51.634598 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d72c032-2888-4aec-a902-486faa1c83a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:51.634879 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d72c032-2888-4aec-a902-486faa1c83a7 None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:51.635942 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d72c032-2888-4aec-a902-486faa1c83a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:51.636260 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d72c032-2888-4aec-a902-486faa1c83a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:51.638568 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d72c032-2888-4aec-a902-486faa1c83a7 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:51.657705 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 305/615] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:10:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:51.666291 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d72c032-2888-4aec-a902-486faa1c83a7 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:51.687135 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ffdabb-be08-4fd9-8c98-055239ae3fe1 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:51.705527 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:51.730704 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d62909-fdbc-43fc-8656-bf283793ff42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:51.732047 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d62909-fdbc-43fc-8656-bf283793ff42 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:51.732541 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d62909-fdbc-43fc-8656-bf283793ff42 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:51.733017 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d62909-fdbc-43fc-8656-bf283793ff42 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:51.739070 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:51.752712 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d62909-fdbc-43fc-8656-bf283793ff42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:10:51.761485 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-d2d62909-fdbc-43fc-8656-bf283793ff42 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:10:51.761485 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:51.761485 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:51.761485 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:51.761485 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:51.761485 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:51.761485 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:51.761485 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:51.761485 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:51.761485 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:51.761485 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:51.761485 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:51.761485 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:10:51.761485 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:10:51.761485 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:10:51.761485 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:10:51.762481 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:10:51.762481 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:10:51.762481 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:10:51.762481 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:10:51.762481 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:51.762481 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:51.762481 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:51.762481 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:51.762481 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:51.762481 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:51.762481 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:51.762481 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:10:51.762481 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:51.762481 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:51.762481 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:51.762481 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:51.763334 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:51.763334 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:51.763334 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:51.763334 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:51.763334 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:10:51.763334 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:10:51.763334 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:10:51.763334 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:10:51.763334 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:10:51.763334 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:10:51.763334 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:10:51.763334 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:10:51.771239 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 311/616] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 14:10:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:10:51.772909 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d62909-fdbc-43fc-8656-bf283793ff42 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:51.822315 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d72c032-2888-4aec-a902-486faa1c83a7 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:51.872730 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103d783e-0dea-432e-8d2a-f5424d1afe76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:51.873641 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103d783e-0dea-432e-8d2a-f5424d1afe76 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:51.876724 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103d783e-0dea-432e-8d2a-f5424d1afe76 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:51.876724 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103d783e-0dea-432e-8d2a-f5424d1afe76 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:51.890231 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103d783e-0dea-432e-8d2a-f5424d1afe76 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:51.890231 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103d783e-0dea-432e-8d2a-f5424d1afe76 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:51.905701 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-103d783e-0dea-432e-8d2a-f5424d1afe76 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 14:10:51.905701 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:51.905701 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:51.905701 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:51.905701 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:51.905701 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:51.905701 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:51.905701 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:51.905701 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:51.905701 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:51.905701 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:51.905701 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:51.905701 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:10:51.905701 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:10:51.905701 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:10:51.905701 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:10:51.906749 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:51.906749 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:51.906749 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:10:51.906749 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:10:51.906749 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:10:51.906749 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:10:51.906749 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:51.906749 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:51.906749 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:51.906749 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:51.906749 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:51.906749 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:10:51.906749 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:51.906749 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:51.906749 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:51.906749 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:51.906749 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:51.906749 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:51.908952 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:51.908952 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:51.908952 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:10:51.908952 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:10:51.908952 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:10:51.908952 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:10:51.908952 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:10:51.908952 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:10:51.908952 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 14:10:51.908952 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:10:51.908952 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 306/617] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:10:51 2022] GET /identity/v3/users/swift => generated 83 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:10:51.916176 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103d783e-0dea-432e-8d2a-f5424d1afe76 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:51.993232 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d62909-fdbc-43fc-8656-bf283793ff42 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:52.046181 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc056b9-fd3a-481d-aa0e-d0805a6e2214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:52.047249 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc056b9-fd3a-481d-aa0e-d0805a6e2214 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:52.047760 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc056b9-fd3a-481d-aa0e-d0805a6e2214 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:52.048242 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc056b9-fd3a-481d-aa0e-d0805a6e2214 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:52.049355 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc056b9-fd3a-481d-aa0e-d0805a6e2214 None admin] RBAC: Adding query filter params (name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:52.049954 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc056b9-fd3a-481d-aa0e-d0805a6e2214 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:52.051638 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc056b9-fd3a-481d-aa0e-d0805a6e2214 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:52.067065 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103d783e-0dea-432e-8d2a-f5424d1afe76 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:52.067065 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:52.067594 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 312/618] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 19 14:10:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:52.069353 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc056b9-fd3a-481d-aa0e-d0805a6e2214 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:52.107118 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327fd2e1-6cc9-4c89-b12a-8b562945284d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:52.107676 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327fd2e1-6cc9-4c89-b12a-8b562945284d None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:52.107963 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327fd2e1-6cc9-4c89-b12a-8b562945284d None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:52.108238 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327fd2e1-6cc9-4c89-b12a-8b562945284d None admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:52.113882 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327fd2e1-6cc9-4c89-b12a-8b562945284d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:52.113882 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327fd2e1-6cc9-4c89-b12a-8b562945284d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:52.113882 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327fd2e1-6cc9-4c89-b12a-8b562945284d None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:52.133871 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:52.134066 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 307/619] 10.222.0.57 () {56 vars in 1135 bytes} [Thu May 19 14:10:51 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:10:52.136750 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327fd2e1-6cc9-4c89-b12a-8b562945284d None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:52.244493 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc056b9-fd3a-481d-aa0e-d0805a6e2214 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:52.248268 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327fd2e1-6cc9-4c89-b12a-8b562945284d None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:52.290034 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6240f2e5-74bd-498f-a66d-309b185064c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:52.290847 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6240f2e5-74bd-498f-a66d-309b185064c5 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:52.291136 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6240f2e5-74bd-498f-a66d-309b185064c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:52.291415 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6240f2e5-74bd-498f-a66d-309b185064c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:52.291518 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb3b8e3-6352-4fec-b20e-6717687e4d64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:52.292405 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb3b8e3-6352-4fec-b20e-6717687e4d64 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:52.292696 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb3b8e3-6352-4fec-b20e-6717687e4d64 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:52.292974 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb3b8e3-6352-4fec-b20e-6717687e4d64 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:52.304719 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb3b8e3-6352-4fec-b20e-6717687e4d64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:52.306515 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb3b8e3-6352-4fec-b20e-6717687e4d64 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:52.311964 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6240f2e5-74bd-498f-a66d-309b185064c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:52.313447 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6240f2e5-74bd-498f-a66d-309b185064c5 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:52.325769 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-2bb3b8e3-6352-4fec-b20e-6717687e4d64 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:10:52.325769 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:52.325769 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:52.325769 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:52.325769 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:52.325769 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:52.325769 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:52.325769 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:52.325769 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:52.325769 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:52.325769 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:52.325769 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:52.325769 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:10:52.325769 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:10:52.325769 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:10:52.326686 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:10:52.326686 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:52.326686 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:52.326686 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:52.326686 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:52.326686 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:52.326686 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:52.326686 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:52.326686 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:10:52.326686 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:52.326686 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:52.326686 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:52.326686 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:52.326686 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:52.326686 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:52.326686 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:52.326686 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:52.327580 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:10:52.327580 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:10:52.327580 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:10:52.327580 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:10:52.327580 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 308/620] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:10:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:10:52.333280 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb3b8e3-6352-4fec-b20e-6717687e4d64 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:52.346974 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-6240f2e5-74bd-498f-a66d-309b185064c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:10:52.346974 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:52.346974 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:52.346974 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:52.346974 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:52.346974 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:52.346974 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:52.346974 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:52.346974 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:52.346974 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:52.346974 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:52.346974 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:52.346974 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:10:52.346974 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:10:52.346974 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:10:52.349022 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:10:52.349022 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:52.349022 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:52.349022 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:52.349022 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:52.349022 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:52.349022 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:52.349022 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:52.349022 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:10:52.349022 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:52.349022 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:52.349022 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:52.349022 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:52.349022 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:52.349022 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:52.349022 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:52.349022 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:52.351152 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:10:52.351152 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:10:52.351152 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:10:52.351152 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:10:52.351152 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 313/621] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:10:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:10:52.351776 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:52.359059 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6240f2e5-74bd-498f-a66d-309b185064c5 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:52.470026 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb3b8e3-6352-4fec-b20e-6717687e4d64 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:52.529865 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:52.542958 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6240f2e5-74bd-498f-a66d-309b185064c5 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:52.559388 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e2c4b5-ae3d-46f6-b00c-e38076c7c1eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:52.560420 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e2c4b5-ae3d-46f6-b00c-e38076c7c1eb None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:52.560920 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e2c4b5-ae3d-46f6-b00c-e38076c7c1eb None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:52.561391 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e2c4b5-ae3d-46f6-b00c-e38076c7c1eb None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:52.562361 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e2c4b5-ae3d-46f6-b00c-e38076c7c1eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:52.562865 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e2c4b5-ae3d-46f6-b00c-e38076c7c1eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:52.564939 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e2c4b5-ae3d-46f6-b00c-e38076c7c1eb None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:52.581867 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33702b42-7667-4602-bd87-43fa0d74208e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:52.581867 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33702b42-7667-4602-bd87-43fa0d74208e None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:52.581867 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33702b42-7667-4602-bd87-43fa0d74208e None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:52.581867 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33702b42-7667-4602-bd87-43fa0d74208e None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:52.582276 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33702b42-7667-4602-bd87-43fa0d74208e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:52.582357 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33702b42-7667-4602-bd87-43fa0d74208e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:52.583772 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33702b42-7667-4602-bd87-43fa0d74208e None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:52.593344 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 309/622] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:10:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:52.596679 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 314/623] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:10:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:10:52.598495 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e2c4b5-ae3d-46f6-b00c-e38076c7c1eb None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:52.601818 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33702b42-7667-4602-bd87-43fa0d74208e None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:52.746125 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33702b42-7667-4602-bd87-43fa0d74208e None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:52.751601 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:52.789459 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5309011f-0d66-412b-a965-3ec4d916de2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:52.790743 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5309011f-0d66-412b-a965-3ec4d916de2f None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:52.791036 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5309011f-0d66-412b-a965-3ec4d916de2f None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:52.791319 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5309011f-0d66-412b-a965-3ec4d916de2f None admin] PATH_INFO: `/v3/users/nova` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:52.800397 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5309011f-0d66-412b-a965-3ec4d916de2f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:52.802511 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5309011f-0d66-412b-a965-3ec4d916de2f None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:52.819987 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-5309011f-0d66-412b-a965-3ec4d916de2f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 14:10:52.819987 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:52.819987 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:52.819987 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:52.819987 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:52.819987 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:52.819987 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:52.819987 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:52.819987 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:52.819987 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:52.819987 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:52.819987 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:52.819987 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:10:52.819987 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:10:52.819987 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:10:52.819987 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:10:52.820980 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:52.820980 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:52.820980 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:10:52.820980 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:10:52.820980 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:10:52.820980 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:10:52.820980 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:52.820980 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:52.820980 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:52.820980 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:52.820980 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:52.820980 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:10:52.820980 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:52.820980 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:52.820980 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:52.820980 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:52.820980 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:52.820980 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:52.822058 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:52.822058 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:52.822058 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:10:52.822058 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:10:52.822058 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:10:52.822058 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:10:52.822058 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:10:52.822058 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:10:52.822058 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 14:10:52.822058 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:10:52.822058 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 315/624] 10.222.0.57 () {56 vars in 1093 bytes} [Thu May 19 14:10:52 2022] GET /identity/v3/users/nova => generated 82 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:10:52.822685 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e2c4b5-ae3d-46f6-b00c-e38076c7c1eb None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:52.827999 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5309011f-0d66-412b-a965-3ec4d916de2f None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:52.845079 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:52.902168 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35341409-b8bf-447f-8c10-ee1f4210c0e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:52.902982 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35341409-b8bf-447f-8c10-ee1f4210c0e5 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:52.903274 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35341409-b8bf-447f-8c10-ee1f4210c0e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:52.903553 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35341409-b8bf-447f-8c10-ee1f4210c0e5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:52.918866 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35341409-b8bf-447f-8c10-ee1f4210c0e5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:52.920353 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35341409-b8bf-447f-8c10-ee1f4210c0e5 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:52.952347 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5309011f-0d66-412b-a965-3ec4d916de2f None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:52.954935 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-35341409-b8bf-447f-8c10-ee1f4210c0e5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:10:52.954935 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:52.954935 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:52.954935 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:52.954935 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:52.954935 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:52.954935 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:52.954935 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:52.954935 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:52.954935 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:52.954935 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:52.954935 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:52.954935 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:10:52.954935 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:10:52.954935 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:10:52.956939 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:10:52.956939 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:52.956939 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:52.956939 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:52.956939 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:52.956939 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:52.956939 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:52.956939 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:52.956939 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:10:52.956939 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:52.956939 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:52.956939 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:52.956939 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:52.956939 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:52.956939 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:52.956939 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:52.956939 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:52.959084 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:10:52.959084 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:10:52.959084 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:10:52.959084 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:10:52.959084 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:10:52.959084 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:10:52.959084 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:10:52.959084 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:10:52.959084 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 310/625] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:10:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 359 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:10:52.963308 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35341409-b8bf-447f-8c10-ee1f4210c0e5 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:53.000230 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f22682-5174-4db5-85bf-ff96ed5ae4a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:53.000917 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f22682-5174-4db5-85bf-ff96ed5ae4a3 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:53.001207 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f22682-5174-4db5-85bf-ff96ed5ae4a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:53.001483 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f22682-5174-4db5-85bf-ff96ed5ae4a3 None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:53.002625 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f22682-5174-4db5-85bf-ff96ed5ae4a3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:53.002940 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f22682-5174-4db5-85bf-ff96ed5ae4a3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:53.004397 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f22682-5174-4db5-85bf-ff96ed5ae4a3 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:53.025393 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 316/626] 10.222.0.57 () {56 vars in 1133 bytes} [Thu May 19 14:10:52 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:53.035411 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f22682-5174-4db5-85bf-ff96ed5ae4a3 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:53.110860 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35341409-b8bf-447f-8c10-ee1f4210c0e5 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:53.140737 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:53.146439 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af103b49-948f-4b50-bf9b-644d06dd9bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:53.147112 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af103b49-948f-4b50-bf9b-644d06dd9bdb None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:53.147392 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af103b49-948f-4b50-bf9b-644d06dd9bdb None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:53.147668 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af103b49-948f-4b50-bf9b-644d06dd9bdb None admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:53.148457 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af103b49-948f-4b50-bf9b-644d06dd9bdb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:53.148766 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af103b49-948f-4b50-bf9b-644d06dd9bdb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:53.150301 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af103b49-948f-4b50-bf9b-644d06dd9bdb None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:53.168479 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 311/627] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:10:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:10:53.173439 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af103b49-948f-4b50-bf9b-644d06dd9bdb None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:53.210237 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:53.275103 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f22682-5174-4db5-85bf-ff96ed5ae4a3 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:53.325041 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd8e9df-7be5-4b98-8074-dbc6909e4bd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:53.325701 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd8e9df-7be5-4b98-8074-dbc6909e4bd7 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:53.326014 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd8e9df-7be5-4b98-8074-dbc6909e4bd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:53.326295 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd8e9df-7be5-4b98-8074-dbc6909e4bd7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:53.365016 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af103b49-948f-4b50-bf9b-644d06dd9bdb None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:53.415622 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff62ba8d-a650-4f8f-836f-d0b8a864f44f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:53.416771 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff62ba8d-a650-4f8f-836f-d0b8a864f44f None admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:53.417060 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff62ba8d-a650-4f8f-836f-d0b8a864f44f None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:53.417333 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff62ba8d-a650-4f8f-836f-d0b8a864f44f None admin] PATH_INFO: `/v3/projects/c6f995ffa0a94b1584f25dbe31e20524/users/09dd957f216a46dd8a3f400fcd2ba944/roles/9c7d4d848dd549adb9f3465d4201f26d` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:53.428334 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd8e9df-7be5-4b98-8074-dbc6909e4bd7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:53.430242 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd8e9df-7be5-4b98-8074-dbc6909e4bd7 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:53.436184 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff62ba8d-a650-4f8f-836f-d0b8a864f44f None admin] RBAC: Authorizing `identity:create_grant(project_id=c6f995ffa0a94b1584f25dbe31e20524, user_id=09dd957f216a46dd8a3f400fcd2ba944, role_id=9c7d4d848dd549adb9f3465d4201f26d)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:53.437636 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff62ba8d-a650-4f8f-836f-d0b8a864f44f None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:53.453999 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 312/628] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 14:10:53 2022] PUT /identity/v3/projects/c6f995ffa0a94b1584f25dbe31e20524/users/09dd957f216a46dd8a3f400fcd2ba944/roles/9c7d4d848dd549adb9f3465d4201f26d => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:10:53.455975 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4e4f08-640b-432f-88d4-f38e3de9c042 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:53.456297 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4e4f08-640b-432f-88d4-f38e3de9c042 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:53.456773 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4e4f08-640b-432f-88d4-f38e3de9c042 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:53.461877 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 313/629] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:10:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:10:53.472556 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4c4ead-4918-4c47-ad22-a72ef3ff2daf None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:53.473216 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4c4ead-4918-4c47-ad22-a72ef3ff2daf None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:53.473709 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4c4ead-4918-4c47-ad22-a72ef3ff2daf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:53.493709 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-ec4c4ead-4918-4c47-ad22-a72ef3ff2daf None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:10:53.495491 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-ec4c4ead-4918-4c47-ad22-a72ef3ff2daf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:10:53.553304 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:53.558374 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:53.598574 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 317/630] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:10:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 564 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:10:53.604897 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd8e9df-7be5-4b98-8074-dbc6909e4bd7 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:53.635519 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 314/631] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:10:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:10:53.697644 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4c4ead-4918-4c47-ad22-a72ef3ff2daf None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:53.752980 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd8e9df-7be5-4b98-8074-dbc6909e4bd7 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:53.790597 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a6857a-f905-4a2b-a6ec-c43fa295860b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:53.791403 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a6857a-f905-4a2b-a6ec-c43fa295860b None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:53.791689 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a6857a-f905-4a2b-a6ec-c43fa295860b None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:53.791969 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a6857a-f905-4a2b-a6ec-c43fa295860b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:53.804351 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a6857a-f905-4a2b-a6ec-c43fa295860b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:53.805822 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a6857a-f905-4a2b-a6ec-c43fa295860b None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:53.831511 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-07a6857a-f905-4a2b-a6ec-c43fa295860b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:10:53.831511 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:53.831511 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:53.831511 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:53.831511 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:53.831511 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:53.831511 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:53.831511 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:53.831511 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:53.831511 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:53.831511 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:53.831511 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:53.831511 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:10:53.831511 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:10:53.831511 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:10:53.832467 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:10:53.832467 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:53.832467 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:53.832467 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:53.832467 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:53.832467 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:53.832467 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:53.832467 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:53.832467 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:10:53.832467 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:53.832467 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:53.832467 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:53.832467 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:53.832467 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:53.832467 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:53.832467 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:53.832467 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:53.833381 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:10:53.833381 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:10:53.833381 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:10:53.833381 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:10:53.833605 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4c4ead-4918-4c47-ad22-a72ef3ff2daf None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:53.835758 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 318/632] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:10:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:10:53.837274 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45be1dcb-4ef1-4286-97dd-a1e002e4c64d None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:53.837774 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45be1dcb-4ef1-4286-97dd-a1e002e4c64d None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:53.838288 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45be1dcb-4ef1-4286-97dd-a1e002e4c64d None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:53.840020 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 319/633] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:10:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:10:53.844801 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45be1dcb-4ef1-4286-97dd-a1e002e4c64d None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:53.866007 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:53.875677 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f488f8fa-77d8-4748-ab54-86b85b143c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:53.876493 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f488f8fa-77d8-4748-ab54-86b85b143c4e None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:53.876778 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f488f8fa-77d8-4748-ab54-86b85b143c4e None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:53.877890 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f488f8fa-77d8-4748-ab54-86b85b143c4e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:53.891368 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f488f8fa-77d8-4748-ab54-86b85b143c4e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:10:53.901961 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-f488f8fa-77d8-4748-ab54-86b85b143c4e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:10:53.901961 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:53.901961 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:53.901961 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:53.901961 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:53.901961 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:53.901961 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:53.901961 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:53.901961 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:53.901961 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:53.901961 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:53.901961 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:53.901961 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:10:53.901961 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:10:53.901961 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:10:53.901961 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:10:53.902932 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:10:53.902932 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:10:53.902932 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:10:53.902932 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:10:53.902932 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:53.902932 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:53.902932 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:53.902932 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:53.902932 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:53.902932 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:53.902932 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:53.902932 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:10:53.902932 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:53.902932 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:53.902932 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:53.902932 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:53.903809 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:53.903809 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:53.903809 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:53.903809 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:53.903809 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:10:53.903809 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:10:53.903809 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:10:53.903809 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:10:53.903809 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:10:53.903809 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:10:53.903809 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:10:53.903809 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:10:53.903809 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 315/634] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 14:10:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:10:53.905025 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b296a679-b09b-4c96-af5c-f78ebde95f20 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:53.905025 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b296a679-b09b-4c96-af5c-f78ebde95f20 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:53.905332 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b296a679-b09b-4c96-af5c-f78ebde95f20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:53.926338 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-b296a679-b09b-4c96-af5c-f78ebde95f20 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:10:53.927880 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-b296a679-b09b-4c96-af5c-f78ebde95f20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:10:53.959008 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:53.979114 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45be1dcb-4ef1-4286-97dd-a1e002e4c64d None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:54.018359 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 316/635] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:10:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:10:54.019823 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b296a679-b09b-4c96-af5c-f78ebde95f20 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:54.024420 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8b8064-ae3a-41cf-b936-7ddffc218654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:54.024420 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8b8064-ae3a-41cf-b936-7ddffc218654 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:54.024420 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8b8064-ae3a-41cf-b936-7ddffc218654 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:54.024420 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8b8064-ae3a-41cf-b936-7ddffc218654 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:54.025096 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8b8064-ae3a-41cf-b936-7ddffc218654 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:54.025169 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8b8064-ae3a-41cf-b936-7ddffc218654 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:54.027527 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8b8064-ae3a-41cf-b936-7ddffc218654 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:54.040626 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 320/636] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:10:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:54.046180 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8b8064-ae3a-41cf-b936-7ddffc218654 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:54.155865 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:54.167283 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b296a679-b09b-4c96-af5c-f78ebde95f20 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:54.187999 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8b8064-ae3a-41cf-b936-7ddffc218654 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:54.211335 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51add61-c3b7-47a4-a53d-4282bff10acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:54.212424 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51add61-c3b7-47a4-a53d-4282bff10acd None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:54.213428 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51add61-c3b7-47a4-a53d-4282bff10acd None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:54.213977 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51add61-c3b7-47a4-a53d-4282bff10acd None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:54.215170 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51add61-c3b7-47a4-a53d-4282bff10acd None admin] RBAC: Adding query filter params (name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:54.215890 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51add61-c3b7-47a4-a53d-4282bff10acd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:54.218201 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51add61-c3b7-47a4-a53d-4282bff10acd None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:54.227539 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2b402b-d2a3-4b96-a5b5-cfb57ca1e575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:54.228339 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2b402b-d2a3-4b96-a5b5-cfb57ca1e575 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:54.228626 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2b402b-d2a3-4b96-a5b5-cfb57ca1e575 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:54.228906 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2b402b-d2a3-4b96-a5b5-cfb57ca1e575 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:54.230411 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 317/637] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 19 14:10:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:54.231637 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51add61-c3b7-47a4-a53d-4282bff10acd None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:54.237184 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2b402b-d2a3-4b96-a5b5-cfb57ca1e575 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:54.238810 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2b402b-d2a3-4b96-a5b5-cfb57ca1e575 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:54.256096 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-4f2b402b-d2a3-4b96-a5b5-cfb57ca1e575 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:10:54.256096 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:54.256096 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:54.256096 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:54.256096 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:54.256096 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:54.256096 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:54.256096 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:54.256096 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:54.256096 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:54.256096 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:54.256096 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:54.256096 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:10:54.256096 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:10:54.256096 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:10:54.257245 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:10:54.257245 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:54.257245 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:54.257245 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:54.257245 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:54.257245 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:54.257245 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:54.257245 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:54.257245 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:10:54.257245 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:54.257245 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:54.257245 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:54.257245 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:54.257245 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:54.257245 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:54.257245 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:54.257245 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:54.258365 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:10:54.258365 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:10:54.258365 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:10:54.258365 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:10:54.258365 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:10:54.258365 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:10:54.258365 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:10:54.258365 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:10:54.260211 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 321/638] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:10:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:10:54.261426 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2b402b-d2a3-4b96-a5b5-cfb57ca1e575 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:54.266025 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:54.346383 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51add61-c3b7-47a4-a53d-4282bff10acd None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:54.373725 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2b402b-d2a3-4b96-a5b5-cfb57ca1e575 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:54.387272 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413b4ae4-fe88-4526-831d-dc7a7c3c12ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:54.388394 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413b4ae4-fe88-4526-831d-dc7a7c3c12ec None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:54.388860 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413b4ae4-fe88-4526-831d-dc7a7c3c12ec None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:54.389339 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413b4ae4-fe88-4526-831d-dc7a7c3c12ec None admin] PATH_INFO: `/v3/roles/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:54.401383 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413b4ae4-fe88-4526-831d-dc7a7c3c12ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:10:54.409549 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-413b4ae4-fe88-4526-831d-dc7a7c3c12ec None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:10:54.409549 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:54.409549 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:54.409549 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:54.409549 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:54.409549 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:54.409549 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:54.409549 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:54.409549 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:54.409549 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:54.409549 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:54.409549 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:54.409549 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:10:54.409549 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:10:54.409549 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:10:54.409549 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:10:54.410629 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:10:54.410629 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:10:54.410629 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:10:54.410629 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:10:54.410629 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:54.410629 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:54.410629 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:54.410629 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:54.410629 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:54.410629 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:54.410629 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:54.410629 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:10:54.410629 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:54.410629 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:54.410629 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:54.410629 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:54.411649 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:54.411649 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:54.411649 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:54.411649 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:54.411649 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:10:54.411649 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:10:54.411649 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:10:54.411649 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:10:54.411649 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:10:54.411649 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:10:54.411649 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:10:54.411649 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:10:54.414220 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 318/639] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 14:10:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:10:54.415217 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413b4ae4-fe88-4526-831d-dc7a7c3c12ec None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:54.419707 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05362fb-7447-4501-8e33-9899a75aeb1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:54.420760 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05362fb-7447-4501-8e33-9899a75aeb1d None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:54.421294 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05362fb-7447-4501-8e33-9899a75aeb1d None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:54.422130 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05362fb-7447-4501-8e33-9899a75aeb1d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:54.433047 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:54.438853 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05362fb-7447-4501-8e33-9899a75aeb1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:54.442652 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05362fb-7447-4501-8e33-9899a75aeb1d None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:54.464207 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-f05362fb-7447-4501-8e33-9899a75aeb1d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:10:54.464207 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:54.464207 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:54.464207 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:54.464207 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:54.464207 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:54.464207 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:54.464207 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:54.464207 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:54.464207 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:54.464207 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:54.464207 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:54.464207 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:10:54.464207 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:10:54.464207 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:10:54.465243 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:10:54.465243 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:54.465243 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:54.465243 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:54.465243 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:54.465243 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:54.465243 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:54.465243 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:54.465243 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:10:54.465243 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:54.465243 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:54.465243 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:54.465243 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:54.465243 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:54.465243 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:54.465243 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:54.465243 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:54.466581 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:10:54.466581 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:10:54.466581 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:10:54.466581 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:10:54.466581 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 322/640] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:10:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:10:54.466581 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05362fb-7447-4501-8e33-9899a75aeb1d None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:54.519923 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:54.537604 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413b4ae4-fe88-4526-831d-dc7a7c3c12ec None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:54.573647 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dee54ed-5c77-4cc2-8cc6-09789d8002f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:54.574624 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dee54ed-5c77-4cc2-8cc6-09789d8002f7 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:54.574909 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dee54ed-5c77-4cc2-8cc6-09789d8002f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:54.575185 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dee54ed-5c77-4cc2-8cc6-09789d8002f7 None admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:54.575968 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dee54ed-5c77-4cc2-8cc6-09789d8002f7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:54.576523 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dee54ed-5c77-4cc2-8cc6-09789d8002f7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:54.577708 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dee54ed-5c77-4cc2-8cc6-09789d8002f7 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:54.579590 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05362fb-7447-4501-8e33-9899a75aeb1d None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:54.595013 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 319/641] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:10:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:54.595013 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dee54ed-5c77-4cc2-8cc6-09789d8002f7 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:54.624467 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167eedb6-304c-4a56-95a2-3e9d6e5e35f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:54.625137 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167eedb6-304c-4a56-95a2-3e9d6e5e35f4 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:54.625420 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167eedb6-304c-4a56-95a2-3e9d6e5e35f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:54.625694 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167eedb6-304c-4a56-95a2-3e9d6e5e35f4 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:54.626461 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167eedb6-304c-4a56-95a2-3e9d6e5e35f4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:54.626768 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167eedb6-304c-4a56-95a2-3e9d6e5e35f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:54.628255 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167eedb6-304c-4a56-95a2-3e9d6e5e35f4 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:54.640631 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 323/642] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 19 14:10:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:54.640631 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583c6343-ac9f-4c22-ac98-f1b80f915f73 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:54.640631 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583c6343-ac9f-4c22-ac98-f1b80f915f73 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:54.640631 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583c6343-ac9f-4c22-ac98-f1b80f915f73 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:54.640631 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 324/643] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:10:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:10:54.640631 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583c6343-ac9f-4c22-ac98-f1b80f915f73 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:54.698698 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:54.710778 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dee54ed-5c77-4cc2-8cc6-09789d8002f7 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:54.748564 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1eb5a1-5e11-4618-9b5f-cda3fe4efd55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:54.749223 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1eb5a1-5e11-4618-9b5f-cda3fe4efd55 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:54.749509 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1eb5a1-5e11-4618-9b5f-cda3fe4efd55 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:54.749791 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1eb5a1-5e11-4618-9b5f-cda3fe4efd55 None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:54.750555 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1eb5a1-5e11-4618-9b5f-cda3fe4efd55 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:54.750863 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1eb5a1-5e11-4618-9b5f-cda3fe4efd55 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:54.750967 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583c6343-ac9f-4c22-ac98-f1b80f915f73 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:54.752323 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1eb5a1-5e11-4618-9b5f-cda3fe4efd55 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:54.765636 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 320/644] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:10:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:54.766531 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1eb5a1-5e11-4618-9b5f-cda3fe4efd55 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:54.771148 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:54.796721 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6951ed3-9d17-40fd-aabc-e8112955c244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:54.797386 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6951ed3-9d17-40fd-aabc-e8112955c244 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:54.797675 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6951ed3-9d17-40fd-aabc-e8112955c244 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:54.797998 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6951ed3-9d17-40fd-aabc-e8112955c244 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:54.799430 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6951ed3-9d17-40fd-aabc-e8112955c244 None admin] RBAC: Adding query filter params (role.id=9c7d4d848dd549adb9f3465d4201f26d, scope.project.id=c6f995ffa0a94b1584f25dbe31e20524, user.id=8c87f361a92d4834854ca07f652be2ff) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:54.799839 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6951ed3-9d17-40fd-aabc-e8112955c244 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:54.801543 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6951ed3-9d17-40fd-aabc-e8112955c244 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:54.807887 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 325/645] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:10:54 2022] GET /identity/v3/role_assignments?user.id=8c87f361a92d4834854ca07f652be2ff&scope.project.id=c6f995ffa0a94b1584f25dbe31e20524&role.id=9c7d4d848dd549adb9f3465d4201f26d => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:54.809117 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb81022a-5cb4-4880-990d-5340f022c8c5 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:54.809408 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb81022a-5cb4-4880-990d-5340f022c8c5 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:54.809689 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb81022a-5cb4-4880-990d-5340f022c8c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:54.829461 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-eb81022a-5cb4-4880-990d-5340f022c8c5 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:10:54.831439 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-eb81022a-5cb4-4880-990d-5340f022c8c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:10:54.903416 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1eb5a1-5e11-4618-9b5f-cda3fe4efd55 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:54.925919 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 326/646] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:10:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:10:54.925919 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb81022a-5cb4-4880-990d-5340f022c8c5 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:54.962752 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ff30ed-cc7c-471d-b645-c5d388d73006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:54.964540 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ff30ed-cc7c-471d-b645-c5d388d73006 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:54.965120 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ff30ed-cc7c-471d-b645-c5d388d73006 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:54.965571 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ff30ed-cc7c-471d-b645-c5d388d73006 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:54.983486 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ff30ed-cc7c-471d-b645-c5d388d73006 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:54.986416 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ff30ed-cc7c-471d-b645-c5d388d73006 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:54.990396 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:55.013124 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-a8ff30ed-cc7c-471d-b645-c5d388d73006 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:10:55.013124 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:55.013124 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:55.013124 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:55.013124 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:55.013124 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:55.013124 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:55.013124 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:55.013124 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:55.013124 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:55.013124 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:55.013124 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:55.013124 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:10:55.013124 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:10:55.013124 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:10:55.014139 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:10:55.014139 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:55.014139 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:55.014139 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:55.014139 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:55.014139 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:55.014139 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:55.014139 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:55.014139 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:10:55.014139 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:55.014139 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:55.014139 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:55.014139 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:55.014139 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:55.014139 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:55.014139 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:55.014139 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:55.015217 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:10:55.015217 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:10:55.015217 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:10:55.015217 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:10:55.017224 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 321/647] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:10:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:10:55.018894 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ff30ed-cc7c-471d-b645-c5d388d73006 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:55.060920 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb81022a-5cb4-4880-990d-5340f022c8c5 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:55.086730 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:55.115404 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80aa03a3-da5b-4003-a3cc-76d57fc45af1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:55.115404 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aa03a3-da5b-4003-a3cc-76d57fc45af1 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:55.115404 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aa03a3-da5b-4003-a3cc-76d57fc45af1 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:55.115404 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aa03a3-da5b-4003-a3cc-76d57fc45af1 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:55.127387 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80aa03a3-da5b-4003-a3cc-76d57fc45af1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:55.129750 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80aa03a3-da5b-4003-a3cc-76d57fc45af1 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:55.144236 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ff30ed-cc7c-471d-b645-c5d388d73006 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:55.147719 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-80aa03a3-da5b-4003-a3cc-76d57fc45af1 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 19 14:10:55.147719 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:55.147719 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:55.147719 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:55.147719 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:55.147719 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:55.147719 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:55.147719 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:55.147719 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:55.147719 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:55.147719 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:55.147719 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:55.147719 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:10:55.147719 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:10:55.147719 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:10:55.147719 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:10:55.148657 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:55.148657 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:55.148657 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:10:55.148657 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:10:55.148657 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:10:55.148657 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:10:55.148657 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:55.148657 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:55.148657 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:55.148657 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:55.148657 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:55.148657 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:10:55.148657 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:55.148657 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:55.148657 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:55.148657 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:55.148657 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:55.148657 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:55.149606 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:55.149606 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:55.149606 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:10:55.149606 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:10:55.149606 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:10:55.149606 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:10:55.149606 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:10:55.149606 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:10:55.149606 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 19 14:10:55.149606 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:10:55.149606 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 327/648] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:10:54 2022] GET /identity/v3/users/glance => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:10:55.150357 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80aa03a3-da5b-4003-a3cc-76d57fc45af1 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:55.190532 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b1a0f6-fdeb-4028-89ce-639a32dc4e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:55.191684 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b1a0f6-fdeb-4028-89ce-639a32dc4e0a None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:55.192065 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b1a0f6-fdeb-4028-89ce-639a32dc4e0a None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:55.192465 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b1a0f6-fdeb-4028-89ce-639a32dc4e0a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:55.207385 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b1a0f6-fdeb-4028-89ce-639a32dc4e0a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:10:55.217240 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-58b1a0f6-fdeb-4028-89ce-639a32dc4e0a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:10:55.217240 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:55.217240 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:55.217240 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:55.217240 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:55.217240 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:55.217240 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:55.217240 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:55.217240 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:55.217240 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:55.217240 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:55.217240 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:55.217240 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:10:55.217240 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:10:55.217240 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:10:55.217240 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:10:55.218556 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:10:55.218556 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:10:55.218556 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:10:55.218556 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:10:55.218556 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:55.218556 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:55.218556 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:55.218556 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:55.218556 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:55.218556 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:55.218556 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:55.218556 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:10:55.218556 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:55.218556 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:55.218556 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:55.218556 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:55.220022 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:55.220022 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:55.220022 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:55.220022 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:55.220022 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:10:55.220022 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:10:55.220022 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:10:55.220022 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:10:55.220022 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:10:55.220022 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:10:55.220022 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:10:55.220022 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:10:55.220022 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 322/649] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 14:10:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:10:55.221354 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b1a0f6-fdeb-4028-89ce-639a32dc4e0a None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:55.275792 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80aa03a3-da5b-4003-a3cc-76d57fc45af1 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:55.287014 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:55.323899 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e1d235-e055-485a-930c-30f235de0f75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:55.324817 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e1d235-e055-485a-930c-30f235de0f75 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:55.325166 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e1d235-e055-485a-930c-30f235de0f75 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:55.325535 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e1d235-e055-485a-930c-30f235de0f75 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:55.326669 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e1d235-e055-485a-930c-30f235de0f75 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:55.327049 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e1d235-e055-485a-930c-30f235de0f75 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:55.328511 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e1d235-e055-485a-930c-30f235de0f75 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:55.342582 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b1a0f6-fdeb-4028-89ce-639a32dc4e0a None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:55.344913 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 328/650] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:10:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:55.346699 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e1d235-e055-485a-930c-30f235de0f75 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:55.361018 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:55.385767 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ae00a8-2bfb-4787-b426-05ca7ac59177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:55.386475 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ae00a8-2bfb-4787-b426-05ca7ac59177 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:55.386753 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ae00a8-2bfb-4787-b426-05ca7ac59177 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:55.387022 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ae00a8-2bfb-4787-b426-05ca7ac59177 None admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:55.387883 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ae00a8-2bfb-4787-b426-05ca7ac59177 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:55.388183 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ae00a8-2bfb-4787-b426-05ca7ac59177 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:55.389608 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ae00a8-2bfb-4787-b426-05ca7ac59177 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:55.410340 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 323/651] 10.222.0.57 () {56 vars in 1137 bytes} [Thu May 19 14:10:55 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:55.411164 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ae00a8-2bfb-4787-b426-05ca7ac59177 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:55.487132 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e1d235-e055-485a-930c-30f235de0f75 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:55.530021 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ae00a8-2bfb-4787-b426-05ca7ac59177 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:55.533645 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4f96e3-491a-43b2-9c95-46a97f5df963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:55.534579 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4f96e3-491a-43b2-9c95-46a97f5df963 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:55.535066 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4f96e3-491a-43b2-9c95-46a97f5df963 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:55.535558 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4f96e3-491a-43b2-9c95-46a97f5df963 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:55.536509 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4f96e3-491a-43b2-9c95-46a97f5df963 None admin] RBAC: Adding query filter params (name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:55.537042 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4f96e3-491a-43b2-9c95-46a97f5df963 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:55.539211 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4f96e3-491a-43b2-9c95-46a97f5df963 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:55.555846 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 329/652] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 19 14:10:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:55.555846 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4f96e3-491a-43b2-9c95-46a97f5df963 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:55.571529 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf90083-ed82-4eb1-9edb-318cacf1c6d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:55.572379 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf90083-ed82-4eb1-9edb-318cacf1c6d5 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:55.572667 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf90083-ed82-4eb1-9edb-318cacf1c6d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:55.572940 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf90083-ed82-4eb1-9edb-318cacf1c6d5 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:55.582648 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf90083-ed82-4eb1-9edb-318cacf1c6d5 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:55.584079 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf90083-ed82-4eb1-9edb-318cacf1c6d5 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:55.595410 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:55.599497 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-4bf90083-ed82-4eb1-9edb-318cacf1c6d5 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 14:10:55.599497 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:55.599497 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:55.599497 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:55.599497 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:55.599497 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:55.599497 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:55.599497 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:55.599497 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:55.599497 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:55.599497 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:55.599497 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:55.599497 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:10:55.599497 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:10:55.599497 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:10:55.599497 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:10:55.600423 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:55.600423 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:55.600423 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:10:55.600423 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:10:55.600423 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:10:55.600423 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:10:55.600423 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:55.600423 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:55.600423 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:55.600423 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:55.600423 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:55.600423 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:10:55.600423 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:55.600423 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:55.600423 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:55.600423 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:55.600423 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:55.600423 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:55.601381 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:55.601381 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:55.601381 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:10:55.601381 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:10:55.601381 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:10:55.601381 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:10:55.601381 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:10:55.601381 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:10:55.601381 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 14:10:55.601381 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:10:55.601381 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 324/653] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:10:55 2022] GET /identity/v3/users/cinder => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:10:55.602706 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf90083-ed82-4eb1-9edb-318cacf1c6d5 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:55.641560 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:55.690506 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4f96e3-491a-43b2-9c95-46a97f5df963 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:55.727124 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cf55c1-3c6e-465c-9517-c24aa5a831f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:55.727918 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cf55c1-3c6e-465c-9517-c24aa5a831f8 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:55.728146 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cf55c1-3c6e-465c-9517-c24aa5a831f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:55.728423 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cf55c1-3c6e-465c-9517-c24aa5a831f8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:55.741432 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf90083-ed82-4eb1-9edb-318cacf1c6d5 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:55.780989 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1499b4-e8bf-45f3-978d-733b2b58d6cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:55.781974 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1499b4-e8bf-45f3-978d-733b2b58d6cb None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:55.782327 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1499b4-e8bf-45f3-978d-733b2b58d6cb None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:55.782705 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1499b4-e8bf-45f3-978d-733b2b58d6cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:55.794719 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1499b4-e8bf-45f3-978d-733b2b58d6cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:55.796276 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1499b4-e8bf-45f3-978d-733b2b58d6cb None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:55.816289 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-cd1499b4-e8bf-45f3-978d-733b2b58d6cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:10:55.816289 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:55.816289 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:55.816289 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:55.816289 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:55.816289 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:55.816289 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:55.816289 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:55.816289 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:55.816289 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:55.816289 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:55.816289 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:55.816289 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:10:55.816289 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:10:55.816289 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:10:55.817266 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:10:55.817266 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:55.817266 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:55.817266 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:55.817266 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:55.817266 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:55.817266 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:55.817266 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:55.817266 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:10:55.817266 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:55.817266 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:55.817266 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:55.817266 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:55.817266 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:55.817266 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:55.817266 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:55.817266 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:55.818210 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:10:55.818210 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:10:55.818210 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:10:55.818210 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:10:55.822243 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 325/654] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:10:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:10:55.822765 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cf55c1-3c6e-465c-9517-c24aa5a831f8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:55.823248 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1499b4-e8bf-45f3-978d-733b2b58d6cb None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:55.824779 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cf55c1-3c6e-465c-9517-c24aa5a831f8 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:55.863443 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:55.927732 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:55.946354 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1499b4-e8bf-45f3-978d-733b2b58d6cb None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:55.948084 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 330/655] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:10:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:10:55.948935 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cf55c1-3c6e-465c-9517-c24aa5a831f8 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:55.988361 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6cdc68-ba1f-4492-9d5c-0ec031a15427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:55.989287 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6cdc68-ba1f-4492-9d5c-0ec031a15427 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:55.989577 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6cdc68-ba1f-4492-9d5c-0ec031a15427 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:55.989873 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6cdc68-ba1f-4492-9d5c-0ec031a15427 None admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:55.990711 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6cdc68-ba1f-4492-9d5c-0ec031a15427 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:55.991080 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6cdc68-ba1f-4492-9d5c-0ec031a15427 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:55.992573 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6cdc68-ba1f-4492-9d5c-0ec031a15427 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:56.014487 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 326/656] 10.222.0.57 () {56 vars in 1137 bytes} [Thu May 19 14:10:55 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:56.014487 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2056b1-ece2-4b90-9669-7a5569049953 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:56.014487 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2056b1-ece2-4b90-9669-7a5569049953 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:56.014487 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2056b1-ece2-4b90-9669-7a5569049953 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:56.014487 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 327/657] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:10:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:10:56.014487 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2056b1-ece2-4b90-9669-7a5569049953 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:56.065889 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cf55c1-3c6e-465c-9517-c24aa5a831f8 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:56.099437 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:56.110072 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d65efc3-a7e3-422f-bf89-592a724b34d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:56.110905 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d65efc3-a7e3-422f-bf89-592a724b34d5 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:56.111311 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d65efc3-a7e3-422f-bf89-592a724b34d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:56.112955 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d65efc3-a7e3-422f-bf89-592a724b34d5 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:56.113698 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d65efc3-a7e3-422f-bf89-592a724b34d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:56.114287 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d65efc3-a7e3-422f-bf89-592a724b34d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:56.115682 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d65efc3-a7e3-422f-bf89-592a724b34d5 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:56.129087 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 331/658] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:10:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:56.130007 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d65efc3-a7e3-422f-bf89-592a724b34d5 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:56.138260 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2056b1-ece2-4b90-9669-7a5569049953 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:56.179077 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd17a3e-98e9-4418-bbcd-9ebc82e67135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:56.179873 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd17a3e-98e9-4418-bbcd-9ebc82e67135 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:56.180147 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd17a3e-98e9-4418-bbcd-9ebc82e67135 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:56.180416 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd17a3e-98e9-4418-bbcd-9ebc82e67135 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:56.193137 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd17a3e-98e9-4418-bbcd-9ebc82e67135 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:56.194623 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd17a3e-98e9-4418-bbcd-9ebc82e67135 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:56.197330 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:56.213676 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-7fd17a3e-98e9-4418-bbcd-9ebc82e67135 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:10:56.213676 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:56.213676 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:56.213676 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:56.213676 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:56.213676 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:56.213676 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:56.213676 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:56.213676 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:56.213676 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:56.213676 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:56.213676 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:56.213676 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:10:56.213676 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:10:56.213676 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:10:56.214653 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:10:56.214653 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:56.214653 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:56.214653 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:56.214653 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:56.214653 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:56.214653 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:56.214653 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:56.214653 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:10:56.214653 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:56.214653 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:56.214653 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:56.214653 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:56.214653 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:56.214653 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:56.214653 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:56.214653 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:56.215552 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:10:56.215552 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:10:56.215552 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:10:56.215552 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:10:56.218192 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 328/659] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:10:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:10:56.221230 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9348f4-ef5d-48a9-9fed-f50a1073b40b None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:56.221523 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9348f4-ef5d-48a9-9fed-f50a1073b40b None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:56.222254 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9348f4-ef5d-48a9-9fed-f50a1073b40b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:56.244277 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d65efc3-a7e3-422f-bf89-592a724b34d5 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:56.245728 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-4b9348f4-ef5d-48a9-9fed-f50a1073b40b None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:10:56.247864 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-4b9348f4-ef5d-48a9-9fed-f50a1073b40b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:10:56.280549 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44d04f9-c3ae-47f6-995b-e21a95c1850d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:56.281199 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44d04f9-c3ae-47f6-995b-e21a95c1850d None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:56.281488 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44d04f9-c3ae-47f6-995b-e21a95c1850d None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:56.281767 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44d04f9-c3ae-47f6-995b-e21a95c1850d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:56.320362 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 329/660] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:10:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:10:56.321197 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9348f4-ef5d-48a9-9fed-f50a1073b40b None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:56.357875 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:56.374277 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44d04f9-c3ae-47f6-995b-e21a95c1850d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:56.375754 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44d04f9-c3ae-47f6-995b-e21a95c1850d None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:56.429386 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9348f4-ef5d-48a9-9fed-f50a1073b40b None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:56.456082 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:56.466518 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42be0b88-ef83-406c-894a-47fedfd6ce75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:56.467535 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42be0b88-ef83-406c-894a-47fedfd6ce75 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:56.467828 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42be0b88-ef83-406c-894a-47fedfd6ce75 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:56.468108 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42be0b88-ef83-406c-894a-47fedfd6ce75 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:56.477385 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42be0b88-ef83-406c-894a-47fedfd6ce75 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:56.479094 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42be0b88-ef83-406c-894a-47fedfd6ce75 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:56.490124 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 332/661] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:10:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:10:56.491196 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44d04f9-c3ae-47f6-995b-e21a95c1850d None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:56.494684 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-42be0b88-ef83-406c-894a-47fedfd6ce75 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 19 14:10:56.494684 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:56.494684 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:56.494684 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:56.494684 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:56.494684 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:56.494684 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:56.494684 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:56.494684 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:56.494684 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:56.494684 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:56.494684 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:56.494684 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:10:56.494684 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:10:56.494684 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:10:56.494684 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:10:56.495604 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:56.495604 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:56.495604 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:10:56.495604 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:10:56.495604 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:10:56.495604 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:10:56.495604 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:56.495604 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:56.495604 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:56.495604 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:56.495604 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:56.495604 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:10:56.495604 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:56.495604 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:56.495604 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:56.495604 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:56.495604 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:56.495604 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:56.496548 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:56.496548 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:56.496548 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:10:56.496548 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:10:56.496548 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:10:56.496548 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:10:56.496548 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:10:56.496548 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:10:56.496548 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 19 14:10:56.496548 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:10:56.500251 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 330/662] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 14:10:56 2022] GET /identity/v3/users/neutron => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:10:56.500921 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42be0b88-ef83-406c-894a-47fedfd6ce75 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:56.590326 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:56.604320 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44d04f9-c3ae-47f6-995b-e21a95c1850d None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:56.604706 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42be0b88-ef83-406c-894a-47fedfd6ce75 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:56.641003 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636f439a-281d-4aad-a424-3791ecc1e568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:56.641583 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503af6d9-5370-4c6b-8983-8047ff3e91aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:56.641725 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636f439a-281d-4aad-a424-3791ecc1e568 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:56.641955 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636f439a-281d-4aad-a424-3791ecc1e568 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:56.642230 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636f439a-281d-4aad-a424-3791ecc1e568 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:56.642934 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503af6d9-5370-4c6b-8983-8047ff3e91aa None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:56.643228 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503af6d9-5370-4c6b-8983-8047ff3e91aa None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:56.643506 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503af6d9-5370-4c6b-8983-8047ff3e91aa None admin] PATH_INFO: `/v3/roles/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:56.643807 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636f439a-281d-4aad-a424-3791ecc1e568 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:56.644115 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636f439a-281d-4aad-a424-3791ecc1e568 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:56.645491 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636f439a-281d-4aad-a424-3791ecc1e568 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:56.654451 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503af6d9-5370-4c6b-8983-8047ff3e91aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:10:56.657232 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 333/663] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:10:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:56.657831 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636f439a-281d-4aad-a424-3791ecc1e568 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:56.667263 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-503af6d9-5370-4c6b-8983-8047ff3e91aa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:10:56.667263 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:56.667263 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:56.667263 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:56.667263 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:56.667263 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:56.667263 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:56.667263 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:56.667263 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:56.667263 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:56.667263 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:56.667263 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:56.667263 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:10:56.667263 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:10:56.667263 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:10:56.667263 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:10:56.668415 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:10:56.668415 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:10:56.668415 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:10:56.668415 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:10:56.668415 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:56.668415 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:56.668415 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:56.668415 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:56.668415 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:56.668415 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:56.668415 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:56.668415 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:10:56.668415 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:56.668415 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:56.668415 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:56.668415 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:56.669257 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:56.669257 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:56.669257 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:56.669257 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:56.669257 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:10:56.669257 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:10:56.669257 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:10:56.669257 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:10:56.669257 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:10:56.669257 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:10:56.669257 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:10:56.669257 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:10:56.669257 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 331/664] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 14:10:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:10:56.669257 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503af6d9-5370-4c6b-8983-8047ff3e91aa None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:56.719054 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:56.772039 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636f439a-281d-4aad-a424-3791ecc1e568 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:56.781263 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503af6d9-5370-4c6b-8983-8047ff3e91aa None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:56.808186 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed13e962-9e8d-41e1-8c50-6632e995aac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:56.808983 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed13e962-9e8d-41e1-8c50-6632e995aac0 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:56.809264 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed13e962-9e8d-41e1-8c50-6632e995aac0 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:56.809537 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed13e962-9e8d-41e1-8c50-6632e995aac0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:56.819192 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2e153f-d67c-4db5-b66d-4ccce428f65b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:56.820085 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2e153f-d67c-4db5-b66d-4ccce428f65b None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:56.820370 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2e153f-d67c-4db5-b66d-4ccce428f65b None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:56.820652 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2e153f-d67c-4db5-b66d-4ccce428f65b None admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:56.821306 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed13e962-9e8d-41e1-8c50-6632e995aac0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:56.821498 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2e153f-d67c-4db5-b66d-4ccce428f65b None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:56.821784 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2e153f-d67c-4db5-b66d-4ccce428f65b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:56.822892 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed13e962-9e8d-41e1-8c50-6632e995aac0 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:56.823303 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2e153f-d67c-4db5-b66d-4ccce428f65b None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:56.832619 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:56.842677 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-ed13e962-9e8d-41e1-8c50-6632e995aac0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:10:56.842677 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:56.842677 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:56.842677 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:56.842677 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:56.842677 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:56.842677 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:56.842677 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:56.842677 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:56.842677 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:56.842677 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:56.842677 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:56.842677 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:10:56.842677 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:10:56.842677 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:10:56.843671 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:10:56.843671 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:56.843671 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:56.843671 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:56.843671 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:56.843671 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:56.843671 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:56.843671 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:56.843671 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:10:56.843671 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:56.843671 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:56.843671 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:56.843671 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:56.843671 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:56.843671 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:56.843671 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:56.843671 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:56.844617 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:10:56.844617 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:10:56.844617 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:10:56.844617 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:10:56.847511 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 334/665] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:10:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:10:56.848391 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed13e962-9e8d-41e1-8c50-6632e995aac0 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:56.853983 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 332/666] 10.222.0.57 () {56 vars in 1139 bytes} [Thu May 19 14:10:56 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:56.854864 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2e153f-d67c-4db5-b66d-4ccce428f65b None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:56.961648 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2e153f-d67c-4db5-b66d-4ccce428f65b None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:56.962547 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed13e962-9e8d-41e1-8c50-6632e995aac0 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:56.964964 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:56.999097 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb28a970-c1b9-4c10-8a3c-52478ceabf8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:56.999895 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb28a970-c1b9-4c10-8a3c-52478ceabf8b None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:57.000182 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb28a970-c1b9-4c10-8a3c-52478ceabf8b None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:57.000459 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb28a970-c1b9-4c10-8a3c-52478ceabf8b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:57.002452 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53be7c10-78b3-43c0-b26d-cca43995f25f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:57.003325 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53be7c10-78b3-43c0-b26d-cca43995f25f None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:57.003607 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53be7c10-78b3-43c0-b26d-cca43995f25f None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:57.003874 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53be7c10-78b3-43c0-b26d-cca43995f25f None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:57.004612 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53be7c10-78b3-43c0-b26d-cca43995f25f None admin] RBAC: Adding query filter params (name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:57.004917 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53be7c10-78b3-43c0-b26d-cca43995f25f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:57.006319 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53be7c10-78b3-43c0-b26d-cca43995f25f None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:57.008851 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb28a970-c1b9-4c10-8a3c-52478ceabf8b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:57.010353 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb28a970-c1b9-4c10-8a3c-52478ceabf8b None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:57.015030 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 333/667] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 19 14:10:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:57.020806 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53be7c10-78b3-43c0-b26d-cca43995f25f None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:57.026883 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-fb28a970-c1b9-4c10-8a3c-52478ceabf8b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:10:57.026883 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:57.026883 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:57.026883 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:57.026883 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:57.026883 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:57.026883 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:57.026883 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:57.026883 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:57.026883 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:57.026883 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:57.026883 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:57.026883 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:10:57.026883 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:10:57.026883 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:10:57.027803 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:10:57.027803 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:57.027803 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:57.027803 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:57.027803 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:57.027803 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:57.027803 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:57.027803 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:57.027803 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:10:57.027803 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:57.027803 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:57.027803 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:57.027803 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:57.027803 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:57.027803 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:57.027803 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:57.027803 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:57.029333 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:10:57.029333 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:10:57.029333 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:10:57.029333 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:10:57.029333 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:10:57.029333 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:10:57.029333 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:10:57.029333 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:10:57.029333 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 335/668] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:10:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:10:57.029333 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb28a970-c1b9-4c10-8a3c-52478ceabf8b None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:57.099173 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:57.141114 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53be7c10-78b3-43c0-b26d-cca43995f25f None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:57.147573 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb28a970-c1b9-4c10-8a3c-52478ceabf8b None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:57.180931 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6ce9f2-e2b1-4857-bc6f-42d0da8d87d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:57.181655 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6ce9f2-e2b1-4857-bc6f-42d0da8d87d2 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:57.181969 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6ce9f2-e2b1-4857-bc6f-42d0da8d87d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:57.182255 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6ce9f2-e2b1-4857-bc6f-42d0da8d87d2 None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:57.182552 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4da76e-5bf1-45e5-8b58-de960b09e6db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:57.182984 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6ce9f2-e2b1-4857-bc6f-42d0da8d87d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:57.183202 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4da76e-5bf1-45e5-8b58-de960b09e6db None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:57.183309 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6ce9f2-e2b1-4857-bc6f-42d0da8d87d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:57.183489 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4da76e-5bf1-45e5-8b58-de960b09e6db None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:57.183758 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4da76e-5bf1-45e5-8b58-de960b09e6db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:57.184681 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6ce9f2-e2b1-4857-bc6f-42d0da8d87d2 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:57.199654 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 334/669] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:10:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:57.200425 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6ce9f2-e2b1-4857-bc6f-42d0da8d87d2 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:57.228184 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:57.260072 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4da76e-5bf1-45e5-8b58-de960b09e6db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:57.261535 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4da76e-5bf1-45e5-8b58-de960b09e6db None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:57.310559 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6ce9f2-e2b1-4857-bc6f-42d0da8d87d2 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:57.319409 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:57.344838 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7430addd-25ba-47b6-8cb7-e12baec39c21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:57.345713 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7430addd-25ba-47b6-8cb7-e12baec39c21 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:57.346436 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7430addd-25ba-47b6-8cb7-e12baec39c21 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:57.346714 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7430addd-25ba-47b6-8cb7-e12baec39c21 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:57.356560 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7430addd-25ba-47b6-8cb7-e12baec39c21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:57.358020 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7430addd-25ba-47b6-8cb7-e12baec39c21 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:57.360462 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 336/670] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:10:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:10:57.362109 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4da76e-5bf1-45e5-8b58-de960b09e6db None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:57.376778 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-7430addd-25ba-47b6-8cb7-e12baec39c21 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:10:57.376778 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:57.376778 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:57.376778 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:57.376778 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:57.376778 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:57.376778 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:57.376778 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:57.376778 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:57.376778 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:57.376778 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:57.376778 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:57.376778 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:10:57.376778 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:10:57.376778 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:10:57.378463 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:10:57.378463 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:57.378463 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:57.378463 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:57.378463 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:57.378463 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:57.378463 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:57.378463 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:57.378463 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:10:57.378463 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:57.378463 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:57.378463 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:57.378463 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:57.378463 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:57.378463 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:57.378463 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:57.378463 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:57.379688 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:10:57.379688 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:10:57.379688 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:10:57.379688 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:10:57.379688 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 335/671] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:10:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:10:57.379688 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7430addd-25ba-47b6-8cb7-e12baec39c21 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:57.457587 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:57.468560 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4da76e-5bf1-45e5-8b58-de960b09e6db None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:57.487843 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7430addd-25ba-47b6-8cb7-e12baec39c21 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:57.510394 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17faa51-4d92-4293-8003-7d0571b1ce3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:57.511051 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17faa51-4d92-4293-8003-7d0571b1ce3c None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:57.511331 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17faa51-4d92-4293-8003-7d0571b1ce3c None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:57.511605 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17faa51-4d92-4293-8003-7d0571b1ce3c None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:57.512463 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17faa51-4d92-4293-8003-7d0571b1ce3c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:57.512772 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17faa51-4d92-4293-8003-7d0571b1ce3c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:57.514235 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17faa51-4d92-4293-8003-7d0571b1ce3c None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:57.522601 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354306d4-a7aa-4edf-9683-1483c6ad84e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:57.523391 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354306d4-a7aa-4edf-9683-1483c6ad84e3 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:57.523670 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354306d4-a7aa-4edf-9683-1483c6ad84e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:57.523938 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354306d4-a7aa-4edf-9683-1483c6ad84e3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:57.526304 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 337/672] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:10:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:57.527672 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c8f56d-c5c0-4d96-bb38-944fb1b062cf None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:57.527972 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c8f56d-c5c0-4d96-bb38-944fb1b062cf None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:57.528255 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c8f56d-c5c0-4d96-bb38-944fb1b062cf None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:57.529641 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 338/673] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:10:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:10:57.530267 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c8f56d-c5c0-4d96-bb38-944fb1b062cf None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:57.535848 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354306d4-a7aa-4edf-9683-1483c6ad84e3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:57.537301 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354306d4-a7aa-4edf-9683-1483c6ad84e3 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:57.552216 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-354306d4-a7aa-4edf-9683-1483c6ad84e3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:10:57.552216 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:57.552216 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:57.552216 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:57.552216 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:57.552216 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:57.552216 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:57.552216 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:57.552216 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:57.552216 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:57.552216 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:57.552216 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:57.552216 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:10:57.552216 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:10:57.552216 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:10:57.553299 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:10:57.553299 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:57.553299 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:57.553299 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:57.553299 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:57.553299 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:57.553299 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:57.553299 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:57.553299 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:10:57.553299 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:57.553299 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:57.553299 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:57.553299 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:57.553299 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:57.553299 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:57.553299 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:57.553299 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:57.554276 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:10:57.554276 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:10:57.554276 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:10:57.554276 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:10:57.554276 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:10:57.554276 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:10:57.554276 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:10:57.554276 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:10:57.554716 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:57.554791 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 336/674] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:10:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:10:57.554791 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354306d4-a7aa-4edf-9683-1483c6ad84e3 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:57.634645 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c8f56d-c5c0-4d96-bb38-944fb1b062cf None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:57.653429 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354306d4-a7aa-4edf-9683-1483c6ad84e3 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:57.668160 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3961d0b-7427-4c2a-b735-0167bff9fde1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:57.668993 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3961d0b-7427-4c2a-b735-0167bff9fde1 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:57.669282 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3961d0b-7427-4c2a-b735-0167bff9fde1 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:57.669560 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3961d0b-7427-4c2a-b735-0167bff9fde1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:57.680501 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3961d0b-7427-4c2a-b735-0167bff9fde1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:57.681978 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3961d0b-7427-4c2a-b735-0167bff9fde1 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:57.687181 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607e039b-8274-420e-a471-0aaa211d6c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:57.687840 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607e039b-8274-420e-a471-0aaa211d6c14 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:57.688131 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607e039b-8274-420e-a471-0aaa211d6c14 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:57.688406 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607e039b-8274-420e-a471-0aaa211d6c14 None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:57.689118 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607e039b-8274-420e-a471-0aaa211d6c14 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:57.689447 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607e039b-8274-420e-a471-0aaa211d6c14 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:57.690990 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607e039b-8274-420e-a471-0aaa211d6c14 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:57.697728 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:57.699050 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-b3961d0b-7427-4c2a-b735-0167bff9fde1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:10:57.699050 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:57.699050 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:57.699050 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:57.699050 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:57.699050 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:57.699050 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:57.699050 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:57.699050 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:57.699050 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:57.699050 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:57.699050 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:57.699050 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:10:57.699050 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:10:57.699050 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:10:57.699944 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:10:57.699944 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:57.699944 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:57.699944 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:57.699944 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:57.699944 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:57.699944 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:57.699944 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:57.699944 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:10:57.699944 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:57.699944 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:57.699944 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:57.699944 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:57.699944 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:57.699944 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:57.699944 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:57.699944 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:57.700857 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:10:57.700857 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:10:57.700857 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:10:57.700857 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:10:57.700857 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 339/675] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:10:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:10:57.701529 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3961d0b-7427-4c2a-b735-0167bff9fde1 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:57.707205 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 337/676] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:10:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:57.708457 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd03264-e3e3-4b85-92b0-665e1ebb83c0 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:57.708742 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd03264-e3e3-4b85-92b0-665e1ebb83c0 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:57.709013 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd03264-e3e3-4b85-92b0-665e1ebb83c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:57.725646 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-3dd03264-e3e3-4b85-92b0-665e1ebb83c0 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:10:57.726891 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-3dd03264-e3e3-4b85-92b0-665e1ebb83c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:10:57.773196 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:57.792623 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 338/677] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:10:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:10:57.793441 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd03264-e3e3-4b85-92b0-665e1ebb83c0 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:57.812955 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3961d0b-7427-4c2a-b735-0167bff9fde1 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:57.847372 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53959217-a3b9-4cb5-a7a3-801e5fee48b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:57.848073 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53959217-a3b9-4cb5-a7a3-801e5fee48b7 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:57.848361 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53959217-a3b9-4cb5-a7a3-801e5fee48b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:57.848648 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53959217-a3b9-4cb5-a7a3-801e5fee48b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:57.849475 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53959217-a3b9-4cb5-a7a3-801e5fee48b7 None admin] RBAC: Adding query filter params (role.id=9c7d4d848dd549adb9f3465d4201f26d, scope.project.id=c6f995ffa0a94b1584f25dbe31e20524, user.id=0b91de04b5014f379b758fd83435b263) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:57.849789 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53959217-a3b9-4cb5-a7a3-801e5fee48b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:57.851250 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53959217-a3b9-4cb5-a7a3-801e5fee48b7 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:57.856965 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 340/678] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:10:57 2022] GET /identity/v3/role_assignments?user.id=0b91de04b5014f379b758fd83435b263&scope.project.id=c6f995ffa0a94b1584f25dbe31e20524&role.id=9c7d4d848dd549adb9f3465d4201f26d => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:57.857744 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53959217-a3b9-4cb5-a7a3-801e5fee48b7 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:57.902840 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd03264-e3e3-4b85-92b0-665e1ebb83c0 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:57.935353 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:57.944059 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9098f9-0159-4384-9a59-b794d272c143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:57.944733 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9098f9-0159-4384-9a59-b794d272c143 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:57.945015 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9098f9-0159-4384-9a59-b794d272c143 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:57.945289 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9098f9-0159-4384-9a59-b794d272c143 None admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:57.946132 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9098f9-0159-4384-9a59-b794d272c143 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:57.946445 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9098f9-0159-4384-9a59-b794d272c143 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:57.947941 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9098f9-0159-4384-9a59-b794d272c143 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:57.962265 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 339/679] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:10:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:57.963119 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9098f9-0159-4384-9a59-b794d272c143 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:57.992599 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53959217-a3b9-4cb5-a7a3-801e5fee48b7 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:58.032979 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018ba129-258a-4fc9-a2f8-c77e7b54e4b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:58.034570 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018ba129-258a-4fc9-a2f8-c77e7b54e4b6 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:58.034757 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018ba129-258a-4fc9-a2f8-c77e7b54e4b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:58.035029 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018ba129-258a-4fc9-a2f8-c77e7b54e4b6 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:58.035759 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018ba129-258a-4fc9-a2f8-c77e7b54e4b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:58.036225 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018ba129-258a-4fc9-a2f8-c77e7b54e4b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:58.037474 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018ba129-258a-4fc9-a2f8-c77e7b54e4b6 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:58.040744 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:58.053999 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 341/680] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:10:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:58.055208 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018ba129-258a-4fc9-a2f8-c77e7b54e4b6 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:58.089660 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9098f9-0159-4384-9a59-b794d272c143 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:58.134990 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc2fa73-b823-400c-9ecc-2547af58afd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:58.135796 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc2fa73-b823-400c-9ecc-2547af58afd5 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:58.136082 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc2fa73-b823-400c-9ecc-2547af58afd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:58.136361 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc2fa73-b823-400c-9ecc-2547af58afd5 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:58.145137 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc2fa73-b823-400c-9ecc-2547af58afd5 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:58.146658 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc2fa73-b823-400c-9ecc-2547af58afd5 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:58.163643 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-abc2fa73-b823-400c-9ecc-2547af58afd5 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 14:10:58.163643 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:58.163643 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:58.163643 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:58.163643 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:58.163643 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:58.163643 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:58.163643 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:58.163643 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:58.163643 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:58.163643 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:58.163643 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:58.163643 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:10:58.163643 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:10:58.163643 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:10:58.163643 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:10:58.164598 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:58.164598 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:58.164598 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:10:58.164598 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:10:58.164598 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:10:58.164598 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:10:58.164598 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:58.164598 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:58.164598 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:58.164598 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:58.164598 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:58.164598 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:10:58.164598 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:58.164598 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:58.164598 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:58.164598 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:58.164598 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:58.164598 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:58.165539 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:58.165539 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:58.165539 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:10:58.165539 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:10:58.165539 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:10:58.165539 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:10:58.165539 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:10:58.165539 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:10:58.165539 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 14:10:58.165539 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:10:58.165539 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 340/681] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:10:57 2022] GET /identity/v3/users/swift => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:10:58.165539 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc2fa73-b823-400c-9ecc-2547af58afd5 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:58.179037 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018ba129-258a-4fc9-a2f8-c77e7b54e4b6 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:58.216845 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99934ea-b462-4567-9ba4-4f0ab8b2d7ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:58.217937 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99934ea-b462-4567-9ba4-4f0ab8b2d7ba None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:58.218379 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99934ea-b462-4567-9ba4-4f0ab8b2d7ba None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:58.218658 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99934ea-b462-4567-9ba4-4f0ab8b2d7ba None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:58.220727 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:58.229633 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99934ea-b462-4567-9ba4-4f0ab8b2d7ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:10:58.237329 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-f99934ea-b462-4567-9ba4-4f0ab8b2d7ba None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:10:58.237329 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:58.237329 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:58.237329 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:58.237329 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:58.237329 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:58.237329 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:58.237329 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:58.237329 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:58.237329 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:58.237329 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:58.237329 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:58.237329 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:10:58.237329 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:10:58.237329 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:10:58.237329 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:10:58.238512 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:10:58.238512 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:10:58.238512 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:10:58.238512 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:10:58.238512 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:58.238512 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:58.238512 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:58.238512 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:58.238512 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:58.238512 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:58.238512 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:58.238512 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:10:58.238512 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:58.238512 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:58.238512 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:58.238512 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:58.239883 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:58.239883 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:58.239883 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:58.239883 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:58.239883 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:10:58.239883 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:10:58.239883 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:10:58.239883 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:10:58.239883 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:10:58.239883 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:10:58.239883 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:10:58.239883 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:10:58.239883 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 342/682] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:10:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:10:58.240598 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99934ea-b462-4567-9ba4-4f0ab8b2d7ba None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:58.285208 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc2fa73-b823-400c-9ecc-2547af58afd5 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:58.301831 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:58.323693 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742c40ca-a4d4-48c6-a591-6a47084a0a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:58.324356 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742c40ca-a4d4-48c6-a591-6a47084a0a04 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:58.324638 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742c40ca-a4d4-48c6-a591-6a47084a0a04 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:58.324917 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742c40ca-a4d4-48c6-a591-6a47084a0a04 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:58.325743 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742c40ca-a4d4-48c6-a591-6a47084a0a04 None admin] RBAC: Adding query filter params (role.id=9c7d4d848dd549adb9f3465d4201f26d, scope.project.id=c6f995ffa0a94b1584f25dbe31e20524, user.id=fefdec9076444495ad679120510fe271) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:58.326087 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742c40ca-a4d4-48c6-a591-6a47084a0a04 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:58.327567 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742c40ca-a4d4-48c6-a591-6a47084a0a04 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:58.345872 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 341/683] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:10:58 2022] GET /identity/v3/role_assignments?user.id=fefdec9076444495ad679120510fe271&scope.project.id=c6f995ffa0a94b1584f25dbe31e20524&role.id=9c7d4d848dd549adb9f3465d4201f26d => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:58.345872 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742c40ca-a4d4-48c6-a591-6a47084a0a04 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:58.350709 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99934ea-b462-4567-9ba4-4f0ab8b2d7ba None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:58.394110 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945a9921-91fc-4911-bdd0-e8806caa0f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:58.395017 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945a9921-91fc-4911-bdd0-e8806caa0f2f None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:58.395438 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945a9921-91fc-4911-bdd0-e8806caa0f2f None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:58.395791 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945a9921-91fc-4911-bdd0-e8806caa0f2f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:58.403814 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945a9921-91fc-4911-bdd0-e8806caa0f2f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:58.405432 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945a9921-91fc-4911-bdd0-e8806caa0f2f None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:58.421244 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-945a9921-91fc-4911-bdd0-e8806caa0f2f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:10:58.421244 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:58.421244 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:58.421244 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:58.421244 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:58.421244 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:58.421244 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:58.421244 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:58.421244 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:58.421244 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:58.421244 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:58.421244 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:58.421244 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:10:58.421244 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:10:58.421244 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:10:58.422430 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:10:58.422430 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:58.422430 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:58.422430 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:58.422430 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:58.422430 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:58.422430 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:58.422430 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:58.422430 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:10:58.422430 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:58.422430 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:58.422430 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:58.422430 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:58.422430 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:58.422430 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:58.422430 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:58.422430 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:58.423626 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:10:58.423626 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:10:58.423626 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:10:58.423626 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:10:58.423626 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:10:58.423626 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:10:58.423626 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:10:58.423626 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:10:58.424895 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 343/684] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:10:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:10:58.425819 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945a9921-91fc-4911-bdd0-e8806caa0f2f None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:58.492596 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742c40ca-a4d4-48c6-a591-6a47084a0a04 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:58.513483 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:58.544495 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa82e250-3f4e-43c6-b9a4-f88b71d37f96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:58.545147 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa82e250-3f4e-43c6-b9a4-f88b71d37f96 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:58.545508 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa82e250-3f4e-43c6-b9a4-f88b71d37f96 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:58.545877 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa82e250-3f4e-43c6-b9a4-f88b71d37f96 None admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:58.546862 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa82e250-3f4e-43c6-b9a4-f88b71d37f96 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:58.547296 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa82e250-3f4e-43c6-b9a4-f88b71d37f96 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:58.549578 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa82e250-3f4e-43c6-b9a4-f88b71d37f96 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:58.565507 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945a9921-91fc-4911-bdd0-e8806caa0f2f None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:58.572057 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 342/685] 10.222.0.57 () {56 vars in 1135 bytes} [Thu May 19 14:10:58 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:58.573087 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa82e250-3f4e-43c6-b9a4-f88b71d37f96 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:58.585879 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:58.613035 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af02cbf0-dfde-43a6-b9dd-1c66fb262e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:58.613715 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af02cbf0-dfde-43a6-b9dd-1c66fb262e49 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:58.614282 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af02cbf0-dfde-43a6-b9dd-1c66fb262e49 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:58.614567 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af02cbf0-dfde-43a6-b9dd-1c66fb262e49 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:58.615326 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af02cbf0-dfde-43a6-b9dd-1c66fb262e49 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:58.615635 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af02cbf0-dfde-43a6-b9dd-1c66fb262e49 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:58.617039 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af02cbf0-dfde-43a6-b9dd-1c66fb262e49 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:58.629308 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 344/686] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:10:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:58.634342 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af02cbf0-dfde-43a6-b9dd-1c66fb262e49 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:58.691970 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa82e250-3f4e-43c6-b9a4-f88b71d37f96 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:58.727799 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d99ad05-7b77-4d2d-b7d8-752d9d66460b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:58.728881 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d99ad05-7b77-4d2d-b7d8-752d9d66460b None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:58.729370 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d99ad05-7b77-4d2d-b7d8-752d9d66460b None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:58.729833 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d99ad05-7b77-4d2d-b7d8-752d9d66460b None admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:58.730852 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d99ad05-7b77-4d2d-b7d8-752d9d66460b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:58.731352 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d99ad05-7b77-4d2d-b7d8-752d9d66460b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:58.733006 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d99ad05-7b77-4d2d-b7d8-752d9d66460b None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:58.747116 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 343/687] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:10:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:58.753323 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d99ad05-7b77-4d2d-b7d8-752d9d66460b None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:58.780046 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af02cbf0-dfde-43a6-b9dd-1c66fb262e49 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:58.789215 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:58.821209 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f8efbc-0fbb-4284-93e7-003e42932c0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:58.822492 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f8efbc-0fbb-4284-93e7-003e42932c0f None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:58.823009 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f8efbc-0fbb-4284-93e7-003e42932c0f None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:58.823482 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f8efbc-0fbb-4284-93e7-003e42932c0f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:58.885992 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d99ad05-7b77-4d2d-b7d8-752d9d66460b None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:58.894216 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:58.898506 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f8efbc-0fbb-4284-93e7-003e42932c0f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:58.900261 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f8efbc-0fbb-4284-93e7-003e42932c0f None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:58.925384 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db81fb9-a6ab-4e9b-8adb-8a8a9d419196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:58.926552 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db81fb9-a6ab-4e9b-8adb-8a8a9d419196 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:58.927064 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db81fb9-a6ab-4e9b-8adb-8a8a9d419196 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:58.927559 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db81fb9-a6ab-4e9b-8adb-8a8a9d419196 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:58.939106 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db81fb9-a6ab-4e9b-8adb-8a8a9d419196 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:58.941570 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db81fb9-a6ab-4e9b-8adb-8a8a9d419196 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:58.961279 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-2db81fb9-a6ab-4e9b-8adb-8a8a9d419196 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:10:58.961279 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:58.961279 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:58.961279 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:58.961279 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:58.961279 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:58.961279 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:58.961279 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:58.961279 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:58.961279 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:58.961279 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:58.961279 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:58.961279 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:10:58.961279 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:10:58.961279 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:10:58.962244 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:10:58.962244 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:58.962244 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:58.962244 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:58.962244 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:58.962244 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:58.962244 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:58.962244 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:58.962244 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:10:58.962244 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:58.962244 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:58.962244 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:58.962244 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:58.962244 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:58.962244 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:58.962244 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:58.962244 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:58.963149 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:10:58.963149 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:10:58.963149 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:10:58.963149 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:10:58.963149 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 344/688] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:10:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:10:58.963870 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db81fb9-a6ab-4e9b-8adb-8a8a9d419196 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:59.030772 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 345/689] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:10:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:10:59.031908 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f8efbc-0fbb-4284-93e7-003e42932c0f None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:59.078322 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:59.104040 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db81fb9-a6ab-4e9b-8adb-8a8a9d419196 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:59.141643 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:59.154542 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5297c9d-120f-4ce4-a252-a2c29337a451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:59.155207 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5297c9d-120f-4ce4-a252-a2c29337a451 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:59.155490 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5297c9d-120f-4ce4-a252-a2c29337a451 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:59.155765 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5297c9d-120f-4ce4-a252-a2c29337a451 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:59.156638 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5297c9d-120f-4ce4-a252-a2c29337a451 None admin] RBAC: Adding query filter params (role.id=9c7d4d848dd549adb9f3465d4201f26d, scope.project.id=c6f995ffa0a94b1584f25dbe31e20524, user.id=56ce23600bfe42e6b98eeebfe7470560) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:59.156960 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5297c9d-120f-4ce4-a252-a2c29337a451 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:59.159017 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5297c9d-120f-4ce4-a252-a2c29337a451 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:59.165767 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 345/690] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:10:58 2022] GET /identity/v3/role_assignments?user.id=56ce23600bfe42e6b98eeebfe7470560&scope.project.id=c6f995ffa0a94b1584f25dbe31e20524&role.id=9c7d4d848dd549adb9f3465d4201f26d => generated 608 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:59.166761 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5297c9d-120f-4ce4-a252-a2c29337a451 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:59.172757 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f8efbc-0fbb-4284-93e7-003e42932c0f None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:59.214096 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dff8096-2338-470a-8b10-3e060fedc8f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:59.215768 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dff8096-2338-470a-8b10-3e060fedc8f5 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:59.216285 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dff8096-2338-470a-8b10-3e060fedc8f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:59.216956 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dff8096-2338-470a-8b10-3e060fedc8f5 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:59.218053 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dff8096-2338-470a-8b10-3e060fedc8f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:59.218765 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dff8096-2338-470a-8b10-3e060fedc8f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:59.221008 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dff8096-2338-470a-8b10-3e060fedc8f5 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:59.235482 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 346/691] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:10:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:59.242515 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dff8096-2338-470a-8b10-3e060fedc8f5 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:59.279188 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5297c9d-120f-4ce4-a252-a2c29337a451 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:59.316913 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fceb11-72f2-48ad-9249-3014f624afae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:59.317887 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fceb11-72f2-48ad-9249-3014f624afae None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:59.318174 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fceb11-72f2-48ad-9249-3014f624afae None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:59.318454 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fceb11-72f2-48ad-9249-3014f624afae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:59.333423 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fceb11-72f2-48ad-9249-3014f624afae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:59.334927 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fceb11-72f2-48ad-9249-3014f624afae None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:59.343611 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:59.361153 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-42fceb11-72f2-48ad-9249-3014f624afae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:10:59.361153 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:59.361153 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:59.361153 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:59.361153 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:59.361153 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:59.361153 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:59.361153 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:59.361153 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:59.361153 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:59.361153 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:59.361153 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:59.361153 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:10:59.361153 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:10:59.361153 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:10:59.362354 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:10:59.362354 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:59.362354 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:59.362354 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:59.362354 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:59.362354 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:59.362354 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:59.362354 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:59.362354 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:10:59.362354 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:59.362354 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:59.362354 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:59.362354 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:59.362354 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:59.362354 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:59.362354 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:59.362354 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:59.363326 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:10:59.363326 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:10:59.363326 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:10:59.363326 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:10:59.365045 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dff8096-2338-470a-8b10-3e060fedc8f5 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:59.367701 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 346/692] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:10:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:10:59.372716 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fceb11-72f2-48ad-9249-3014f624afae None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:59.404803 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9febfbbf-87a6-4b59-b7d7-655008f0c061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:59.405628 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9febfbbf-87a6-4b59-b7d7-655008f0c061 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:59.405944 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9febfbbf-87a6-4b59-b7d7-655008f0c061 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:59.406224 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9febfbbf-87a6-4b59-b7d7-655008f0c061 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:59.417744 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9febfbbf-87a6-4b59-b7d7-655008f0c061 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:59.419482 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9febfbbf-87a6-4b59-b7d7-655008f0c061 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:59.422026 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:59.438105 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-9febfbbf-87a6-4b59-b7d7-655008f0c061 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 14:10:59.438105 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:59.438105 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:59.438105 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:59.438105 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:59.438105 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:59.438105 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:59.438105 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:59.438105 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:59.438105 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:59.438105 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:59.438105 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:59.438105 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:10:59.438105 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:10:59.438105 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:10:59.438105 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:10:59.439271 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:59.439271 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:59.439271 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:10:59.439271 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:10:59.439271 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:10:59.439271 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:10:59.439271 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:59.439271 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:59.439271 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:59.439271 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:59.439271 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:59.439271 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:10:59.439271 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:59.439271 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:59.439271 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:59.439271 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:59.439271 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:59.439271 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:59.440477 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:59.440477 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:59.440477 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:10:59.440477 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:10:59.440477 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:10:59.440477 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:10:59.440477 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:10:59.440477 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:10:59.440477 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 14:10:59.440477 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:10:59.444260 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 347/693] 10.222.0.57 () {56 vars in 1093 bytes} [Thu May 19 14:10:59 2022] GET /identity/v3/users/nova => generated 82 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:10:59.445614 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9febfbbf-87a6-4b59-b7d7-655008f0c061 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:59.558564 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fceb11-72f2-48ad-9249-3014f624afae None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:59.588012 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9febfbbf-87a6-4b59-b7d7-655008f0c061 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:59.598761 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e712507-c8f4-4c1e-8423-33cf55d7b18c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:59.599771 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e712507-c8f4-4c1e-8423-33cf55d7b18c None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:59.600367 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e712507-c8f4-4c1e-8423-33cf55d7b18c None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:59.600849 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e712507-c8f4-4c1e-8423-33cf55d7b18c None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:59.601908 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e712507-c8f4-4c1e-8423-33cf55d7b18c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:59.602422 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e712507-c8f4-4c1e-8423-33cf55d7b18c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:59.604119 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e712507-c8f4-4c1e-8423-33cf55d7b18c None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:59.618189 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 347/694] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:10:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:10:59.623417 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e712507-c8f4-4c1e-8423-33cf55d7b18c None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:59.631093 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11273bda-998a-4f4b-b69b-3af46f960c49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:59.631880 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11273bda-998a-4f4b-b69b-3af46f960c49 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:59.632241 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11273bda-998a-4f4b-b69b-3af46f960c49 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:59.632596 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11273bda-998a-4f4b-b69b-3af46f960c49 None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:59.633516 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11273bda-998a-4f4b-b69b-3af46f960c49 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:10:59.633928 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11273bda-998a-4f4b-b69b-3af46f960c49 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:59.636030 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11273bda-998a-4f4b-b69b-3af46f960c49 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:59.660764 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 348/695] 10.222.0.57 () {56 vars in 1133 bytes} [Thu May 19 14:10:59 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:10:59.667681 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11273bda-998a-4f4b-b69b-3af46f960c49 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:59.703829 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:59.731778 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:59.756707 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e712507-c8f4-4c1e-8423-33cf55d7b18c None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:59.786587 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11273bda-998a-4f4b-b69b-3af46f960c49 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:10:59.806160 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d529c9-9ed5-41d4-b0fb-384ca051e2c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:59.807396 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d529c9-9ed5-41d4-b0fb-384ca051e2c3 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:59.807892 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d529c9-9ed5-41d4-b0fb-384ca051e2c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:59.808376 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d529c9-9ed5-41d4-b0fb-384ca051e2c3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:10:59.817181 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d529c9-9ed5-41d4-b0fb-384ca051e2c3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:10:59.819059 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d529c9-9ed5-41d4-b0fb-384ca051e2c3 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:10:59.835597 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-a2d529c9-9ed5-41d4-b0fb-384ca051e2c3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:10:59.835597 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:10:59.835597 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:10:59.835597 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:10:59.835597 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:10:59.835597 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:10:59.835597 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:10:59.835597 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:10:59.835597 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:10:59.835597 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:10:59.835597 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:10:59.835597 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:10:59.835597 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:10:59.835597 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:10:59.835597 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:10:59.836561 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:10:59.836561 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:10:59.836561 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:10:59.836561 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:10:59.836561 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:10:59.836561 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:10:59.836561 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:10:59.836561 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:10:59.836561 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:10:59.836561 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:10:59.836561 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:10:59.836561 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:10:59.836561 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:10:59.836561 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:10:59.836561 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:10:59.836561 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:10:59.836561 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:10:59.837475 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:10:59.837475 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:10:59.837475 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:10:59.837475 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:10:59.837475 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:10:59.837475 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:10:59.837475 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:10:59.837475 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:10:59.840271 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 348/696] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:10:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:10:59.851027 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d529c9-9ed5-41d4-b0fb-384ca051e2c3 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:10:59.974308 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580221ba-c708-40f5-9a2b-46b1259b02d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:10:59.974974 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580221ba-c708-40f5-9a2b-46b1259b02d2 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:10:59.975258 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580221ba-c708-40f5-9a2b-46b1259b02d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:10:59.975536 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580221ba-c708-40f5-9a2b-46b1259b02d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:00.042737 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d529c9-9ed5-41d4-b0fb-384ca051e2c3 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:00.085990 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:00.114299 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c928324e-3900-4622-ab53-7de492bb8716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:00.114990 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c928324e-3900-4622-ab53-7de492bb8716 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:00.115276 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c928324e-3900-4622-ab53-7de492bb8716 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:00.115554 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c928324e-3900-4622-ab53-7de492bb8716 None admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:00.116363 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c928324e-3900-4622-ab53-7de492bb8716 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:00.116676 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c928324e-3900-4622-ab53-7de492bb8716 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:00.119656 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c928324e-3900-4622-ab53-7de492bb8716 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:00.123086 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580221ba-c708-40f5-9a2b-46b1259b02d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:00.123210 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580221ba-c708-40f5-9a2b-46b1259b02d2 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:00.150159 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 349/697] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:10:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:00.154252 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c928324e-3900-4622-ab53-7de492bb8716 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:00.254919 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:00.292176 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c928324e-3900-4622-ab53-7de492bb8716 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:00.332421 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e347f378-f020-433c-b90f-07338fc40010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:00.333792 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e347f378-f020-433c-b90f-07338fc40010 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:00.334295 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e347f378-f020-433c-b90f-07338fc40010 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:00.334780 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e347f378-f020-433c-b90f-07338fc40010 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:00.336444 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e347f378-f020-433c-b90f-07338fc40010 None admin] RBAC: Adding query filter params (role.id=9c7d4d848dd549adb9f3465d4201f26d, scope.project.id=c6f995ffa0a94b1584f25dbe31e20524, user.id=09dd957f216a46dd8a3f400fcd2ba944) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:00.336951 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e347f378-f020-433c-b90f-07338fc40010 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:00.339169 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e347f378-f020-433c-b90f-07338fc40010 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:00.353871 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 349/698] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:10:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 684 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:11:00.353871 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b16374-69cc-4e63-b42e-9cecdf0824d6 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:00.353871 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b16374-69cc-4e63-b42e-9cecdf0824d6 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:00.353871 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b16374-69cc-4e63-b42e-9cecdf0824d6 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:00.354290 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 350/699] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:11:00 2022] GET /identity/v3/role_assignments?user.id=09dd957f216a46dd8a3f400fcd2ba944&scope.project.id=c6f995ffa0a94b1584f25dbe31e20524&role.id=9c7d4d848dd549adb9f3465d4201f26d => generated 608 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:00.355432 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 350/700] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:11:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:11:00.356622 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e347f378-f020-433c-b90f-07338fc40010 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:00.371720 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8484c37c-044b-49af-bba1-59b26a360be2 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:00.372373 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8484c37c-044b-49af-bba1-59b26a360be2 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:00.372862 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8484c37c-044b-49af-bba1-59b26a360be2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:00.405168 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-8484c37c-044b-49af-bba1-59b26a360be2 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:11:00.408523 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-8484c37c-044b-49af-bba1-59b26a360be2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:11:00.454269 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:00.565691 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 351/701] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:11:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:11:00.572130 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e347f378-f020-433c-b90f-07338fc40010 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:00.590940 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8484c37c-044b-49af-bba1-59b26a360be2 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:00.610793 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c37634-d713-43bc-b1a5-b6ac40e82688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:00.611989 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c37634-d713-43bc-b1a5-b6ac40e82688 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:00.612482 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c37634-d713-43bc-b1a5-b6ac40e82688 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:00.612980 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c37634-d713-43bc-b1a5-b6ac40e82688 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:00.624339 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c37634-d713-43bc-b1a5-b6ac40e82688 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:00.627442 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c37634-d713-43bc-b1a5-b6ac40e82688 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:00.646548 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-c4c37634-d713-43bc-b1a5-b6ac40e82688 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:11:00.646548 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:00.646548 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:11:00.646548 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:11:00.646548 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:11:00.646548 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:11:00.646548 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:11:00.646548 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:11:00.646548 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:11:00.646548 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:11:00.646548 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:11:00.646548 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:11:00.646548 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:11:00.646548 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:11:00.646548 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:11:00.647477 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:11:00.647477 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:11:00.647477 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:11:00.647477 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:11:00.647477 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:11:00.647477 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:11:00.647477 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:11:00.647477 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:11:00.647477 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:11:00.647477 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:11:00.647477 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:11:00.647477 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:11:00.647477 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:11:00.647477 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:11:00.647477 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:11:00.647477 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:11:00.647477 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:11:00.648497 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:11:00.648497 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:11:00.648497 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:11:00.648497 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:11:00.650845 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 351/702] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:11:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:11:00.655981 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c37634-d713-43bc-b1a5-b6ac40e82688 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:00.698684 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:00.717965 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8484c37c-044b-49af-bba1-59b26a360be2 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:00.759365 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278c4ee1-8b6e-4caa-aa77-4f8041f2082a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:00.760167 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278c4ee1-8b6e-4caa-aa77-4f8041f2082a None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:00.760453 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278c4ee1-8b6e-4caa-aa77-4f8041f2082a None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:00.760731 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278c4ee1-8b6e-4caa-aa77-4f8041f2082a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:00.776501 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278c4ee1-8b6e-4caa-aa77-4f8041f2082a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:00.778532 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278c4ee1-8b6e-4caa-aa77-4f8041f2082a None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:00.796534 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c37634-d713-43bc-b1a5-b6ac40e82688 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:00.800646 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-278c4ee1-8b6e-4caa-aa77-4f8041f2082a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:11:00.800646 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:00.800646 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:11:00.800646 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:11:00.800646 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:11:00.800646 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:11:00.800646 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:11:00.800646 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:11:00.800646 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:11:00.800646 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:11:00.800646 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:11:00.800646 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:11:00.800646 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:11:00.800646 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:11:00.800646 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:11:00.801634 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:11:00.801634 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:11:00.801634 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:11:00.801634 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:11:00.801634 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:11:00.801634 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:11:00.801634 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:11:00.801634 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:11:00.801634 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:11:00.801634 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:11:00.801634 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:11:00.801634 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:11:00.801634 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:11:00.801634 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:11:00.801634 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:11:00.801634 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:11:00.801634 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:11:00.802605 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:11:00.802605 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:11:00.802605 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:11:00.802605 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:11:00.802846 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:00.803354 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 352/703] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:11:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:11:00.809442 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278c4ee1-8b6e-4caa-aa77-4f8041f2082a None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:00.850977 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12f4ffe-144f-480c-bf4b-7e0b0e4e15e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:00.851985 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12f4ffe-144f-480c-bf4b-7e0b0e4e15e3 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:00.852439 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12f4ffe-144f-480c-bf4b-7e0b0e4e15e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:00.852867 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12f4ffe-144f-480c-bf4b-7e0b0e4e15e3 None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:00.854048 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12f4ffe-144f-480c-bf4b-7e0b0e4e15e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:00.854563 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12f4ffe-144f-480c-bf4b-7e0b0e4e15e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:00.856940 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12f4ffe-144f-480c-bf4b-7e0b0e4e15e3 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:00.874260 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 352/704] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:11:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:00.880176 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12f4ffe-144f-480c-bf4b-7e0b0e4e15e3 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:00.956088 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278c4ee1-8b6e-4caa-aa77-4f8041f2082a None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:01.020023 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6b1636-036e-4d93-9a07-b95b81e10bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:01.020701 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6b1636-036e-4d93-9a07-b95b81e10bb9 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:01.020986 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6b1636-036e-4d93-9a07-b95b81e10bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:01.021261 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6b1636-036e-4d93-9a07-b95b81e10bb9 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:01.022154 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6b1636-036e-4d93-9a07-b95b81e10bb9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:01.022471 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6b1636-036e-4d93-9a07-b95b81e10bb9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:01.023980 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6b1636-036e-4d93-9a07-b95b81e10bb9 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:01.031650 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:01.038539 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 353/705] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:11:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:01.043422 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38364604-b670-4329-9d2a-082f367853f2 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:01.043422 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38364604-b670-4329-9d2a-082f367853f2 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:01.043422 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38364604-b670-4329-9d2a-082f367853f2 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:01.043422 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 354/706] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:11:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:11:01.044364 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38364604-b670-4329-9d2a-082f367853f2 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:01.053245 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12f4ffe-144f-480c-bf4b-7e0b0e4e15e3 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:01.089514 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236b15fb-b8db-4b03-b316-2a6d938945cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:01.091090 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236b15fb-b8db-4b03-b316-2a6d938945cd None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:01.091090 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236b15fb-b8db-4b03-b316-2a6d938945cd None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:01.091090 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236b15fb-b8db-4b03-b316-2a6d938945cd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:01.098150 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236b15fb-b8db-4b03-b316-2a6d938945cd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:01.099590 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236b15fb-b8db-4b03-b316-2a6d938945cd None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:01.115172 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-236b15fb-b8db-4b03-b316-2a6d938945cd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:11:01.115172 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:01.115172 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:11:01.115172 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:11:01.115172 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:11:01.115172 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:11:01.115172 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:11:01.115172 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:11:01.115172 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:11:01.115172 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:11:01.115172 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:11:01.115172 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:11:01.115172 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:11:01.115172 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:11:01.115172 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:11:01.116065 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:11:01.116065 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:11:01.116065 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:11:01.116065 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:11:01.116065 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:11:01.116065 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:11:01.116065 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:11:01.116065 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:11:01.116065 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:11:01.116065 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:11:01.116065 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:11:01.116065 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:11:01.116065 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:11:01.116065 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:11:01.116065 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:11:01.116065 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:11:01.116065 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:11:01.116991 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:11:01.116991 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:11:01.116991 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:11:01.116991 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:11:01.116991 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:11:01.116991 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:11:01.116991 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:11:01.116991 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:11:01.121901 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 353/707] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:11:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:11:01.123062 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa7f8b1-ac47-45e3-ba89-965ca35d0888 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:01.123343 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa7f8b1-ac47-45e3-ba89-965ca35d0888 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:01.123622 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa7f8b1-ac47-45e3-ba89-965ca35d0888 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:01.142653 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-5aa7f8b1-ac47-45e3-ba89-965ca35d0888 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:11:01.143880 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-5aa7f8b1-ac47-45e3-ba89-965ca35d0888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:11:01.154233 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:01.178608 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38364604-b670-4329-9d2a-082f367853f2 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:01.216243 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26721318-6499-4ff8-8c42-628f4b13fbfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:01.216984 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 354/708] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:11:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:11:01.217141 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26721318-6499-4ff8-8c42-628f4b13fbfc None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:01.217523 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26721318-6499-4ff8-8c42-628f4b13fbfc None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:01.217924 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26721318-6499-4ff8-8c42-628f4b13fbfc None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:01.218617 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa7f8b1-ac47-45e3-ba89-965ca35d0888 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:01.218826 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26721318-6499-4ff8-8c42-628f4b13fbfc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:01.221063 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26721318-6499-4ff8-8c42-628f4b13fbfc None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:01.248049 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 355/709] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 19 14:11:01 2022] POST /identity/v3/users => generated 254 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 14:11:01.257970 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26721318-6499-4ff8-8c42-628f4b13fbfc None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:01.322427 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:01.374206 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26721318-6499-4ff8-8c42-628f4b13fbfc None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:01.376360 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa7f8b1-ac47-45e3-ba89-965ca35d0888 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:01.413805 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236ee2c3-6f4f-445c-bff9-b434b950d51d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:01.414967 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236ee2c3-6f4f-445c-bff9-b434b950d51d None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:01.415400 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236ee2c3-6f4f-445c-bff9-b434b950d51d None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:01.415827 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236ee2c3-6f4f-445c-bff9-b434b950d51d None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:01.416677 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236ee2c3-6f4f-445c-bff9-b434b950d51d None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:01.418275 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236ee2c3-6f4f-445c-bff9-b434b950d51d None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:01.434150 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-236ee2c3-6f4f-445c-bff9-b434b950d51d None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 14:11:01.434150 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:01.434150 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:11:01.434150 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:11:01.434150 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:11:01.434150 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:11:01.434150 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:11:01.434150 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:11:01.434150 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 14:11:01.434150 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:11:01.434150 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:11:01.434150 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:11:01.434150 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:01.434150 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:11:01.434150 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:11:01.434150 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:11:01.435263 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:11:01.435263 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:11:01.435263 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:11:01.435263 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:11:01.435263 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:11:01.435263 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:11:01.435263 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:11:01.435263 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:11:01.435263 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:11:01.435263 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:11:01.435263 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:11:01.435263 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:11:01.435263 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:11:01.435263 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:11:01.435263 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:11:01.435263 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:11:01.436159 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:11:01.436159 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:11:01.436159 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:11:01.436159 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:11:01.436159 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:11:01.436159 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:11:01.436159 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:11:01.436159 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:11:01.436159 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:11:01.436159 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:11:01.436159 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:11:01.436159 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:11:01.436159 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:11:01.436159 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 14:11:01.436159 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:11:01.439093 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 356/710] 10.222.0.57 () {56 vars in 1129 bytes} [Thu May 19 14:11:01 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:11:01.445011 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236ee2c3-6f4f-445c-bff9-b434b950d51d None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:01.462384 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7388c7-c70b-4527-8527-21260ab971a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:01.463442 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7388c7-c70b-4527-8527-21260ab971a2 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:01.463910 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7388c7-c70b-4527-8527-21260ab971a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:01.464388 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7388c7-c70b-4527-8527-21260ab971a2 None admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:01.465394 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7388c7-c70b-4527-8527-21260ab971a2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:01.468729 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7388c7-c70b-4527-8527-21260ab971a2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:01.471091 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7388c7-c70b-4527-8527-21260ab971a2 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:01.495663 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 355/711] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:11:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:01.501213 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7388c7-c70b-4527-8527-21260ab971a2 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:01.505707 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:01.577830 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236ee2c3-6f4f-445c-bff9-b434b950d51d None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:01.597863 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:01.616026 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b67820-4705-4dde-861d-4c1e3fc07aae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:01.617000 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b67820-4705-4dde-861d-4c1e3fc07aae None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:01.617469 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b67820-4705-4dde-861d-4c1e3fc07aae None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:01.618048 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b67820-4705-4dde-861d-4c1e3fc07aae None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:01.618949 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b67820-4705-4dde-861d-4c1e3fc07aae None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:01.619507 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b67820-4705-4dde-861d-4c1e3fc07aae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:01.621086 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b67820-4705-4dde-861d-4c1e3fc07aae None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:01.626579 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 357/712] 10.222.0.57 () {56 vars in 1124 bytes} [Thu May 19 14:11:01 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:01.632976 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b67820-4705-4dde-861d-4c1e3fc07aae None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:01.633125 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7388c7-c70b-4527-8527-21260ab971a2 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:01.683222 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4696ca4b-1bd6-4762-a966-a8fbfd1932ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:01.684267 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4696ca4b-1bd6-4762-a966-a8fbfd1932ff None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:01.684741 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4696ca4b-1bd6-4762-a966-a8fbfd1932ff None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:01.685264 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4696ca4b-1bd6-4762-a966-a8fbfd1932ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:01.686349 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4696ca4b-1bd6-4762-a966-a8fbfd1932ff None admin] RBAC: Adding query filter params (role.id=4946cf0ff7164307b36efeee7c787146, scope.project.id=c6f995ffa0a94b1584f25dbe31e20524, user.id=8c87f361a92d4834854ca07f652be2ff) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:01.686881 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4696ca4b-1bd6-4762-a966-a8fbfd1932ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:01.688575 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4696ca4b-1bd6-4762-a966-a8fbfd1932ff None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:01.695647 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 356/713] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:11:01 2022] GET /identity/v3/role_assignments?user.id=8c87f361a92d4834854ca07f652be2ff&scope.project.id=c6f995ffa0a94b1584f25dbe31e20524&role.id=4946cf0ff7164307b36efeee7c787146 => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:01.770884 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b67820-4705-4dde-861d-4c1e3fc07aae None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:01.812366 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8391318f-f8c1-44ca-b989-d46087ed8260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:01.813425 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8391318f-f8c1-44ca-b989-d46087ed8260 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:01.814031 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8391318f-f8c1-44ca-b989-d46087ed8260 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:01.815074 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8391318f-f8c1-44ca-b989-d46087ed8260 None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:01.816084 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8391318f-f8c1-44ca-b989-d46087ed8260 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:01.816610 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8391318f-f8c1-44ca-b989-d46087ed8260 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:01.818239 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8391318f-f8c1-44ca-b989-d46087ed8260 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:01.825293 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 358/714] 10.222.0.57 () {56 vars in 1124 bytes} [Thu May 19 14:11:01 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:02.184295 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843e5a8c-0283-42dd-8adb-075673dd8b88 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:02.184934 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843e5a8c-0283-42dd-8adb-075673dd8b88 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:02.185424 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843e5a8c-0283-42dd-8adb-075673dd8b88 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:02.187088 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 357/715] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:11:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:11:02.198529 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851f7c7f-9d9d-421b-b5ae-1b30a451ee53 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:02.198828 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851f7c7f-9d9d-421b-b5ae-1b30a451ee53 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:02.199095 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851f7c7f-9d9d-421b-b5ae-1b30a451ee53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:02.222853 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-851f7c7f-9d9d-421b-b5ae-1b30a451ee53 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:11:02.224057 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-851f7c7f-9d9d-421b-b5ae-1b30a451ee53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:11:02.279936 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:02.316672 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 359/716] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:11:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:11:02.352017 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843e5a8c-0283-42dd-8adb-075673dd8b88 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:02.448869 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:02.488282 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843e5a8c-0283-42dd-8adb-075673dd8b88 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:02.543777 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8a5679-a573-4fdb-ac0e-151a8b7ae2c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:02.544584 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8a5679-a573-4fdb-ac0e-151a8b7ae2c6 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:02.544871 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8a5679-a573-4fdb-ac0e-151a8b7ae2c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:02.545151 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8a5679-a573-4fdb-ac0e-151a8b7ae2c6 None admin] PATH_INFO: `/v3/services/network` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:02.545812 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8a5679-a573-4fdb-ac0e-151a8b7ae2c6 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:02.547240 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8a5679-a573-4fdb-ac0e-151a8b7ae2c6 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:02.563647 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-0e8a5679-a573-4fdb-ac0e-151a8b7ae2c6 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 19 14:11:02.563647 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:02.563647 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:11:02.563647 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:11:02.563647 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:11:02.563647 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:11:02.563647 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:11:02.563647 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:11:02.563647 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 14:11:02.563647 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:11:02.563647 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:11:02.563647 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:11:02.563647 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:02.563647 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:11:02.563647 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:11:02.563647 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:11:02.563647 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:11:02.564794 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:11:02.564794 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:11:02.564794 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:11:02.564794 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:11:02.564794 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:11:02.564794 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:11:02.564794 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:11:02.564794 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:11:02.564794 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:11:02.564794 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:11:02.564794 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:11:02.564794 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:11:02.564794 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:11:02.564794 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:11:02.564794 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:11:02.564794 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:11:02.565681 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:11:02.565681 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:11:02.565681 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:11:02.565681 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:11:02.565681 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:11:02.565681 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:11:02.565681 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:11:02.565681 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:11:02.565681 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:11:02.565681 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:11:02.565681 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:11:02.565681 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:11:02.565681 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 14:11:02.565681 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:11:02.565681 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 358/717] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:11:02 2022] GET /identity/v3/services/network => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:11:02.572881 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851f7c7f-9d9d-421b-b5ae-1b30a451ee53 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:02.702549 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851f7c7f-9d9d-421b-b5ae-1b30a451ee53 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:02.749319 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1a65c0-5eb5-4bce-9f1a-b16f5508cc34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:02.750017 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1a65c0-5eb5-4bce-9f1a-b16f5508cc34 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:02.750301 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1a65c0-5eb5-4bce-9f1a-b16f5508cc34 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:02.750575 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1a65c0-5eb5-4bce-9f1a-b16f5508cc34 None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:02.751288 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1a65c0-5eb5-4bce-9f1a-b16f5508cc34 None admin] RBAC: Adding query filter params (name=network) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:02.752121 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1a65c0-5eb5-4bce-9f1a-b16f5508cc34 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:02.754155 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1a65c0-5eb5-4bce-9f1a-b16f5508cc34 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:02.759247 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 360/718] 10.222.0.57 () {56 vars in 1112 bytes} [Thu May 19 14:11:02 2022] GET /identity/v3/services?name=network => generated 124 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:02.764630 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8a5679-a573-4fdb-ac0e-151a8b7ae2c6 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:02.916017 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8a5679-a573-4fdb-ac0e-151a8b7ae2c6 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:02.961183 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:02.965159 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006af3f1-266b-49eb-8919-cdb014629260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:02.965826 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006af3f1-266b-49eb-8919-cdb014629260 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:02.966129 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006af3f1-266b-49eb-8919-cdb014629260 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:02.966402 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006af3f1-266b-49eb-8919-cdb014629260 None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:02.967102 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006af3f1-266b-49eb-8919-cdb014629260 None admin] RBAC: Adding query filter params (type=network) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:02.967406 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006af3f1-266b-49eb-8919-cdb014629260 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:02.968881 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006af3f1-266b-49eb-8919-cdb014629260 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:02.978673 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 359/719] 10.222.0.57 () {56 vars in 1112 bytes} [Thu May 19 14:11:02 2022] GET /identity/v3/services?type=network => generated 124 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:03.468750 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274b2a2d-3d04-4095-9ad5-eb4c173bdd72 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:03.469267 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274b2a2d-3d04-4095-9ad5-eb4c173bdd72 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:03.469345 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274b2a2d-3d04-4095-9ad5-eb4c173bdd72 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:03.471068 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 361/720] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:11:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:11:03.476547 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d242d5-b517-438a-b069-eb04addeb8f5 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:03.476973 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d242d5-b517-438a-b069-eb04addeb8f5 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:03.477145 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d242d5-b517-438a-b069-eb04addeb8f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:03.499081 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-99d242d5-b517-438a-b069-eb04addeb8f5 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:11:03.500270 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-99d242d5-b517-438a-b069-eb04addeb8f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:11:03.601978 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 360/721] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:11:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:11:03.621422 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274b2a2d-3d04-4095-9ad5-eb4c173bdd72 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:03.676800 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:03.742537 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274b2a2d-3d04-4095-9ad5-eb4c173bdd72 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:03.778103 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846c25f4-2cf9-44fe-a3fd-d3de2c4289d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:03.778900 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846c25f4-2cf9-44fe-a3fd-d3de2c4289d2 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:03.779234 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846c25f4-2cf9-44fe-a3fd-d3de2c4289d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:03.779507 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846c25f4-2cf9-44fe-a3fd-d3de2c4289d2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:03.790034 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846c25f4-2cf9-44fe-a3fd-d3de2c4289d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:11:03.797639 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-846c25f4-2cf9-44fe-a3fd-d3de2c4289d2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:11:03.797639 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:03.797639 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:11:03.797639 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:11:03.797639 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:11:03.797639 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:11:03.797639 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:11:03.797639 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:11:03.797639 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:11:03.797639 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:11:03.797639 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:11:03.797639 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:11:03.797639 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:11:03.797639 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:11:03.797639 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:11:03.797639 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:11:03.798771 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:11:03.798771 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:11:03.798771 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:11:03.798771 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:11:03.798771 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:11:03.798771 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:11:03.798771 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:11:03.798771 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:11:03.798771 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:11:03.798771 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:11:03.798771 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:11:03.798771 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:11:03.798771 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:11:03.798771 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:11:03.798771 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:11:03.798771 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:11:03.799624 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:11:03.799624 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:11:03.799624 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:11:03.799624 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:11:03.799624 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:11:03.799624 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:11:03.799624 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:11:03.799624 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:11:03.799624 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:11:03.799624 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:11:03.799624 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:11:03.799624 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:11:03.803989 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 362/722] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:11:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:11:03.805444 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d242d5-b517-438a-b069-eb04addeb8f5 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:03.927162 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d242d5-b517-438a-b069-eb04addeb8f5 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:03.964201 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:03.978220 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3c71b2-43ea-4677-a6bb-1ca2bc84c82d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:03.978891 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3c71b2-43ea-4677-a6bb-1ca2bc84c82d None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:03.979171 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3c71b2-43ea-4677-a6bb-1ca2bc84c82d None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:03.979444 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3c71b2-43ea-4677-a6bb-1ca2bc84c82d None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:03.980196 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3c71b2-43ea-4677-a6bb-1ca2bc84c82d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:03.980499 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3c71b2-43ea-4677-a6bb-1ca2bc84c82d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:03.981932 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3c71b2-43ea-4677-a6bb-1ca2bc84c82d None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:03.995166 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 361/723] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:11:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:04.000644 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846c25f4-2cf9-44fe-a3fd-d3de2c4289d2 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:04.143207 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:04.146426 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846c25f4-2cf9-44fe-a3fd-d3de2c4289d2 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:04.190509 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec9449a-87c4-4c02-bbbc-e7f95f4c09cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:04.191304 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec9449a-87c4-4c02-bbbc-e7f95f4c09cd None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:04.191585 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec9449a-87c4-4c02-bbbc-e7f95f4c09cd None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:04.191860 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec9449a-87c4-4c02-bbbc-e7f95f4c09cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:04.202735 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec9449a-87c4-4c02-bbbc-e7f95f4c09cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:04.204194 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec9449a-87c4-4c02-bbbc-e7f95f4c09cd None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:04.222459 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-eec9449a-87c4-4c02-bbbc-e7f95f4c09cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:11:04.222459 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:04.222459 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:11:04.222459 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:11:04.222459 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:11:04.222459 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:11:04.222459 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:11:04.222459 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:11:04.222459 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:11:04.222459 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:11:04.222459 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:11:04.222459 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:11:04.222459 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:11:04.222459 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:11:04.222459 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:11:04.223415 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:11:04.223415 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:11:04.223415 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:11:04.223415 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:11:04.223415 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:11:04.223415 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:11:04.223415 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:11:04.223415 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:11:04.223415 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:11:04.223415 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:11:04.223415 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:11:04.223415 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:11:04.223415 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:11:04.223415 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:11:04.223415 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:11:04.223415 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:11:04.223415 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:11:04.224415 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:11:04.224415 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:11:04.224415 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:11:04.224415 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:11:04.228469 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 363/724] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:11:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:11:04.229953 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3c71b2-43ea-4677-a6bb-1ca2bc84c82d None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:04.373947 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3c71b2-43ea-4677-a6bb-1ca2bc84c82d None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:04.409902 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa5a221-4818-4cce-925a-b7d3365030b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:04.410570 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa5a221-4818-4cce-925a-b7d3365030b1 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:04.410849 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa5a221-4818-4cce-925a-b7d3365030b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:04.411119 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa5a221-4818-4cce-925a-b7d3365030b1 None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:04.411834 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa5a221-4818-4cce-925a-b7d3365030b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:04.412134 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa5a221-4818-4cce-925a-b7d3365030b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:04.413507 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa5a221-4818-4cce-925a-b7d3365030b1 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:04.554667 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc22b124-8929-4271-af25-1d2fa6fda868 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:04.555226 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc22b124-8929-4271-af25-1d2fa6fda868 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:04.555226 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc22b124-8929-4271-af25-1d2fa6fda868 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:04.555574 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 362/725] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:11:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:04.556946 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa5a221-4818-4cce-925a-b7d3365030b1 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:04.557198 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 364/726] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:11:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:11:04.562634 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f99324-67d6-46f4-a002-3a2caa1186eb None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:04.562932 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f99324-67d6-46f4-a002-3a2caa1186eb None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:04.563216 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f99324-67d6-46f4-a002-3a2caa1186eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:04.588450 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-78f99324-67d6-46f4-a002-3a2caa1186eb None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:11:04.589662 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-78f99324-67d6-46f4-a002-3a2caa1186eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:11:04.626301 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:04.663066 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 365/727] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:11:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:11:04.664549 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a240f459-54ed-42d9-bb63-e6ad86958d0f None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:04.664836 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a240f459-54ed-42d9-bb63-e6ad86958d0f None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:04.665282 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a240f459-54ed-42d9-bb63-e6ad86958d0f None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:04.666833 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 366/728] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:11:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:11:04.676388 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bfd409-e5b9-4b47-8987-8d94bff45e62 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:04.676765 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bfd409-e5b9-4b47-8987-8d94bff45e62 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:04.677118 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bfd409-e5b9-4b47-8987-8d94bff45e62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:04.696845 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-f6bfd409-e5b9-4b47-8987-8d94bff45e62 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:11:04.698607 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-f6bfd409-e5b9-4b47-8987-8d94bff45e62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:11:04.704937 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa5a221-4818-4cce-925a-b7d3365030b1 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:04.743694 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190ec97b-2800-46db-b31a-a6ca4d6b4f62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:04.744589 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190ec97b-2800-46db-b31a-a6ca4d6b4f62 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:04.744947 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190ec97b-2800-46db-b31a-a6ca4d6b4f62 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:04.745293 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190ec97b-2800-46db-b31a-a6ca4d6b4f62 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:04.754127 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190ec97b-2800-46db-b31a-a6ca4d6b4f62 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:04.755575 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190ec97b-2800-46db-b31a-a6ca4d6b4f62 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:04.770196 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-190ec97b-2800-46db-b31a-a6ca4d6b4f62 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 14:11:04.770196 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:04.770196 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:11:04.770196 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:11:04.770196 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:11:04.770196 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:11:04.770196 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:11:04.770196 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:11:04.770196 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:11:04.770196 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:11:04.770196 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:11:04.770196 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:11:04.770196 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:11:04.770196 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:11:04.770196 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:11:04.770196 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:11:04.771178 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:11:04.771178 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:11:04.771178 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:11:04.771178 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:11:04.771178 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:11:04.771178 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:11:04.771178 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:11:04.771178 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:11:04.771178 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:11:04.771178 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:11:04.771178 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:11:04.771178 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:11:04.771178 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:11:04.771178 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:11:04.771178 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:11:04.771178 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:11:04.771178 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:11:04.771178 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:11:04.772389 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:11:04.772389 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:11:04.772389 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:11:04.772389 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:11:04.772389 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:11:04.772389 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:11:04.772389 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:11:04.772389 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:11:04.772389 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 14:11:04.772389 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:11:04.772389 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 363/729] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:11:04 2022] GET /identity/v3/users/swift => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:11:04.772389 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190ec97b-2800-46db-b31a-a6ca4d6b4f62 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:04.785249 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 367/730] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:11:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:11:04.787042 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63456f51-480e-4332-86fd-c227f0a66fcf None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:04.787534 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63456f51-480e-4332-86fd-c227f0a66fcf None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:04.788005 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63456f51-480e-4332-86fd-c227f0a66fcf None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:04.789905 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 368/731] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:11:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:11:04.791201 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63456f51-480e-4332-86fd-c227f0a66fcf None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:04.818446 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:04.887847 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190ec97b-2800-46db-b31a-a6ca4d6b4f62 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:04.897744 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:04.909215 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63456f51-480e-4332-86fd-c227f0a66fcf None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:04.924333 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1968a0e8-9b75-4050-afa9-a67829728284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:04.925144 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1968a0e8-9b75-4050-afa9-a67829728284 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:04.925428 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1968a0e8-9b75-4050-afa9-a67829728284 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:04.925707 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1968a0e8-9b75-4050-afa9-a67829728284 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:04.937870 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1968a0e8-9b75-4050-afa9-a67829728284 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:11:04.945918 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d892f781-7be8-41a2-a5c6-17ba02157efa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:04.946198 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-1968a0e8-9b75-4050-afa9-a67829728284 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:11:04.946198 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:04.946198 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:11:04.946198 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:11:04.946198 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:11:04.946198 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:11:04.946198 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:11:04.946198 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:11:04.946198 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:11:04.946198 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:11:04.946198 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:11:04.946198 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:11:04.946198 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:11:04.946198 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:11:04.946198 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:11:04.946198 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:11:04.947111 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:11:04.947111 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:11:04.947111 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:11:04.947111 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:11:04.947111 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:11:04.947111 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:11:04.947111 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:11:04.947111 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:11:04.947111 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:11:04.947111 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:11:04.947111 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:11:04.947111 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:11:04.947111 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:11:04.947111 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:11:04.947111 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:11:04.947111 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:11:04.947995 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:11:04.947995 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:11:04.947995 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:11:04.947995 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:11:04.947995 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:11:04.947995 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:11:04.947995 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:11:04.947995 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:11:04.947995 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:11:04.947995 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:11:04.947995 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:11:04.947995 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:11:04.949174 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d892f781-7be8-41a2-a5c6-17ba02157efa None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:04.949174 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d892f781-7be8-41a2-a5c6-17ba02157efa None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:04.949174 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d892f781-7be8-41a2-a5c6-17ba02157efa None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:04.949174 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d892f781-7be8-41a2-a5c6-17ba02157efa None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:04.949174 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d892f781-7be8-41a2-a5c6-17ba02157efa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:04.949674 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d892f781-7be8-41a2-a5c6-17ba02157efa None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:04.953078 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 364/732] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 14:11:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:11:04.953344 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412b979b-7f77-404e-a414-547f970ec98f None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:04.953641 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412b979b-7f77-404e-a414-547f970ec98f None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:04.953952 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412b979b-7f77-404e-a414-547f970ec98f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:04.971573 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-412b979b-7f77-404e-a414-547f970ec98f None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:11:04.972783 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-412b979b-7f77-404e-a414-547f970ec98f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:11:04.974309 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 369/733] 10.222.0.57 () {56 vars in 1135 bytes} [Thu May 19 14:11:04 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:04.975393 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d892f781-7be8-41a2-a5c6-17ba02157efa None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:05.047217 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 365/734] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:11:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:11:05.049039 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412b979b-7f77-404e-a414-547f970ec98f None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:05.050465 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:05.086573 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d892f781-7be8-41a2-a5c6-17ba02157efa None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:05.130290 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830a0a1f-ee51-4672-a962-01320fe6990e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:05.131215 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830a0a1f-ee51-4672-a962-01320fe6990e None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:05.131659 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830a0a1f-ee51-4672-a962-01320fe6990e None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:05.131939 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830a0a1f-ee51-4672-a962-01320fe6990e None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:05.132700 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830a0a1f-ee51-4672-a962-01320fe6990e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:05.134143 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830a0a1f-ee51-4672-a962-01320fe6990e None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:05.150705 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 370/735] 10.222.0.57 () {60 vars in 1141 bytes} [Thu May 19 14:11:04 2022] POST /identity/v3/services => generated 251 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:11:05.151784 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830a0a1f-ee51-4672-a962-01320fe6990e None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:05.182260 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:05.187278 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412b979b-7f77-404e-a414-547f970ec98f None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:05.268822 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124fcde2-114b-46a8-b193-b03cc888625f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:05.270118 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124fcde2-114b-46a8-b193-b03cc888625f None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:05.271017 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124fcde2-114b-46a8-b193-b03cc888625f None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:05.271784 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124fcde2-114b-46a8-b193-b03cc888625f None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:05.272946 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124fcde2-114b-46a8-b193-b03cc888625f None admin] RBAC: Adding query filter params (name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:05.273481 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124fcde2-114b-46a8-b193-b03cc888625f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:05.275808 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124fcde2-114b-46a8-b193-b03cc888625f None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:05.280480 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830a0a1f-ee51-4672-a962-01320fe6990e None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:05.288681 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 366/736] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 19 14:11:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:05.289988 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124fcde2-114b-46a8-b193-b03cc888625f None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:05.319669 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:05.327823 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d127ed5-dcec-4586-bfa2-8c8468b465b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:05.328495 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d127ed5-dcec-4586-bfa2-8c8468b465b7 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:05.329065 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d127ed5-dcec-4586-bfa2-8c8468b465b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:05.329349 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d127ed5-dcec-4586-bfa2-8c8468b465b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:05.426058 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d127ed5-dcec-4586-bfa2-8c8468b465b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:05.428184 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d127ed5-dcec-4586-bfa2-8c8468b465b7 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:05.432604 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124fcde2-114b-46a8-b193-b03cc888625f None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:05.475694 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bca27ee-0f0a-4e01-9508-743652faadac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:05.476589 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bca27ee-0f0a-4e01-9508-743652faadac None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:05.476877 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bca27ee-0f0a-4e01-9508-743652faadac None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:05.477153 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bca27ee-0f0a-4e01-9508-743652faadac None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:05.489255 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bca27ee-0f0a-4e01-9508-743652faadac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:11:05.498075 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-5bca27ee-0f0a-4e01-9508-743652faadac None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:11:05.498075 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:05.498075 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:11:05.498075 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:11:05.498075 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:11:05.498075 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:11:05.498075 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:11:05.498075 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:11:05.498075 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:11:05.498075 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:11:05.498075 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:11:05.498075 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:11:05.498075 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:11:05.498075 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:11:05.498075 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:11:05.498075 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:11:05.499134 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:11:05.499134 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:11:05.499134 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:11:05.499134 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:11:05.499134 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:11:05.499134 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:11:05.499134 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:11:05.499134 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:11:05.499134 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:11:05.499134 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:11:05.499134 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:11:05.499134 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:11:05.499134 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:11:05.499134 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:11:05.499134 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:11:05.499134 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:11:05.499984 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:11:05.499984 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:11:05.499984 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:11:05.499984 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:11:05.499984 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:11:05.499984 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:11:05.499984 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:11:05.499984 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:11:05.499984 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:11:05.499984 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:11:05.499984 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:11:05.499984 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:11:05.499984 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 367/737] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:11:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:11:05.500701 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bca27ee-0f0a-4e01-9508-743652faadac None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:05.526839 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:05.570189 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 371/738] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:11:05 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:11:05.571040 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d127ed5-dcec-4586-bfa2-8c8468b465b7 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:05.603941 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:05.639489 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bca27ee-0f0a-4e01-9508-743652faadac None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:05.676224 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d127ed5-dcec-4586-bfa2-8c8468b465b7 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:05.679555 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023bcfe9-e985-43e4-9be5-4169ce68fe56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:05.680601 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023bcfe9-e985-43e4-9be5-4169ce68fe56 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:05.681109 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023bcfe9-e985-43e4-9be5-4169ce68fe56 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:05.681611 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023bcfe9-e985-43e4-9be5-4169ce68fe56 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:05.695203 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023bcfe9-e985-43e4-9be5-4169ce68fe56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:05.697002 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023bcfe9-e985-43e4-9be5-4169ce68fe56 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:05.712712 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8f662f-bd6b-44c5-8816-5392e629a568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:05.713379 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8f662f-bd6b-44c5-8816-5392e629a568 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:05.713662 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8f662f-bd6b-44c5-8816-5392e629a568 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:05.713964 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8f662f-bd6b-44c5-8816-5392e629a568 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:05.714704 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8f662f-bd6b-44c5-8816-5392e629a568 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:05.715006 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8f662f-bd6b-44c5-8816-5392e629a568 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:05.716465 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8f662f-bd6b-44c5-8816-5392e629a568 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:05.719454 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-023bcfe9-e985-43e4-9be5-4169ce68fe56 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:11:05.719454 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:05.719454 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:11:05.719454 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:11:05.719454 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:11:05.719454 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:11:05.719454 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:11:05.719454 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:11:05.719454 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:11:05.719454 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:11:05.719454 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:11:05.719454 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:11:05.719454 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:11:05.719454 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:11:05.719454 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:11:05.720377 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:11:05.720377 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:11:05.720377 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:11:05.720377 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:11:05.720377 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:11:05.720377 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:11:05.720377 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:11:05.720377 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:11:05.720377 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:11:05.720377 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:11:05.720377 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:11:05.720377 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:11:05.720377 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:11:05.720377 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:11:05.720377 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:11:05.720377 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:11:05.720377 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:11:05.721495 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:11:05.721495 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:11:05.721495 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:11:05.721495 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:11:05.728287 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 368/740] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:11:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:11:05.729582 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023bcfe9-e985-43e4-9be5-4169ce68fe56 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:05.729911 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 372/740] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:11:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:05.730841 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8f662f-bd6b-44c5-8816-5392e629a568 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:05.808542 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:05.831868 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:05.841230 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8f662f-bd6b-44c5-8816-5392e629a568 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:05.853660 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023bcfe9-e985-43e4-9be5-4169ce68fe56 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:05.879419 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682e963c-5a3b-4c05-b4c8-199546ddf830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:05.879762 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682e963c-5a3b-4c05-b4c8-199546ddf830 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:05.880052 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682e963c-5a3b-4c05-b4c8-199546ddf830 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:05.880383 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682e963c-5a3b-4c05-b4c8-199546ddf830 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:05.881102 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682e963c-5a3b-4c05-b4c8-199546ddf830 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:05.881447 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682e963c-5a3b-4c05-b4c8-199546ddf830 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:05.882802 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682e963c-5a3b-4c05-b4c8-199546ddf830 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:05.896101 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b450fe34-9225-4629-9e3b-6e95448441ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:05.897136 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b450fe34-9225-4629-9e3b-6e95448441ec None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:05.897597 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b450fe34-9225-4629-9e3b-6e95448441ec None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:05.898058 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b450fe34-9225-4629-9e3b-6e95448441ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:05.901817 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 373/741] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:11:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:05.902793 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682e963c-5a3b-4c05-b4c8-199546ddf830 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:05.911156 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b450fe34-9225-4629-9e3b-6e95448441ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:05.912955 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b450fe34-9225-4629-9e3b-6e95448441ec None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:05.932752 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-b450fe34-9225-4629-9e3b-6e95448441ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:11:05.932752 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:05.932752 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:11:05.932752 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:11:05.932752 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:11:05.932752 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:11:05.932752 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:11:05.932752 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:11:05.932752 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:11:05.932752 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:11:05.932752 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:11:05.932752 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:11:05.932752 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:11:05.932752 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:11:05.932752 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:11:05.933901 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:11:05.933901 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:11:05.933901 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:11:05.933901 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:11:05.933901 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:11:05.933901 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:11:05.933901 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:11:05.933901 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:11:05.933901 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:11:05.933901 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:11:05.933901 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:11:05.933901 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:11:05.933901 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:11:05.933901 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:11:05.933901 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:11:05.933901 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:11:05.933901 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:11:05.934920 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:11:05.934920 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:11:05.934920 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:11:05.934920 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:11:05.936828 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 369/742] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:11:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:11:05.937978 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b450fe34-9225-4629-9e3b-6e95448441ec None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:06.008554 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682e963c-5a3b-4c05-b4c8-199546ddf830 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:06.046373 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffdd79d-c2b9-4167-943b-641b3e1a75b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:06.047165 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffdd79d-c2b9-4167-943b-641b3e1a75b5 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:06.047447 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffdd79d-c2b9-4167-943b-641b3e1a75b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:06.047720 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffdd79d-c2b9-4167-943b-641b3e1a75b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:06.051797 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b450fe34-9225-4629-9e3b-6e95448441ec None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:06.058444 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffdd79d-c2b9-4167-943b-641b3e1a75b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:06.059956 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffdd79d-c2b9-4167-943b-641b3e1a75b5 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:06.074008 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:06.080191 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-5ffdd79d-c2b9-4167-943b-641b3e1a75b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:11:06.080191 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:06.080191 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:11:06.080191 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:11:06.080191 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:11:06.080191 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:11:06.080191 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:11:06.080191 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:11:06.080191 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:11:06.080191 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:11:06.080191 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:11:06.080191 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:11:06.080191 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:11:06.080191 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:11:06.080191 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:11:06.081143 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:11:06.081143 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:11:06.081143 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:11:06.081143 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:11:06.081143 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:11:06.081143 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:11:06.081143 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:11:06.081143 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:11:06.081143 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:11:06.081143 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:11:06.081143 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:11:06.081143 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:11:06.081143 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:11:06.081143 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:11:06.081143 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:11:06.081143 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:11:06.081143 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:11:06.082111 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:11:06.082111 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:11:06.082111 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:11:06.082111 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:11:06.082111 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 374/743] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:11:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:11:06.082399 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffdd79d-c2b9-4167-943b-641b3e1a75b5 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:06.086441 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:06.097495 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9e8276-bc06-4358-b455-59677e1eb87a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:06.098343 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9e8276-bc06-4358-b455-59677e1eb87a None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:06.098624 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9e8276-bc06-4358-b455-59677e1eb87a None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:06.098895 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9e8276-bc06-4358-b455-59677e1eb87a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:06.106899 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9e8276-bc06-4358-b455-59677e1eb87a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:06.108348 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9e8276-bc06-4358-b455-59677e1eb87a None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:06.124419 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-ad9e8276-bc06-4358-b455-59677e1eb87a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 14:11:06.124419 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:06.124419 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:11:06.124419 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:11:06.124419 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:11:06.124419 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:11:06.124419 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:11:06.124419 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:11:06.124419 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:11:06.124419 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:11:06.124419 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:11:06.124419 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:11:06.124419 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:11:06.124419 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:11:06.124419 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:11:06.124419 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:11:06.125411 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:11:06.125411 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:11:06.125411 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:11:06.125411 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:11:06.125411 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:11:06.125411 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:11:06.125411 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:11:06.125411 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:11:06.125411 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:11:06.125411 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:11:06.125411 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:11:06.125411 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:11:06.125411 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:11:06.125411 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:11:06.125411 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:11:06.125411 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:11:06.125411 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:11:06.125411 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:11:06.127046 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:11:06.127046 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:11:06.127046 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:11:06.127046 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:11:06.127046 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:11:06.127046 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:11:06.127046 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:11:06.127046 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:11:06.127046 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 14:11:06.127046 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:11:06.127046 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 370/744] 10.222.0.57 () {56 vars in 1117 bytes} [Thu May 19 14:11:05 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:11:06.127656 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dab2724-6918-4322-91fe-06bfca9bc186 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:06.127814 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dab2724-6918-4322-91fe-06bfca9bc186 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:06.128425 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dab2724-6918-4322-91fe-06bfca9bc186 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:06.129364 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 371/745] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:11:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:11:06.129905 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dab2724-6918-4322-91fe-06bfca9bc186 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:06.207699 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffdd79d-c2b9-4167-943b-641b3e1a75b5 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:06.239773 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dab2724-6918-4322-91fe-06bfca9bc186 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:06.243720 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86296bb-67b8-4f18-8369-d327f71ff345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:06.244253 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86296bb-67b8-4f18-8369-d327f71ff345 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:06.244536 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86296bb-67b8-4f18-8369-d327f71ff345 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:06.244812 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86296bb-67b8-4f18-8369-d327f71ff345 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:06.245527 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86296bb-67b8-4f18-8369-d327f71ff345 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:06.245834 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86296bb-67b8-4f18-8369-d327f71ff345 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:06.247346 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86296bb-67b8-4f18-8369-d327f71ff345 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:06.265018 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 375/746] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:11:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:06.265753 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86296bb-67b8-4f18-8369-d327f71ff345 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:06.274101 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7568133-08d8-4b14-9b06-b2fc75516e7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:06.274764 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7568133-08d8-4b14-9b06-b2fc75516e7a None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:06.275046 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7568133-08d8-4b14-9b06-b2fc75516e7a None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:06.275308 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7568133-08d8-4b14-9b06-b2fc75516e7a None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:06.276026 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7568133-08d8-4b14-9b06-b2fc75516e7a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:06.276328 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7568133-08d8-4b14-9b06-b2fc75516e7a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:06.277680 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7568133-08d8-4b14-9b06-b2fc75516e7a None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:06.290093 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 372/747] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:11:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:06.291297 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b50629-840b-4be9-b0ea-8355cae8f18a None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:06.291582 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b50629-840b-4be9-b0ea-8355cae8f18a None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:06.291854 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b50629-840b-4be9-b0ea-8355cae8f18a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:06.310987 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-f0b50629-840b-4be9-b0ea-8355cae8f18a None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:11:06.312555 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-f0b50629-840b-4be9-b0ea-8355cae8f18a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:11:06.335705 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:06.352167 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:06.380630 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86296bb-67b8-4f18-8369-d327f71ff345 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:06.398630 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 373/748] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:11:06 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:11:06.399795 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b50629-840b-4be9-b0ea-8355cae8f18a None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:06.425655 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222d2609-66b1-4cdc-8d60-6303d9a5c0b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:06.426362 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222d2609-66b1-4cdc-8d60-6303d9a5c0b8 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:06.426643 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222d2609-66b1-4cdc-8d60-6303d9a5c0b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:06.426917 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222d2609-66b1-4cdc-8d60-6303d9a5c0b8 None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:06.427757 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222d2609-66b1-4cdc-8d60-6303d9a5c0b8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:06.428063 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222d2609-66b1-4cdc-8d60-6303d9a5c0b8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:06.429495 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222d2609-66b1-4cdc-8d60-6303d9a5c0b8 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:06.447706 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 376/749] 10.222.0.57 () {56 vars in 1149 bytes} [Thu May 19 14:11:06 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:06.452266 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222d2609-66b1-4cdc-8d60-6303d9a5c0b8 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:06.511219 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b50629-840b-4be9-b0ea-8355cae8f18a None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:06.546784 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a09d1c-f222-4448-ae3f-ae07c3c1f5c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:06.547580 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a09d1c-f222-4448-ae3f-ae07c3c1f5c4 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:06.547857 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a09d1c-f222-4448-ae3f-ae07c3c1f5c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:06.548126 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a09d1c-f222-4448-ae3f-ae07c3c1f5c4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:06.555481 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a09d1c-f222-4448-ae3f-ae07c3c1f5c4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:06.556913 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a09d1c-f222-4448-ae3f-ae07c3c1f5c4 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:06.564242 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222d2609-66b1-4cdc-8d60-6303d9a5c0b8 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:06.570296 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-b0a09d1c-f222-4448-ae3f-ae07c3c1f5c4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:11:06.570296 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:06.570296 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:11:06.570296 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:11:06.570296 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:11:06.570296 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:11:06.570296 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:11:06.570296 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:11:06.570296 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:11:06.570296 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:11:06.570296 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:11:06.570296 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:11:06.570296 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:11:06.570296 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:11:06.570296 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:11:06.571345 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:11:06.571345 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:11:06.571345 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:11:06.571345 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:11:06.571345 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:11:06.571345 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:11:06.571345 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:11:06.571345 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:11:06.571345 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:11:06.571345 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:11:06.571345 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:11:06.571345 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:11:06.571345 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:11:06.571345 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:11:06.571345 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:11:06.571345 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:11:06.571345 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:11:06.572435 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:11:06.572435 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:11:06.572435 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:11:06.572435 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:11:06.572435 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:11:06.572435 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:11:06.572435 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:11:06.572435 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:11:06.576239 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 374/750] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:11:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:11:06.577072 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a09d1c-f222-4448-ae3f-ae07c3c1f5c4 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:06.592581 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:06.606662 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99324eb-7ba8-4015-a25c-6bfdd50e3383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:06.607465 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99324eb-7ba8-4015-a25c-6bfdd50e3383 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:06.607753 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99324eb-7ba8-4015-a25c-6bfdd50e3383 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:06.608027 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99324eb-7ba8-4015-a25c-6bfdd50e3383 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:06.611535 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:06.616567 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99324eb-7ba8-4015-a25c-6bfdd50e3383 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:06.618218 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99324eb-7ba8-4015-a25c-6bfdd50e3383 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:06.634675 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-c99324eb-7ba8-4015-a25c-6bfdd50e3383 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 14:11:06.634675 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:06.634675 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:11:06.634675 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:11:06.634675 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:11:06.634675 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:11:06.634675 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:11:06.634675 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:11:06.634675 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:11:06.634675 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:11:06.634675 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:11:06.634675 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:11:06.634675 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:11:06.634675 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:11:06.634675 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:11:06.634675 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:11:06.635800 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:11:06.635800 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:11:06.635800 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:11:06.635800 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:11:06.635800 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:11:06.635800 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:11:06.635800 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:11:06.635800 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:11:06.635800 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:11:06.635800 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:11:06.635800 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:11:06.635800 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:11:06.635800 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:11:06.635800 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:11:06.635800 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:11:06.635800 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:11:06.635800 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:11:06.635800 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:11:06.636923 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:11:06.636923 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:11:06.636923 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:11:06.636923 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:11:06.636923 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:11:06.636923 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:11:06.636923 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:11:06.636923 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:11:06.636923 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 14:11:06.636923 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:11:06.643603 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 377/751] 10.222.0.57 () {56 vars in 1093 bytes} [Thu May 19 14:11:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:11:06.644595 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99324eb-7ba8-4015-a25c-6bfdd50e3383 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:06.687914 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a09d1c-f222-4448-ae3f-ae07c3c1f5c4 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:06.729950 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16da8f1-ed7c-4002-a8e5-ae20620e1411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:06.730627 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16da8f1-ed7c-4002-a8e5-ae20620e1411 None admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:06.730912 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16da8f1-ed7c-4002-a8e5-ae20620e1411 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:06.731201 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16da8f1-ed7c-4002-a8e5-ae20620e1411 None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:06.732120 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16da8f1-ed7c-4002-a8e5-ae20620e1411 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:06.733277 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16da8f1-ed7c-4002-a8e5-ae20620e1411 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:06.749635 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99324eb-7ba8-4015-a25c-6bfdd50e3383 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:06.750311 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 375/752] 10.222.0.57 () {60 vars in 1141 bytes} [Thu May 19 14:11:06 2022] POST /identity/v3/services => generated 240 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:11:06.750983 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16da8f1-ed7c-4002-a8e5-ae20620e1411 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:06.804370 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd98547-d2e1-487e-9165-9808e835ffd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:06.805251 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd98547-d2e1-487e-9165-9808e835ffd8 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:06.805593 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd98547-d2e1-487e-9165-9808e835ffd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:06.806210 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd98547-d2e1-487e-9165-9808e835ffd8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:06.864535 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:06.884256 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:06.899736 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd98547-d2e1-487e-9165-9808e835ffd8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:06.901602 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd98547-d2e1-487e-9165-9808e835ffd8 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:06.902124 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16da8f1-ed7c-4002-a8e5-ae20620e1411 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:06.952556 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f41d28b-32de-4e65-9646-5fd2432b1f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:06.953227 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f41d28b-32de-4e65-9646-5fd2432b1f22 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:06.953514 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f41d28b-32de-4e65-9646-5fd2432b1f22 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:06.953791 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f41d28b-32de-4e65-9646-5fd2432b1f22 None admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:06.955194 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f41d28b-32de-4e65-9646-5fd2432b1f22 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:06.956189 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f41d28b-32de-4e65-9646-5fd2432b1f22 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:06.957486 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f41d28b-32de-4e65-9646-5fd2432b1f22 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:06.970947 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 376/753] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:11:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:06.971799 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f41d28b-32de-4e65-9646-5fd2432b1f22 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:07.037213 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 378/754] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:11:06 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:11:07.038326 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd98547-d2e1-487e-9165-9808e835ffd8 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:07.096445 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f41d28b-32de-4e65-9646-5fd2432b1f22 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:07.144423 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09850005-7782-4cc1-8353-95d99279fd28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:07.145102 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09850005-7782-4cc1-8353-95d99279fd28 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:07.145387 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09850005-7782-4cc1-8353-95d99279fd28 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:07.145665 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09850005-7782-4cc1-8353-95d99279fd28 None admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:07.146567 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09850005-7782-4cc1-8353-95d99279fd28 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:07.146881 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09850005-7782-4cc1-8353-95d99279fd28 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:07.148321 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09850005-7782-4cc1-8353-95d99279fd28 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:07.154433 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:07.168680 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 377/755] 10.222.0.57 () {56 vars in 1133 bytes} [Thu May 19 14:11:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:07.169522 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09850005-7782-4cc1-8353-95d99279fd28 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:07.181426 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:07.187071 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd98547-d2e1-487e-9165-9808e835ffd8 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:07.233245 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b404da4-c18c-41fe-aa78-70cc4a328380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:07.234472 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b404da4-c18c-41fe-aa78-70cc4a328380 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:07.235250 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b404da4-c18c-41fe-aa78-70cc4a328380 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:07.236470 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b404da4-c18c-41fe-aa78-70cc4a328380 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:07.237678 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b404da4-c18c-41fe-aa78-70cc4a328380 None admin] RBAC: Adding query filter params (role.id=4946cf0ff7164307b36efeee7c787146, scope.project.id=c6f995ffa0a94b1584f25dbe31e20524, user.id=09dd957f216a46dd8a3f400fcd2ba944) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:07.238560 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b404da4-c18c-41fe-aa78-70cc4a328380 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:07.240757 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b404da4-c18c-41fe-aa78-70cc4a328380 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:07.249117 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 379/756] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:11:07 2022] GET /identity/v3/role_assignments?user.id=09dd957f216a46dd8a3f400fcd2ba944&scope.project.id=c6f995ffa0a94b1584f25dbe31e20524&role.id=4946cf0ff7164307b36efeee7c787146 => generated 259 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:07.250220 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b404da4-c18c-41fe-aa78-70cc4a328380 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:07.319804 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09850005-7782-4cc1-8353-95d99279fd28 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:07.378877 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0ea8d0-492e-4562-91c6-358140eb3f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:07.380005 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0ea8d0-492e-4562-91c6-358140eb3f8e None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:07.382378 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0ea8d0-492e-4562-91c6-358140eb3f8e None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:07.382702 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0ea8d0-492e-4562-91c6-358140eb3f8e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:07.409772 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0ea8d0-492e-4562-91c6-358140eb3f8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:07.411282 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0ea8d0-492e-4562-91c6-358140eb3f8e None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:07.446471 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-0e0ea8d0-492e-4562-91c6-358140eb3f8e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:11:07.446471 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:07.446471 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:11:07.446471 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:11:07.446471 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:11:07.446471 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:11:07.446471 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:11:07.446471 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:11:07.446471 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:11:07.446471 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:11:07.446471 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:11:07.446471 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:11:07.446471 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:11:07.446471 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:11:07.446471 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:11:07.447441 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:11:07.447441 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:11:07.447441 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:11:07.447441 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:11:07.447441 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:11:07.447441 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:11:07.447441 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:11:07.447441 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:11:07.447441 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:11:07.447441 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:11:07.447441 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:11:07.447441 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:11:07.447441 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:11:07.447441 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:11:07.447441 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:11:07.447441 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:11:07.447441 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:11:07.448345 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:11:07.448345 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:11:07.448345 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:11:07.448345 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:11:07.448345 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 378/757] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:11:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:11:07.460351 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0ea8d0-492e-4562-91c6-358140eb3f8e None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:07.492492 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b404da4-c18c-41fe-aa78-70cc4a328380 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:07.552873 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:07.561088 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b471262-64ac-490a-a6f3-4a95be2fc6f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:07.561920 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b471262-64ac-490a-a6f3-4a95be2fc6f4 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:07.562215 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b471262-64ac-490a-a6f3-4a95be2fc6f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:07.562896 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b471262-64ac-490a-a6f3-4a95be2fc6f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:07.574062 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b471262-64ac-490a-a6f3-4a95be2fc6f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:07.575539 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b471262-64ac-490a-a6f3-4a95be2fc6f4 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:07.578277 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:07.595872 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-5b471262-64ac-490a-a6f3-4a95be2fc6f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:11:07.595872 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:07.595872 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:11:07.595872 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:11:07.595872 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:11:07.595872 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:11:07.595872 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:11:07.595872 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:11:07.595872 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:11:07.595872 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:11:07.595872 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:11:07.595872 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:11:07.595872 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:11:07.595872 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:11:07.595872 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:11:07.597754 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:11:07.597754 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:11:07.597754 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:11:07.597754 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:11:07.597754 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:11:07.597754 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:11:07.597754 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:11:07.597754 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:11:07.597754 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:11:07.597754 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:11:07.597754 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:11:07.597754 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:11:07.597754 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:11:07.597754 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:11:07.597754 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:11:07.597754 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:11:07.597754 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:11:07.599069 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:11:07.599069 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:11:07.599069 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:11:07.599069 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:11:07.601800 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 380/758] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:11:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:11:07.603381 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa5c84d-644c-49bd-b372-8afa88c01b3c None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:07.603690 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa5c84d-644c-49bd-b372-8afa88c01b3c None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:07.604014 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa5c84d-644c-49bd-b372-8afa88c01b3c None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:07.606364 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0ea8d0-492e-4562-91c6-358140eb3f8e None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:07.609777 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 381/759] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:11:07 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:11:07.614465 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa5c84d-644c-49bd-b372-8afa88c01b3c None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:07.643968 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f7145e-fd28-4918-bb80-cefa60324877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:07.645069 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f7145e-fd28-4918-bb80-cefa60324877 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:07.645613 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f7145e-fd28-4918-bb80-cefa60324877 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:07.646115 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f7145e-fd28-4918-bb80-cefa60324877 None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:07.647030 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f7145e-fd28-4918-bb80-cefa60324877 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:07.647624 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f7145e-fd28-4918-bb80-cefa60324877 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:07.649421 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f7145e-fd28-4918-bb80-cefa60324877 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:07.663444 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 379/760] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:11:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:07.665060 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18445ba-d1d6-4b9d-a82f-f6509605d511 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:07.665550 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18445ba-d1d6-4b9d-a82f-f6509605d511 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:07.666118 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18445ba-d1d6-4b9d-a82f-f6509605d511 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:07.686614 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-a18445ba-d1d6-4b9d-a82f-f6509605d511 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:11:07.688365 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-a18445ba-d1d6-4b9d-a82f-f6509605d511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:11:07.727307 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa5c84d-644c-49bd-b372-8afa88c01b3c None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:07.763093 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 380/761] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:11:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:11:07.764257 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18445ba-d1d6-4b9d-a82f-f6509605d511 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:07.770735 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d69401-b9de-4d9b-b7a7-fb15c03e2685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:07.771421 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d69401-b9de-4d9b-b7a7-fb15c03e2685 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:07.771717 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d69401-b9de-4d9b-b7a7-fb15c03e2685 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:07.772002 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d69401-b9de-4d9b-b7a7-fb15c03e2685 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:07.772739 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d69401-b9de-4d9b-b7a7-fb15c03e2685 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:07.773054 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d69401-b9de-4d9b-b7a7-fb15c03e2685 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:07.774486 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d69401-b9de-4d9b-b7a7-fb15c03e2685 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:07.801419 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 382/762] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:11:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:07.802825 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d69401-b9de-4d9b-b7a7-fb15c03e2685 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:07.823115 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:07.878232 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18445ba-d1d6-4b9d-a82f-f6509605d511 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:07.885192 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:07.919235 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcce795-aaba-4d82-b375-b9d445d8d201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:07.920348 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcce795-aaba-4d82-b375-b9d445d8d201 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:07.920825 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcce795-aaba-4d82-b375-b9d445d8d201 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:07.921288 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcce795-aaba-4d82-b375-b9d445d8d201 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:07.928623 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcce795-aaba-4d82-b375-b9d445d8d201 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:07.930460 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcce795-aaba-4d82-b375-b9d445d8d201 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:07.943697 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d69401-b9de-4d9b-b7a7-fb15c03e2685 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:07.945918 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-2fcce795-aaba-4d82-b375-b9d445d8d201 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:11:07.945918 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:07.945918 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:11:07.945918 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:11:07.945918 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:11:07.945918 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:11:07.945918 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:11:07.945918 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:11:07.945918 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:11:07.945918 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:11:07.945918 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:11:07.945918 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:11:07.945918 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:11:07.945918 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:11:07.945918 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:11:07.946927 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:11:07.946927 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:11:07.946927 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:11:07.946927 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:11:07.946927 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:11:07.946927 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:11:07.946927 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:11:07.946927 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:11:07.946927 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:11:07.946927 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:11:07.946927 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:11:07.946927 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:11:07.946927 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:11:07.946927 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:11:07.946927 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:11:07.946927 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:11:07.946927 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:11:07.947814 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:11:07.947814 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:11:07.947814 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:11:07.947814 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:11:07.947814 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:11:07.947814 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:11:07.947814 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:11:07.947814 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:11:07.950331 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 381/763] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:11:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:11:07.951380 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcce795-aaba-4d82-b375-b9d445d8d201 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:07.988451 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b7b0ed-63ec-4432-87c3-1c9f420d38d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:07.989828 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b7b0ed-63ec-4432-87c3-1c9f420d38d6 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:07.990143 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b7b0ed-63ec-4432-87c3-1c9f420d38d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:07.990424 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b7b0ed-63ec-4432-87c3-1c9f420d38d6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:08.001773 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b7b0ed-63ec-4432-87c3-1c9f420d38d6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:08.003726 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b7b0ed-63ec-4432-87c3-1c9f420d38d6 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:08.025394 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-33b7b0ed-63ec-4432-87c3-1c9f420d38d6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:11:08.025394 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:08.025394 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:11:08.025394 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:11:08.025394 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:11:08.025394 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:11:08.025394 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:11:08.025394 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:11:08.025394 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:11:08.025394 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:11:08.025394 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:11:08.025394 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:11:08.025394 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:11:08.025394 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:11:08.025394 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:11:08.026384 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:11:08.026384 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:11:08.026384 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:11:08.026384 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:11:08.026384 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:11:08.026384 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:11:08.026384 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:11:08.026384 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:11:08.026384 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:11:08.026384 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:11:08.026384 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:11:08.026384 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:11:08.026384 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:11:08.026384 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:11:08.026384 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:11:08.026384 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:11:08.026384 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:11:08.027316 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:11:08.027316 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:11:08.027316 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:11:08.027316 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:11:08.027316 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:11:08.027316 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:11:08.027316 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:11:08.027316 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:11:08.029914 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 383/764] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:11:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:11:08.030946 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b7b0ed-63ec-4432-87c3-1c9f420d38d6 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:08.085782 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcce795-aaba-4d82-b375-b9d445d8d201 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:08.103004 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:08.146649 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d786b7-a06e-42ff-83ed-8eee44179520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:08.148219 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d786b7-a06e-42ff-83ed-8eee44179520 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:08.148708 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d786b7-a06e-42ff-83ed-8eee44179520 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:08.149182 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d786b7-a06e-42ff-83ed-8eee44179520 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:08.150599 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d786b7-a06e-42ff-83ed-8eee44179520 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:08.152487 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d786b7-a06e-42ff-83ed-8eee44179520 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:08.163063 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b7b0ed-63ec-4432-87c3-1c9f420d38d6 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:08.169682 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-14d786b7-a06e-42ff-83ed-8eee44179520 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 14:11:08.169682 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:08.169682 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:11:08.169682 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:11:08.169682 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:11:08.169682 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:11:08.169682 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:11:08.169682 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:11:08.169682 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 14:11:08.169682 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:11:08.169682 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:11:08.169682 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:11:08.169682 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:08.169682 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:11:08.169682 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:11:08.169682 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:11:08.170688 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:11:08.170688 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:11:08.170688 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:11:08.170688 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:11:08.170688 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:11:08.170688 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:11:08.170688 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:11:08.170688 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:11:08.170688 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:11:08.170688 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:11:08.170688 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:11:08.170688 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:11:08.170688 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:11:08.170688 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:11:08.170688 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:11:08.170688 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:11:08.171612 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:11:08.171612 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:11:08.171612 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:11:08.171612 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:11:08.171612 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:11:08.171612 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:11:08.171612 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:11:08.171612 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:11:08.171612 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:11:08.171612 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:11:08.171612 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:11:08.171612 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:11:08.171612 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:11:08.171612 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 14:11:08.171612 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:11:08.174179 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 382/765] 10.222.0.57 () {56 vars in 1129 bytes} [Thu May 19 14:11:07 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:11:08.175260 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d786b7-a06e-42ff-83ed-8eee44179520 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:08.189248 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:08.212150 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d641ac2-8baa-4153-9937-d75f52a54a0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:08.212869 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d641ac2-8baa-4153-9937-d75f52a54a0d None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:08.213220 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d641ac2-8baa-4153-9937-d75f52a54a0d None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:08.213474 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d641ac2-8baa-4153-9937-d75f52a54a0d None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:08.214357 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d641ac2-8baa-4153-9937-d75f52a54a0d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:08.214668 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d641ac2-8baa-4153-9937-d75f52a54a0d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:08.216418 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d641ac2-8baa-4153-9937-d75f52a54a0d None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:08.232033 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 384/766] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:11:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:08.233610 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d641ac2-8baa-4153-9937-d75f52a54a0d None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:08.311515 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d786b7-a06e-42ff-83ed-8eee44179520 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:08.356626 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eb0875-48f0-4c1a-83d0-7d7c702916b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:08.357562 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eb0875-48f0-4c1a-83d0-7d7c702916b9 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:08.357863 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eb0875-48f0-4c1a-83d0-7d7c702916b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:08.358147 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eb0875-48f0-4c1a-83d0-7d7c702916b9 None admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:08.359188 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04eb0875-48f0-4c1a-83d0-7d7c702916b9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:08.359908 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04eb0875-48f0-4c1a-83d0-7d7c702916b9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:08.362184 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04eb0875-48f0-4c1a-83d0-7d7c702916b9 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:08.367173 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d641ac2-8baa-4153-9937-d75f52a54a0d None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:08.378282 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 383/767] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:11:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:08.379133 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eb0875-48f0-4c1a-83d0-7d7c702916b9 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:08.418469 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1591ef41-1dd1-4b24-a085-547c3708b9ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:08.419399 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1591ef41-1dd1-4b24-a085-547c3708b9ce None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:08.420082 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1591ef41-1dd1-4b24-a085-547c3708b9ce None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:08.420308 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1591ef41-1dd1-4b24-a085-547c3708b9ce None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:08.421306 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1591ef41-1dd1-4b24-a085-547c3708b9ce None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:08.421618 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1591ef41-1dd1-4b24-a085-547c3708b9ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:08.423742 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:08.424482 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1591ef41-1dd1-4b24-a085-547c3708b9ce None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:08.433933 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 385/768] 10.222.0.57 () {56 vars in 1124 bytes} [Thu May 19 14:11:08 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:08.434785 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1591ef41-1dd1-4b24-a085-547c3708b9ce None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:08.486861 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:08.504429 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eb0875-48f0-4c1a-83d0-7d7c702916b9 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:08.541514 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc9d9b7-8512-46d2-bb7c-5a0599bbf9f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:08.542225 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc9d9b7-8512-46d2-bb7c-5a0599bbf9f8 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:08.542512 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc9d9b7-8512-46d2-bb7c-5a0599bbf9f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:08.542791 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc9d9b7-8512-46d2-bb7c-5a0599bbf9f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:08.543630 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc9d9b7-8512-46d2-bb7c-5a0599bbf9f8 None admin] RBAC: Adding query filter params (role.id=9c7d4d848dd549adb9f3465d4201f26d, scope.project.id=c6f995ffa0a94b1584f25dbe31e20524, user.id=1623987d87ab4cf19b923d254a68c2c6) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:08.543949 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc9d9b7-8512-46d2-bb7c-5a0599bbf9f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:08.545388 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc9d9b7-8512-46d2-bb7c-5a0599bbf9f8 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:08.552389 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 384/769] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:11:08 2022] GET /identity/v3/role_assignments?user.id=1623987d87ab4cf19b923d254a68c2c6&scope.project.id=c6f995ffa0a94b1584f25dbe31e20524&role.id=9c7d4d848dd549adb9f3465d4201f26d => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:08.553204 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc9d9b7-8512-46d2-bb7c-5a0599bbf9f8 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:08.561938 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1591ef41-1dd1-4b24-a085-547c3708b9ce None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:08.636141 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56096bd2-a07f-4142-ae77-e9f18625982f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:08.637050 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56096bd2-a07f-4142-ae77-e9f18625982f None admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:08.637331 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56096bd2-a07f-4142-ae77-e9f18625982f None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:08.637603 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56096bd2-a07f-4142-ae77-e9f18625982f None admin] PATH_INFO: `/v3/projects/c6f995ffa0a94b1584f25dbe31e20524/users/8c87f361a92d4834854ca07f652be2ff/roles/4946cf0ff7164307b36efeee7c787146` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:08.643524 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56096bd2-a07f-4142-ae77-e9f18625982f None admin] RBAC: Authorizing `identity:create_grant(project_id=c6f995ffa0a94b1584f25dbe31e20524, user_id=8c87f361a92d4834854ca07f652be2ff, role_id=4946cf0ff7164307b36efeee7c787146)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:08.644993 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56096bd2-a07f-4142-ae77-e9f18625982f None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:08.662289 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:08.666747 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc9d9b7-8512-46d2-bb7c-5a0599bbf9f8 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:08.676675 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 386/770] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 14:11:08 2022] PUT /identity/v3/projects/c6f995ffa0a94b1584f25dbe31e20524/users/8c87f361a92d4834854ca07f652be2ff/roles/4946cf0ff7164307b36efeee7c787146 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:11:08.714592 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45424d5a-f37e-42fe-859c-239fa9ecf661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:08.715334 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45424d5a-f37e-42fe-859c-239fa9ecf661 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:08.715605 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45424d5a-f37e-42fe-859c-239fa9ecf661 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:08.715871 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45424d5a-f37e-42fe-859c-239fa9ecf661 None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:08.716661 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45424d5a-f37e-42fe-859c-239fa9ecf661 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:08.716959 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45424d5a-f37e-42fe-859c-239fa9ecf661 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:08.718437 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45424d5a-f37e-42fe-859c-239fa9ecf661 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:08.730022 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 385/771] 10.222.0.57 () {56 vars in 1124 bytes} [Thu May 19 14:11:08 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:08.733895 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56096bd2-a07f-4142-ae77-e9f18625982f None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:08.878387 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:08.882582 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56096bd2-a07f-4142-ae77-e9f18625982f None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:08.922401 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf527b73-4787-4f5c-bab7-1515304195cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:08.923683 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf527b73-4787-4f5c-bab7-1515304195cf None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:08.924116 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf527b73-4787-4f5c-bab7-1515304195cf None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:08.924395 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf527b73-4787-4f5c-bab7-1515304195cf None admin] PATH_INFO: `/v3/endpoints` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:08.925213 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf527b73-4787-4f5c-bab7-1515304195cf None admin] RBAC: Adding query filter params (interface=public, service_id=26e7e0127d494b83a59b8ce9bc0ba9de, region_id=RegionOne) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:08.925527 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf527b73-4787-4f5c-bab7-1515304195cf None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:08.927009 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf527b73-4787-4f5c-bab7-1515304195cf None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:08.936059 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 387/772] 10.222.0.57 () {56 vars in 1251 bytes} [Thu May 19 14:11:08 2022] GET /identity/v3/endpoints?service_id=26e7e0127d494b83a59b8ce9bc0ba9de&interface=public®ion_id=RegionOne => generated 194 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:08.942044 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45424d5a-f37e-42fe-859c-239fa9ecf661 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:09.075396 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45424d5a-f37e-42fe-859c-239fa9ecf661 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:09.115675 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e22ad6-fd7c-4eaf-8c22-bf3910b8067e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:09.116365 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e22ad6-fd7c-4eaf-8c22-bf3910b8067e None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:09.116650 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e22ad6-fd7c-4eaf-8c22-bf3910b8067e None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:09.116930 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e22ad6-fd7c-4eaf-8c22-bf3910b8067e None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:09.117632 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e22ad6-fd7c-4eaf-8c22-bf3910b8067e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:09.119346 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e22ad6-fd7c-4eaf-8c22-bf3910b8067e None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:09.129993 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 386/773] 10.222.0.57 () {56 vars in 1087 bytes} [Thu May 19 14:11:08 2022] GET /identity/v3/services => generated 772 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:09.155618 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792b96aa-4ad6-46dd-a3c0-46afbedd2871 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:09.155948 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792b96aa-4ad6-46dd-a3c0-46afbedd2871 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:09.156181 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792b96aa-4ad6-46dd-a3c0-46afbedd2871 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:09.158310 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 388/774] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:11:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:11:09.164798 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4cbbcb-e92e-4a84-bf24-9801cb23aa03 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:09.165121 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4cbbcb-e92e-4a84-bf24-9801cb23aa03 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:09.165370 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4cbbcb-e92e-4a84-bf24-9801cb23aa03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:09.187058 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-9b4cbbcb-e92e-4a84-bf24-9801cb23aa03 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:11:09.188205 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-9b4cbbcb-e92e-4a84-bf24-9801cb23aa03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:11:09.215608 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:09.284002 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 387/775] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:11:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:11:09.319232 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792b96aa-4ad6-46dd-a3c0-46afbedd2871 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:09.473149 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792b96aa-4ad6-46dd-a3c0-46afbedd2871 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:09.525638 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a194964f-b485-4e70-b727-b169f3d07655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:09.526877 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a194964f-b485-4e70-b727-b169f3d07655 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:09.527373 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a194964f-b485-4e70-b727-b169f3d07655 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:09.527858 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a194964f-b485-4e70-b727-b169f3d07655 None admin] PATH_INFO: `/v3/services/network` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:09.528784 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a194964f-b485-4e70-b727-b169f3d07655 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:09.530493 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a194964f-b485-4e70-b727-b169f3d07655 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:09.546200 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-a194964f-b485-4e70-b727-b169f3d07655 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 19 14:11:09.546200 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:09.546200 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:11:09.546200 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:11:09.546200 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:11:09.546200 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:11:09.546200 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:11:09.546200 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:11:09.546200 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 14:11:09.546200 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:11:09.546200 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:11:09.546200 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:11:09.546200 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:09.546200 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:11:09.546200 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:11:09.546200 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:11:09.546200 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:11:09.547243 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:11:09.547243 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:11:09.547243 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:11:09.547243 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:11:09.547243 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:11:09.547243 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:11:09.547243 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:11:09.547243 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:11:09.547243 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:11:09.547243 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:11:09.547243 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:11:09.547243 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:11:09.547243 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:11:09.547243 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:11:09.547243 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:11:09.547243 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:11:09.548134 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:11:09.548134 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:11:09.548134 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:11:09.548134 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:11:09.548134 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:11:09.548134 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:11:09.548134 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:11:09.548134 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:11:09.548134 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:11:09.548134 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:11:09.548134 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:11:09.548134 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:11:09.548134 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 14:11:09.548134 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:11:09.550772 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 389/776] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:11:09 2022] GET /identity/v3/services/network => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:11:09.556784 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4cbbcb-e92e-4a84-bf24-9801cb23aa03 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:09.694928 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4cbbcb-e92e-4a84-bf24-9801cb23aa03 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:09.700530 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00733b1f-4037-407d-847c-105ab23c2bc9 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:09.701017 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00733b1f-4037-407d-847c-105ab23c2bc9 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:09.701125 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00733b1f-4037-407d-847c-105ab23c2bc9 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:09.702748 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 390/777] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:11:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:11:09.708722 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9a7e84-0b85-40be-94d7-c3c05caa9381 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:09.709016 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9a7e84-0b85-40be-94d7-c3c05caa9381 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:09.709295 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9a7e84-0b85-40be-94d7-c3c05caa9381 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:09.727732 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-3e9a7e84-0b85-40be-94d7-c3c05caa9381 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:11:09.728857 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-3e9a7e84-0b85-40be-94d7-c3c05caa9381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:11:09.732648 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8458a25a-0a1d-46a2-abd0-ff15a22518d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:09.733324 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8458a25a-0a1d-46a2-abd0-ff15a22518d3 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:09.733611 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8458a25a-0a1d-46a2-abd0-ff15a22518d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:09.733916 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8458a25a-0a1d-46a2-abd0-ff15a22518d3 None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:09.734688 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8458a25a-0a1d-46a2-abd0-ff15a22518d3 None admin] RBAC: Adding query filter params (name=network) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:09.735004 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8458a25a-0a1d-46a2-abd0-ff15a22518d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:09.736451 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8458a25a-0a1d-46a2-abd0-ff15a22518d3 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:09.742355 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 388/778] 10.222.0.57 () {56 vars in 1112 bytes} [Thu May 19 14:11:09 2022] GET /identity/v3/services?name=network => generated 124 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:09.754927 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8458a25a-0a1d-46a2-abd0-ff15a22518d3 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:09.755951 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:09.790476 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:09.813225 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 391/779] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:11:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:11:09.871441 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9a7e84-0b85-40be-94d7-c3c05caa9381 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:09.932144 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8458a25a-0a1d-46a2-abd0-ff15a22518d3 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:10.008086 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6dfa3b-e176-424c-a486-a3f6863e6157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:10.008684 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6dfa3b-e176-424c-a486-a3f6863e6157 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:10.008972 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6dfa3b-e176-424c-a486-a3f6863e6157 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:10.009247 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6dfa3b-e176-424c-a486-a3f6863e6157 None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:10.009996 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6dfa3b-e176-424c-a486-a3f6863e6157 None admin] RBAC: Adding query filter params (type=network) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:10.010309 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6dfa3b-e176-424c-a486-a3f6863e6157 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:10.011683 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6dfa3b-e176-424c-a486-a3f6863e6157 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:10.018420 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9a7e84-0b85-40be-94d7-c3c05caa9381 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:10.029871 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 389/780] 10.222.0.57 () {56 vars in 1112 bytes} [Thu May 19 14:11:09 2022] GET /identity/v3/services?type=network => generated 350 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:10.029871 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6dfa3b-e176-424c-a486-a3f6863e6157 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:10.091004 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1317538-2283-4081-9ed2-d053ff55faee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:10.091811 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1317538-2283-4081-9ed2-d053ff55faee None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:10.092096 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1317538-2283-4081-9ed2-d053ff55faee None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:10.092373 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1317538-2283-4081-9ed2-d053ff55faee None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:10.113879 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1317538-2283-4081-9ed2-d053ff55faee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:11:10.121777 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-b1317538-2283-4081-9ed2-d053ff55faee None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:11:10.121777 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:10.121777 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:11:10.121777 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:11:10.121777 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:11:10.121777 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:11:10.121777 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:11:10.121777 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:11:10.121777 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:11:10.121777 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:11:10.121777 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:11:10.121777 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:11:10.121777 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:11:10.121777 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:11:10.121777 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:11:10.121777 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:11:10.123025 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:11:10.123025 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:11:10.123025 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:11:10.123025 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:11:10.123025 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:11:10.123025 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:11:10.123025 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:11:10.123025 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:11:10.123025 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:11:10.123025 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:11:10.123025 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:11:10.123025 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:11:10.123025 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:11:10.123025 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:11:10.123025 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:11:10.123025 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:11:10.124036 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:11:10.124036 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:11:10.124036 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:11:10.124036 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:11:10.124036 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:11:10.124036 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:11:10.124036 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:11:10.124036 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:11:10.124036 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:11:10.124036 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:11:10.124036 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:11:10.124036 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:11:10.124904 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 392/781] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:11:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:11:10.133888 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1317538-2283-4081-9ed2-d053ff55faee None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:10.165356 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:10.194258 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:10.201653 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6dfa3b-e176-424c-a486-a3f6863e6157 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:10.268993 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f9e5dd-df21-4279-9e04-ffa91e811536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:10.269579 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f9e5dd-df21-4279-9e04-ffa91e811536 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:10.269879 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f9e5dd-df21-4279-9e04-ffa91e811536 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:10.270183 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f9e5dd-df21-4279-9e04-ffa91e811536 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:10.270983 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f9e5dd-df21-4279-9e04-ffa91e811536 None admin] RBAC: Adding query filter params (interface=public, service_id=c3fcada9c39c4ec8b7350e18d5345731, region_id=RegionOne) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:10.271295 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f9e5dd-df21-4279-9e04-ffa91e811536 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:10.272662 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f9e5dd-df21-4279-9e04-ffa91e811536 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:10.289864 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 390/782] 10.222.0.57 () {56 vars in 1251 bytes} [Thu May 19 14:11:10 2022] GET /identity/v3/endpoints?service_id=c3fcada9c39c4ec8b7350e18d5345731&interface=public®ion_id=RegionOne => generated 194 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:11:10.300944 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f9e5dd-df21-4279-9e04-ffa91e811536 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:10.319138 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1317538-2283-4081-9ed2-d053ff55faee None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:10.378164 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96156be-b06d-4448-a5b5-5497e2c7ef9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:10.378894 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96156be-b06d-4448-a5b5-5497e2c7ef9b None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:10.379184 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96156be-b06d-4448-a5b5-5497e2c7ef9b None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:10.379465 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96156be-b06d-4448-a5b5-5497e2c7ef9b None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:10.380218 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96156be-b06d-4448-a5b5-5497e2c7ef9b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:10.380529 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96156be-b06d-4448-a5b5-5497e2c7ef9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:10.384792 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96156be-b06d-4448-a5b5-5497e2c7ef9b None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:10.403833 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 393/783] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:11:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:10.410407 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96156be-b06d-4448-a5b5-5497e2c7ef9b None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:10.475626 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f9e5dd-df21-4279-9e04-ffa91e811536 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:10.512626 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aab889c-62bf-4a2a-a574-bf17ed1ef275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:10.513299 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aab889c-62bf-4a2a-a574-bf17ed1ef275 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:10.513588 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aab889c-62bf-4a2a-a574-bf17ed1ef275 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:10.513888 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aab889c-62bf-4a2a-a574-bf17ed1ef275 None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:10.514917 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aab889c-62bf-4a2a-a574-bf17ed1ef275 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:10.516483 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aab889c-62bf-4a2a-a574-bf17ed1ef275 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:10.522613 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 391/784] 10.222.0.57 () {56 vars in 1087 bytes} [Thu May 19 14:11:10 2022] GET /identity/v3/services => generated 772 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:10.598503 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:10.637526 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96156be-b06d-4448-a5b5-5497e2c7ef9b None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:10.703778 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee4412f-3411-4822-a211-330aa829873b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:10.705207 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee4412f-3411-4822-a211-330aa829873b None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:10.705458 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee4412f-3411-4822-a211-330aa829873b None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:10.705738 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee4412f-3411-4822-a211-330aa829873b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:10.726503 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee4412f-3411-4822-a211-330aa829873b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:10.727982 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee4412f-3411-4822-a211-330aa829873b None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:10.749191 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-aee4412f-3411-4822-a211-330aa829873b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:11:10.749191 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:10.749191 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:11:10.749191 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:11:10.749191 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:11:10.749191 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:11:10.749191 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:11:10.749191 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:11:10.749191 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:11:10.749191 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:11:10.749191 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:11:10.749191 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:11:10.749191 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:11:10.749191 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:11:10.749191 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:11:10.750142 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:11:10.750142 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:11:10.750142 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:11:10.750142 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:11:10.750142 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:11:10.750142 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:11:10.750142 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:11:10.750142 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:11:10.750142 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:11:10.750142 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:11:10.750142 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:11:10.750142 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:11:10.750142 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:11:10.750142 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:11:10.750142 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:11:10.750142 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:11:10.750142 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:11:10.751110 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:11:10.751110 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:11:10.751110 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:11:10.751110 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:11:10.751110 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 394/785] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:11:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:11:10.756713 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aab889c-62bf-4a2a-a574-bf17ed1ef275 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:10.775239 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:10.871054 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aab889c-62bf-4a2a-a574-bf17ed1ef275 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:10.913028 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd27107b-09ef-46a5-88d5-91f4762b085f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:10.914133 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd27107b-09ef-46a5-88d5-91f4762b085f None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:10.914663 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd27107b-09ef-46a5-88d5-91f4762b085f None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:10.915146 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd27107b-09ef-46a5-88d5-91f4762b085f None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:10.916104 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd27107b-09ef-46a5-88d5-91f4762b085f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:10.916630 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd27107b-09ef-46a5-88d5-91f4762b085f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:10.918228 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd27107b-09ef-46a5-88d5-91f4762b085f None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:10.940272 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 392/786] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:11:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:10.945931 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee4412f-3411-4822-a211-330aa829873b None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:11.101459 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee4412f-3411-4822-a211-330aa829873b None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:11.146353 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:11.153088 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f047afe4-9b58-4d27-9153-bb03a7cfa7a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:11.154463 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f047afe4-9b58-4d27-9153-bb03a7cfa7a9 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:11.155198 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f047afe4-9b58-4d27-9153-bb03a7cfa7a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:11.155482 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f047afe4-9b58-4d27-9153-bb03a7cfa7a9 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:11.167026 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f047afe4-9b58-4d27-9153-bb03a7cfa7a9 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:11.168611 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f047afe4-9b58-4d27-9153-bb03a7cfa7a9 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:11.187187 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-f047afe4-9b58-4d27-9153-bb03a7cfa7a9 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 14:11:11.187187 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:11.187187 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:11:11.187187 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:11:11.187187 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:11:11.187187 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:11:11.187187 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:11:11.187187 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:11:11.187187 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:11:11.187187 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:11:11.187187 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:11:11.187187 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:11:11.187187 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:11:11.187187 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:11:11.187187 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:11:11.187187 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:11:11.188220 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:11:11.188220 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:11:11.188220 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:11:11.188220 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:11:11.188220 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:11:11.188220 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:11:11.188220 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:11:11.188220 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:11:11.188220 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:11:11.188220 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:11:11.188220 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:11:11.188220 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:11:11.188220 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:11:11.188220 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:11:11.188220 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:11:11.188220 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:11:11.188220 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:11:11.188220 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:11:11.190493 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:11:11.190493 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:11:11.190493 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:11:11.190493 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:11:11.190493 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:11:11.190493 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:11:11.190493 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:11:11.190493 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:11:11.190493 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 14:11:11.190493 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:11:11.193584 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 395/787] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:11:10 2022] GET /identity/v3/users/swift => generated 83 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:11:11.194938 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd27107b-09ef-46a5-88d5-91f4762b085f None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:11.307301 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:11.346238 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd27107b-09ef-46a5-88d5-91f4762b085f None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:11.404938 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d11ae3-bc95-4da7-aec4-0e1971f797f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:11.405623 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d11ae3-bc95-4da7-aec4-0e1971f797f8 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:11.405931 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d11ae3-bc95-4da7-aec4-0e1971f797f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:11.406212 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d11ae3-bc95-4da7-aec4-0e1971f797f8 None admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:11.407060 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d11ae3-bc95-4da7-aec4-0e1971f797f8 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:11.407405 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d11ae3-bc95-4da7-aec4-0e1971f797f8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:11.408985 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d11ae3-bc95-4da7-aec4-0e1971f797f8 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:11.441049 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 393/788] 10.222.0.57 () {56 vars in 1135 bytes} [Thu May 19 14:11:11 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:11:11.446682 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f047afe4-9b58-4d27-9153-bb03a7cfa7a9 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:11.571196 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30492441-518c-4ede-9962-c0c113a4ad56 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:11.571862 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30492441-518c-4ede-9962-c0c113a4ad56 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:11.572350 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30492441-518c-4ede-9962-c0c113a4ad56 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:11.574077 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 394/789] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:11:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:11:11.582876 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66b8338-a980-499c-bda5-2c30ec17f9c6 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:11.583218 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66b8338-a980-499c-bda5-2c30ec17f9c6 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:11.583450 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66b8338-a980-499c-bda5-2c30ec17f9c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:11.596933 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f047afe4-9b58-4d27-9153-bb03a7cfa7a9 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:11.622938 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-f66b8338-a980-499c-bda5-2c30ec17f9c6 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:11:11.624209 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-f66b8338-a980-499c-bda5-2c30ec17f9c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:11:11.640386 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2e4279-c22b-4f55-a336-363b58445d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:11.641520 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2e4279-c22b-4f55-a336-363b58445d19 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:11.642240 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2e4279-c22b-4f55-a336-363b58445d19 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:11.643766 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2e4279-c22b-4f55-a336-363b58445d19 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:11.664707 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2e4279-c22b-4f55-a336-363b58445d19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:11.666087 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2e4279-c22b-4f55-a336-363b58445d19 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:11.711051 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-1c2e4279-c22b-4f55-a336-363b58445d19 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:11:11.711051 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:11.711051 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:11:11.711051 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:11:11.711051 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:11:11.711051 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:11:11.711051 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:11:11.711051 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:11:11.711051 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:11:11.711051 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:11:11.711051 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:11:11.711051 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:11:11.711051 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:11:11.711051 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:11:11.711051 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:11:11.712458 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:11:11.712458 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:11:11.712458 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:11:11.712458 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:11:11.712458 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:11:11.712458 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:11:11.712458 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:11:11.712458 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:11:11.712458 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:11:11.712458 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:11:11.712458 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:11:11.712458 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:11:11.712458 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:11:11.712458 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:11:11.712458 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:11:11.712458 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:11:11.712458 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:11:11.713375 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:11:11.713375 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:11:11.713375 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:11:11.713375 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:11:11.717626 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 396/790] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:11:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:11:11.723503 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2e4279-c22b-4f55-a336-363b58445d19 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:11.756853 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 395/791] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:11:11 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:11:11.767701 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325411a2-1e55-4025-ba3f-61c484bea5ef None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:11.768309 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325411a2-1e55-4025-ba3f-61c484bea5ef None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:11.768787 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325411a2-1e55-4025-ba3f-61c484bea5ef None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:11.770436 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 396/792] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:11:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:11:11.776300 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8c464b-5275-418f-be6f-3ba630ea76bd None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:11.776592 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8c464b-5275-418f-be6f-3ba630ea76bd None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:11.776876 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8c464b-5275-418f-be6f-3ba630ea76bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:11.801467 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-0f8c464b-5275-418f-be6f-3ba630ea76bd None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:11:11.802983 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-0f8c464b-5275-418f-be6f-3ba630ea76bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:11:11.808822 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:11.868405 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:11.883107 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2e4279-c22b-4f55-a336-363b58445d19 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:11.885992 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 397/793] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:11:11 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:11:11.886841 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8c464b-5275-418f-be6f-3ba630ea76bd None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:11.923992 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9da1d49-035c-499b-8d59-2d29f43a989b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:11.924669 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9da1d49-035c-499b-8d59-2d29f43a989b None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:11.924957 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9da1d49-035c-499b-8d59-2d29f43a989b None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:11.925244 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9da1d49-035c-499b-8d59-2d29f43a989b None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:11.926592 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9da1d49-035c-499b-8d59-2d29f43a989b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:11.926592 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9da1d49-035c-499b-8d59-2d29f43a989b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:11.927674 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9da1d49-035c-499b-8d59-2d29f43a989b None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:11.939971 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 397/794] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:11:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:11.941266 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d316161-d15f-4e9c-afa0-b7cf7cc89790 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:11.941556 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d316161-d15f-4e9c-afa0-b7cf7cc89790 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:11.941835 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d316161-d15f-4e9c-afa0-b7cf7cc89790 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:11.943217 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 398/795] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:11:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:11:11.943761 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d316161-d15f-4e9c-afa0-b7cf7cc89790 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:12.007422 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8c464b-5275-418f-be6f-3ba630ea76bd None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:12.045920 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a948a973-aacc-4a82-9eb6-b196f86b8bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:12.047050 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a948a973-aacc-4a82-9eb6-b196f86b8bcc None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:12.047332 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a948a973-aacc-4a82-9eb6-b196f86b8bcc None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:12.047606 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a948a973-aacc-4a82-9eb6-b196f86b8bcc None admin] PATH_INFO: `/v3/roles/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:12.056394 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d316161-d15f-4e9c-afa0-b7cf7cc89790 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:12.058083 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:12.058654 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a948a973-aacc-4a82-9eb6-b196f86b8bcc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:11:12.067022 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-a948a973-aacc-4a82-9eb6-b196f86b8bcc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:11:12.067022 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:12.067022 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:11:12.067022 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:11:12.067022 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:11:12.067022 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:11:12.067022 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:11:12.067022 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:11:12.067022 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:11:12.067022 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:11:12.067022 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:11:12.067022 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:11:12.067022 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:11:12.067022 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:11:12.067022 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:11:12.067022 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:11:12.068063 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:11:12.068063 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:11:12.068063 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:11:12.068063 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:11:12.068063 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:11:12.068063 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:11:12.068063 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:11:12.068063 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:11:12.068063 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:11:12.068063 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:11:12.068063 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:11:12.068063 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:11:12.068063 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:11:12.068063 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:11:12.068063 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:11:12.068063 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:11:12.068952 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:11:12.068952 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:11:12.068952 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:11:12.068952 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:11:12.068952 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:11:12.068952 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:11:12.068952 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:11:12.068952 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:11:12.068952 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:11:12.068952 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:11:12.068952 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:11:12.068952 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:11:12.073460 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 398/796] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 14:11:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:11:12.074505 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a948a973-aacc-4a82-9eb6-b196f86b8bcc None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:12.112438 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b7c38c-fd8c-45e9-bb0d-358db76a7570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:12.113815 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b7c38c-fd8c-45e9-bb0d-358db76a7570 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:12.114307 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b7c38c-fd8c-45e9-bb0d-358db76a7570 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:12.114745 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b7c38c-fd8c-45e9-bb0d-358db76a7570 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:12.128021 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b7c38c-fd8c-45e9-bb0d-358db76a7570 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:11:12.135962 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-34b7c38c-fd8c-45e9-bb0d-358db76a7570 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:11:12.135962 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:12.135962 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:11:12.135962 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:11:12.135962 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:11:12.135962 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:11:12.135962 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:11:12.135962 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:11:12.135962 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:11:12.135962 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:11:12.135962 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:11:12.135962 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:11:12.135962 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:11:12.135962 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:11:12.135962 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:11:12.135962 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:11:12.137070 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:11:12.137070 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:11:12.137070 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:11:12.137070 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:11:12.137070 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:11:12.137070 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:11:12.137070 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:11:12.137070 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:11:12.137070 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:11:12.137070 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:11:12.137070 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:11:12.137070 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:11:12.137070 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:11:12.137070 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:11:12.137070 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:11:12.137070 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:11:12.137961 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:11:12.137961 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:11:12.137961 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:11:12.137961 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:11:12.137961 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:11:12.137961 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:11:12.137961 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:11:12.137961 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:11:12.137961 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:11:12.137961 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:11:12.137961 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:11:12.137961 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:11:12.137961 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 399/797] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:11:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:11:12.138662 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d4a027-df09-4d1a-82cb-17c526c2df00 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:12.138759 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d4a027-df09-4d1a-82cb-17c526c2df00 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:12.139048 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d4a027-df09-4d1a-82cb-17c526c2df00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:12.156441 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-c0d4a027-df09-4d1a-82cb-17c526c2df00 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:11:12.158229 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-c0d4a027-df09-4d1a-82cb-17c526c2df00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:11:12.161691 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:12.231375 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a948a973-aacc-4a82-9eb6-b196f86b8bcc None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:12.238052 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 400/798] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:11:12 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:11:12.238052 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d4a027-df09-4d1a-82cb-17c526c2df00 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:12.274538 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7712b15e-9ea9-4f54-a8c4-bb05d79b5db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:12.275389 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7712b15e-9ea9-4f54-a8c4-bb05d79b5db4 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:12.275671 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7712b15e-9ea9-4f54-a8c4-bb05d79b5db4 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:12.276094 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7712b15e-9ea9-4f54-a8c4-bb05d79b5db4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:12.283360 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7712b15e-9ea9-4f54-a8c4-bb05d79b5db4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:12.285423 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7712b15e-9ea9-4f54-a8c4-bb05d79b5db4 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:12.300510 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-7712b15e-9ea9-4f54-a8c4-bb05d79b5db4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:11:12.300510 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:12.300510 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:11:12.300510 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:11:12.300510 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:11:12.300510 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:11:12.300510 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:11:12.300510 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:11:12.300510 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:11:12.300510 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:11:12.300510 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:11:12.300510 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:11:12.300510 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:11:12.300510 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:11:12.300510 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:11:12.301418 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:11:12.301418 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:11:12.301418 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:11:12.301418 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:11:12.301418 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:11:12.301418 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:11:12.301418 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:11:12.301418 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:11:12.301418 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:11:12.301418 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:11:12.301418 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:11:12.301418 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:11:12.301418 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:11:12.301418 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:11:12.301418 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:11:12.301418 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:11:12.301418 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:11:12.302691 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:11:12.302691 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:11:12.302691 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:11:12.302691 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:11:12.302691 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:11:12.302691 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:11:12.302691 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:11:12.302691 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:11:12.302691 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 399/799] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:11:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:11:12.302691 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7712b15e-9ea9-4f54-a8c4-bb05d79b5db4 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:12.340604 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:12.350673 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d4a027-df09-4d1a-82cb-17c526c2df00 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:12.389053 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f231633-128b-40bb-955a-03103c1ff96c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:12.389833 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f231633-128b-40bb-955a-03103c1ff96c None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:12.390142 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f231633-128b-40bb-955a-03103c1ff96c None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:12.390427 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f231633-128b-40bb-955a-03103c1ff96c None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:12.391169 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f231633-128b-40bb-955a-03103c1ff96c None admin] RBAC: Adding query filter params (name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:12.391476 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f231633-128b-40bb-955a-03103c1ff96c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:12.393611 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f231633-128b-40bb-955a-03103c1ff96c None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:12.406587 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 401/800] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 19 14:11:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:12.406587 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f231633-128b-40bb-955a-03103c1ff96c None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:12.417885 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7712b15e-9ea9-4f54-a8c4-bb05d79b5db4 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:12.429229 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:12.454190 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ddc4c9-9833-4952-baa1-be828d6c2b4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:12.455450 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ddc4c9-9833-4952-baa1-be828d6c2b4b None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:12.455739 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ddc4c9-9833-4952-baa1-be828d6c2b4b None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:12.456012 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ddc4c9-9833-4952-baa1-be828d6c2b4b None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:12.456748 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ddc4c9-9833-4952-baa1-be828d6c2b4b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:12.457053 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ddc4c9-9833-4952-baa1-be828d6c2b4b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:12.458457 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ddc4c9-9833-4952-baa1-be828d6c2b4b None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:12.466865 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 400/801] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:11:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:12.467540 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ddc4c9-9833-4952-baa1-be828d6c2b4b None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:12.513303 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f231633-128b-40bb-955a-03103c1ff96c None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:12.548730 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fe455a-56b5-4b50-9ae3-bac35744f8d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:12.549547 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fe455a-56b5-4b50-9ae3-bac35744f8d5 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:12.549834 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fe455a-56b5-4b50-9ae3-bac35744f8d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:12.550138 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fe455a-56b5-4b50-9ae3-bac35744f8d5 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:12.550791 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fe455a-56b5-4b50-9ae3-bac35744f8d5 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:12.552165 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fe455a-56b5-4b50-9ae3-bac35744f8d5 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:12.564796 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-b5fe455a-56b5-4b50-9ae3-bac35744f8d5 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 14:11:12.564796 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:12.564796 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:11:12.564796 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:11:12.564796 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:11:12.564796 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:11:12.564796 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:11:12.564796 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:11:12.564796 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 14:11:12.564796 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:11:12.564796 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:11:12.564796 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:11:12.564796 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:12.564796 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:11:12.564796 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:11:12.564796 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:11:12.565754 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:11:12.565754 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:11:12.565754 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:11:12.565754 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:11:12.565754 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:11:12.565754 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:11:12.565754 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:11:12.565754 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:11:12.565754 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:11:12.565754 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:11:12.565754 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:11:12.565754 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:11:12.565754 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:11:12.565754 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:11:12.565754 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:11:12.565754 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:11:12.566628 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:11:12.566628 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:11:12.566628 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:11:12.566628 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:11:12.566628 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:11:12.566628 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:11:12.566628 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:11:12.566628 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:11:12.566628 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:11:12.566628 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:11:12.566628 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:11:12.566628 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:11:12.566628 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:11:12.566628 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 14:11:12.566628 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:11:12.566628 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 402/802] 10.222.0.57 () {56 vars in 1129 bytes} [Thu May 19 14:11:12 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:11:12.568067 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fe455a-56b5-4b50-9ae3-bac35744f8d5 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:12.577230 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ddc4c9-9833-4952-baa1-be828d6c2b4b None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:12.587787 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:12.615231 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b775cd2-7e78-4e33-b048-a0f342eebf9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:12.615904 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b775cd2-7e78-4e33-b048-a0f342eebf9d None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:12.616186 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b775cd2-7e78-4e33-b048-a0f342eebf9d None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:12.616470 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b775cd2-7e78-4e33-b048-a0f342eebf9d None admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:12.617254 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b775cd2-7e78-4e33-b048-a0f342eebf9d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:12.617565 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b775cd2-7e78-4e33-b048-a0f342eebf9d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:12.619037 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b775cd2-7e78-4e33-b048-a0f342eebf9d None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:12.632184 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 401/803] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:11:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:12.637041 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b775cd2-7e78-4e33-b048-a0f342eebf9d None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:12.668385 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:12.684886 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fe455a-56b5-4b50-9ae3-bac35744f8d5 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:12.721707 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe77317d-8add-4635-85f4-11ffe706716f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:12.722519 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe77317d-8add-4635-85f4-11ffe706716f None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:12.722801 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe77317d-8add-4635-85f4-11ffe706716f None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:12.723074 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe77317d-8add-4635-85f4-11ffe706716f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:12.733891 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe77317d-8add-4635-85f4-11ffe706716f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:12.735396 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe77317d-8add-4635-85f4-11ffe706716f None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:12.743217 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b775cd2-7e78-4e33-b048-a0f342eebf9d None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:12.753674 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-fe77317d-8add-4635-85f4-11ffe706716f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:11:12.753674 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:12.753674 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:11:12.753674 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:11:12.753674 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:11:12.753674 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:11:12.753674 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:11:12.753674 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:11:12.753674 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:11:12.753674 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:11:12.753674 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:11:12.753674 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:11:12.753674 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:11:12.753674 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:11:12.753674 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:11:12.754675 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:11:12.754675 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:11:12.754675 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:11:12.754675 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:11:12.754675 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:11:12.754675 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:11:12.754675 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:11:12.754675 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:11:12.754675 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:11:12.754675 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:11:12.754675 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:11:12.754675 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:11:12.754675 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:11:12.754675 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:11:12.754675 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:11:12.754675 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:11:12.754675 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:11:12.755605 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:11:12.755605 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:11:12.755605 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:11:12.755605 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:11:12.755605 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 403/804] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:11:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:11:12.756325 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe77317d-8add-4635-85f4-11ffe706716f None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:12.784538 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7c7e78-95fb-4b4c-8a49-40cac1069075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:12.785251 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7c7e78-95fb-4b4c-8a49-40cac1069075 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:12.785539 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7c7e78-95fb-4b4c-8a49-40cac1069075 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:12.785816 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7c7e78-95fb-4b4c-8a49-40cac1069075 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:12.796968 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7c7e78-95fb-4b4c-8a49-40cac1069075 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:12.798879 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7c7e78-95fb-4b4c-8a49-40cac1069075 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:12.817230 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-8c7c7e78-95fb-4b4c-8a49-40cac1069075 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:11:12.817230 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:12.817230 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:11:12.817230 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:11:12.817230 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:11:12.817230 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:11:12.817230 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:11:12.817230 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:11:12.817230 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:11:12.817230 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:11:12.817230 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:11:12.817230 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:11:12.817230 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:11:12.817230 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:11:12.817230 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:11:12.818232 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:11:12.818232 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:11:12.818232 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:11:12.818232 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:11:12.818232 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:11:12.818232 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:11:12.818232 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:11:12.818232 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:11:12.818232 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:11:12.818232 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:11:12.818232 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:11:12.818232 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:11:12.818232 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:11:12.818232 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:11:12.818232 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:11:12.818232 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:11:12.818232 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:11:12.819144 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:11:12.819144 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:11:12.819144 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:11:12.819144 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:11:12.823034 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 402/805] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:11:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:11:12.823857 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7c7e78-95fb-4b4c-8a49-40cac1069075 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:12.832163 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:12.868968 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe77317d-8add-4635-85f4-11ffe706716f None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:12.904957 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c769311e-400b-4384-a9ce-b726deda6fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:12.905627 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c769311e-400b-4384-a9ce-b726deda6fae None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:12.905938 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c769311e-400b-4384-a9ce-b726deda6fae None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:12.906219 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c769311e-400b-4384-a9ce-b726deda6fae None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:12.906925 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c769311e-400b-4384-a9ce-b726deda6fae None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:12.907233 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c769311e-400b-4384-a9ce-b726deda6fae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:12.908625 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c769311e-400b-4384-a9ce-b726deda6fae None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:12.911328 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:12.918584 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 404/806] 10.222.0.57 () {56 vars in 1124 bytes} [Thu May 19 14:11:12 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:12.918584 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c769311e-400b-4384-a9ce-b726deda6fae None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:12.932394 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7c7e78-95fb-4b4c-8a49-40cac1069075 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:12.968368 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22f898b-8ea7-409d-ac7b-f0ae92042d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:12.969256 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22f898b-8ea7-409d-ac7b-f0ae92042d53 None admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:12.969542 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22f898b-8ea7-409d-ac7b-f0ae92042d53 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:12.969881 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22f898b-8ea7-409d-ac7b-f0ae92042d53 None admin] PATH_INFO: `/v3/projects/c6f995ffa0a94b1584f25dbe31e20524/users/09dd957f216a46dd8a3f400fcd2ba944/roles/4946cf0ff7164307b36efeee7c787146` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:12.971861 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22f898b-8ea7-409d-ac7b-f0ae92042d53 None admin] RBAC: Authorizing `identity:create_grant(project_id=c6f995ffa0a94b1584f25dbe31e20524, user_id=09dd957f216a46dd8a3f400fcd2ba944, role_id=4946cf0ff7164307b36efeee7c787146)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:12.973297 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22f898b-8ea7-409d-ac7b-f0ae92042d53 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:12.986995 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 403/807] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 14:11:12 2022] PUT /identity/v3/projects/c6f995ffa0a94b1584f25dbe31e20524/users/09dd957f216a46dd8a3f400fcd2ba944/roles/4946cf0ff7164307b36efeee7c787146 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:11:12.987572 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22f898b-8ea7-409d-ac7b-f0ae92042d53 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:13.045661 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c769311e-400b-4384-a9ce-b726deda6fae None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:13.099983 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996ae77e-4160-4628-b6f8-8062c3a65fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:13.101053 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996ae77e-4160-4628-b6f8-8062c3a65fc6 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:13.101538 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996ae77e-4160-4628-b6f8-8062c3a65fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:13.102034 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996ae77e-4160-4628-b6f8-8062c3a65fc6 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:13.103010 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996ae77e-4160-4628-b6f8-8062c3a65fc6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:13.103530 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996ae77e-4160-4628-b6f8-8062c3a65fc6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:13.106370 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996ae77e-4160-4628-b6f8-8062c3a65fc6 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:13.112188 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:13.136075 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 405/808] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:11:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:13.136075 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996ae77e-4160-4628-b6f8-8062c3a65fc6 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:13.157558 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22f898b-8ea7-409d-ac7b-f0ae92042d53 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:13.207285 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e785a56-369d-49f6-b4fc-439cdd731ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:13.207966 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e785a56-369d-49f6-b4fc-439cdd731ed7 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:13.208442 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e785a56-369d-49f6-b4fc-439cdd731ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:13.208565 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e785a56-369d-49f6-b4fc-439cdd731ed7 None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:13.209240 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e785a56-369d-49f6-b4fc-439cdd731ed7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:13.209583 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e785a56-369d-49f6-b4fc-439cdd731ed7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:13.210959 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e785a56-369d-49f6-b4fc-439cdd731ed7 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:13.226065 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 404/809] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:11:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:13.226914 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e785a56-369d-49f6-b4fc-439cdd731ed7 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:13.230561 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:13.262551 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996ae77e-4160-4628-b6f8-8062c3a65fc6 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:13.300113 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c894ecdc-41fe-4433-bcb8-8374789bd20a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:13.300710 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c894ecdc-41fe-4433-bcb8-8374789bd20a None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:13.300993 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c894ecdc-41fe-4433-bcb8-8374789bd20a None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:13.301271 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c894ecdc-41fe-4433-bcb8-8374789bd20a None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:13.302019 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c894ecdc-41fe-4433-bcb8-8374789bd20a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:13.302329 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c894ecdc-41fe-4433-bcb8-8374789bd20a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:13.303707 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c894ecdc-41fe-4433-bcb8-8374789bd20a None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:13.311870 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 406/810] 10.222.0.57 () {56 vars in 1124 bytes} [Thu May 19 14:11:13 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:13.313669 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0ce40d-a315-4ff3-b17c-62773fe07bb8 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:13.314184 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0ce40d-a315-4ff3-b17c-62773fe07bb8 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:13.314697 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0ce40d-a315-4ff3-b17c-62773fe07bb8 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:13.316508 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 407/811] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:11:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:11:13.317419 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0ce40d-a315-4ff3-b17c-62773fe07bb8 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:13.351180 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e785a56-369d-49f6-b4fc-439cdd731ed7 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:13.384937 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:13.391060 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00f6539-521d-45e4-880f-8182b31762ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:13.392136 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00f6539-521d-45e4-880f-8182b31762ee None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:13.392136 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00f6539-521d-45e4-880f-8182b31762ee None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:13.392430 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00f6539-521d-45e4-880f-8182b31762ee None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:13.400560 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00f6539-521d-45e4-880f-8182b31762ee None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:13.402037 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00f6539-521d-45e4-880f-8182b31762ee None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:13.420568 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-d00f6539-521d-45e4-880f-8182b31762ee None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 14:11:13.420568 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:13.420568 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:11:13.420568 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:11:13.420568 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:11:13.420568 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:11:13.420568 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:11:13.420568 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:11:13.420568 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:11:13.420568 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:11:13.420568 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:11:13.420568 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:11:13.420568 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:11:13.420568 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:11:13.420568 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:11:13.420568 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:11:13.421588 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:11:13.421588 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:11:13.421588 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:11:13.421588 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:11:13.421588 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:11:13.421588 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:11:13.421588 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:11:13.421588 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:11:13.421588 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:11:13.421588 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:11:13.421588 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:11:13.421588 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:11:13.421588 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:11:13.421588 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:11:13.421588 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:11:13.421588 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:11:13.421588 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:11:13.421588 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:11:13.422575 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:11:13.422575 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:11:13.422575 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:11:13.422575 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:11:13.422575 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:11:13.422575 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:11:13.422575 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:11:13.422575 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:11:13.422575 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 14:11:13.422575 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:11:13.422575 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 405/812] 10.222.0.57 () {56 vars in 1117 bytes} [Thu May 19 14:11:13 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:11:13.423751 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aabe9f3-8526-4baa-b49c-10f6833fba7f None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:13.423822 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aabe9f3-8526-4baa-b49c-10f6833fba7f None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:13.424438 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aabe9f3-8526-4baa-b49c-10f6833fba7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:13.436823 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0ce40d-a315-4ff3-b17c-62773fe07bb8 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:13.441197 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-3aabe9f3-8526-4baa-b49c-10f6833fba7f None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:11:13.442416 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-3aabe9f3-8526-4baa-b49c-10f6833fba7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:11:13.471317 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92288d6d-af4e-4498-a6a8-65e0e7c764c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:13.472112 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92288d6d-af4e-4498-a6a8-65e0e7c764c4 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:13.472391 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92288d6d-af4e-4498-a6a8-65e0e7c764c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:13.472671 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92288d6d-af4e-4498-a6a8-65e0e7c764c4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:13.479904 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92288d6d-af4e-4498-a6a8-65e0e7c764c4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:13.481343 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92288d6d-af4e-4498-a6a8-65e0e7c764c4 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:13.496619 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-92288d6d-af4e-4498-a6a8-65e0e7c764c4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 14:11:13.496619 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:13.496619 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:11:13.496619 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:11:13.496619 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:11:13.496619 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:11:13.496619 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:11:13.496619 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:11:13.496619 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:11:13.496619 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:11:13.496619 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:11:13.496619 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:11:13.496619 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:11:13.496619 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:11:13.496619 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:11:13.496619 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:11:13.497637 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:11:13.497637 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:11:13.497637 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:11:13.497637 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:11:13.497637 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:11:13.497637 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:11:13.497637 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:11:13.497637 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:11:13.497637 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:11:13.497637 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:11:13.497637 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:11:13.497637 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:11:13.497637 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:11:13.497637 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:11:13.497637 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:11:13.497637 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:11:13.497637 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:11:13.497637 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:11:13.498640 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:11:13.498640 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:11:13.498640 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:11:13.498640 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:11:13.498640 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:11:13.498640 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:11:13.498640 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:11:13.498640 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:11:13.498640 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 14:11:13.498640 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:11:13.498640 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 408/813] 10.222.0.57 () {56 vars in 1093 bytes} [Thu May 19 14:11:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:11:13.498640 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92288d6d-af4e-4498-a6a8-65e0e7c764c4 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:13.512602 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:13.522939 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 406/814] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:11:13 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:11:13.524136 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aabe9f3-8526-4baa-b49c-10f6833fba7f None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:13.620096 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92288d6d-af4e-4498-a6a8-65e0e7c764c4 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:13.638190 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:13.655652 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aabe9f3-8526-4baa-b49c-10f6833fba7f None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:13.656756 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3429bdf7-f955-4772-97e5-7b2f3846bca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:13.658019 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3429bdf7-f955-4772-97e5-7b2f3846bca7 None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:13.658289 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3429bdf7-f955-4772-97e5-7b2f3846bca7 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:13.658619 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3429bdf7-f955-4772-97e5-7b2f3846bca7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:13.659378 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3429bdf7-f955-4772-97e5-7b2f3846bca7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:13.660767 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3429bdf7-f955-4772-97e5-7b2f3846bca7 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:13.683732 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 409/815] 10.222.0.57 () {60 vars in 1144 bytes} [Thu May 19 14:11:13 2022] POST /identity/v3/endpoints => generated 346 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:11:13.684564 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3429bdf7-f955-4772-97e5-7b2f3846bca7 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:13.712694 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beda0959-f49f-4aa7-b766-c4b60f795bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:13.713792 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beda0959-f49f-4aa7-b766-c4b60f795bbf None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:13.714188 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beda0959-f49f-4aa7-b766-c4b60f795bbf None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:13.714535 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beda0959-f49f-4aa7-b766-c4b60f795bbf None admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:13.715472 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beda0959-f49f-4aa7-b766-c4b60f795bbf None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:13.715852 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beda0959-f49f-4aa7-b766-c4b60f795bbf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:13.717517 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beda0959-f49f-4aa7-b766-c4b60f795bbf None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:13.746597 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 407/816] 10.222.0.57 () {56 vars in 1149 bytes} [Thu May 19 14:11:13 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:13.747699 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beda0959-f49f-4aa7-b766-c4b60f795bbf None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:13.811864 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:13.827871 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3429bdf7-f955-4772-97e5-7b2f3846bca7 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:13.878981 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefd5204-b60c-44f4-9f90-9dd738dedb1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:13.879674 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefd5204-b60c-44f4-9f90-9dd738dedb1b None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:13.879962 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefd5204-b60c-44f4-9f90-9dd738dedb1b None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:13.880239 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefd5204-b60c-44f4-9f90-9dd738dedb1b None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:13.881117 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefd5204-b60c-44f4-9f90-9dd738dedb1b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:13.881428 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefd5204-b60c-44f4-9f90-9dd738dedb1b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:13.881810 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beda0959-f49f-4aa7-b766-c4b60f795bbf None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:13.883681 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefd5204-b60c-44f4-9f90-9dd738dedb1b None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:13.903775 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 410/817] 10.222.0.57 () {56 vars in 1133 bytes} [Thu May 19 14:11:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:13.904965 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefd5204-b60c-44f4-9f90-9dd738dedb1b None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:13.924780 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749062ad-bc4f-4b71-a7df-d17a109ca1ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:13.925948 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749062ad-bc4f-4b71-a7df-d17a109ca1ae None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:13.926372 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749062ad-bc4f-4b71-a7df-d17a109ca1ae None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:13.926783 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749062ad-bc4f-4b71-a7df-d17a109ca1ae None admin] PATH_INFO: `/v3/services/network` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:13.927603 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749062ad-bc4f-4b71-a7df-d17a109ca1ae None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:13.929217 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749062ad-bc4f-4b71-a7df-d17a109ca1ae None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:13.942848 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:13.943016 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-749062ad-bc4f-4b71-a7df-d17a109ca1ae None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 19 14:11:13.943016 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:13.943016 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:11:13.943016 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:11:13.943016 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:11:13.943016 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:11:13.943016 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:11:13.943016 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:11:13.943016 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 14:11:13.943016 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:11:13.943016 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:11:13.943016 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:11:13.943016 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:13.943016 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:11:13.943016 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:11:13.943016 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:11:13.943016 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:11:13.944032 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:11:13.944032 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:11:13.944032 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:11:13.944032 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:11:13.944032 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:11:13.944032 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:11:13.944032 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:11:13.944032 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:11:13.944032 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:11:13.944032 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:11:13.944032 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:11:13.944032 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:11:13.944032 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:11:13.944032 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:11:13.944032 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:11:13.944032 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:11:13.945073 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:11:13.945073 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:11:13.945073 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:11:13.945073 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:11:13.945073 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:11:13.945073 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:11:13.945073 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:11:13.945073 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:11:13.945073 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:11:13.945073 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:11:13.945073 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:11:13.945073 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:11:13.945073 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 14:11:13.945073 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:11:13.945073 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 408/818] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:11:13 2022] GET /identity/v3/services/network => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:11:13.946024 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749062ad-bc4f-4b71-a7df-d17a109ca1ae None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:14.019020 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefd5204-b60c-44f4-9f90-9dd738dedb1b None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:14.056505 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad231d3-daff-40fa-8b3d-d415078d4d60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:14.058098 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad231d3-daff-40fa-8b3d-d415078d4d60 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:14.058482 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad231d3-daff-40fa-8b3d-d415078d4d60 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:14.058855 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad231d3-daff-40fa-8b3d-d415078d4d60 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:14.062926 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749062ad-bc4f-4b71-a7df-d17a109ca1ae None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:14.073094 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad231d3-daff-40fa-8b3d-d415078d4d60 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:14.075582 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad231d3-daff-40fa-8b3d-d415078d4d60 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:14.077957 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:14.101443 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-6ad231d3-daff-40fa-8b3d-d415078d4d60 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:11:14.101443 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:14.101443 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:11:14.101443 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:11:14.101443 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:11:14.101443 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:11:14.101443 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:11:14.101443 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:11:14.101443 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:11:14.101443 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:11:14.101443 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:11:14.101443 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:11:14.101443 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:11:14.101443 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:11:14.101443 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:11:14.102614 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:11:14.102614 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:11:14.102614 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:11:14.102614 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:11:14.102614 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:11:14.102614 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:11:14.102614 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:11:14.102614 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:11:14.102614 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:11:14.102614 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:11:14.102614 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:11:14.102614 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:11:14.102614 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:11:14.102614 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:11:14.102614 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:11:14.102614 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:11:14.102614 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:11:14.103792 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:11:14.103792 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:11:14.103792 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:11:14.103792 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:11:14.107481 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbd8a5b-6458-40b5-8819-266f1f7e297b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:14.107481 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbd8a5b-6458-40b5-8819-266f1f7e297b None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:14.107481 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbd8a5b-6458-40b5-8819-266f1f7e297b None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:14.107481 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbd8a5b-6458-40b5-8819-266f1f7e297b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:14.117960 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 411/819] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:11:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:11:14.117960 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad231d3-daff-40fa-8b3d-d415078d4d60 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:14.186678 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbd8a5b-6458-40b5-8819-266f1f7e297b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:14.188193 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbd8a5b-6458-40b5-8819-266f1f7e297b None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:14.244504 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad231d3-daff-40fa-8b3d-d415078d4d60 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:14.274421 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:14.284986 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a647ef7-825d-4f6f-bba2-fd458bd3c33d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:14.285912 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a647ef7-825d-4f6f-bba2-fd458bd3c33d None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:14.286343 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a647ef7-825d-4f6f-bba2-fd458bd3c33d None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:14.286763 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a647ef7-825d-4f6f-bba2-fd458bd3c33d None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:14.287644 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a647ef7-825d-4f6f-bba2-fd458bd3c33d None admin] RBAC: Adding query filter params (name=network) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:14.288139 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a647ef7-825d-4f6f-bba2-fd458bd3c33d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:14.289704 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a647ef7-825d-4f6f-bba2-fd458bd3c33d None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:14.299288 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 409/820] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:11:13 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:11:14.299288 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbd8a5b-6458-40b5-8819-266f1f7e297b None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:14.299954 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 412/821] 10.222.0.57 () {56 vars in 1112 bytes} [Thu May 19 14:11:14 2022] GET /identity/v3/services?name=network => generated 124 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:14.301080 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a647ef7-825d-4f6f-bba2-fd458bd3c33d None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:14.320018 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:14.405377 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a647ef7-825d-4f6f-bba2-fd458bd3c33d None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:14.405577 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbd8a5b-6458-40b5-8819-266f1f7e297b None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:14.442182 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a0021b-4172-4f59-9dc8-6234a128e8de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:14.443165 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b788b90-ba18-4694-a829-b1bc7eff921b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:14.443291 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a0021b-4172-4f59-9dc8-6234a128e8de None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:14.443762 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a0021b-4172-4f59-9dc8-6234a128e8de None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:14.443987 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b788b90-ba18-4694-a829-b1bc7eff921b None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:14.444340 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b788b90-ba18-4694-a829-b1bc7eff921b None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:14.444589 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a0021b-4172-4f59-9dc8-6234a128e8de None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:14.444689 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b788b90-ba18-4694-a829-b1bc7eff921b None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:14.445468 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b788b90-ba18-4694-a829-b1bc7eff921b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:14.445726 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a0021b-4172-4f59-9dc8-6234a128e8de None admin] RBAC: Adding query filter params (type=network) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:14.445862 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b788b90-ba18-4694-a829-b1bc7eff921b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:14.446367 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a0021b-4172-4f59-9dc8-6234a128e8de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:14.447400 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b788b90-ba18-4694-a829-b1bc7eff921b None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:14.448132 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a0021b-4172-4f59-9dc8-6234a128e8de None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:14.458230 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 413/822] 10.222.0.57 () {56 vars in 1112 bytes} [Thu May 19 14:11:14 2022] GET /identity/v3/services?type=network => generated 350 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:14.459339 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a0021b-4172-4f59-9dc8-6234a128e8de None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:14.460132 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 410/823] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:11:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:14.461001 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b788b90-ba18-4694-a829-b1bc7eff921b None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:14.475395 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:14.538821 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:14.565808 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a0021b-4172-4f59-9dc8-6234a128e8de None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:14.575031 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b788b90-ba18-4694-a829-b1bc7eff921b None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:14.601711 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191ce0c5-4be8-4067-a172-86125a9a59c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:14.602679 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191ce0c5-4be8-4067-a172-86125a9a59c2 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:14.603117 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191ce0c5-4be8-4067-a172-86125a9a59c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:14.603553 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191ce0c5-4be8-4067-a172-86125a9a59c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:14.615052 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191ce0c5-4be8-4067-a172-86125a9a59c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:14.615359 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1184cd56-1796-4ee0-816e-edd5148e30c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:14.616179 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1184cd56-1796-4ee0-816e-edd5148e30c6 None admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:14.616605 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1184cd56-1796-4ee0-816e-edd5148e30c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:14.617004 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191ce0c5-4be8-4067-a172-86125a9a59c2 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:14.617175 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1184cd56-1796-4ee0-816e-edd5148e30c6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:14.618287 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1184cd56-1796-4ee0-816e-edd5148e30c6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:14.619993 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1184cd56-1796-4ee0-816e-edd5148e30c6 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:14.636488 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-191ce0c5-4be8-4067-a172-86125a9a59c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:11:14.636488 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:14.636488 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:11:14.636488 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:11:14.636488 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:11:14.636488 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:11:14.636488 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:11:14.636488 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:11:14.636488 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:11:14.636488 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:11:14.636488 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:11:14.636488 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:11:14.636488 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:11:14.636488 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:11:14.636488 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:11:14.637475 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:11:14.637475 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:11:14.637475 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:11:14.637475 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:11:14.637475 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:11:14.637475 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:11:14.637475 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:11:14.637475 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:11:14.637475 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:11:14.637475 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:11:14.637475 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:11:14.637475 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:11:14.637475 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:11:14.637475 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:11:14.637475 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:11:14.637475 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:11:14.637475 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:11:14.638422 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:11:14.638422 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:11:14.638422 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:11:14.638422 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:11:14.638422 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 414/824] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:11:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:11:14.639276 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191ce0c5-4be8-4067-a172-86125a9a59c2 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:14.647097 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 411/825] 10.222.0.57 () {60 vars in 1144 bytes} [Thu May 19 14:11:14 2022] POST /identity/v3/endpoints => generated 327 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:11:14.648137 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1184cd56-1796-4ee0-816e-edd5148e30c6 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:14.756108 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:14.777209 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1184cd56-1796-4ee0-816e-edd5148e30c6 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:14.804988 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:14.820431 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8ce7e5-6230-4016-9d7c-2b66f84be1d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:14.822241 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8ce7e5-6230-4016-9d7c-2b66f84be1d7 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:14.822764 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8ce7e5-6230-4016-9d7c-2b66f84be1d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:14.823258 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8ce7e5-6230-4016-9d7c-2b66f84be1d7 None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:14.824000 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191ce0c5-4be8-4067-a172-86125a9a59c2 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:14.824469 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8ce7e5-6230-4016-9d7c-2b66f84be1d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:14.824990 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8ce7e5-6230-4016-9d7c-2b66f84be1d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:14.828006 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8ce7e5-6230-4016-9d7c-2b66f84be1d7 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:14.843289 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 412/826] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:11:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:14.850956 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8ce7e5-6230-4016-9d7c-2b66f84be1d7 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:14.900794 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2baba07-c996-478f-b327-20695da8f28d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:14.901622 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2baba07-c996-478f-b327-20695da8f28d None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:14.901952 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2baba07-c996-478f-b327-20695da8f28d None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:14.902236 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2baba07-c996-478f-b327-20695da8f28d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:14.918855 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2baba07-c996-478f-b327-20695da8f28d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:14.920385 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2baba07-c996-478f-b327-20695da8f28d None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:14.936519 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-f2baba07-c996-478f-b327-20695da8f28d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:11:14.936519 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:14.936519 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:11:14.936519 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:11:14.936519 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:11:14.936519 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:11:14.936519 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:11:14.936519 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:11:14.936519 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:11:14.936519 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:11:14.936519 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:11:14.936519 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:11:14.936519 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:11:14.936519 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:11:14.936519 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:11:14.937504 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:11:14.937504 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:11:14.937504 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:11:14.937504 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:11:14.937504 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:11:14.937504 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:11:14.937504 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:11:14.937504 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:11:14.937504 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:11:14.937504 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:11:14.937504 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:11:14.937504 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:11:14.937504 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:11:14.937504 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:11:14.937504 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:11:14.937504 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:11:14.937504 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:11:14.938516 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:11:14.938516 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:11:14.938516 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:11:14.938516 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:11:14.938516 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:11:14.938516 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:11:14.938516 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:11:14.938516 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:11:14.938516 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 415/827] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:11:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:11:14.945659 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2baba07-c996-478f-b327-20695da8f28d None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:15.020446 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8ce7e5-6230-4016-9d7c-2b66f84be1d7 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:15.067599 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7d686f-fa48-401d-a7ea-0541fdb311a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:15.068488 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7d686f-fa48-401d-a7ea-0541fdb311a8 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:15.068770 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7d686f-fa48-401d-a7ea-0541fdb311a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:15.069041 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7d686f-fa48-401d-a7ea-0541fdb311a8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:15.081622 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7d686f-fa48-401d-a7ea-0541fdb311a8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:15.081801 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2baba07-c996-478f-b327-20695da8f28d None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:15.083805 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7d686f-fa48-401d-a7ea-0541fdb311a8 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:15.101581 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-3c7d686f-fa48-401d-a7ea-0541fdb311a8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:11:15.101581 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:15.101581 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:11:15.101581 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:11:15.101581 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:11:15.101581 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:11:15.101581 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:11:15.101581 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:11:15.101581 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:11:15.101581 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:11:15.101581 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:11:15.101581 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:11:15.101581 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:11:15.101581 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:11:15.101581 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:11:15.102703 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:11:15.102703 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:11:15.102703 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:11:15.102703 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:11:15.102703 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:11:15.102703 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:11:15.102703 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:11:15.102703 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:11:15.102703 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:11:15.102703 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:11:15.102703 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:11:15.102703 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:11:15.102703 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:11:15.102703 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:11:15.102703 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:11:15.102703 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:11:15.102703 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:11:15.103801 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:11:15.103801 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:11:15.103801 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:11:15.103801 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:11:15.103801 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:11:15.103801 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:11:15.103801 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:11:15.103801 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:11:15.103801 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 413/828] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:11:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:11:15.110884 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7d686f-fa48-401d-a7ea-0541fdb311a8 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:15.121487 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:15.142349 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc527d5-3aa6-4f75-a2f7-1299067cf990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:15.143213 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc527d5-3aa6-4f75-a2f7-1299067cf990 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:15.143820 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc527d5-3aa6-4f75-a2f7-1299067cf990 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:15.144110 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc527d5-3aa6-4f75-a2f7-1299067cf990 None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:15.144921 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc527d5-3aa6-4f75-a2f7-1299067cf990 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:15.146352 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc527d5-3aa6-4f75-a2f7-1299067cf990 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:15.147842 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc527d5-3aa6-4f75-a2f7-1299067cf990 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:15.165791 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 416/829] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:11:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:15.172400 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc527d5-3aa6-4f75-a2f7-1299067cf990 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:15.190357 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:15.272507 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7d686f-fa48-401d-a7ea-0541fdb311a8 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:15.294627 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc527d5-3aa6-4f75-a2f7-1299067cf990 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:15.318695 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9af375-04eb-42b5-aacc-8ab291d4141b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:15.319900 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9af375-04eb-42b5-aacc-8ab291d4141b None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:15.320452 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9af375-04eb-42b5-aacc-8ab291d4141b None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:15.320741 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9af375-04eb-42b5-aacc-8ab291d4141b None admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:15.321543 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9af375-04eb-42b5-aacc-8ab291d4141b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:15.322142 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9af375-04eb-42b5-aacc-8ab291d4141b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:15.324342 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9af375-04eb-42b5-aacc-8ab291d4141b None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:15.332398 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ba83e0-74b8-4134-bfa1-e7af8fb1f04e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:15.333982 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ba83e0-74b8-4134-bfa1-e7af8fb1f04e None admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:15.334273 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ba83e0-74b8-4134-bfa1-e7af8fb1f04e None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:15.334554 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ba83e0-74b8-4134-bfa1-e7af8fb1f04e None admin] PATH_INFO: `/v3/projects/c6f995ffa0a94b1584f25dbe31e20524/users/1623987d87ab4cf19b923d254a68c2c6/roles/9c7d4d848dd549adb9f3465d4201f26d` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:15.339340 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 414/830] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:11:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:15.345962 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9af375-04eb-42b5-aacc-8ab291d4141b None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:15.352975 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ba83e0-74b8-4134-bfa1-e7af8fb1f04e None admin] RBAC: Authorizing `identity:create_grant(project_id=c6f995ffa0a94b1584f25dbe31e20524, user_id=1623987d87ab4cf19b923d254a68c2c6, role_id=9c7d4d848dd549adb9f3465d4201f26d)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:15.354483 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ba83e0-74b8-4134-bfa1-e7af8fb1f04e None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:15.369878 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 417/831] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 14:11:15 2022] PUT /identity/v3/projects/c6f995ffa0a94b1584f25dbe31e20524/users/1623987d87ab4cf19b923d254a68c2c6/roles/9c7d4d848dd549adb9f3465d4201f26d => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:11:15.371000 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49584b95-8719-46b4-8037-23a842fc1888 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:15.371293 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49584b95-8719-46b4-8037-23a842fc1888 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:15.371577 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49584b95-8719-46b4-8037-23a842fc1888 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:15.373032 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 418/832] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:11:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:11:15.379070 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b963080e-ff72-4f22-874c-b12e05cdb84f None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:15.379383 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b963080e-ff72-4f22-874c-b12e05cdb84f None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:15.379639 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b963080e-ff72-4f22-874c-b12e05cdb84f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:15.405981 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-b963080e-ff72-4f22-874c-b12e05cdb84f None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:11:15.409024 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-b963080e-ff72-4f22-874c-b12e05cdb84f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:11:15.451581 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:15.479170 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9af375-04eb-42b5-aacc-8ab291d4141b None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:15.482701 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:15.517872 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 419/833] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:11:15 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:11:15.532967 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbb5f70-da6f-4abc-8688-0df2b4008f25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:15.533662 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbb5f70-da6f-4abc-8688-0df2b4008f25 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:15.533984 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbb5f70-da6f-4abc-8688-0df2b4008f25 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:15.534267 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbb5f70-da6f-4abc-8688-0df2b4008f25 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:15.535180 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbb5f70-da6f-4abc-8688-0df2b4008f25 None admin] RBAC: Adding query filter params (role.id=4946cf0ff7164307b36efeee7c787146, scope.project.id=c6f995ffa0a94b1584f25dbe31e20524, user.id=8c87f361a92d4834854ca07f652be2ff) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:15.535502 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbb5f70-da6f-4abc-8688-0df2b4008f25 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:15.537014 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbb5f70-da6f-4abc-8688-0df2b4008f25 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:15.540339 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b963080e-ff72-4f22-874c-b12e05cdb84f None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:15.547170 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 415/834] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:11:15 2022] GET /identity/v3/role_assignments?user.id=8c87f361a92d4834854ca07f652be2ff&scope.project.id=c6f995ffa0a94b1584f25dbe31e20524&role.id=4946cf0ff7164307b36efeee7c787146 => generated 608 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:15.664524 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b963080e-ff72-4f22-874c-b12e05cdb84f None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:15.705483 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6a0da8-df83-4bee-b449-ece5d74f2016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:15.706384 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6a0da8-df83-4bee-b449-ece5d74f2016 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:15.706673 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6a0da8-df83-4bee-b449-ece5d74f2016 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:15.706955 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6a0da8-df83-4bee-b449-ece5d74f2016 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:15.719247 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6a0da8-df83-4bee-b449-ece5d74f2016 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:11:15.727444 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-8f6a0da8-df83-4bee-b449-ece5d74f2016 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:11:15.727444 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:15.727444 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:11:15.727444 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:11:15.727444 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:11:15.727444 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:11:15.727444 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:11:15.727444 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:11:15.727444 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:11:15.727444 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:11:15.727444 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:11:15.727444 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:11:15.727444 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:11:15.727444 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:11:15.727444 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:11:15.727444 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:11:15.728457 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:11:15.728457 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:11:15.728457 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:11:15.728457 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:11:15.728457 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:11:15.728457 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:11:15.728457 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:11:15.728457 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:11:15.728457 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:11:15.728457 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:11:15.728457 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:11:15.728457 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:11:15.728457 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:11:15.728457 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:11:15.728457 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:11:15.728457 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:11:15.729304 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:11:15.729304 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:11:15.729304 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:11:15.729304 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:11:15.729304 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:11:15.729304 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:11:15.729304 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:11:15.729304 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:11:15.729304 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:11:15.729304 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:11:15.729304 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:11:15.729304 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:11:15.729304 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 420/835] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:11:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:11:15.736542 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbb5f70-da6f-4abc-8688-0df2b4008f25 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:15.850713 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbb5f70-da6f-4abc-8688-0df2b4008f25 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:15.890471 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d0573f-37b0-4981-99d3-fdda329df84e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:15.891248 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d0573f-37b0-4981-99d3-fdda329df84e None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:15.891877 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d0573f-37b0-4981-99d3-fdda329df84e None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:15.892314 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d0573f-37b0-4981-99d3-fdda329df84e None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:15.893280 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d0573f-37b0-4981-99d3-fdda329df84e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:15.893594 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d0573f-37b0-4981-99d3-fdda329df84e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:15.895012 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d0573f-37b0-4981-99d3-fdda329df84e None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:15.904266 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 416/836] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:11:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:15.909878 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6a0da8-df83-4bee-b449-ece5d74f2016 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:15.937634 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:16.031067 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6a0da8-df83-4bee-b449-ece5d74f2016 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:16.067711 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fc19f5-7872-4a08-b8cf-eaa5778a6e83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:16.068599 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fc19f5-7872-4a08-b8cf-eaa5778a6e83 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:16.068880 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fc19f5-7872-4a08-b8cf-eaa5778a6e83 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:16.069149 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fc19f5-7872-4a08-b8cf-eaa5778a6e83 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:16.080217 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fc19f5-7872-4a08-b8cf-eaa5778a6e83 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:16.081673 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fc19f5-7872-4a08-b8cf-eaa5778a6e83 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:16.098998 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-42fc19f5-7872-4a08-b8cf-eaa5778a6e83 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:11:16.098998 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:16.098998 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:11:16.098998 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:11:16.098998 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:11:16.098998 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:11:16.098998 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:11:16.098998 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:11:16.098998 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:11:16.098998 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:11:16.098998 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:11:16.098998 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:11:16.098998 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:11:16.098998 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:11:16.098998 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:11:16.099929 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:11:16.099929 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:11:16.099929 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:11:16.099929 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:11:16.099929 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:11:16.099929 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:11:16.099929 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:11:16.099929 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:11:16.099929 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:11:16.099929 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:11:16.099929 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:11:16.099929 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:11:16.099929 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:11:16.099929 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:11:16.099929 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:11:16.099929 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:11:16.099929 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:11:16.100882 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:11:16.100882 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:11:16.100882 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:11:16.100882 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:11:16.100882 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 421/837] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:11:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:11:16.107236 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d0573f-37b0-4981-99d3-fdda329df84e None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:16.153310 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:16.173924 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69c7b27-22ba-49b3-a80b-607a8400cc3e None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:16.174290 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69c7b27-22ba-49b3-a80b-607a8400cc3e None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:16.174575 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69c7b27-22ba-49b3-a80b-607a8400cc3e None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:16.176194 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 422/838] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:11:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:11:16.181871 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f243cae-8716-4912-a520-45525f5dec57 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:16.182163 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f243cae-8716-4912-a520-45525f5dec57 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:16.182556 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f243cae-8716-4912-a520-45525f5dec57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:16.201744 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-6f243cae-8716-4912-a520-45525f5dec57 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:11:16.202963 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-6f243cae-8716-4912-a520-45525f5dec57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:11:16.241737 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d0573f-37b0-4981-99d3-fdda329df84e None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:16.277477 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 423/839] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:11:16 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:11:16.280213 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6186a605-cf50-4189-873e-d6b9aa233afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:16.281033 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6186a605-cf50-4189-873e-d6b9aa233afc None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:16.281442 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6186a605-cf50-4189-873e-d6b9aa233afc None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:16.281857 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6186a605-cf50-4189-873e-d6b9aa233afc None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:16.283877 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6186a605-cf50-4189-873e-d6b9aa233afc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:16.284344 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6186a605-cf50-4189-873e-d6b9aa233afc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:16.285857 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6186a605-cf50-4189-873e-d6b9aa233afc None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:16.300098 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 417/840] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:11:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:16.301450 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f243cae-8716-4912-a520-45525f5dec57 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:16.304867 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6186a605-cf50-4189-873e-d6b9aa233afc None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:16.312562 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:16.411224 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:16.422079 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6186a605-cf50-4189-873e-d6b9aa233afc None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:16.434259 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f243cae-8716-4912-a520-45525f5dec57 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:16.458472 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dd3e5a-10c3-489b-bd42-d87b059e5862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:16.459368 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dd3e5a-10c3-489b-bd42-d87b059e5862 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:16.459650 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dd3e5a-10c3-489b-bd42-d87b059e5862 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:16.459927 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dd3e5a-10c3-489b-bd42-d87b059e5862 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:16.468242 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dd3e5a-10c3-489b-bd42-d87b059e5862 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:16.470407 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dd3e5a-10c3-489b-bd42-d87b059e5862 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:16.471775 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af80ba6e-5f73-48da-8a66-90d2056543ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:16.472682 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af80ba6e-5f73-48da-8a66-90d2056543ce None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:16.473084 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af80ba6e-5f73-48da-8a66-90d2056543ce None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:16.473460 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af80ba6e-5f73-48da-8a66-90d2056543ce None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:16.474249 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af80ba6e-5f73-48da-8a66-90d2056543ce None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:16.475722 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af80ba6e-5f73-48da-8a66-90d2056543ce None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:16.486371 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-40dd3e5a-10c3-489b-bd42-d87b059e5862 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 14:11:16.486371 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:16.486371 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:11:16.486371 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:11:16.486371 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:11:16.486371 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:11:16.486371 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:11:16.486371 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:11:16.486371 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:11:16.486371 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:11:16.486371 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:11:16.486371 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:11:16.486371 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:11:16.486371 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:11:16.486371 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:11:16.486371 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:11:16.487393 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:11:16.487393 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:11:16.487393 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:11:16.487393 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:11:16.487393 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:11:16.487393 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:11:16.487393 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:11:16.487393 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:11:16.487393 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:11:16.487393 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:11:16.487393 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:11:16.487393 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:11:16.487393 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:11:16.487393 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:11:16.487393 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:11:16.487393 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:11:16.487393 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:11:16.487393 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:11:16.489167 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:11:16.489167 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:11:16.489167 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:11:16.489167 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:11:16.489167 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:11:16.489167 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:11:16.489167 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:11:16.489167 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:11:16.489167 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 14:11:16.489167 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:11:16.489167 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 418/841] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:11:16 2022] GET /identity/v3/users/swift => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:11:16.489916 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-af80ba6e-5f73-48da-8a66-90d2056543ce None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 14:11:16.489916 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:16.489916 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:11:16.489916 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:11:16.489916 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:11:16.489916 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:11:16.489916 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:11:16.489916 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:11:16.489916 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 14:11:16.489916 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:11:16.489916 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:11:16.489916 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:11:16.489916 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:16.489916 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:11:16.489916 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:11:16.489916 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:11:16.489916 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:11:16.491171 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:11:16.491171 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:11:16.491171 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:11:16.491171 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:11:16.491171 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:11:16.491171 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:11:16.491171 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:11:16.491171 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:11:16.491171 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:11:16.491171 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:11:16.491171 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:11:16.491171 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:11:16.491171 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:11:16.491171 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:11:16.491171 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:11:16.491171 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:11:16.492230 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:11:16.492230 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:11:16.492230 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:11:16.492230 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:11:16.492230 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:11:16.492230 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:11:16.492230 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:11:16.492230 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:11:16.492230 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:11:16.492230 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:11:16.492230 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:11:16.492230 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:11:16.492230 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 14:11:16.492230 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:11:16.496512 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 424/842] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 19 14:11:16 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:11:16.496718 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dd3e5a-10c3-489b-bd42-d87b059e5862 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:16.502330 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af80ba6e-5f73-48da-8a66-90d2056543ce None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:16.592283 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:16.625309 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dd3e5a-10c3-489b-bd42-d87b059e5862 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:16.631422 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af80ba6e-5f73-48da-8a66-90d2056543ce None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:16.662292 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fd786f-1b64-4c88-a636-b26640e3aa9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:16.662892 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fd786f-1b64-4c88-a636-b26640e3aa9f None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:16.663175 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fd786f-1b64-4c88-a636-b26640e3aa9f None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:16.663444 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fd786f-1b64-4c88-a636-b26640e3aa9f None admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:16.664492 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fd786f-1b64-4c88-a636-b26640e3aa9f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:16.664807 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fd786f-1b64-4c88-a636-b26640e3aa9f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:16.666274 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fd786f-1b64-4c88-a636-b26640e3aa9f None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:16.669529 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ac9519-0ddb-45c3-97ec-48606bbe3496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:16.670270 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ac9519-0ddb-45c3-97ec-48606bbe3496 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:16.670551 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ac9519-0ddb-45c3-97ec-48606bbe3496 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:16.670827 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ac9519-0ddb-45c3-97ec-48606bbe3496 None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:16.671535 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ac9519-0ddb-45c3-97ec-48606bbe3496 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:16.671838 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ac9519-0ddb-45c3-97ec-48606bbe3496 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:16.673234 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ac9519-0ddb-45c3-97ec-48606bbe3496 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:16.683269 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 425/843] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 19 14:11:16 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:16.684414 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ac9519-0ddb-45c3-97ec-48606bbe3496 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:16.691384 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 419/844] 10.222.0.57 () {56 vars in 1135 bytes} [Thu May 19 14:11:16 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:11:16.692432 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fd786f-1b64-4c88-a636-b26640e3aa9f None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:16.703714 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:16.798007 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ac9519-0ddb-45c3-97ec-48606bbe3496 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:16.824553 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fd786f-1b64-4c88-a636-b26640e3aa9f None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:16.836974 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c42067-5966-44cc-af42-1985df0ff6fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:16.837637 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c42067-5966-44cc-af42-1985df0ff6fb None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:16.837946 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c42067-5966-44cc-af42-1985df0ff6fb None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:16.838222 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c42067-5966-44cc-af42-1985df0ff6fb None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:16.838918 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c42067-5966-44cc-af42-1985df0ff6fb None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:16.839222 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c42067-5966-44cc-af42-1985df0ff6fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:16.840602 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c42067-5966-44cc-af42-1985df0ff6fb None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:16.842084 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:16.854188 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 426/845] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 19 14:11:16 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:16.870991 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4b0fc6-dca1-4def-97b8-6d4273c4284c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:16.871656 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4b0fc6-dca1-4def-97b8-6d4273c4284c None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:16.871938 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4b0fc6-dca1-4def-97b8-6d4273c4284c None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:16.872213 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4b0fc6-dca1-4def-97b8-6d4273c4284c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:16.952793 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4b0fc6-dca1-4def-97b8-6d4273c4284c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:16.954822 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4b0fc6-dca1-4def-97b8-6d4273c4284c None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:16.966334 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:17.067035 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 420/846] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:11:16 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 19 14:11:17.072589 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c42067-5966-44cc-af42-1985df0ff6fb None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:17.195971 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c42067-5966-44cc-af42-1985df0ff6fb None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:17.233084 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cc099c-6104-492a-9635-66b4a515f963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:17.233930 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cc099c-6104-492a-9635-66b4a515f963 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:17.234215 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cc099c-6104-492a-9635-66b4a515f963 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:17.234491 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cc099c-6104-492a-9635-66b4a515f963 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:17.246507 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cc099c-6104-492a-9635-66b4a515f963 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:17.247966 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cc099c-6104-492a-9635-66b4a515f963 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:17.267962 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-70cc099c-6104-492a-9635-66b4a515f963 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:11:17.267962 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:17.267962 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:11:17.267962 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:11:17.267962 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:11:17.267962 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:11:17.267962 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:11:17.267962 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:11:17.267962 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:11:17.267962 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:11:17.267962 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:11:17.267962 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:11:17.267962 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:11:17.267962 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:11:17.267962 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:11:17.270341 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:11:17.270341 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:11:17.270341 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:11:17.270341 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:11:17.270341 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:11:17.270341 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:11:17.270341 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:11:17.270341 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:11:17.270341 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:11:17.270341 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:11:17.270341 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:11:17.270341 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:11:17.270341 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:11:17.270341 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:11:17.270341 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:11:17.270341 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:11:17.270341 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:11:17.272514 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:11:17.272514 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:11:17.272514 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:11:17.272514 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:11:17.272514 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 427/847] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:11:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:11:17.275035 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4b0fc6-dca1-4def-97b8-6d4273c4284c None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:17.378708 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4b0fc6-dca1-4def-97b8-6d4273c4284c None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:17.388816 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:17.418714 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308c0011-779b-4a2a-b525-945ccddc82be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:17.419444 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308c0011-779b-4a2a-b525-945ccddc82be None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:17.419728 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308c0011-779b-4a2a-b525-945ccddc82be None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:17.420002 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308c0011-779b-4a2a-b525-945ccddc82be None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:17.420717 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308c0011-779b-4a2a-b525-945ccddc82be None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:17.421022 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308c0011-779b-4a2a-b525-945ccddc82be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:17.422409 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308c0011-779b-4a2a-b525-945ccddc82be None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:17.435273 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 421/848] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:11:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:11:17.441830 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cc099c-6104-492a-9635-66b4a515f963 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:17.496545 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:17.559875 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cc099c-6104-492a-9635-66b4a515f963 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:17.600238 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d8bf14-b9b9-4904-b8ff-7816a9b37e96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:17.601803 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d8bf14-b9b9-4904-b8ff-7816a9b37e96 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:17.602335 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d8bf14-b9b9-4904-b8ff-7816a9b37e96 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:17.602806 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d8bf14-b9b9-4904-b8ff-7816a9b37e96 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:17.611441 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d8bf14-b9b9-4904-b8ff-7816a9b37e96 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:17.613233 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d8bf14-b9b9-4904-b8ff-7816a9b37e96 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:17.629521 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-01d8bf14-b9b9-4904-b8ff-7816a9b37e96 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:11:17.629521 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:17.629521 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:11:17.629521 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:11:17.629521 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:11:17.629521 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:11:17.629521 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:11:17.629521 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:11:17.629521 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:11:17.629521 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:11:17.629521 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:11:17.629521 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:11:17.629521 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:11:17.629521 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:11:17.629521 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:11:17.630471 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:11:17.630471 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:11:17.630471 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:11:17.630471 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:11:17.630471 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:11:17.630471 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:11:17.630471 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:11:17.630471 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:11:17.630471 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:11:17.630471 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:11:17.630471 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:11:17.630471 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:11:17.630471 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:11:17.630471 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:11:17.630471 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:11:17.630471 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:11:17.630471 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:11:17.631407 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:11:17.631407 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:11:17.631407 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:11:17.631407 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:11:17.631407 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:11:17.631407 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:11:17.631407 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:11:17.631407 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:11:17.638034 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 428/849] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:11:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:11:17.639816 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308c0011-779b-4a2a-b525-945ccddc82be None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:17.753419 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697bc3e8-8408-4e6f-a0f4-582fc5d1825c None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:17.754146 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697bc3e8-8408-4e6f-a0f4-582fc5d1825c None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:17.754649 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697bc3e8-8408-4e6f-a0f4-582fc5d1825c None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:17.756586 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 429/850] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:11:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:11:17.761589 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308c0011-779b-4a2a-b525-945ccddc82be None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:17.763218 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475d5e8f-73a2-47eb-9c1f-dd0facbc5e46 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:17.763744 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475d5e8f-73a2-47eb-9c1f-dd0facbc5e46 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:17.764249 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475d5e8f-73a2-47eb-9c1f-dd0facbc5e46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:17.784633 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-475d5e8f-73a2-47eb-9c1f-dd0facbc5e46 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:11:17.786427 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-475d5e8f-73a2-47eb-9c1f-dd0facbc5e46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:11:17.793809 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee019192-3583-4511-b0cd-01d66f950bf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:17.794503 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee019192-3583-4511-b0cd-01d66f950bf9 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:17.794779 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee019192-3583-4511-b0cd-01d66f950bf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:17.795061 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee019192-3583-4511-b0cd-01d66f950bf9 None admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:17.795847 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee019192-3583-4511-b0cd-01d66f950bf9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:17.796500 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee019192-3583-4511-b0cd-01d66f950bf9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:17.797624 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee019192-3583-4511-b0cd-01d66f950bf9 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:17.813891 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 422/851] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:11:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:17.818106 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee019192-3583-4511-b0cd-01d66f950bf9 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:17.852422 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:17.877023 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 430/852] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:11:17 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:11:17.900861 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475d5e8f-73a2-47eb-9c1f-dd0facbc5e46 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:17.932969 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee019192-3583-4511-b0cd-01d66f950bf9 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:17.949013 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:17.972869 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2580976-67dc-467a-9f4d-c420ced4ef4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:17.973938 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2580976-67dc-467a-9f4d-c420ced4ef4d None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:17.974425 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2580976-67dc-467a-9f4d-c420ced4ef4d None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:17.974895 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2580976-67dc-467a-9f4d-c420ced4ef4d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:17.975925 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2580976-67dc-467a-9f4d-c420ced4ef4d None admin] RBAC: Adding query filter params (role.id=4946cf0ff7164307b36efeee7c787146, scope.project.id=c6f995ffa0a94b1584f25dbe31e20524, user.id=09dd957f216a46dd8a3f400fcd2ba944) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:17.976428 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2580976-67dc-467a-9f4d-c420ced4ef4d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:17.978280 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2580976-67dc-467a-9f4d-c420ced4ef4d None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:17.985735 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 423/853] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:11:17 2022] GET /identity/v3/role_assignments?user.id=09dd957f216a46dd8a3f400fcd2ba944&scope.project.id=c6f995ffa0a94b1584f25dbe31e20524&role.id=4946cf0ff7164307b36efeee7c787146 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:11:18.022070 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f44dcb6-b58f-4200-9a59-68db80e7712c None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:18.022721 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f44dcb6-b58f-4200-9a59-68db80e7712c None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:18.023187 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f44dcb6-b58f-4200-9a59-68db80e7712c None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:18.024907 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475d5e8f-73a2-47eb-9c1f-dd0facbc5e46 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:18.025403 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 424/854] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:11:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:11:18.031626 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837095cc-36fa-4fe7-b14c-fd435d85859b None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:18.032884 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837095cc-36fa-4fe7-b14c-fd435d85859b None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:18.033672 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837095cc-36fa-4fe7-b14c-fd435d85859b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:18.053441 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-837095cc-36fa-4fe7-b14c-fd435d85859b None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:11:18.055168 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-837095cc-36fa-4fe7-b14c-fd435d85859b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:11:18.063790 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf33537-755d-4f6a-8eda-50865ec4c871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:18.064697 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf33537-755d-4f6a-8eda-50865ec4c871 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:18.064988 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf33537-755d-4f6a-8eda-50865ec4c871 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:18.065270 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf33537-755d-4f6a-8eda-50865ec4c871 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:18.077219 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf33537-755d-4f6a-8eda-50865ec4c871 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:11:18.086085 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-0cf33537-755d-4f6a-8eda-50865ec4c871 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:11:18.086085 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:18.086085 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:11:18.086085 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:11:18.086085 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:11:18.086085 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:11:18.086085 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:11:18.086085 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:11:18.086085 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:11:18.086085 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:11:18.086085 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:11:18.086085 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:11:18.086085 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:11:18.086085 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:11:18.086085 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:11:18.086085 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:11:18.087197 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:11:18.087197 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:11:18.087197 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:11:18.087197 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:11:18.087197 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:11:18.087197 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:11:18.087197 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:11:18.087197 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:11:18.087197 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:11:18.087197 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:11:18.087197 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:11:18.087197 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:11:18.087197 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:11:18.087197 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:11:18.087197 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:11:18.087197 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:11:18.088119 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:11:18.088119 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:11:18.088119 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:11:18.088119 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:11:18.088119 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:11:18.088119 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:11:18.088119 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:11:18.088119 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:11:18.088119 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:11:18.088119 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:11:18.088119 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:11:18.088119 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:11:18.088119 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 431/855] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 14:11:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:11:18.095447 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf33537-755d-4f6a-8eda-50865ec4c871 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:18.150424 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 425/856] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:11:18 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:11:18.170332 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837095cc-36fa-4fe7-b14c-fd435d85859b None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:18.197751 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:18.233568 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:18.240560 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf33537-755d-4f6a-8eda-50865ec4c871 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:18.289216 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a9a7ab-396d-4e46-8653-f9755304cada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:18.289817 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a9a7ab-396d-4e46-8653-f9755304cada None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:18.290194 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a9a7ab-396d-4e46-8653-f9755304cada None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:18.290473 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a9a7ab-396d-4e46-8653-f9755304cada None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:18.291227 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a9a7ab-396d-4e46-8653-f9755304cada None admin] RBAC: Adding query filter params (name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:18.291537 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a9a7ab-396d-4e46-8653-f9755304cada None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:18.292926 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a9a7ab-396d-4e46-8653-f9755304cada None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:18.300917 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837095cc-36fa-4fe7-b14c-fd435d85859b None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:18.302213 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 432/857] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 19 14:11:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:18.310292 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a9a7ab-396d-4e46-8653-f9755304cada None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:18.350551 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab134221-4569-4d90-a014-d15c408c77a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:18.351443 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab134221-4569-4d90-a014-d15c408c77a2 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:18.351723 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab134221-4569-4d90-a014-d15c408c77a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:18.351997 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab134221-4569-4d90-a014-d15c408c77a2 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:18.353205 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab134221-4569-4d90-a014-d15c408c77a2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:18.355046 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab134221-4569-4d90-a014-d15c408c77a2 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:18.372490 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-ab134221-4569-4d90-a014-d15c408c77a2 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 14:11:18.372490 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:18.372490 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:11:18.372490 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:11:18.372490 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:11:18.372490 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:11:18.372490 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:11:18.372490 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:11:18.372490 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 14:11:18.372490 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:11:18.372490 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:11:18.372490 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:11:18.372490 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:18.372490 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:11:18.372490 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:11:18.372490 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:11:18.373730 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:11:18.373730 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:11:18.373730 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:11:18.373730 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:11:18.373730 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:11:18.373730 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:11:18.373730 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:11:18.373730 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:11:18.373730 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:11:18.373730 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:11:18.373730 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:11:18.373730 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:11:18.373730 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:11:18.373730 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:11:18.373730 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:11:18.373730 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:11:18.374924 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:11:18.374924 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:11:18.374924 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:11:18.374924 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:11:18.374924 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:11:18.374924 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:11:18.374924 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:11:18.374924 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:11:18.374924 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:11:18.374924 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:11:18.374924 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:11:18.374924 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:11:18.374924 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:11:18.374924 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 14:11:18.374924 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:11:18.374924 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 426/858] 10.222.0.57 () {56 vars in 1132 bytes} [Thu May 19 14:11:18 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:11:18.380524 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab134221-4569-4d90-a014-d15c408c77a2 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:18.425787 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a9a7ab-396d-4e46-8653-f9755304cada None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:18.462630 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49bc07d-5a22-4105-aaac-f39534b923d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:18.463499 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49bc07d-5a22-4105-aaac-f39534b923d3 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:18.463788 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49bc07d-5a22-4105-aaac-f39534b923d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:18.464065 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49bc07d-5a22-4105-aaac-f39534b923d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:18.472526 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:18.475969 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49bc07d-5a22-4105-aaac-f39534b923d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:18.477558 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49bc07d-5a22-4105-aaac-f39534b923d3 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:18.495021 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab134221-4569-4d90-a014-d15c408c77a2 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:18.495909 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-e49bc07d-5a22-4105-aaac-f39534b923d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:11:18.495909 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:18.495909 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:11:18.495909 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:11:18.495909 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:11:18.495909 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:11:18.495909 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:11:18.495909 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:11:18.495909 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:11:18.495909 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:11:18.495909 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:11:18.495909 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:11:18.495909 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:11:18.495909 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:11:18.495909 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:11:18.496827 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:11:18.496827 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:11:18.496827 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:11:18.496827 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:11:18.496827 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:11:18.496827 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:11:18.496827 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:11:18.496827 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:11:18.496827 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:11:18.496827 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:11:18.496827 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:11:18.496827 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:11:18.496827 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:11:18.496827 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:11:18.496827 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:11:18.496827 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:11:18.496827 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:11:18.497752 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:11:18.497752 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:11:18.497752 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:11:18.497752 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:11:18.502434 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 433/859] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:11:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:11:18.504093 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49bc07d-5a22-4105-aaac-f39534b923d3 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:18.522640 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:18.534200 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a2cf8d-52f5-4e0c-b13d-0ee3dbd97b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:18.534973 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a2cf8d-52f5-4e0c-b13d-0ee3dbd97b64 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:18.535264 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a2cf8d-52f5-4e0c-b13d-0ee3dbd97b64 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:18.535543 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a2cf8d-52f5-4e0c-b13d-0ee3dbd97b64 None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:18.536254 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a2cf8d-52f5-4e0c-b13d-0ee3dbd97b64 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:18.536562 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a2cf8d-52f5-4e0c-b13d-0ee3dbd97b64 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:18.538160 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a2cf8d-52f5-4e0c-b13d-0ee3dbd97b64 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:18.543745 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 427/860] 10.222.0.57 () {56 vars in 1126 bytes} [Thu May 19 14:11:18 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:18.549166 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a2cf8d-52f5-4e0c-b13d-0ee3dbd97b64 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:18.618640 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49bc07d-5a22-4105-aaac-f39534b923d3 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:18.658168 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef65b964-09f7-42fa-b15c-8cb468d8b5eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:18.658941 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef65b964-09f7-42fa-b15c-8cb468d8b5eb None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:18.659226 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef65b964-09f7-42fa-b15c-8cb468d8b5eb None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:18.659532 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef65b964-09f7-42fa-b15c-8cb468d8b5eb None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:18.660315 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef65b964-09f7-42fa-b15c-8cb468d8b5eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:18.660619 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef65b964-09f7-42fa-b15c-8cb468d8b5eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:18.662020 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef65b964-09f7-42fa-b15c-8cb468d8b5eb None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:18.662528 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a2cf8d-52f5-4e0c-b13d-0ee3dbd97b64 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:18.675461 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 434/861] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:11:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:18.680917 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef65b964-09f7-42fa-b15c-8cb468d8b5eb None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:18.699800 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65aa244b-36a4-410b-874d-7e2002feb0a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:18.700476 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aa244b-36a4-410b-874d-7e2002feb0a3 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:18.700765 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aa244b-36a4-410b-874d-7e2002feb0a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:18.701043 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aa244b-36a4-410b-874d-7e2002feb0a3 None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:18.701888 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65aa244b-36a4-410b-874d-7e2002feb0a3 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:18.702356 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65aa244b-36a4-410b-874d-7e2002feb0a3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:18.703830 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65aa244b-36a4-410b-874d-7e2002feb0a3 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:18.709903 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 428/862] 10.222.0.57 () {56 vars in 1126 bytes} [Thu May 19 14:11:18 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:18.774523 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:18.803950 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef65b964-09f7-42fa-b15c-8cb468d8b5eb None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:18.843712 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd93848-e187-45f2-bb78-cd600058661c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:18.844592 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd93848-e187-45f2-bb78-cd600058661c None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:18.845496 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd93848-e187-45f2-bb78-cd600058661c None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:18.845774 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd93848-e187-45f2-bb78-cd600058661c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:18.854246 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd93848-e187-45f2-bb78-cd600058661c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:18.855738 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd93848-e187-45f2-bb78-cd600058661c None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:18.874423 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-abd93848-e187-45f2-bb78-cd600058661c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 14:11:18.874423 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:18.874423 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:11:18.874423 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:11:18.874423 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:11:18.874423 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:11:18.874423 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:11:18.874423 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:11:18.874423 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:11:18.874423 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:11:18.874423 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:11:18.874423 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:11:18.874423 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:11:18.874423 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:11:18.874423 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:11:18.874423 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:11:18.875498 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:11:18.875498 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:11:18.875498 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:11:18.875498 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:11:18.875498 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:11:18.875498 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:11:18.875498 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:11:18.875498 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:11:18.875498 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:11:18.875498 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:11:18.875498 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:11:18.875498 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:11:18.875498 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:11:18.875498 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:11:18.875498 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:11:18.875498 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:11:18.875498 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:11:18.875498 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:11:18.877146 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:11:18.877146 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:11:18.877146 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:11:18.877146 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:11:18.877146 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:11:18.877146 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:11:18.877146 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:11:18.877146 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:11:18.877146 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 14:11:18.877146 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:11:18.877737 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 435/863] 10.222.0.57 () {56 vars in 1117 bytes} [Thu May 19 14:11:18 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:11:18.883054 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65aa244b-36a4-410b-874d-7e2002feb0a3 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:18.903623 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:18.997432 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65aa244b-36a4-410b-874d-7e2002feb0a3 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:19.033141 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09003d36-9247-4d97-ae87-bd289b81ff70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:19.034181 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09003d36-9247-4d97-ae87-bd289b81ff70 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:19.034666 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09003d36-9247-4d97-ae87-bd289b81ff70 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:19.035152 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09003d36-9247-4d97-ae87-bd289b81ff70 None admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:19.036166 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09003d36-9247-4d97-ae87-bd289b81ff70 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:19.036675 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09003d36-9247-4d97-ae87-bd289b81ff70 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:19.038397 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09003d36-9247-4d97-ae87-bd289b81ff70 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:19.060190 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 429/864] 10.222.0.57 () {56 vars in 1149 bytes} [Thu May 19 14:11:18 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:19.066969 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd93848-e187-45f2-bb78-cd600058661c None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:19.191938 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd93848-e187-45f2-bb78-cd600058661c None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:19.228392 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635cc4bc-e9a8-408a-9e9d-8cf0385bb301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:19.229413 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635cc4bc-e9a8-408a-9e9d-8cf0385bb301 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:19.229938 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635cc4bc-e9a8-408a-9e9d-8cf0385bb301 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:19.230420 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635cc4bc-e9a8-408a-9e9d-8cf0385bb301 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:19.233421 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:19.245245 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7509715-7c8c-4a51-b265-57f61ef57427 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:19.245582 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7509715-7c8c-4a51-b265-57f61ef57427 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:19.245818 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7509715-7c8c-4a51-b265-57f61ef57427 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:19.247287 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 430/865] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:11:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:11:19.254027 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e253b85d-059a-4a52-a2e0-8cb5ce516a68 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:19.254322 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e253b85d-059a-4a52-a2e0-8cb5ce516a68 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:19.254604 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e253b85d-059a-4a52-a2e0-8cb5ce516a68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:19.273062 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-e253b85d-059a-4a52-a2e0-8cb5ce516a68 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:11:19.274395 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-e253b85d-059a-4a52-a2e0-8cb5ce516a68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:11:19.315766 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635cc4bc-e9a8-408a-9e9d-8cf0385bb301 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:19.317184 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635cc4bc-e9a8-408a-9e9d-8cf0385bb301 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:19.371422 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 431/866] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:11:19 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:11:19.393483 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e253b85d-059a-4a52-a2e0-8cb5ce516a68 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:19.396009 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:19.432382 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 436/867] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:11:19 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:11:19.437772 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635cc4bc-e9a8-408a-9e9d-8cf0385bb301 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:19.478286 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:19.514641 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e253b85d-059a-4a52-a2e0-8cb5ce516a68 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:19.553530 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb146119-6874-4284-ae07-426728dde2ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:19.554526 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb146119-6874-4284-ae07-426728dde2ea None admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:19.554993 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb146119-6874-4284-ae07-426728dde2ea None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:19.555481 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb146119-6874-4284-ae07-426728dde2ea None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:19.556401 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb146119-6874-4284-ae07-426728dde2ea None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:19.558010 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb146119-6874-4284-ae07-426728dde2ea None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:19.562583 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635cc4bc-e9a8-408a-9e9d-8cf0385bb301 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:19.584779 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 432/868] 10.222.0.57 () {60 vars in 1141 bytes} [Thu May 19 14:11:19 2022] POST /identity/v3/services => generated 251 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:11:19.622720 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef2e3ca-2c59-445a-80eb-77d350118c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:19.624078 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef2e3ca-2c59-445a-80eb-77d350118c17 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:19.624581 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef2e3ca-2c59-445a-80eb-77d350118c17 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:19.625092 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef2e3ca-2c59-445a-80eb-77d350118c17 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:19.637712 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef2e3ca-2c59-445a-80eb-77d350118c17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:19.639777 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef2e3ca-2c59-445a-80eb-77d350118c17 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:19.662682 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-eef2e3ca-2c59-445a-80eb-77d350118c17 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:11:19.662682 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:19.662682 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:11:19.662682 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:11:19.662682 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:11:19.662682 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:11:19.662682 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:11:19.662682 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:11:19.662682 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:11:19.662682 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:11:19.662682 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:11:19.662682 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:11:19.662682 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:11:19.662682 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:11:19.662682 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:11:19.663680 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:11:19.663680 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:11:19.663680 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:11:19.663680 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:11:19.663680 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:11:19.663680 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:11:19.663680 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:11:19.663680 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:11:19.663680 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:11:19.663680 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:11:19.663680 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:11:19.663680 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:11:19.663680 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:11:19.663680 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:11:19.663680 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:11:19.663680 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:11:19.663680 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:11:19.664606 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:11:19.664606 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:11:19.664606 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:11:19.664606 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:11:19.667124 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 437/869] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:11:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:11:19.673076 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb146119-6874-4284-ae07-426728dde2ea None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:19.744759 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:19.791914 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb146119-6874-4284-ae07-426728dde2ea None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:19.827215 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5369a78-bf6b-45fc-b038-bf48b0f10bc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:19.827885 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5369a78-bf6b-45fc-b038-bf48b0f10bc0 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:19.828169 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5369a78-bf6b-45fc-b038-bf48b0f10bc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:19.828443 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5369a78-bf6b-45fc-b038-bf48b0f10bc0 None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:19.829691 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5369a78-bf6b-45fc-b038-bf48b0f10bc0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:19.830034 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5369a78-bf6b-45fc-b038-bf48b0f10bc0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:19.831441 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5369a78-bf6b-45fc-b038-bf48b0f10bc0 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:19.843920 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 433/870] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:11:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:19.850031 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef2e3ca-2c59-445a-80eb-77d350118c17 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:19.937426 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:19.957987 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef2e3ca-2c59-445a-80eb-77d350118c17 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:19.995300 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc106f16-44f8-4a49-8472-2f2eafe6b7eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:19.996478 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc106f16-44f8-4a49-8472-2f2eafe6b7eb None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:19.996958 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc106f16-44f8-4a49-8472-2f2eafe6b7eb None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:19.997428 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc106f16-44f8-4a49-8472-2f2eafe6b7eb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:20.005512 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc106f16-44f8-4a49-8472-2f2eafe6b7eb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:20.007248 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc106f16-44f8-4a49-8472-2f2eafe6b7eb None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:20.022274 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-cc106f16-44f8-4a49-8472-2f2eafe6b7eb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:11:20.022274 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:20.022274 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:11:20.022274 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:11:20.022274 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:11:20.022274 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:11:20.022274 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:11:20.022274 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:11:20.022274 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:11:20.022274 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:11:20.022274 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:11:20.022274 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:11:20.022274 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:11:20.022274 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:11:20.022274 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:11:20.023217 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:11:20.023217 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:11:20.023217 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:11:20.023217 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:11:20.023217 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:11:20.023217 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:11:20.023217 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:11:20.023217 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:11:20.023217 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:11:20.023217 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:11:20.023217 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:11:20.023217 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:11:20.023217 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:11:20.023217 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:11:20.023217 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:11:20.023217 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:11:20.023217 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:11:20.024113 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:11:20.024113 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:11:20.024113 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:11:20.024113 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:11:20.024113 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:11:20.024113 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:11:20.024113 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:11:20.024113 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:11:20.026671 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 438/871] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:11:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:11:20.036235 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5369a78-bf6b-45fc-b038-bf48b0f10bc0 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:20.143976 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5369a78-bf6b-45fc-b038-bf48b0f10bc0 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:20.168054 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:20.181652 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59dada4-2d56-410c-b5c3-8cedff2426b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:20.182349 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59dada4-2d56-410c-b5c3-8cedff2426b5 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:20.182629 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59dada4-2d56-410c-b5c3-8cedff2426b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:20.182900 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59dada4-2d56-410c-b5c3-8cedff2426b5 None admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:20.183684 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59dada4-2d56-410c-b5c3-8cedff2426b5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:20.183991 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59dada4-2d56-410c-b5c3-8cedff2426b5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:20.185421 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59dada4-2d56-410c-b5c3-8cedff2426b5 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:20.198455 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 434/872] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:11:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:20.204908 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc106f16-44f8-4a49-8472-2f2eafe6b7eb None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:20.319834 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc106f16-44f8-4a49-8472-2f2eafe6b7eb None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:20.355333 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fab94f-c495-4bd6-bc30-107bcc097e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:20.355997 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fab94f-c495-4bd6-bc30-107bcc097e71 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:20.356280 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fab94f-c495-4bd6-bc30-107bcc097e71 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:20.356557 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fab94f-c495-4bd6-bc30-107bcc097e71 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:20.357391 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fab94f-c495-4bd6-bc30-107bcc097e71 None admin] RBAC: Adding query filter params (role.id=9c7d4d848dd549adb9f3465d4201f26d, scope.project.id=c6f995ffa0a94b1584f25dbe31e20524, user.id=1623987d87ab4cf19b923d254a68c2c6) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:20.357706 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fab94f-c495-4bd6-bc30-107bcc097e71 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:20.359226 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fab94f-c495-4bd6-bc30-107bcc097e71 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:20.366339 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 439/873] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:11:20 2022] GET /identity/v3/role_assignments?user.id=1623987d87ab4cf19b923d254a68c2c6&scope.project.id=c6f995ffa0a94b1584f25dbe31e20524&role.id=9c7d4d848dd549adb9f3465d4201f26d => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:20.442795 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de2550b-ddca-4cb0-b8be-8521e1b7cc4f None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:20.443139 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de2550b-ddca-4cb0-b8be-8521e1b7cc4f None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:20.443364 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de2550b-ddca-4cb0-b8be-8521e1b7cc4f None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:20.445092 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 435/874] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:11:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:11:20.450871 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e37e35-14a7-4ccb-b63b-e1f977b39814 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:20.451177 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e37e35-14a7-4ccb-b63b-e1f977b39814 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:20.451447 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e37e35-14a7-4ccb-b63b-e1f977b39814 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:20.464045 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:20.470982 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-c0e37e35-14a7-4ccb-b63b-e1f977b39814 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:11:20.472224 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-c0e37e35-14a7-4ccb-b63b-e1f977b39814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:11:20.548502 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 440/875] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:11:20 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:11:20.571406 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de2550b-ddca-4cb0-b8be-8521e1b7cc4f None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:20.683778 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de2550b-ddca-4cb0-b8be-8521e1b7cc4f None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:20.721388 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cd0edb-70b9-49ee-a36f-6d759e7be992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:20.722663 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cd0edb-70b9-49ee-a36f-6d759e7be992 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:20.723162 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cd0edb-70b9-49ee-a36f-6d759e7be992 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:20.723636 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cd0edb-70b9-49ee-a36f-6d759e7be992 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:20.724499 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cd0edb-70b9-49ee-a36f-6d759e7be992 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:20.726110 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cd0edb-70b9-49ee-a36f-6d759e7be992 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:20.740132 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-36cd0edb-70b9-49ee-a36f-6d759e7be992 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 14:11:20.740132 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:20.740132 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:11:20.740132 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:11:20.740132 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:11:20.740132 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:11:20.740132 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:11:20.740132 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:11:20.740132 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 14:11:20.740132 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:11:20.740132 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:11:20.740132 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:11:20.740132 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:20.740132 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:11:20.740132 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:11:20.740132 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:11:20.740132 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:11:20.741147 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:11:20.741147 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:11:20.741147 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:11:20.741147 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:11:20.741147 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:11:20.741147 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:11:20.741147 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:11:20.741147 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:11:20.741147 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:11:20.741147 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:11:20.741147 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:11:20.741147 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:11:20.741147 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:11:20.741147 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:11:20.741147 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:11:20.741147 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:11:20.742022 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:11:20.742022 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:11:20.742022 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:11:20.742022 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:11:20.742022 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:11:20.742022 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:11:20.742022 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:11:20.742022 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:11:20.742022 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:11:20.742022 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:11:20.742022 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:11:20.742022 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:11:20.742022 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 14:11:20.742022 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:11:20.744911 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 436/876] 10.222.0.57 () {56 vars in 1126 bytes} [Thu May 19 14:11:20 2022] GET /identity/v3/services/object-store => generated 93 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:11:20.752766 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e37e35-14a7-4ccb-b63b-e1f977b39814 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:20.855496 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e37e35-14a7-4ccb-b63b-e1f977b39814 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:20.891174 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bbd335-ea29-41e2-a206-7429424bcd61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:20.891892 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bbd335-ea29-41e2-a206-7429424bcd61 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:20.892178 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bbd335-ea29-41e2-a206-7429424bcd61 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:20.892457 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bbd335-ea29-41e2-a206-7429424bcd61 None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:20.893185 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bbd335-ea29-41e2-a206-7429424bcd61 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:20.893497 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bbd335-ea29-41e2-a206-7429424bcd61 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:20.894991 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bbd335-ea29-41e2-a206-7429424bcd61 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:20.897288 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:20.901272 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 441/877] 10.222.0.57 () {56 vars in 1122 bytes} [Thu May 19 14:11:20 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:20.909362 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cd0edb-70b9-49ee-a36f-6d759e7be992 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:20.963966 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:21.023881 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cd0edb-70b9-49ee-a36f-6d759e7be992 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:21.062721 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c907df8b-8590-48bf-ae8c-eaf686888608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:21.063549 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c907df8b-8590-48bf-ae8c-eaf686888608 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:21.063787 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c907df8b-8590-48bf-ae8c-eaf686888608 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:21.063952 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c907df8b-8590-48bf-ae8c-eaf686888608 None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:21.064667 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c907df8b-8590-48bf-ae8c-eaf686888608 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:21.065017 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c907df8b-8590-48bf-ae8c-eaf686888608 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:21.066492 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c907df8b-8590-48bf-ae8c-eaf686888608 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:21.071607 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 437/878] 10.222.0.57 () {56 vars in 1122 bytes} [Thu May 19 14:11:20 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:21.098221 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3267e2a3-c2d5-443b-bf4b-2e299f4a16a6 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:21.098545 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3267e2a3-c2d5-443b-bf4b-2e299f4a16a6 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:21.098792 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3267e2a3-c2d5-443b-bf4b-2e299f4a16a6 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:21.101907 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 442/879] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:11:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:11:21.110956 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca4ce45-9b1c-4fa3-afee-51f18a02349f None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:21.111250 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca4ce45-9b1c-4fa3-afee-51f18a02349f None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:21.111538 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca4ce45-9b1c-4fa3-afee-51f18a02349f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:21.136059 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-cca4ce45-9b1c-4fa3-afee-51f18a02349f None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:11:21.137289 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-cca4ce45-9b1c-4fa3-afee-51f18a02349f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:11:21.214764 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 438/880] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:11:21 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:11:21.238700 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3267e2a3-c2d5-443b-bf4b-2e299f4a16a6 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:21.350559 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3267e2a3-c2d5-443b-bf4b-2e299f4a16a6 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:21.387127 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e83dd8-b1c2-47ac-b0d6-6808a5a27cc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:21.387849 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e83dd8-b1c2-47ac-b0d6-6808a5a27cc9 None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:21.388133 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e83dd8-b1c2-47ac-b0d6-6808a5a27cc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:21.388461 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e83dd8-b1c2-47ac-b0d6-6808a5a27cc9 None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:21.389209 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e83dd8-b1c2-47ac-b0d6-6808a5a27cc9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:21.390612 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e83dd8-b1c2-47ac-b0d6-6808a5a27cc9 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:21.409211 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 443/881] 10.222.0.57 () {60 vars in 1141 bytes} [Thu May 19 14:11:21 2022] POST /identity/v3/services => generated 269 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:11:22.058969 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff213a2-6a73-484c-9b34-45334722fb37 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:22.059357 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff213a2-6a73-484c-9b34-45334722fb37 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:22.059576 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff213a2-6a73-484c-9b34-45334722fb37 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:22.061154 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 439/882] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:11:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:11:22.067000 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea246e26-76ca-48ad-a280-b3f41d529dfd None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:22.067296 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea246e26-76ca-48ad-a280-b3f41d529dfd None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:22.067586 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea246e26-76ca-48ad-a280-b3f41d529dfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:22.088744 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-ea246e26-76ca-48ad-a280-b3f41d529dfd None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:11:22.089963 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-ea246e26-76ca-48ad-a280-b3f41d529dfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:11:22.161657 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:22.179712 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 444/883] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:11:22 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:11:22.204295 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff213a2-6a73-484c-9b34-45334722fb37 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:22.250360 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:22.317716 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff213a2-6a73-484c-9b34-45334722fb37 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:22.353430 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ec0159-5238-4b3f-b4e2-0d7a41977ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:22.354256 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ec0159-5238-4b3f-b4e2-0d7a41977ae6 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:22.354536 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ec0159-5238-4b3f-b4e2-0d7a41977ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:22.354806 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ec0159-5238-4b3f-b4e2-0d7a41977ae6 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:22.355456 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ec0159-5238-4b3f-b4e2-0d7a41977ae6 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:22.356817 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ec0159-5238-4b3f-b4e2-0d7a41977ae6 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:22.369785 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-30ec0159-5238-4b3f-b4e2-0d7a41977ae6 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 14:11:22.369785 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:22.369785 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:11:22.369785 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:11:22.369785 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:11:22.369785 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:11:22.369785 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:11:22.369785 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:11:22.369785 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 14:11:22.369785 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:11:22.369785 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:11:22.369785 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:11:22.369785 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:22.369785 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:11:22.369785 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:11:22.369785 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:11:22.369785 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:11:22.370870 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:11:22.370870 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:11:22.370870 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:11:22.370870 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:11:22.370870 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:11:22.370870 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:11:22.370870 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:11:22.370870 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:11:22.370870 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:11:22.370870 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:11:22.370870 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:11:22.370870 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:11:22.370870 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:11:22.370870 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:11:22.370870 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:11:22.370870 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:11:22.371742 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:11:22.371742 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:11:22.371742 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:11:22.371742 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:11:22.371742 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:11:22.371742 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:11:22.371742 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:11:22.371742 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:11:22.371742 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:11:22.371742 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:11:22.371742 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:11:22.371742 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:11:22.371742 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 14:11:22.371742 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:11:22.371742 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 440/884] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 19 14:11:22 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:11:22.377104 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea246e26-76ca-48ad-a280-b3f41d529dfd None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:22.480646 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea246e26-76ca-48ad-a280-b3f41d529dfd None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:22.515057 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb14276-f3a0-4074-8579-2dc99aae020a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:22.515725 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb14276-f3a0-4074-8579-2dc99aae020a None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:22.516005 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb14276-f3a0-4074-8579-2dc99aae020a None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:22.516310 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb14276-f3a0-4074-8579-2dc99aae020a None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:22.517044 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb14276-f3a0-4074-8579-2dc99aae020a None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:22.517345 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb14276-f3a0-4074-8579-2dc99aae020a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:22.518741 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb14276-f3a0-4074-8579-2dc99aae020a None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:22.523694 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 445/885] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 19 14:11:22 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:22.530964 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ec0159-5238-4b3f-b4e2-0d7a41977ae6 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:22.637511 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ec0159-5238-4b3f-b4e2-0d7a41977ae6 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:22.650172 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:22.673563 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174a45d5-5763-4ce4-b3c7-c199e38d94df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:22.674304 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174a45d5-5763-4ce4-b3c7-c199e38d94df None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:22.674595 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174a45d5-5763-4ce4-b3c7-c199e38d94df None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:22.675284 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174a45d5-5763-4ce4-b3c7-c199e38d94df None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:22.676086 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174a45d5-5763-4ce4-b3c7-c199e38d94df None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:22.676931 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174a45d5-5763-4ce4-b3c7-c199e38d94df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:22.678402 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174a45d5-5763-4ce4-b3c7-c199e38d94df None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:22.684621 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 441/886] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 19 14:11:22 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:22.690456 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb14276-f3a0-4074-8579-2dc99aae020a None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:22.744336 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:22.801361 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb14276-f3a0-4074-8579-2dc99aae020a None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:22.837335 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53e77e0-6af1-40e2-a790-13f3de7d18df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:22.838041 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53e77e0-6af1-40e2-a790-13f3de7d18df None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:22.838331 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53e77e0-6af1-40e2-a790-13f3de7d18df None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:22.838618 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53e77e0-6af1-40e2-a790-13f3de7d18df None admin] PATH_INFO: `/v3/endpoints` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:22.839417 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53e77e0-6af1-40e2-a790-13f3de7d18df None admin] RBAC: Adding query filter params (interface=public, service_id=2a33cbf07cc44759b4f0238f63f2d1bb, region_id=RegionOne) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:22.839732 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53e77e0-6af1-40e2-a790-13f3de7d18df None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:22.841120 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53e77e0-6af1-40e2-a790-13f3de7d18df None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:22.847519 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 446/887] 10.222.0.57 () {56 vars in 1251 bytes} [Thu May 19 14:11:22 2022] GET /identity/v3/endpoints?service_id=2a33cbf07cc44759b4f0238f63f2d1bb&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:22.855247 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174a45d5-5763-4ce4-b3c7-c199e38d94df None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:22.877710 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7907f2-3d0d-4ac5-9079-6b54e3754991 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:22.878044 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7907f2-3d0d-4ac5-9079-6b54e3754991 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:22.878307 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7907f2-3d0d-4ac5-9079-6b54e3754991 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:22.879916 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 447/888] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:11:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:11:22.887107 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03138c02-007f-459f-adbd-a1c44d0918b2 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:22.887506 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03138c02-007f-459f-adbd-a1c44d0918b2 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:22.887846 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03138c02-007f-459f-adbd-a1c44d0918b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:22.909541 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-03138c02-007f-459f-adbd-a1c44d0918b2 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:11:22.910880 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-03138c02-007f-459f-adbd-a1c44d0918b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:11:22.966586 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174a45d5-5763-4ce4-b3c7-c199e38d94df None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:22.984284 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 448/889] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:11:22 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:11:23.009020 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03138c02-007f-459f-adbd-a1c44d0918b2 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:23.010257 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44deaea-a707-4428-9c18-73838b636d41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:23.011067 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44deaea-a707-4428-9c18-73838b636d41 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:23.011451 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44deaea-a707-4428-9c18-73838b636d41 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:23.011828 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44deaea-a707-4428-9c18-73838b636d41 None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:23.012626 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44deaea-a707-4428-9c18-73838b636d41 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:23.014162 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44deaea-a707-4428-9c18-73838b636d41 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:23.026245 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 442/890] 10.222.0.57 () {56 vars in 1087 bytes} [Thu May 19 14:11:22 2022] GET /identity/v3/services => generated 1268 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 14:11:23.064789 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:23.134011 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03138c02-007f-459f-adbd-a1c44d0918b2 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:23.168661 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a289bb2e-ce4b-43ce-b1b2-7525b5f437a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:23.169440 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a289bb2e-ce4b-43ce-b1b2-7525b5f437a9 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:23.169732 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a289bb2e-ce4b-43ce-b1b2-7525b5f437a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:23.170049 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a289bb2e-ce4b-43ce-b1b2-7525b5f437a9 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:23.184064 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a289bb2e-ce4b-43ce-b1b2-7525b5f437a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:11:23.193186 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-a289bb2e-ce4b-43ce-b1b2-7525b5f437a9 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 14:11:23.193186 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:23.193186 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:11:23.193186 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:11:23.193186 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:11:23.193186 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:11:23.193186 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:11:23.193186 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:11:23.193186 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:11:23.193186 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:11:23.193186 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:11:23.193186 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:11:23.193186 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:11:23.193186 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:11:23.193186 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:11:23.193186 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:11:23.194262 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:11:23.194262 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:11:23.194262 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:11:23.194262 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:11:23.194262 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:11:23.194262 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:11:23.194262 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:11:23.194262 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:11:23.194262 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:11:23.194262 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:11:23.194262 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:11:23.194262 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:11:23.194262 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:11:23.194262 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:11:23.194262 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:11:23.194262 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:11:23.195103 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:11:23.195103 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:11:23.195103 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:11:23.195103 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:11:23.195103 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:11:23.195103 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:11:23.195103 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:11:23.195103 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:11:23.195103 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:11:23.195103 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:11:23.195103 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 14:11:23.195103 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:11:23.195103 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 449/891] 10.222.0.57 () {56 vars in 1120 bytes} [Thu May 19 14:11:23 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:11:23.201041 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44deaea-a707-4428-9c18-73838b636d41 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:23.249173 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:23.330317 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44deaea-a707-4428-9c18-73838b636d41 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:23.366272 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53371879-e834-42a8-9152-5f98036308ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:23.366947 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53371879-e834-42a8-9152-5f98036308ce None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:23.367240 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53371879-e834-42a8-9152-5f98036308ce None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:23.367512 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53371879-e834-42a8-9152-5f98036308ce None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:23.368320 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53371879-e834-42a8-9152-5f98036308ce None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:23.368639 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53371879-e834-42a8-9152-5f98036308ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:23.370041 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53371879-e834-42a8-9152-5f98036308ce None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:23.379608 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 443/892] 10.222.0.57 () {56 vars in 1115 bytes} [Thu May 19 14:11:23 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:23.384887 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a289bb2e-ce4b-43ce-b1b2-7525b5f437a9 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:23.451462 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082a880b-3df5-4be4-ba25-19ed63289fd2 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:23.451752 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082a880b-3df5-4be4-ba25-19ed63289fd2 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:23.451920 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082a880b-3df5-4be4-ba25-19ed63289fd2 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:23.453902 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 444/893] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:11:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:11:23.459187 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabbecad-9247-4f92-ade3-c5bd81fb4a35 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:23.459748 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabbecad-9247-4f92-ade3-c5bd81fb4a35 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:23.460236 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabbecad-9247-4f92-ade3-c5bd81fb4a35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:23.477614 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-fabbecad-9247-4f92-ade3-c5bd81fb4a35 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:11:23.478971 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-fabbecad-9247-4f92-ade3-c5bd81fb4a35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:11:23.497467 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a289bb2e-ce4b-43ce-b1b2-7525b5f437a9 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:23.500473 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:23.538849 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a3cb23-ae9f-471a-8c5f-4f9be99c70c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:23.539828 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a3cb23-ae9f-471a-8c5f-4f9be99c70c2 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:23.540247 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a3cb23-ae9f-471a-8c5f-4f9be99c70c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:23.540617 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a3cb23-ae9f-471a-8c5f-4f9be99c70c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:23.552500 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a3cb23-ae9f-471a-8c5f-4f9be99c70c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:23.554125 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a3cb23-ae9f-471a-8c5f-4f9be99c70c2 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:23.567125 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 445/894] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:11:23 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:11:23.581865 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-30a3cb23-ae9f-471a-8c5f-4f9be99c70c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:11:23.581865 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:23.581865 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:11:23.581865 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:11:23.581865 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:11:23.581865 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:11:23.581865 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:11:23.581865 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:11:23.581865 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:11:23.581865 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:11:23.581865 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:11:23.581865 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:11:23.581865 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:11:23.581865 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:11:23.581865 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:11:23.582811 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:11:23.582811 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:11:23.582811 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:11:23.582811 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:11:23.582811 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:11:23.582811 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:11:23.582811 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:11:23.582811 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:11:23.582811 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:11:23.582811 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:11:23.582811 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:11:23.582811 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:11:23.582811 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:11:23.582811 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:11:23.582811 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:11:23.582811 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:11:23.582811 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:11:23.583718 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:11:23.583718 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:11:23.583718 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:11:23.583718 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:11:23.583718 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 450/895] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:11:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:11:23.587105 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabbecad-9247-4f92-ade3-c5bd81fb4a35 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:23.601373 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a3cb23-ae9f-471a-8c5f-4f9be99c70c2 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:23.619788 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:23.727524 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabbecad-9247-4f92-ade3-c5bd81fb4a35 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:23.739072 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a3cb23-ae9f-471a-8c5f-4f9be99c70c2 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:23.775389 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c78739-8675-4fc0-9874-6c16b96247b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:23.776028 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5f45e8-6c9d-47d2-9718-493c5287064e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:23.776159 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c78739-8675-4fc0-9874-6c16b96247b5 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:23.776364 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c78739-8675-4fc0-9874-6c16b96247b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:23.776645 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c78739-8675-4fc0-9874-6c16b96247b5 None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:23.777372 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c78739-8675-4fc0-9874-6c16b96247b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:23.777632 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5f45e8-6c9d-47d2-9718-493c5287064e None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:23.777732 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c78739-8675-4fc0-9874-6c16b96247b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:23.778222 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5f45e8-6c9d-47d2-9718-493c5287064e None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:23.778684 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5f45e8-6c9d-47d2-9718-493c5287064e None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:23.779070 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c78739-8675-4fc0-9874-6c16b96247b5 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:23.779846 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5f45e8-6c9d-47d2-9718-493c5287064e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:23.781410 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5f45e8-6c9d-47d2-9718-493c5287064e None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:23.798043 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 446/896] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:11:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:23.798043 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95298c82-cd44-4851-8b2b-2b53152c3faa None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:23.798043 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95298c82-cd44-4851-8b2b-2b53152c3faa None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:23.798043 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95298c82-cd44-4851-8b2b-2b53152c3faa None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:23.801768 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 447/897] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:11:23 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:11:23.803560 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95298c82-cd44-4851-8b2b-2b53152c3faa None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:23.806443 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 451/898] 10.222.0.57 () {60 vars in 1141 bytes} [Thu May 19 14:11:23 2022] POST /identity/v3/services => generated 241 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 14:11:23.807875 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3036ebe-3e40-495d-8676-bce1572a0be3 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:23.808170 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3036ebe-3e40-495d-8676-bce1572a0be3 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:23.808501 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3036ebe-3e40-495d-8676-bce1572a0be3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:23.835900 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-c3036ebe-3e40-495d-8676-bce1572a0be3 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:11:23.837252 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-c3036ebe-3e40-495d-8676-bce1572a0be3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:11:23.851501 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:23.936527 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 452/899] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:11:23 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:11:23.938643 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:23.956677 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3036ebe-3e40-495d-8676-bce1572a0be3 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:23.960833 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95298c82-cd44-4851-8b2b-2b53152c3faa None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:24.010969 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06454e42-e749-474c-aa73-24f7d3cb49bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:24.012756 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06454e42-e749-474c-aa73-24f7d3cb49bc None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:24.013103 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06454e42-e749-474c-aa73-24f7d3cb49bc None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:24.013450 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06454e42-e749-474c-aa73-24f7d3cb49bc None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:24.023151 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06454e42-e749-474c-aa73-24f7d3cb49bc None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:24.025093 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06454e42-e749-474c-aa73-24f7d3cb49bc None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:24.042676 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-06454e42-e749-474c-aa73-24f7d3cb49bc None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 14:11:24.042676 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:24.042676 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:11:24.042676 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:11:24.042676 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:11:24.042676 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:11:24.042676 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:11:24.042676 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:11:24.042676 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:11:24.042676 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:11:24.042676 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:11:24.042676 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:11:24.042676 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:11:24.042676 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:11:24.042676 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:11:24.042676 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:11:24.043677 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:11:24.043677 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:11:24.043677 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:11:24.043677 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:11:24.043677 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:11:24.043677 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:11:24.043677 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:11:24.043677 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:11:24.043677 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:11:24.043677 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:11:24.043677 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:11:24.043677 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:11:24.043677 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:11:24.043677 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:11:24.043677 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:11:24.043677 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:11:24.043677 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:11:24.043677 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:11:24.044627 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:11:24.044627 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:11:24.044627 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:11:24.044627 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:11:24.044627 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:11:24.044627 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:11:24.044627 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:11:24.044627 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:11:24.044627 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 14:11:24.044627 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:11:24.044627 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 448/900] 10.222.0.57 () {56 vars in 1117 bytes} [Thu May 19 14:11:23 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:11:24.050043 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06454e42-e749-474c-aa73-24f7d3cb49bc None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:24.083205 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3036ebe-3e40-495d-8676-bce1572a0be3 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:24.132014 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8368607-8dcb-4356-bd85-52c7dfe2c748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:24.133153 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8368607-8dcb-4356-bd85-52c7dfe2c748 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:24.133716 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8368607-8dcb-4356-bd85-52c7dfe2c748 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:24.134205 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8368607-8dcb-4356-bd85-52c7dfe2c748 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:24.135125 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8368607-8dcb-4356-bd85-52c7dfe2c748 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:24.136819 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8368607-8dcb-4356-bd85-52c7dfe2c748 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:24.153647 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-c8368607-8dcb-4356-bd85-52c7dfe2c748 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 14:11:24.153647 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:24.153647 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:11:24.153647 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:11:24.153647 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:11:24.153647 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:11:24.153647 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:11:24.153647 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:11:24.153647 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 14:11:24.153647 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:11:24.153647 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:11:24.153647 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:11:24.153647 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:24.153647 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:11:24.153647 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:11:24.153647 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:11:24.154756 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:11:24.154756 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:11:24.154756 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:11:24.154756 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:11:24.154756 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:11:24.154756 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:11:24.154756 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:11:24.154756 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:11:24.154756 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:11:24.154756 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:11:24.154756 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:11:24.154756 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:11:24.154756 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:11:24.154756 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:11:24.154756 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:11:24.154756 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:11:24.155604 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:11:24.155604 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:11:24.155604 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:11:24.155604 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:11:24.155604 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:11:24.155604 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:11:24.155604 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:11:24.155604 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:11:24.155604 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:11:24.155604 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:11:24.155604 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:11:24.155604 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:11:24.155604 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:11:24.155604 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 14:11:24.155604 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:11:24.159653 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 453/901] 10.222.0.57 () {56 vars in 1132 bytes} [Thu May 19 14:11:23 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:11:24.165163 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8368607-8dcb-4356-bd85-52c7dfe2c748 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:24.178024 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:24.189874 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06454e42-e749-474c-aa73-24f7d3cb49bc None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:24.229408 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e403c51c-43e4-4469-a797-6d573a66bc41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:24.230418 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e403c51c-43e4-4469-a797-6d573a66bc41 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:24.230552 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e403c51c-43e4-4469-a797-6d573a66bc41 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:24.230788 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e403c51c-43e4-4469-a797-6d573a66bc41 None admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:24.231641 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e403c51c-43e4-4469-a797-6d573a66bc41 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:24.232003 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e403c51c-43e4-4469-a797-6d573a66bc41 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:24.233450 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e403c51c-43e4-4469-a797-6d573a66bc41 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:24.259041 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 449/902] 10.222.0.57 () {56 vars in 1149 bytes} [Thu May 19 14:11:24 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:11:24.261046 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e403c51c-43e4-4469-a797-6d573a66bc41 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:24.265031 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:24.306904 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8368607-8dcb-4356-bd85-52c7dfe2c748 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:24.343875 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f9c58f-9c07-428f-b9dc-4ae0d4a5f060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:24.345106 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f9c58f-9c07-428f-b9dc-4ae0d4a5f060 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:24.345106 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f9c58f-9c07-428f-b9dc-4ae0d4a5f060 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:24.345760 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f9c58f-9c07-428f-b9dc-4ae0d4a5f060 None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:24.346663 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f9c58f-9c07-428f-b9dc-4ae0d4a5f060 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:24.346857 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f9c58f-9c07-428f-b9dc-4ae0d4a5f060 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:24.348194 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f9c58f-9c07-428f-b9dc-4ae0d4a5f060 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:24.358270 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 454/903] 10.222.0.57 () {56 vars in 1126 bytes} [Thu May 19 14:11:24 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:11:24.358991 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f9c58f-9c07-428f-b9dc-4ae0d4a5f060 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:24.404281 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e403c51c-43e4-4469-a797-6d573a66bc41 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:24.443762 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fa10dd-ec78-4fc7-988f-fda8810aa88d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:24.444168 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:24.444625 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fa10dd-ec78-4fc7-988f-fda8810aa88d None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:24.444715 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fa10dd-ec78-4fc7-988f-fda8810aa88d None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:24.444948 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fa10dd-ec78-4fc7-988f-fda8810aa88d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:24.487839 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f9c58f-9c07-428f-b9dc-4ae0d4a5f060 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:24.524458 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cd1ee3-b071-47bd-b34d-7c289be151ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:24.525049 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cd1ee3-b071-47bd-b34d-7c289be151ea None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:24.525338 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cd1ee3-b071-47bd-b34d-7c289be151ea None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:24.525619 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cd1ee3-b071-47bd-b34d-7c289be151ea None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:24.526434 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cd1ee3-b071-47bd-b34d-7c289be151ea None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:24.526724 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cd1ee3-b071-47bd-b34d-7c289be151ea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:24.528203 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cd1ee3-b071-47bd-b34d-7c289be151ea None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:24.530806 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fa10dd-ec78-4fc7-988f-fda8810aa88d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:24.533323 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fa10dd-ec78-4fc7-988f-fda8810aa88d None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:24.535440 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:24.538450 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 455/904] 10.222.0.57 () {56 vars in 1126 bytes} [Thu May 19 14:11:24 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:24.539426 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cd1ee3-b071-47bd-b34d-7c289be151ea None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:24.660473 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cd1ee3-b071-47bd-b34d-7c289be151ea None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:24.666543 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 450/905] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:11:24 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:11:24.667737 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fa10dd-ec78-4fc7-988f-fda8810aa88d None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:24.708462 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bb38f7-33fe-477f-ba6a-3f812e373b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:24.709149 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bb38f7-33fe-477f-ba6a-3f812e373b18 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:24.709685 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bb38f7-33fe-477f-ba6a-3f812e373b18 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:24.710004 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bb38f7-33fe-477f-ba6a-3f812e373b18 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:24.710803 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bb38f7-33fe-477f-ba6a-3f812e373b18 None admin] RBAC: Adding query filter params (interface=public, service_id=a10ac092028f4574b01ad923cbbc9b7d, region_id=RegionOne) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:24.711123 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bb38f7-33fe-477f-ba6a-3f812e373b18 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:24.712524 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bb38f7-33fe-477f-ba6a-3f812e373b18 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:24.715222 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:24.725346 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 456/906] 10.222.0.57 () {56 vars in 1251 bytes} [Thu May 19 14:11:24 2022] GET /identity/v3/endpoints?service_id=a10ac092028f4574b01ad923cbbc9b7d&interface=public®ion_id=RegionOne => generated 194 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:24.726397 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bb38f7-33fe-477f-ba6a-3f812e373b18 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:24.801574 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fa10dd-ec78-4fc7-988f-fda8810aa88d None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:24.803260 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:24.847660 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa2e30c-85cd-45d0-8dcb-9d375a1fffeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:24.849373 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa2e30c-85cd-45d0-8dcb-9d375a1fffeb None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:24.849898 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa2e30c-85cd-45d0-8dcb-9d375a1fffeb None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:24.850608 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa2e30c-85cd-45d0-8dcb-9d375a1fffeb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:24.862769 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bb38f7-33fe-477f-ba6a-3f812e373b18 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:24.867802 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa2e30c-85cd-45d0-8dcb-9d375a1fffeb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:24.869691 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa2e30c-85cd-45d0-8dcb-9d375a1fffeb None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:24.895757 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-0fa2e30c-85cd-45d0-8dcb-9d375a1fffeb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:11:24.895757 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:24.895757 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:11:24.895757 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:11:24.895757 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:11:24.895757 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:11:24.895757 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:11:24.895757 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:11:24.895757 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:11:24.895757 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:11:24.895757 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:11:24.895757 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:11:24.895757 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:11:24.895757 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:11:24.895757 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:11:24.896717 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:11:24.896717 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:11:24.896717 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:11:24.896717 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:11:24.896717 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:11:24.896717 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:11:24.896717 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:11:24.896717 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:11:24.896717 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:11:24.896717 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:11:24.896717 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:11:24.896717 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:11:24.896717 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:11:24.896717 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:11:24.896717 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:11:24.896717 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:11:24.896717 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:11:24.897628 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:11:24.897628 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:11:24.897628 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:11:24.897628 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:11:24.901888 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 451/907] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:11:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:11:24.904703 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa2e30c-85cd-45d0-8dcb-9d375a1fffeb None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:24.910327 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca41056e-c6f3-45b8-b886-4a779407016c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:24.911314 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca41056e-c6f3-45b8-b886-4a779407016c None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:24.911804 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca41056e-c6f3-45b8-b886-4a779407016c None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:24.912299 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca41056e-c6f3-45b8-b886-4a779407016c None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:24.913190 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca41056e-c6f3-45b8-b886-4a779407016c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:24.915357 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca41056e-c6f3-45b8-b886-4a779407016c None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:24.930298 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 457/908] 10.222.0.57 () {56 vars in 1087 bytes} [Thu May 19 14:11:24 2022] GET /identity/v3/services => generated 1497 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 14:11:25.027320 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa2e30c-85cd-45d0-8dcb-9d375a1fffeb None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:25.066690 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dcb2ef-e496-41ee-a5c5-20d6175b89d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:25.067290 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dcb2ef-e496-41ee-a5c5-20d6175b89d3 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:25.067583 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dcb2ef-e496-41ee-a5c5-20d6175b89d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:25.067923 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dcb2ef-e496-41ee-a5c5-20d6175b89d3 None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:25.068763 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dcb2ef-e496-41ee-a5c5-20d6175b89d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:25.069131 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dcb2ef-e496-41ee-a5c5-20d6175b89d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:25.070562 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dcb2ef-e496-41ee-a5c5-20d6175b89d3 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:25.084097 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 452/909] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:11:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:11:25.090288 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca41056e-c6f3-45b8-b886-4a779407016c None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:25.164602 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:25.211547 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca41056e-c6f3-45b8-b886-4a779407016c None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:25.250191 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a6f281-f6cd-41cf-b7d8-4870b92f792c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:25.251023 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a6f281-f6cd-41cf-b7d8-4870b92f792c None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:25.251303 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a6f281-f6cd-41cf-b7d8-4870b92f792c None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:25.251576 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a6f281-f6cd-41cf-b7d8-4870b92f792c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:25.258943 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a6f281-f6cd-41cf-b7d8-4870b92f792c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:25.260403 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a6f281-f6cd-41cf-b7d8-4870b92f792c None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:25.275681 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-40a6f281-f6cd-41cf-b7d8-4870b92f792c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:11:25.275681 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:25.275681 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:11:25.275681 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:11:25.275681 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:11:25.275681 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:11:25.275681 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:11:25.275681 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:11:25.275681 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:11:25.275681 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:11:25.275681 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:11:25.275681 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:11:25.275681 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:11:25.275681 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:11:25.275681 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:11:25.276614 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:11:25.276614 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:11:25.276614 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:11:25.276614 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:11:25.276614 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:11:25.276614 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:11:25.276614 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:11:25.276614 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:11:25.276614 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:11:25.276614 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:11:25.276614 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:11:25.276614 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:11:25.276614 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:11:25.276614 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:11:25.276614 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:11:25.276614 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:11:25.276614 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:11:25.277672 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:11:25.277672 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:11:25.277672 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:11:25.277672 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:11:25.277672 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:11:25.277672 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:11:25.277672 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:11:25.277672 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:11:25.277672 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 458/910] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:11:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:11:25.283130 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dcb2ef-e496-41ee-a5c5-20d6175b89d3 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:25.301525 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:25.393746 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dcb2ef-e496-41ee-a5c5-20d6175b89d3 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:25.433913 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b166909-ac32-4625-8793-787d80e98ea8 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:25.434224 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b166909-ac32-4625-8793-787d80e98ea8 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:25.434489 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b166909-ac32-4625-8793-787d80e98ea8 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:25.436124 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 459/911] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:11:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:11:25.436315 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1bd51f-1cde-497b-825b-114fc4a107e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:25.436889 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1bd51f-1cde-497b-825b-114fc4a107e1 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:25.437181 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1bd51f-1cde-497b-825b-114fc4a107e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:25.437463 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1bd51f-1cde-497b-825b-114fc4a107e1 None admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:25.438504 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1bd51f-1cde-497b-825b-114fc4a107e1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:25.438818 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1bd51f-1cde-497b-825b-114fc4a107e1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:25.440264 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1bd51f-1cde-497b-825b-114fc4a107e1 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:25.441386 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb8cd6d-0911-44c8-b96e-0229d17e0f78 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:25.441681 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb8cd6d-0911-44c8-b96e-0229d17e0f78 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:25.442199 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb8cd6d-0911-44c8-b96e-0229d17e0f78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:25.458154 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 453/912] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:11:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:11:25.459838 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1bd51f-1cde-497b-825b-114fc4a107e1 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:25.461655 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-3bb8cd6d-0911-44c8-b96e-0229d17e0f78 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:11:25.463307 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-3bb8cd6d-0911-44c8-b96e-0229d17e0f78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:11:25.543599 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:25.555059 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 460/913] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:11:25 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:11:25.577577 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1bd51f-1cde-497b-825b-114fc4a107e1 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:25.580015 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb8cd6d-0911-44c8-b96e-0229d17e0f78 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:25.614759 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50b649e-04dc-45bd-9abc-726cf5e51a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:25.615670 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50b649e-04dc-45bd-9abc-726cf5e51a20 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:25.616084 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50b649e-04dc-45bd-9abc-726cf5e51a20 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:25.616745 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50b649e-04dc-45bd-9abc-726cf5e51a20 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:25.617740 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50b649e-04dc-45bd-9abc-726cf5e51a20 None admin] RBAC: Adding query filter params (role.id=82029ff85919459780b85031cdfc369f, scope.project.id=c6f995ffa0a94b1584f25dbe31e20524, user.id=1623987d87ab4cf19b923d254a68c2c6) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:25.618211 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50b649e-04dc-45bd-9abc-726cf5e51a20 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:25.619857 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50b649e-04dc-45bd-9abc-726cf5e51a20 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:25.626429 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 454/914] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:11:25 2022] GET /identity/v3/role_assignments?user.id=1623987d87ab4cf19b923d254a68c2c6&scope.project.id=c6f995ffa0a94b1584f25dbe31e20524&role.id=82029ff85919459780b85031cdfc369f => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:11:25.647503 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:25.701970 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb8cd6d-0911-44c8-b96e-0229d17e0f78 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:25.742316 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cfe44e-d122-4bbd-aa3d-0aae0f466680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:25.743139 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cfe44e-d122-4bbd-aa3d-0aae0f466680 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:25.743422 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cfe44e-d122-4bbd-aa3d-0aae0f466680 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:25.743699 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cfe44e-d122-4bbd-aa3d-0aae0f466680 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:25.744358 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cfe44e-d122-4bbd-aa3d-0aae0f466680 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:25.745762 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cfe44e-d122-4bbd-aa3d-0aae0f466680 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:25.758735 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-b1cfe44e-d122-4bbd-aa3d-0aae0f466680 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 14:11:25.758735 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:25.758735 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:11:25.758735 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:11:25.758735 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:11:25.758735 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:11:25.758735 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:11:25.758735 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:11:25.758735 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 14:11:25.758735 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:11:25.758735 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:11:25.758735 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:11:25.758735 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:25.758735 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:11:25.758735 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:11:25.758735 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:11:25.758735 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:11:25.759806 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:11:25.759806 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:11:25.759806 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:11:25.759806 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:11:25.759806 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:11:25.759806 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:11:25.759806 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:11:25.759806 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:11:25.759806 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:11:25.759806 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:11:25.759806 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:11:25.759806 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:11:25.759806 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:11:25.759806 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:11:25.759806 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:11:25.759806 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:11:25.760893 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:11:25.760893 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:11:25.760893 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:11:25.760893 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:11:25.760893 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:11:25.760893 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:11:25.760893 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:11:25.760893 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:11:25.760893 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:11:25.760893 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:11:25.760893 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:11:25.760893 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:11:25.760893 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 14:11:25.760893 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:11:25.760893 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 461/915] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 19 14:11:25 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:11:25.766815 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50b649e-04dc-45bd-9abc-726cf5e51a20 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:25.879915 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50b649e-04dc-45bd-9abc-726cf5e51a20 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:25.915029 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e3d41c-1ec6-43f4-b113-fcaa429d3dc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:25.916035 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e3d41c-1ec6-43f4-b113-fcaa429d3dc8 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:25.916532 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e3d41c-1ec6-43f4-b113-fcaa429d3dc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:25.916998 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e3d41c-1ec6-43f4-b113-fcaa429d3dc8 None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:25.917943 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e3d41c-1ec6-43f4-b113-fcaa429d3dc8 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:25.918445 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e3d41c-1ec6-43f4-b113-fcaa429d3dc8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:25.920009 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e3d41c-1ec6-43f4-b113-fcaa429d3dc8 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:25.926180 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 455/916] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 19 14:11:25 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:25.933864 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cfe44e-d122-4bbd-aa3d-0aae0f466680 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:26.048862 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cfe44e-d122-4bbd-aa3d-0aae0f466680 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:26.065678 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:26.084525 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b656ae9-f97d-4cd3-85f5-209721208557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:26.085193 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b656ae9-f97d-4cd3-85f5-209721208557 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:26.085476 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b656ae9-f97d-4cd3-85f5-209721208557 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:26.085753 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b656ae9-f97d-4cd3-85f5-209721208557 None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:26.086490 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b656ae9-f97d-4cd3-85f5-209721208557 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:26.086795 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b656ae9-f97d-4cd3-85f5-209721208557 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:26.088169 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b656ae9-f97d-4cd3-85f5-209721208557 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:26.093565 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 462/917] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 19 14:11:25 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:26.099608 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e3d41c-1ec6-43f4-b113-fcaa429d3dc8 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:26.102516 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:26.181200 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ac6d83-7df9-4c8e-a80b-9fd4b1a78476 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:26.181388 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ac6d83-7df9-4c8e-a80b-9fd4b1a78476 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:26.181674 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ac6d83-7df9-4c8e-a80b-9fd4b1a78476 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:26.183316 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 463/918] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:11:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:11:26.190410 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d07dc89-c911-486c-86c4-519a7879be9d None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:26.190701 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d07dc89-c911-486c-86c4-519a7879be9d None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:26.190976 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d07dc89-c911-486c-86c4-519a7879be9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:26.216671 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-2d07dc89-c911-486c-86c4-519a7879be9d None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:11:26.217882 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-2d07dc89-c911-486c-86c4-519a7879be9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:11:26.223722 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e3d41c-1ec6-43f4-b113-fcaa429d3dc8 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:26.265804 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59792ba6-14d9-44bb-918e-f0a61f03fce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:26.266619 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59792ba6-14d9-44bb-918e-f0a61f03fce0 None admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:26.266947 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59792ba6-14d9-44bb-918e-f0a61f03fce0 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:26.267226 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59792ba6-14d9-44bb-918e-f0a61f03fce0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:26.268017 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59792ba6-14d9-44bb-918e-f0a61f03fce0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:26.270354 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59792ba6-14d9-44bb-918e-f0a61f03fce0 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:26.292410 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 456/919] 10.222.0.57 () {60 vars in 1144 bytes} [Thu May 19 14:11:26 2022] POST /identity/v3/endpoints => generated 346 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:11:26.313532 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 464/920] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:11:26 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:11:26.338356 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59792ba6-14d9-44bb-918e-f0a61f03fce0 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:26.409186 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:26.457979 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59792ba6-14d9-44bb-918e-f0a61f03fce0 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:26.496502 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d74af6-53da-407c-aded-ef2e8e1e3523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:26.497383 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d74af6-53da-407c-aded-ef2e8e1e3523 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:26.497661 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d74af6-53da-407c-aded-ef2e8e1e3523 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:26.497963 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d74af6-53da-407c-aded-ef2e8e1e3523 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:26.498654 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d74af6-53da-407c-aded-ef2e8e1e3523 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:26.500025 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d74af6-53da-407c-aded-ef2e8e1e3523 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:26.512626 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-f8d74af6-53da-407c-aded-ef2e8e1e3523 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 14:11:26.512626 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:26.512626 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:11:26.512626 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:11:26.512626 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:11:26.512626 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:11:26.512626 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:11:26.512626 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:11:26.512626 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 14:11:26.512626 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:11:26.512626 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:11:26.512626 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:11:26.512626 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:26.512626 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:11:26.512626 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:11:26.512626 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:11:26.512626 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:11:26.513670 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:11:26.513670 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:11:26.513670 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:11:26.513670 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:11:26.513670 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:11:26.513670 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:11:26.513670 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:11:26.513670 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:11:26.513670 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:11:26.513670 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:11:26.513670 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:11:26.513670 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:11:26.513670 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:11:26.513670 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:11:26.513670 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:11:26.513670 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:11:26.514551 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:11:26.514551 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:11:26.514551 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:11:26.514551 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:11:26.514551 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:11:26.514551 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:11:26.514551 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:11:26.514551 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:11:26.514551 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:11:26.514551 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:11:26.514551 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:11:26.514551 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:11:26.514551 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 14:11:26.514551 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:11:26.514551 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 457/921] 10.222.0.57 () {56 vars in 1126 bytes} [Thu May 19 14:11:26 2022] GET /identity/v3/services/object-store => generated 93 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:11:26.520353 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d07dc89-c911-486c-86c4-519a7879be9d None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:26.634440 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d07dc89-c911-486c-86c4-519a7879be9d None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:26.643253 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:26.672520 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5e8e41-b0ae-47d1-865c-2a55c6d87442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:26.673184 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5e8e41-b0ae-47d1-865c-2a55c6d87442 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:26.673470 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5e8e41-b0ae-47d1-865c-2a55c6d87442 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:26.673749 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5e8e41-b0ae-47d1-865c-2a55c6d87442 None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:26.674488 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5e8e41-b0ae-47d1-865c-2a55c6d87442 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:26.674797 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5e8e41-b0ae-47d1-865c-2a55c6d87442 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:26.676184 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5e8e41-b0ae-47d1-865c-2a55c6d87442 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:26.681559 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 465/922] 10.222.0.57 () {56 vars in 1122 bytes} [Thu May 19 14:11:26 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:11:26.687154 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d74af6-53da-407c-aded-ef2e8e1e3523 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:26.796400 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d74af6-53da-407c-aded-ef2e8e1e3523 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:26.822478 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:26.836059 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5165cb1-cf0b-48ec-8adc-3963c0d3871b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:26.836735 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5165cb1-cf0b-48ec-8adc-3963c0d3871b None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:26.837017 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5165cb1-cf0b-48ec-8adc-3963c0d3871b None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:26.837291 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5165cb1-cf0b-48ec-8adc-3963c0d3871b None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:26.838034 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5165cb1-cf0b-48ec-8adc-3963c0d3871b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:26.838344 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5165cb1-cf0b-48ec-8adc-3963c0d3871b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:26.839773 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5165cb1-cf0b-48ec-8adc-3963c0d3871b None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:26.845255 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 458/923] 10.222.0.57 () {56 vars in 1122 bytes} [Thu May 19 14:11:26 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:26.850708 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5e8e41-b0ae-47d1-865c-2a55c6d87442 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:26.964760 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5e8e41-b0ae-47d1-865c-2a55c6d87442 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:27.002853 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaec157-c806-4439-be82-905072417302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:27.003530 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaec157-c806-4439-be82-905072417302 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:27.003816 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaec157-c806-4439-be82-905072417302 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:27.004091 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaec157-c806-4439-be82-905072417302 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:27.004882 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaec157-c806-4439-be82-905072417302 None admin] RBAC: Adding query filter params (interface=public, service_id=37c91f0d9e274221b0c859342f5ed13e, region_id=RegionOne) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:27.005196 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaec157-c806-4439-be82-905072417302 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:27.006600 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaec157-c806-4439-be82-905072417302 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:27.012360 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 466/924] 10.222.0.57 () {56 vars in 1251 bytes} [Thu May 19 14:11:26 2022] GET /identity/v3/endpoints?service_id=37c91f0d9e274221b0c859342f5ed13e&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:27.018546 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5165cb1-cf0b-48ec-8adc-3963c0d3871b None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:27.141827 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5165cb1-cf0b-48ec-8adc-3963c0d3871b None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:27.177961 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09329547-8ead-4d15-a15a-f20c77417ec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:27.178876 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09329547-8ead-4d15-a15a-f20c77417ec2 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:27.179309 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09329547-8ead-4d15-a15a-f20c77417ec2 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:27.179733 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09329547-8ead-4d15-a15a-f20c77417ec2 None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:27.180590 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09329547-8ead-4d15-a15a-f20c77417ec2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:27.182210 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09329547-8ead-4d15-a15a-f20c77417ec2 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:27.189480 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 459/925] 10.222.0.57 () {56 vars in 1087 bytes} [Thu May 19 14:11:27 2022] GET /identity/v3/services => generated 1497 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 14:11:27.457315 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0933c333-396b-4786-aef1-8f66dc4b935f None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:27.458040 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0933c333-396b-4786-aef1-8f66dc4b935f None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:27.458559 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0933c333-396b-4786-aef1-8f66dc4b935f None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:27.460409 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 467/926] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:11:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:11:27.465980 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d575d421-a727-46c3-b083-1c675dc1fcac None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:27.466846 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d575d421-a727-46c3-b083-1c675dc1fcac None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:27.467330 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d575d421-a727-46c3-b083-1c675dc1fcac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:27.486925 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-d575d421-a727-46c3-b083-1c675dc1fcac None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:11:27.488752 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-d575d421-a727-46c3-b083-1c675dc1fcac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:11:27.537636 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:27.562229 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 460/927] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:11:27 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:11:27.585475 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0933c333-396b-4786-aef1-8f66dc4b935f None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:27.624952 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:27.707036 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0933c333-396b-4786-aef1-8f66dc4b935f None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:27.750908 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3403af17-55a4-442c-aa4c-83f423b661ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:27.752005 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3403af17-55a4-442c-aa4c-83f423b661ce None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:27.752459 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3403af17-55a4-442c-aa4c-83f423b661ce None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:27.752883 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3403af17-55a4-442c-aa4c-83f423b661ce None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:27.753761 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3403af17-55a4-442c-aa4c-83f423b661ce None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:27.755388 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3403af17-55a4-442c-aa4c-83f423b661ce None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:27.770206 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-3403af17-55a4-442c-aa4c-83f423b661ce None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 14:11:27.770206 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:27.770206 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:11:27.770206 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:11:27.770206 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:11:27.770206 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:11:27.770206 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:11:27.770206 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:11:27.770206 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 14:11:27.770206 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:11:27.770206 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:11:27.770206 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:11:27.770206 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:27.770206 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:11:27.770206 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:11:27.770206 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:11:27.771460 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:11:27.771460 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:11:27.771460 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:11:27.771460 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:11:27.771460 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:11:27.771460 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:11:27.771460 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:11:27.771460 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:11:27.771460 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:11:27.771460 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:11:27.771460 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:11:27.771460 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:11:27.771460 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:11:27.771460 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:11:27.771460 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:11:27.771460 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:11:27.772501 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:11:27.772501 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:11:27.772501 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:11:27.772501 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:11:27.772501 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:11:27.772501 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:11:27.772501 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:11:27.772501 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:11:27.772501 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:11:27.772501 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:11:27.772501 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:11:27.772501 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:11:27.772501 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:11:27.772501 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 14:11:27.772501 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:11:27.775198 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 468/928] 10.222.0.57 () {56 vars in 1132 bytes} [Thu May 19 14:11:27 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:11:27.781420 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d575d421-a727-46c3-b083-1c675dc1fcac None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:27.890421 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d575d421-a727-46c3-b083-1c675dc1fcac None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:27.933279 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f0fb20-947f-4a27-9883-dca26391f28c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:27.933982 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f0fb20-947f-4a27-9883-dca26391f28c None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:27.934267 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f0fb20-947f-4a27-9883-dca26391f28c None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:27.934546 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f0fb20-947f-4a27-9883-dca26391f28c None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:27.935260 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f0fb20-947f-4a27-9883-dca26391f28c None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:27.935565 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f0fb20-947f-4a27-9883-dca26391f28c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:27.936938 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f0fb20-947f-4a27-9883-dca26391f28c None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:27.940440 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2b8dae-f1bd-4b40-b8b7-c3b6f22b73b3 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:27.940732 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2b8dae-f1bd-4b40-b8b7-c3b6f22b73b3 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:27.941010 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2b8dae-f1bd-4b40-b8b7-c3b6f22b73b3 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:27.943153 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 461/930] 10.222.0.57 () {56 vars in 1126 bytes} [Thu May 19 14:11:27 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:27.943279 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 469/930] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:11:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:11:27.950800 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2b8dae-f1bd-4b40-b8b7-c3b6f22b73b3 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:27.951634 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f234f293-305a-43c3-b60e-d506102c7f48 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:27.952084 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f234f293-305a-43c3-b60e-d506102c7f48 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:27.952528 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f234f293-305a-43c3-b60e-d506102c7f48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:27.971626 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-f234f293-305a-43c3-b60e-d506102c7f48 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:11:27.972835 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-f234f293-305a-43c3-b60e-d506102c7f48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:11:28.018878 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:28.049280 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 462/931] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:11:27 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:11:28.071255 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f234f293-305a-43c3-b60e-d506102c7f48 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:28.084145 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:28.090649 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2b8dae-f1bd-4b40-b8b7-c3b6f22b73b3 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:28.140721 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ed8f6b-ab7d-45b9-ba6d-78669553b350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:28.141936 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ed8f6b-ab7d-45b9-ba6d-78669553b350 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:28.142426 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ed8f6b-ab7d-45b9-ba6d-78669553b350 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:28.142845 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ed8f6b-ab7d-45b9-ba6d-78669553b350 None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:28.143684 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ed8f6b-ab7d-45b9-ba6d-78669553b350 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:28.144130 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ed8f6b-ab7d-45b9-ba6d-78669553b350 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:28.145631 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ed8f6b-ab7d-45b9-ba6d-78669553b350 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:28.151342 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 470/932] 10.222.0.57 () {56 vars in 1126 bytes} [Thu May 19 14:11:27 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:28.158137 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ed8f6b-ab7d-45b9-ba6d-78669553b350 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:28.214156 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f234f293-305a-43c3-b60e-d506102c7f48 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:28.263233 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f55681e-b6be-4537-ba55-fb4e68d305e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:28.264380 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f55681e-b6be-4537-ba55-fb4e68d305e6 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:28.264730 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f55681e-b6be-4537-ba55-fb4e68d305e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:28.265076 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f55681e-b6be-4537-ba55-fb4e68d305e6 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:28.269469 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ed8f6b-ab7d-45b9-ba6d-78669553b350 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:28.279614 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f55681e-b6be-4537-ba55-fb4e68d305e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:11:28.291239 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-7f55681e-b6be-4537-ba55-fb4e68d305e6 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 14:11:28.291239 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:28.291239 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:11:28.291239 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:11:28.291239 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:11:28.291239 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:11:28.291239 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:11:28.291239 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:11:28.291239 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:11:28.291239 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:11:28.291239 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:11:28.291239 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:11:28.291239 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:11:28.291239 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:11:28.291239 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:11:28.291239 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:11:28.292396 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:11:28.292396 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:11:28.292396 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:11:28.292396 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:11:28.292396 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:11:28.292396 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:11:28.292396 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:11:28.292396 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:11:28.292396 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:11:28.292396 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:11:28.292396 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:11:28.292396 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:11:28.292396 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:11:28.292396 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:11:28.292396 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:11:28.292396 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:11:28.293296 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:11:28.293296 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:11:28.293296 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:11:28.293296 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:11:28.293296 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:11:28.293296 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:11:28.293296 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:11:28.293296 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:11:28.293296 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:11:28.293296 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:11:28.293296 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 14:11:28.293296 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:11:28.298036 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 463/933] 10.222.0.57 () {56 vars in 1120 bytes} [Thu May 19 14:11:28 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:11:28.300703 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f55681e-b6be-4537-ba55-fb4e68d305e6 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:28.314602 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f8dd86-5b8f-4ef0-b0a0-a197294e676f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:28.315620 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f8dd86-5b8f-4ef0-b0a0-a197294e676f None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:28.316098 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f8dd86-5b8f-4ef0-b0a0-a197294e676f None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:28.316574 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f8dd86-5b8f-4ef0-b0a0-a197294e676f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:28.317505 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f8dd86-5b8f-4ef0-b0a0-a197294e676f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:28.319122 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f8dd86-5b8f-4ef0-b0a0-a197294e676f None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:28.345031 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 471/934] 10.222.0.57 () {60 vars in 1144 bytes} [Thu May 19 14:11:28 2022] POST /identity/v3/endpoints => generated 347 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:11:28.353898 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:28.439764 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f55681e-b6be-4537-ba55-fb4e68d305e6 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:28.487486 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed18153-50cf-455f-ae40-1ed09552a3ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:28.489113 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed18153-50cf-455f-ae40-1ed09552a3ef None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:28.489570 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed18153-50cf-455f-ae40-1ed09552a3ef None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:28.490033 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed18153-50cf-455f-ae40-1ed09552a3ef None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:28.490940 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed18153-50cf-455f-ae40-1ed09552a3ef None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:28.491399 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed18153-50cf-455f-ae40-1ed09552a3ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:28.492967 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed18153-50cf-455f-ae40-1ed09552a3ef None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:28.503550 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 464/935] 10.222.0.57 () {56 vars in 1115 bytes} [Thu May 19 14:11:28 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:28.509621 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f8dd86-5b8f-4ef0-b0a0-a197294e676f None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:28.531990 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:28.627110 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f8dd86-5b8f-4ef0-b0a0-a197294e676f None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:28.667109 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4227954-cada-4b98-a48b-5063d246a4e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:28.668155 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4227954-cada-4b98-a48b-5063d246a4e0 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:28.668615 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4227954-cada-4b98-a48b-5063d246a4e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:28.669037 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4227954-cada-4b98-a48b-5063d246a4e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:28.680645 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4227954-cada-4b98-a48b-5063d246a4e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:28.682388 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4227954-cada-4b98-a48b-5063d246a4e0 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:28.700350 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-e4227954-cada-4b98-a48b-5063d246a4e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:11:28.700350 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:28.700350 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:11:28.700350 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:11:28.700350 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:11:28.700350 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:11:28.700350 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:11:28.700350 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:11:28.700350 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:11:28.700350 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:11:28.700350 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:11:28.700350 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:11:28.700350 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:11:28.700350 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:11:28.700350 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:11:28.701566 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:11:28.701566 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:11:28.701566 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:11:28.701566 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:11:28.701566 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:11:28.701566 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:11:28.701566 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:11:28.701566 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:11:28.701566 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:11:28.701566 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:11:28.701566 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:11:28.701566 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:11:28.701566 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:11:28.701566 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:11:28.701566 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:11:28.701566 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:11:28.701566 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:11:28.702718 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:11:28.702718 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:11:28.702718 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:11:28.702718 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:11:28.705391 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 472/936] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:11:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:11:28.711192 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed18153-50cf-455f-ae40-1ed09552a3ef None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:28.803153 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e804e817-e006-48dc-afa7-d7bf1b249979 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:28.803842 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e804e817-e006-48dc-afa7-d7bf1b249979 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:28.804485 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e804e817-e006-48dc-afa7-d7bf1b249979 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:28.806720 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 473/937] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:11:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:11:28.813632 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17930d8e-3f88-4770-ad6f-4b62b11a6c55 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:28.813955 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17930d8e-3f88-4770-ad6f-4b62b11a6c55 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:28.814235 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17930d8e-3f88-4770-ad6f-4b62b11a6c55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:28.814677 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:28.825826 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed18153-50cf-455f-ae40-1ed09552a3ef None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:28.832942 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-17930d8e-3f88-4770-ad6f-4b62b11a6c55 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:11:28.834269 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-17930d8e-3f88-4770-ad6f-4b62b11a6c55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:11:28.866211 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751d6660-30f7-430b-a6a7-94798198d1b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:28.866885 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751d6660-30f7-430b-a6a7-94798198d1b3 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:28.867169 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751d6660-30f7-430b-a6a7-94798198d1b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:28.867446 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751d6660-30f7-430b-a6a7-94798198d1b3 None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:28.868167 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751d6660-30f7-430b-a6a7-94798198d1b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:28.868472 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751d6660-30f7-430b-a6a7-94798198d1b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:28.869865 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751d6660-30f7-430b-a6a7-94798198d1b3 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:28.884262 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 465/938] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:11:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:28.889669 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751d6660-30f7-430b-a6a7-94798198d1b3 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:28.913721 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:28.923447 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 474/939] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:11:28 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:11:28.947842 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17930d8e-3f88-4770-ad6f-4b62b11a6c55 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:29.019009 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751d6660-30f7-430b-a6a7-94798198d1b3 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:29.056627 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f628d9f1-d4e9-409c-a5d2-5a8f97404ac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:29.057689 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f628d9f1-d4e9-409c-a5d2-5a8f97404ac0 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:29.057689 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f628d9f1-d4e9-409c-a5d2-5a8f97404ac0 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:29.057972 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f628d9f1-d4e9-409c-a5d2-5a8f97404ac0 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:29.065675 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f628d9f1-d4e9-409c-a5d2-5a8f97404ac0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:29.067165 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f628d9f1-d4e9-409c-a5d2-5a8f97404ac0 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:29.072026 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17930d8e-3f88-4770-ad6f-4b62b11a6c55 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:29.083099 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-f628d9f1-d4e9-409c-a5d2-5a8f97404ac0 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 14:11:29.083099 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:29.083099 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:11:29.083099 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:11:29.083099 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:11:29.083099 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:11:29.083099 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:11:29.083099 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:11:29.083099 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:11:29.083099 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:11:29.083099 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:11:29.083099 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:11:29.083099 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:11:29.083099 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:11:29.083099 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:11:29.083099 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:11:29.084094 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:11:29.084094 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:11:29.084094 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:11:29.084094 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:11:29.084094 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:11:29.084094 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:11:29.084094 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:11:29.084094 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:11:29.084094 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:11:29.084094 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:11:29.084094 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:11:29.084094 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:11:29.084094 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:11:29.084094 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:11:29.084094 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:11:29.084094 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:11:29.084094 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:11:29.084094 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:11:29.085081 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:11:29.085081 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:11:29.085081 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:11:29.085081 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:11:29.085081 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:11:29.085081 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:11:29.085081 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:11:29.085081 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:11:29.085081 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 14:11:29.085081 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:11:29.085081 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 466/940] 10.222.0.57 () {56 vars in 1117 bytes} [Thu May 19 14:11:28 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:11:29.090195 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f628d9f1-d4e9-409c-a5d2-5a8f97404ac0 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:29.110840 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:29.127824 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b36cb1c-8708-4050-bcb8-60d114e3f099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:29.129562 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b36cb1c-8708-4050-bcb8-60d114e3f099 None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:29.130096 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b36cb1c-8708-4050-bcb8-60d114e3f099 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:29.130598 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b36cb1c-8708-4050-bcb8-60d114e3f099 None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:29.131528 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b36cb1c-8708-4050-bcb8-60d114e3f099 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:29.133192 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b36cb1c-8708-4050-bcb8-60d114e3f099 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:29.134855 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: WARNING keystone.server.flask.common [None req-5b36cb1c-8708-4050-bcb8-60d114e3f099 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 19 14:11:29.154945 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.sql.core [None req-5b36cb1c-8708-4050-bcb8-60d114e3f099 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 19 14:11:29.154945 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 19 14:11:29.154945 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [parameters: {'id': '3690f7e5865f4b09a1b774d567ad8f50', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 19 14:11:29.154945 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80203) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 19 14:11:29.166437 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-5b36cb1c-8708-4050-bcb8-60d114e3f099 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 3690f7e5865f4b09a1b774d567ad8f50.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 3690f7e5865f4b09a1b774d567ad8f50. May 19 14:11:29.166437 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:29.166437 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 14:11:29.166437 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 14:11:29.166437 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 14:11:29.166437 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 14:11:29.166437 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 14:11:29.166437 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application result = self._query(query) May 19 14:11:29.166437 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 14:11:29.166437 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application conn.query(q) May 19 14:11:29.166437 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 14:11:29.166437 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 14:11:29.166437 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 14:11:29.167278 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application result.read() May 19 14:11:29.167278 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 14:11:29.167278 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 14:11:29.167278 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 14:11:29.167278 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 14:11:29.167278 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 14:11:29.167278 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 14:11:29.167278 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 14:11:29.167278 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 14:11:29.167278 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 19 14:11:29.167278 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:11:29.167278 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 19 14:11:29.167278 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:11:29.167278 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:29.167278 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 19 14:11:29.167278 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 19 14:11:29.167278 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project May 19 14:11:29.167278 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return project_ref.to_dict() May 19 14:11:29.168247 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 14:11:29.168247 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application next(self.gen) May 19 14:11:29.168247 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 19 14:11:29.168247 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application yield resource May 19 14:11:29.168247 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 14:11:29.168247 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application next(self.gen) May 19 14:11:29.168247 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 19 14:11:29.168247 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self.session.rollback() May 19 14:11:29.168247 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 19 14:11:29.168247 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self.force_reraise() May 19 14:11:29.168247 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 19 14:11:29.168247 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise self.value May 19 14:11:29.168247 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 19 14:11:29.168247 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 19 14:11:29.168247 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 19 14:11:29.168247 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application session.commit() May 19 14:11:29.168247 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 19 14:11:29.169169 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 19 14:11:29.169169 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 19 14:11:29.169169 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._prepare_impl() May 19 14:11:29.169169 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 19 14:11:29.169169 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self.session.flush() May 19 14:11:29.169169 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 19 14:11:29.169169 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._flush(objects) May 19 14:11:29.169169 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 19 14:11:29.169169 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 19 14:11:29.169169 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 19 14:11:29.169169 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application compat.raise_( May 19 14:11:29.169169 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 14:11:29.169169 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception May 19 14:11:29.169169 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 19 14:11:29.169169 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application flush_context.execute() May 19 14:11:29.169169 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 19 14:11:29.169169 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rec.execute(self) May 19 14:11:29.170120 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 19 14:11:29.170120 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 19 14:11:29.170120 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 19 14:11:29.170120 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application _emit_insert_statements( May 19 14:11:29.170120 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 19 14:11:29.170120 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application c = connection._execute_20( May 19 14:11:29.170120 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 19 14:11:29.170120 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 19 14:11:29.170120 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 19 14:11:29.170120 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 19 14:11:29.170120 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 19 14:11:29.170120 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ret = self._execute_context( May 19 14:11:29.170120 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 19 14:11:29.170120 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 19 14:11:29.170120 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 19 14:11:29.171837 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 19 14:11:29.171837 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 14:11:29.171837 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception May 19 14:11:29.171837 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 14:11:29.171837 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 14:11:29.171837 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 14:11:29.171837 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 14:11:29.171837 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 14:11:29.171837 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application result = self._query(query) May 19 14:11:29.171837 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 14:11:29.171837 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application conn.query(q) May 19 14:11:29.171837 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 14:11:29.171837 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 14:11:29.171837 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 14:11:29.171837 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application result.read() May 19 14:11:29.171837 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 14:11:29.171837 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 14:11:29.178394 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 14:11:29.178394 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 14:11:29.178394 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 14:11:29.178394 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 14:11:29.178394 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 14:11:29.178394 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 14:11:29.178394 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 19 14:11:29.178394 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 19 14:11:29.178394 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [parameters: {'id': '3690f7e5865f4b09a1b774d567ad8f50', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 19 14:11:29.178394 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 19 14:11:29.178394 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:11:29.178394 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:11:29.178394 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:11:29.178394 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:29.178394 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project May 19 14:11:29.179215 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) May 19 14:11:29.179215 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 19 14:11:29.179215 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 19 14:11:29.179215 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. May 19 14:11:29.179215 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:11:29.179215 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:11:29.179215 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:11:29.179215 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:29.179215 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:11:29.179215 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:11:29.179215 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:11:29.179215 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:11:29.179215 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:11:29.179215 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:11:29.179215 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:11:29.179215 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:11:29.179215 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:11:29.180179 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:11:29.180179 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post May 19 14:11:29.180179 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( May 19 14:11:29.180179 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:11:29.180179 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:11:29.180179 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project May 19 14:11:29.180179 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.Conflict( May 19 14:11:29.180179 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 3690f7e5865f4b09a1b774d567ad8f50. May 19 14:11:29.180179 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:11:29.180179 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 475/941] 10.222.0.57 () {60 vars in 1140 bytes} [Thu May 19 14:11:28 2022] POST /identity/v3/projects => generated 262 bytes in 227 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) May 19 14:11:29.183176 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b36cb1c-8708-4050-bcb8-60d114e3f099 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:29.214585 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f628d9f1-d4e9-409c-a5d2-5a8f97404ac0 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:29.258144 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d80153-6923-485b-9ed0-f0f4d0475250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:29.258945 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:29.260399 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d80153-6923-485b-9ed0-f0f4d0475250 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:29.260729 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d80153-6923-485b-9ed0-f0f4d0475250 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:29.261274 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d80153-6923-485b-9ed0-f0f4d0475250 None admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:29.262536 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d80153-6923-485b-9ed0-f0f4d0475250 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:29.262946 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d80153-6923-485b-9ed0-f0f4d0475250 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:29.266440 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d80153-6923-485b-9ed0-f0f4d0475250 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:29.289899 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 467/942] 10.222.0.57 () {56 vars in 1149 bytes} [Thu May 19 14:11:29 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:29.293537 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d80153-6923-485b-9ed0-f0f4d0475250 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:29.322047 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b36cb1c-8708-4050-bcb8-60d114e3f099 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:29.377904 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:29.381657 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231a776c-68d4-40d1-b34a-b278d79cbd5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:29.382930 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231a776c-68d4-40d1-b34a-b278d79cbd5f None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:29.383217 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231a776c-68d4-40d1-b34a-b278d79cbd5f None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:29.383494 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231a776c-68d4-40d1-b34a-b278d79cbd5f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:29.393094 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231a776c-68d4-40d1-b34a-b278d79cbd5f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:29.395071 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231a776c-68d4-40d1-b34a-b278d79cbd5f None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:29.415684 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-231a776c-68d4-40d1-b34a-b278d79cbd5f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:11:29.415684 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:29.415684 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:11:29.415684 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:11:29.415684 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:11:29.415684 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:11:29.415684 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:11:29.415684 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:11:29.415684 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:11:29.415684 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:11:29.415684 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:11:29.415684 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:11:29.415684 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:11:29.415684 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:11:29.415684 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:11:29.416637 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:11:29.416637 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:11:29.416637 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:11:29.416637 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:11:29.416637 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:11:29.416637 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:11:29.416637 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:11:29.416637 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:11:29.416637 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:11:29.416637 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:11:29.416637 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:11:29.416637 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:11:29.416637 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:11:29.416637 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:11:29.416637 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:11:29.416637 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:11:29.416637 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:11:29.417554 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:11:29.417554 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:11:29.417554 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:11:29.417554 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:11:29.417554 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:11:29.417554 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:11:29.417554 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:11:29.417554 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:11:29.417554 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 476/943] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:11:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:11:29.422595 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d80153-6923-485b-9ed0-f0f4d0475250 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:29.426068 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231a776c-68d4-40d1-b34a-b278d79cbd5f None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:29.471225 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631449d9-f4df-4590-a859-8ce581cc46e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:29.472403 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631449d9-f4df-4590-a859-8ce581cc46e9 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:29.472917 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631449d9-f4df-4590-a859-8ce581cc46e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:29.473460 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631449d9-f4df-4590-a859-8ce581cc46e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:29.485073 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631449d9-f4df-4590-a859-8ce581cc46e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:29.486881 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631449d9-f4df-4590-a859-8ce581cc46e9 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:29.506415 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-631449d9-f4df-4590-a859-8ce581cc46e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:11:29.506415 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:29.506415 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:11:29.506415 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:11:29.506415 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:11:29.506415 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:11:29.506415 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:11:29.506415 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:11:29.506415 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:11:29.506415 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:11:29.506415 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:11:29.506415 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:11:29.506415 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:11:29.506415 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:11:29.506415 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:11:29.507831 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:11:29.507831 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:11:29.507831 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:11:29.507831 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:11:29.507831 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:11:29.507831 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:11:29.507831 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:11:29.507831 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:11:29.507831 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:11:29.507831 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:11:29.507831 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:11:29.507831 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:11:29.507831 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:11:29.507831 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:11:29.507831 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:11:29.507831 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:11:29.507831 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:11:29.508733 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:11:29.508733 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:11:29.508733 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:11:29.508733 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:11:29.511061 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 468/944] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:11:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:11:29.518005 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631449d9-f4df-4590-a859-8ce581cc46e9 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:29.546493 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231a776c-68d4-40d1-b34a-b278d79cbd5f None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:29.565978 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:29.586553 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f940c0d3-e044-48a9-bcf4-44b88f744bc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:29.587492 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f940c0d3-e044-48a9-bcf4-44b88f744bc4 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:29.587784 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f940c0d3-e044-48a9-bcf4-44b88f744bc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:29.588073 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f940c0d3-e044-48a9-bcf4-44b88f744bc4 None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:29.588857 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f940c0d3-e044-48a9-bcf4-44b88f744bc4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:29.589171 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f940c0d3-e044-48a9-bcf4-44b88f744bc4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:29.590699 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f940c0d3-e044-48a9-bcf4-44b88f744bc4 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:29.613811 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 477/945] 10.222.0.57 () {56 vars in 1112 bytes} [Thu May 19 14:11:29 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:29.615467 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f726d4-3b05-4754-845d-72e8ed1072e2 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:29.615752 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f726d4-3b05-4754-845d-72e8ed1072e2 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:29.616026 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f726d4-3b05-4754-845d-72e8ed1072e2 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:29.617461 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 478/946] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:11:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:11:29.623163 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa936dd-b0f7-47de-8dcc-891ee2d8f880 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:29.623454 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa936dd-b0f7-47de-8dcc-891ee2d8f880 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:29.623723 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa936dd-b0f7-47de-8dcc-891ee2d8f880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:29.644867 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631449d9-f4df-4590-a859-8ce581cc46e9 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:29.648701 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-daa936dd-b0f7-47de-8dcc-891ee2d8f880 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:11:29.651196 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-daa936dd-b0f7-47de-8dcc-891ee2d8f880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:11:29.682314 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:29.697817 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd71274-a7f8-4f2a-be70-ec5a18bed37f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:29.698352 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd71274-a7f8-4f2a-be70-ec5a18bed37f None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:29.698640 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd71274-a7f8-4f2a-be70-ec5a18bed37f None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:29.698918 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd71274-a7f8-4f2a-be70-ec5a18bed37f None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:29.699701 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd71274-a7f8-4f2a-be70-ec5a18bed37f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:29.699970 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd71274-a7f8-4f2a-be70-ec5a18bed37f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:29.701384 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd71274-a7f8-4f2a-be70-ec5a18bed37f None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:29.720468 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 469/947] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:11:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:29.721529 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd71274-a7f8-4f2a-be70-ec5a18bed37f None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:29.749094 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 479/948] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:11:29 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:11:29.777545 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa936dd-b0f7-47de-8dcc-891ee2d8f880 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:29.855711 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd71274-a7f8-4f2a-be70-ec5a18bed37f None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:29.896607 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa936dd-b0f7-47de-8dcc-891ee2d8f880 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:29.903054 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895977d4-0e68-4057-80fb-c8e4f1976177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:29.903857 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895977d4-0e68-4057-80fb-c8e4f1976177 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:29.904141 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895977d4-0e68-4057-80fb-c8e4f1976177 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:29.904418 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895977d4-0e68-4057-80fb-c8e4f1976177 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:29.905487 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:29.914699 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895977d4-0e68-4057-80fb-c8e4f1976177 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:29.916469 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895977d4-0e68-4057-80fb-c8e4f1976177 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:29.935854 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-895977d4-0e68-4057-80fb-c8e4f1976177 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:11:29.935854 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:29.935854 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:11:29.935854 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:11:29.935854 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:11:29.935854 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:11:29.935854 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:11:29.935854 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:11:29.935854 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:11:29.935854 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:11:29.935854 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:11:29.935854 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:11:29.935854 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:11:29.935854 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:11:29.935854 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:11:29.936816 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:11:29.936816 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:11:29.936816 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:11:29.936816 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:11:29.936816 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:11:29.936816 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:11:29.936816 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:11:29.936816 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:11:29.936816 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:11:29.936816 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:11:29.936816 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:11:29.936816 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:11:29.936816 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:11:29.936816 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:11:29.936816 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:11:29.936816 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:11:29.936816 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:11:29.938343 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:11:29.938343 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:11:29.938343 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:11:29.938343 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:11:29.938343 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:11:29.938343 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:11:29.938343 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:11:29.938343 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:11:29.938343 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 470/949] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:11:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:11:29.943405 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895977d4-0e68-4057-80fb-c8e4f1976177 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:29.946355 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99416ac0-cf53-461a-b683-d5928d74510f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:29.947996 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99416ac0-cf53-461a-b683-d5928d74510f None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:29.948282 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99416ac0-cf53-461a-b683-d5928d74510f None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:29.948557 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99416ac0-cf53-461a-b683-d5928d74510f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:29.949220 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99416ac0-cf53-461a-b683-d5928d74510f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:29.950636 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99416ac0-cf53-461a-b683-d5928d74510f None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:29.964072 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-99416ac0-cf53-461a-b683-d5928d74510f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 14:11:29.964072 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:29.964072 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:11:29.964072 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:11:29.964072 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:11:29.964072 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:11:29.964072 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:11:29.964072 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:11:29.964072 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 14:11:29.964072 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:11:29.964072 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:11:29.964072 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:11:29.964072 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:29.964072 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:11:29.964072 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:11:29.964072 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:11:29.964072 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:11:29.965145 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:11:29.965145 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:11:29.965145 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:11:29.965145 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:11:29.965145 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:11:29.965145 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:11:29.965145 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:11:29.965145 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:11:29.965145 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:11:29.965145 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:11:29.965145 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:11:29.965145 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:11:29.965145 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:11:29.965145 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:11:29.965145 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:11:29.965145 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:11:29.966033 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:11:29.966033 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:11:29.966033 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:11:29.966033 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:11:29.966033 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:11:29.966033 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:11:29.966033 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:11:29.966033 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:11:29.966033 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:11:29.966033 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:11:29.966033 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:11:29.966033 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:11:29.966033 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 14:11:29.966033 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:11:29.966033 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 480/950] 10.222.0.57 () {56 vars in 1126 bytes} [Thu May 19 14:11:29 2022] GET /identity/v3/services/object-store => generated 93 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:11:29.971743 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99416ac0-cf53-461a-b683-d5928d74510f None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:30.002300 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:30.068983 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895977d4-0e68-4057-80fb-c8e4f1976177 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:30.095605 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99416ac0-cf53-461a-b683-d5928d74510f None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:30.109053 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdaba47-309f-4d3f-bb20-9510a8532730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:30.111363 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdaba47-309f-4d3f-bb20-9510a8532730 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:30.111864 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdaba47-309f-4d3f-bb20-9510a8532730 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:30.112352 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdaba47-309f-4d3f-bb20-9510a8532730 None admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:30.113634 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdaba47-309f-4d3f-bb20-9510a8532730 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:30.114181 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdaba47-309f-4d3f-bb20-9510a8532730 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:30.115839 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdaba47-309f-4d3f-bb20-9510a8532730 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:30.132605 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 471/951] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:11:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:11:30.141895 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3212e7ea-7e4f-4e1e-89c9-548180414680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:30.141895 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3212e7ea-7e4f-4e1e-89c9-548180414680 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:30.141895 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3212e7ea-7e4f-4e1e-89c9-548180414680 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:30.141895 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3212e7ea-7e4f-4e1e-89c9-548180414680 None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:30.142331 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3212e7ea-7e4f-4e1e-89c9-548180414680 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:30.142331 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3212e7ea-7e4f-4e1e-89c9-548180414680 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:30.142331 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3212e7ea-7e4f-4e1e-89c9-548180414680 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:30.142563 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdaba47-309f-4d3f-bb20-9510a8532730 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:30.149657 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 481/952] 10.222.0.57 () {56 vars in 1122 bytes} [Thu May 19 14:11:29 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:11:30.150700 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3212e7ea-7e4f-4e1e-89c9-548180414680 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:30.190886 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:30.267128 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3212e7ea-7e4f-4e1e-89c9-548180414680 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:30.283204 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdaba47-309f-4d3f-bb20-9510a8532730 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:30.285588 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:30.305575 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f455fc-ed81-46bf-8b5f-896f8cccbef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:30.306438 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f455fc-ed81-46bf-8b5f-896f8cccbef5 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:30.306833 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f455fc-ed81-46bf-8b5f-896f8cccbef5 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:30.307220 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f455fc-ed81-46bf-8b5f-896f8cccbef5 None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:30.308026 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f455fc-ed81-46bf-8b5f-896f8cccbef5 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:30.308448 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f455fc-ed81-46bf-8b5f-896f8cccbef5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:30.310242 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f455fc-ed81-46bf-8b5f-896f8cccbef5 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:30.321356 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 482/953] 10.222.0.57 () {56 vars in 1122 bytes} [Thu May 19 14:11:30 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:11:30.322291 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f455fc-ed81-46bf-8b5f-896f8cccbef5 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:30.330724 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7084f1d2-411c-4637-8b05-f053cbc993da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:30.332371 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7084f1d2-411c-4637-8b05-f053cbc993da None admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:30.332828 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7084f1d2-411c-4637-8b05-f053cbc993da None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:30.333256 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7084f1d2-411c-4637-8b05-f053cbc993da None admin] PATH_INFO: `/v3/projects/c6f995ffa0a94b1584f25dbe31e20524/users/1623987d87ab4cf19b923d254a68c2c6/roles/82029ff85919459780b85031cdfc369f` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:30.336956 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7084f1d2-411c-4637-8b05-f053cbc993da None admin] RBAC: Authorizing `identity:create_grant(project_id=c6f995ffa0a94b1584f25dbe31e20524, user_id=1623987d87ab4cf19b923d254a68c2c6, role_id=82029ff85919459780b85031cdfc369f)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:30.338919 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7084f1d2-411c-4637-8b05-f053cbc993da None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:30.366926 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 472/954] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 14:11:30 2022] PUT /identity/v3/projects/c6f995ffa0a94b1584f25dbe31e20524/users/1623987d87ab4cf19b923d254a68c2c6/roles/82029ff85919459780b85031cdfc369f => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:11:30.437149 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:30.443257 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f455fc-ed81-46bf-8b5f-896f8cccbef5 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:30.482287 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455c9284-6bf5-4b72-9865-b756a71f015b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:30.482953 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455c9284-6bf5-4b72-9865-b756a71f015b None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:30.483242 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455c9284-6bf5-4b72-9865-b756a71f015b None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:30.483521 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455c9284-6bf5-4b72-9865-b756a71f015b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:30.484278 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455c9284-6bf5-4b72-9865-b756a71f015b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:30.486553 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455c9284-6bf5-4b72-9865-b756a71f015b None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:30.518070 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 483/955] 10.222.0.57 () {60 vars in 1144 bytes} [Thu May 19 14:11:30 2022] POST /identity/v3/endpoints => generated 349 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:11:30.749934 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b578e9-3c47-4253-a8f0-53a64567c5dc None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:30.750501 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b578e9-3c47-4253-a8f0-53a64567c5dc None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:30.750581 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b578e9-3c47-4253-a8f0-53a64567c5dc None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:30.752238 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 473/956] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:11:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:11:30.759751 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dcb9c7-97a4-49c8-9384-2396eb3cd0d4 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:30.759751 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dcb9c7-97a4-49c8-9384-2396eb3cd0d4 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:30.759751 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dcb9c7-97a4-49c8-9384-2396eb3cd0d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:30.776965 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-84dcb9c7-97a4-49c8-9384-2396eb3cd0d4 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:11:30.778201 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-84dcb9c7-97a4-49c8-9384-2396eb3cd0d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:11:30.855871 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 484/957] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:11:30 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:11:30.879730 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b578e9-3c47-4253-a8f0-53a64567c5dc None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:30.994579 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b578e9-3c47-4253-a8f0-53a64567c5dc None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:31.032232 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd00fe0-c1ed-468e-8f70-e5d4dffc9a63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:31.033113 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd00fe0-c1ed-468e-8f70-e5d4dffc9a63 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:31.033406 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd00fe0-c1ed-468e-8f70-e5d4dffc9a63 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:31.033685 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd00fe0-c1ed-468e-8f70-e5d4dffc9a63 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:31.034379 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd00fe0-c1ed-468e-8f70-e5d4dffc9a63 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:31.035756 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd00fe0-c1ed-468e-8f70-e5d4dffc9a63 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:31.048810 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-edd00fe0-c1ed-468e-8f70-e5d4dffc9a63 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 19 14:11:31.048810 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:31.048810 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:11:31.048810 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:11:31.048810 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:11:31.048810 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:11:31.048810 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:11:31.048810 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:11:31.048810 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 14:11:31.048810 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:11:31.048810 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:11:31.048810 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:11:31.048810 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:31.048810 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:11:31.048810 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:11:31.048810 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:11:31.048810 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:11:31.050005 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:11:31.050005 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:11:31.050005 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:11:31.050005 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:11:31.050005 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:11:31.050005 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:11:31.050005 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:11:31.050005 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:11:31.050005 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:11:31.050005 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:11:31.050005 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:11:31.050005 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:11:31.050005 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:11:31.050005 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:11:31.050005 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:11:31.050005 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:11:31.051909 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:11:31.051909 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:11:31.051909 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:11:31.051909 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:11:31.051909 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:11:31.051909 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:11:31.051909 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:11:31.051909 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:11:31.051909 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:11:31.051909 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:11:31.051909 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:11:31.051909 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:11:31.051909 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 14:11:31.051909 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:11:31.051909 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 474/958] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:11:30 2022] GET /identity/v3/services/compute => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:11:31.058836 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dcb9c7-97a4-49c8-9384-2396eb3cd0d4 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:31.149129 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:31.169306 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dcb9c7-97a4-49c8-9384-2396eb3cd0d4 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:31.206148 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cc03ab-684c-4079-abf2-6acdcc64b91e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:31.206822 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cc03ab-684c-4079-abf2-6acdcc64b91e None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:31.207105 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cc03ab-684c-4079-abf2-6acdcc64b91e None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:31.207381 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cc03ab-684c-4079-abf2-6acdcc64b91e None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:31.208084 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cc03ab-684c-4079-abf2-6acdcc64b91e None admin] RBAC: Adding query filter params (name=compute) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:31.208393 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cc03ab-684c-4079-abf2-6acdcc64b91e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:31.209772 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cc03ab-684c-4079-abf2-6acdcc64b91e None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:31.215082 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 485/959] 10.222.0.57 () {56 vars in 1112 bytes} [Thu May 19 14:11:31 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:31.220424 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd00fe0-c1ed-468e-8f70-e5d4dffc9a63 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:31.250815 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:31.334299 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd00fe0-c1ed-468e-8f70-e5d4dffc9a63 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:31.369464 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d9f397-fe1c-4bce-9581-24e8eeb1d850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:31.370168 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d9f397-fe1c-4bce-9581-24e8eeb1d850 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:31.370454 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d9f397-fe1c-4bce-9581-24e8eeb1d850 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:31.370730 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d9f397-fe1c-4bce-9581-24e8eeb1d850 None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:31.371447 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d9f397-fe1c-4bce-9581-24e8eeb1d850 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:31.371757 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d9f397-fe1c-4bce-9581-24e8eeb1d850 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:31.373128 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d9f397-fe1c-4bce-9581-24e8eeb1d850 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:31.379015 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 475/960] 10.222.0.57 () {56 vars in 1112 bytes} [Thu May 19 14:11:31 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:32.096405 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f74f7fb-7726-43b1-a2c3-67c5bf467f0c None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:32.096924 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f74f7fb-7726-43b1-a2c3-67c5bf467f0c None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:32.096924 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f74f7fb-7726-43b1-a2c3-67c5bf467f0c None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:32.098584 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 486/961] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:11:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:11:32.104010 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26233965-a128-4e3b-8ce2-ce15e257cdd0 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:32.104409 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26233965-a128-4e3b-8ce2-ce15e257cdd0 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:32.104613 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26233965-a128-4e3b-8ce2-ce15e257cdd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:32.128479 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-26233965-a128-4e3b-8ce2-ce15e257cdd0 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:11:32.129701 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-26233965-a128-4e3b-8ce2-ce15e257cdd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:11:32.201946 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 476/962] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:11:32 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:11:32.227701 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f74f7fb-7726-43b1-a2c3-67c5bf467f0c None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:32.336953 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f74f7fb-7726-43b1-a2c3-67c5bf467f0c None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:32.373633 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9683da-d5e3-4a55-8770-2e003fbed7b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:32.374361 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9683da-d5e3-4a55-8770-2e003fbed7b5 None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:32.374649 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9683da-d5e3-4a55-8770-2e003fbed7b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:32.374914 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9683da-d5e3-4a55-8770-2e003fbed7b5 None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:32.375923 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9683da-d5e3-4a55-8770-2e003fbed7b5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:32.377365 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9683da-d5e3-4a55-8770-2e003fbed7b5 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:32.391731 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.sql.core [None req-8b9683da-d5e3-4a55-8770-2e003fbed7b5 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 19 14:11:32.391731 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 19 14:11:32.391731 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [parameters: {'user_id': '01410afceaa94b44a8c566882431f34f', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 19 14:11:32.391731 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80203) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 19 14:11:32.399312 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-8b9683da-d5e3-4a55-8770-2e003fbed7b5 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 19 14:11:32.399312 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:32.399312 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 14:11:32.399312 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 14:11:32.399312 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 14:11:32.399312 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 14:11:32.399312 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 14:11:32.399312 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application result = self._query(query) May 19 14:11:32.399312 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 14:11:32.399312 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application conn.query(q) May 19 14:11:32.399312 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 14:11:32.399312 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 14:11:32.399312 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 14:11:32.399312 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application result.read() May 19 14:11:32.399312 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 14:11:32.400203 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 14:11:32.400203 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 14:11:32.400203 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 14:11:32.400203 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 14:11:32.400203 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 14:11:32.400203 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 14:11:32.400203 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 14:11:32.400203 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 19 14:11:32.400203 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:11:32.400203 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 19 14:11:32.400203 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:11:32.400203 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:32.400203 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 19 14:11:32.400203 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 19 14:11:32.400203 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user May 19 14:11:32.400203 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) May 19 14:11:32.400203 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 14:11:32.400203 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application next(self.gen) May 19 14:11:32.401646 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 19 14:11:32.401646 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application yield resource May 19 14:11:32.401646 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 14:11:32.401646 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application next(self.gen) May 19 14:11:32.401646 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 19 14:11:32.401646 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self.session.rollback() May 19 14:11:32.401646 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 19 14:11:32.401646 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self.force_reraise() May 19 14:11:32.401646 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 19 14:11:32.401646 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise self.value May 19 14:11:32.401646 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 19 14:11:32.401646 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 19 14:11:32.401646 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 19 14:11:32.401646 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application session.commit() May 19 14:11:32.401646 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 19 14:11:32.401646 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 19 14:11:32.401646 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 19 14:11:32.403012 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._prepare_impl() May 19 14:11:32.403012 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 19 14:11:32.403012 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self.session.flush() May 19 14:11:32.403012 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 19 14:11:32.403012 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._flush(objects) May 19 14:11:32.403012 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 19 14:11:32.403012 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 19 14:11:32.403012 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 19 14:11:32.403012 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application compat.raise_( May 19 14:11:32.403012 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 14:11:32.403012 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception May 19 14:11:32.403012 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 19 14:11:32.403012 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application flush_context.execute() May 19 14:11:32.403012 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 19 14:11:32.403012 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rec.execute(self) May 19 14:11:32.403012 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 19 14:11:32.403012 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 19 14:11:32.403928 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 19 14:11:32.403928 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application _emit_insert_statements( May 19 14:11:32.403928 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements May 19 14:11:32.403928 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application result = connection._execute_20( May 19 14:11:32.403928 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 19 14:11:32.403928 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 19 14:11:32.403928 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 19 14:11:32.403928 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 19 14:11:32.403928 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 19 14:11:32.403928 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ret = self._execute_context( May 19 14:11:32.403928 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 19 14:11:32.403928 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 19 14:11:32.403928 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 19 14:11:32.403928 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 19 14:11:32.403928 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 14:11:32.403928 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception May 19 14:11:32.404796 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 14:11:32.404796 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 14:11:32.404796 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 14:11:32.404796 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 14:11:32.404796 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 14:11:32.404796 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application result = self._query(query) May 19 14:11:32.404796 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 14:11:32.404796 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application conn.query(q) May 19 14:11:32.404796 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 14:11:32.404796 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 14:11:32.404796 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 14:11:32.404796 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application result.read() May 19 14:11:32.404796 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 14:11:32.404796 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 14:11:32.404796 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 14:11:32.404796 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 14:11:32.406966 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 14:11:32.406966 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 14:11:32.406966 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 14:11:32.406966 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 14:11:32.406966 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 19 14:11:32.406966 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 19 14:11:32.406966 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [parameters: {'user_id': '01410afceaa94b44a8c566882431f34f', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 19 14:11:32.406966 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 19 14:11:32.406966 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:11:32.406966 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:11:32.406966 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:11:32.406966 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:32.406966 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:11:32.406966 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:11:32.406966 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:11:32.407938 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:11:32.407938 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:11:32.407938 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:11:32.407938 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:11:32.407938 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:11:32.407938 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:11:32.407938 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:11:32.407938 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post May 19 14:11:32.407938 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( May 19 14:11:32.407938 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:11:32.407938 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:11:32.407938 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:11:32.407938 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:11:32.407938 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:11:32.407938 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:11:32.407938 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user May 19 14:11:32.407938 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) May 19 14:11:32.409060 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects May 19 14:11:32.409060 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) May 19 14:11:32.409060 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 19 14:11:32.409060 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 19 14:11:32.409060 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 19 14:11:32.409060 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:11:32.409060 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 487/963] 10.222.0.57 () {60 vars in 1131 bytes} [Thu May 19 14:11:32 2022] POST /identity/v3/users => generated 160 bytes in 174 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 19 14:11:32.409534 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26233965-a128-4e3b-8ce2-ce15e257cdd0 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:32.436665 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:32.523554 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26233965-a128-4e3b-8ce2-ce15e257cdd0 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:32.562472 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882f2f03-3083-402b-b427-ae5416b60e9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:32.563612 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882f2f03-3083-402b-b427-ae5416b60e9f None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:32.563895 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882f2f03-3083-402b-b427-ae5416b60e9f None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:32.564168 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882f2f03-3083-402b-b427-ae5416b60e9f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:32.571933 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882f2f03-3083-402b-b427-ae5416b60e9f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:32.573373 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882f2f03-3083-402b-b427-ae5416b60e9f None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:32.589099 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-882f2f03-3083-402b-b427-ae5416b60e9f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 14:11:32.589099 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:32.589099 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:11:32.589099 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:11:32.589099 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:11:32.589099 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:11:32.589099 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:11:32.589099 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:11:32.589099 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:11:32.589099 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:11:32.589099 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:11:32.589099 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:11:32.589099 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:11:32.589099 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:11:32.589099 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:11:32.589099 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:11:32.591447 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:11:32.591447 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:11:32.591447 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:11:32.591447 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:11:32.591447 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:11:32.591447 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:11:32.591447 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:11:32.591447 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:11:32.591447 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:11:32.591447 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:11:32.591447 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:11:32.591447 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:11:32.591447 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:11:32.591447 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:11:32.591447 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:11:32.591447 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:11:32.591447 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:11:32.591447 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:11:32.594521 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:11:32.594521 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:11:32.594521 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:11:32.594521 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:11:32.594521 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:11:32.594521 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:11:32.594521 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:11:32.594521 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:11:32.594521 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 14:11:32.594521 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:11:32.594521 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 477/964] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:11:32 2022] GET /identity/v3/users/cinder => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:11:32.600747 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9683da-d5e3-4a55-8770-2e003fbed7b5 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:32.611878 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:32.720551 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9683da-d5e3-4a55-8770-2e003fbed7b5 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:32.757124 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c876cdb-45fd-4832-8d24-6d1193a858bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:32.758141 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c876cdb-45fd-4832-8d24-6d1193a858bc None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:32.758632 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c876cdb-45fd-4832-8d24-6d1193a858bc None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:32.759107 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c876cdb-45fd-4832-8d24-6d1193a858bc None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:32.760092 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c876cdb-45fd-4832-8d24-6d1193a858bc None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:32.760579 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c876cdb-45fd-4832-8d24-6d1193a858bc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:32.762436 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c876cdb-45fd-4832-8d24-6d1193a858bc None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:32.802682 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 488/965] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:11:32 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:32.839495 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb9e73b-004b-4d45-a1b7-9a272320d7d2 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:32.840169 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb9e73b-004b-4d45-a1b7-9a272320d7d2 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:32.840657 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb9e73b-004b-4d45-a1b7-9a272320d7d2 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:32.842660 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 478/966] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:11:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:11:32.849163 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd66276-7ff2-4d6c-958a-2f806f44fa6b None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:32.849789 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd66276-7ff2-4d6c-958a-2f806f44fa6b None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:32.850332 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd66276-7ff2-4d6c-958a-2f806f44fa6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:32.870545 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-1bd66276-7ff2-4d6c-958a-2f806f44fa6b None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:11:32.872550 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-1bd66276-7ff2-4d6c-958a-2f806f44fa6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:11:32.879404 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e707d4c5-7a5e-4628-8b57-aa88a5de7b5f None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:32.880033 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e707d4c5-7a5e-4628-8b57-aa88a5de7b5f None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:32.880541 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e707d4c5-7a5e-4628-8b57-aa88a5de7b5f None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:32.882447 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 479/967] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:11:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:11:32.892178 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249a6cde-8339-4ddc-b569-5ae159b5ca04 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:32.892811 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249a6cde-8339-4ddc-b569-5ae159b5ca04 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:32.893302 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249a6cde-8339-4ddc-b569-5ae159b5ca04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:32.911688 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-249a6cde-8339-4ddc-b569-5ae159b5ca04 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:11:32.913270 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-249a6cde-8339-4ddc-b569-5ae159b5ca04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:11:32.949341 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:32.961894 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 489/968] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:11:32 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:11:32.989985 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd66276-7ff2-4d6c-958a-2f806f44fa6b None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:33.023767 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 480/969] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:11:32 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:11:33.048336 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249a6cde-8339-4ddc-b569-5ae159b5ca04 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:33.060112 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:33.118249 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd66276-7ff2-4d6c-958a-2f806f44fa6b None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:33.159906 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df586646-0d09-4547-8e8e-39ad2b73c25c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:33.161108 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df586646-0d09-4547-8e8e-39ad2b73c25c None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:33.161664 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df586646-0d09-4547-8e8e-39ad2b73c25c None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:33.162317 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df586646-0d09-4547-8e8e-39ad2b73c25c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:33.172447 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249a6cde-8339-4ddc-b569-5ae159b5ca04 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:33.176289 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df586646-0d09-4547-8e8e-39ad2b73c25c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:11:33.184582 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-df586646-0d09-4547-8e8e-39ad2b73c25c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 14:11:33.184582 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:33.184582 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:11:33.184582 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:11:33.184582 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:11:33.184582 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:11:33.184582 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:11:33.184582 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:11:33.184582 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:11:33.184582 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:11:33.184582 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:11:33.184582 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:11:33.184582 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:11:33.184582 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:11:33.184582 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:11:33.184582 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:11:33.185546 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:11:33.185546 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:11:33.185546 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:11:33.185546 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:11:33.185546 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:11:33.185546 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:11:33.185546 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:11:33.185546 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:11:33.185546 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:11:33.185546 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:11:33.185546 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:11:33.185546 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:11:33.185546 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:11:33.185546 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:11:33.185546 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:11:33.185546 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:11:33.186480 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:11:33.186480 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:11:33.186480 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:11:33.186480 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:11:33.186480 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:11:33.186480 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:11:33.186480 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:11:33.186480 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:11:33.186480 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:11:33.186480 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:11:33.186480 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 14:11:33.186480 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:11:33.186480 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 490/970] 10.222.0.57 () {56 vars in 1120 bytes} [Thu May 19 14:11:32 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:11:33.192426 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df586646-0d09-4547-8e8e-39ad2b73c25c None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:33.213733 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec945a1-1606-458b-ac2a-bbd40b23fcdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:33.214607 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec945a1-1606-458b-ac2a-bbd40b23fcdf None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:33.214896 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec945a1-1606-458b-ac2a-bbd40b23fcdf None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:33.215176 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec945a1-1606-458b-ac2a-bbd40b23fcdf None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:33.215843 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec945a1-1606-458b-ac2a-bbd40b23fcdf None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:33.217216 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec945a1-1606-458b-ac2a-bbd40b23fcdf None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:33.230100 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-bec945a1-1606-458b-ac2a-bbd40b23fcdf None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 14:11:33.230100 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:33.230100 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:11:33.230100 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:11:33.230100 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:11:33.230100 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:11:33.230100 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:11:33.230100 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:11:33.230100 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 14:11:33.230100 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:11:33.230100 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:11:33.230100 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:11:33.230100 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:33.230100 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:11:33.230100 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:11:33.230100 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:11:33.230100 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:11:33.231114 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:11:33.231114 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:11:33.231114 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:11:33.231114 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:11:33.231114 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:11:33.231114 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:11:33.231114 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:11:33.231114 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:11:33.231114 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:11:33.231114 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:11:33.231114 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:11:33.231114 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:11:33.231114 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:11:33.231114 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:11:33.231114 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:11:33.231114 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:11:33.231968 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:11:33.231968 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:11:33.231968 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:11:33.231968 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:11:33.231968 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:11:33.231968 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:11:33.231968 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:11:33.231968 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:11:33.231968 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:11:33.231968 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:11:33.231968 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:11:33.231968 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:11:33.231968 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 14:11:33.231968 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:11:33.231968 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 481/971] 10.222.0.57 () {56 vars in 1126 bytes} [Thu May 19 14:11:33 2022] GET /identity/v3/services/object-store => generated 93 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:11:33.238311 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec945a1-1606-458b-ac2a-bbd40b23fcdf None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:33.260255 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:33.314840 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df586646-0d09-4547-8e8e-39ad2b73c25c None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:33.317946 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:33.344613 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec945a1-1606-458b-ac2a-bbd40b23fcdf None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:33.360151 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d3baf3-b640-428f-9433-c34f6393ef11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:33.361195 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d3baf3-b640-428f-9433-c34f6393ef11 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:33.361658 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d3baf3-b640-428f-9433-c34f6393ef11 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:33.361979 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d3baf3-b640-428f-9433-c34f6393ef11 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:33.362733 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d3baf3-b640-428f-9433-c34f6393ef11 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:33.363305 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d3baf3-b640-428f-9433-c34f6393ef11 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:33.364686 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d3baf3-b640-428f-9433-c34f6393ef11 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:33.373666 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 491/972] 10.222.0.57 () {56 vars in 1115 bytes} [Thu May 19 14:11:33 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:11:33.379807 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d3baf3-b640-428f-9433-c34f6393ef11 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:33.382446 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d318ab6-cfb1-43a4-bb22-2486460cc030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:33.383104 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d318ab6-cfb1-43a4-bb22-2486460cc030 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:33.383390 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d318ab6-cfb1-43a4-bb22-2486460cc030 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:33.383674 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d318ab6-cfb1-43a4-bb22-2486460cc030 None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:33.384374 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d318ab6-cfb1-43a4-bb22-2486460cc030 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:33.384839 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d318ab6-cfb1-43a4-bb22-2486460cc030 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:33.386380 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d318ab6-cfb1-43a4-bb22-2486460cc030 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:33.391534 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 482/973] 10.222.0.57 () {56 vars in 1122 bytes} [Thu May 19 14:11:33 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:11:33.396910 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d318ab6-cfb1-43a4-bb22-2486460cc030 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:33.496709 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d3baf3-b640-428f-9433-c34f6393ef11 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:33.502513 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d318ab6-cfb1-43a4-bb22-2486460cc030 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:33.519372 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:33.541273 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eec2e6f-1d4b-4f32-8207-e324b49fa062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:33.542101 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eec2e6f-1d4b-4f32-8207-e324b49fa062 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:33.542433 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eec2e6f-1d4b-4f32-8207-e324b49fa062 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:33.542712 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eec2e6f-1d4b-4f32-8207-e324b49fa062 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:33.543856 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e5b1ae-db25-4077-80fd-ccbbdb09f502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:33.544517 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e5b1ae-db25-4077-80fd-ccbbdb09f502 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:33.544977 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e5b1ae-db25-4077-80fd-ccbbdb09f502 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:33.545257 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e5b1ae-db25-4077-80fd-ccbbdb09f502 None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:33.546013 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e5b1ae-db25-4077-80fd-ccbbdb09f502 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:33.546326 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e5b1ae-db25-4077-80fd-ccbbdb09f502 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:33.548257 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e5b1ae-db25-4077-80fd-ccbbdb09f502 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:33.558129 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 483/974] 10.222.0.57 () {56 vars in 1122 bytes} [Thu May 19 14:11:33 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:33.558297 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eec2e6f-1d4b-4f32-8207-e324b49fa062 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:33.558297 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eec2e6f-1d4b-4f32-8207-e324b49fa062 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:33.559720 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e5b1ae-db25-4077-80fd-ccbbdb09f502 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:33.563266 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:33.576489 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-4eec2e6f-1d4b-4f32-8207-e324b49fa062 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:11:33.576489 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:33.576489 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:11:33.576489 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:11:33.576489 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:11:33.576489 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:11:33.576489 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:11:33.576489 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:11:33.576489 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:11:33.576489 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:11:33.576489 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:11:33.576489 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:11:33.576489 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:11:33.576489 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:11:33.576489 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:11:33.577409 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:11:33.577409 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:11:33.577409 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:11:33.577409 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:11:33.577409 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:11:33.577409 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:11:33.577409 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:11:33.577409 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:11:33.577409 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:11:33.577409 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:11:33.577409 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:11:33.577409 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:11:33.577409 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:11:33.577409 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:11:33.577409 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:11:33.577409 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:11:33.577409 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:11:33.579172 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:11:33.579172 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:11:33.579172 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:11:33.579172 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:11:33.579172 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 492/975] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:11:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:11:33.583641 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eec2e6f-1d4b-4f32-8207-e324b49fa062 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:33.682886 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e5b1ae-db25-4077-80fd-ccbbdb09f502 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:33.694529 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eec2e6f-1d4b-4f32-8207-e324b49fa062 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:33.731227 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9203cb02-71f2-4614-abc2-2609b872e251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:33.731891 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9203cb02-71f2-4614-abc2-2609b872e251 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:33.732175 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9203cb02-71f2-4614-abc2-2609b872e251 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:33.732296 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707d6570-ba1b-4b11-b0e2-3cdf4aec8034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:33.732452 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9203cb02-71f2-4614-abc2-2609b872e251 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:33.732938 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707d6570-ba1b-4b11-b0e2-3cdf4aec8034 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:33.733170 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9203cb02-71f2-4614-abc2-2609b872e251 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:33.733278 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707d6570-ba1b-4b11-b0e2-3cdf4aec8034 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:33.733475 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707d6570-ba1b-4b11-b0e2-3cdf4aec8034 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:33.733576 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9203cb02-71f2-4614-abc2-2609b872e251 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:33.734316 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707d6570-ba1b-4b11-b0e2-3cdf4aec8034 None admin] RBAC: Adding query filter params (interface=admin, service_id=37c91f0d9e274221b0c859342f5ed13e, region_id=RegionOne) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:33.735489 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707d6570-ba1b-4b11-b0e2-3cdf4aec8034 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:33.735594 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9203cb02-71f2-4614-abc2-2609b872e251 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:33.736935 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707d6570-ba1b-4b11-b0e2-3cdf4aec8034 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:33.742203 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 484/976] 10.222.0.57 () {56 vars in 1249 bytes} [Thu May 19 14:11:33 2022] GET /identity/v3/endpoints?service_id=37c91f0d9e274221b0c859342f5ed13e&interface=admin®ion_id=RegionOne => generated 193 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:11:33.743404 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd881bd2-70a0-4c3b-b272-c1ace21c7f8f None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:33.743684 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd881bd2-70a0-4c3b-b272-c1ace21c7f8f None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:33.743953 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd881bd2-70a0-4c3b-b272-c1ace21c7f8f None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:33.745151 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 485/977] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:11:33 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:11:33.748179 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd881bd2-70a0-4c3b-b272-c1ace21c7f8f None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:33.752565 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 493/978] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:11:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:11:33.753859 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fe2e35-a38d-495b-9886-0b907d05a9b4 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:33.754150 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fe2e35-a38d-495b-9886-0b907d05a9b4 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:33.754426 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fe2e35-a38d-495b-9886-0b907d05a9b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:33.770122 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-f2fe2e35-a38d-495b-9886-0b907d05a9b4 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:11:33.771236 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-f2fe2e35-a38d-495b-9886-0b907d05a9b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:11:33.794695 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:33.819469 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:33.841499 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 494/979] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:11:33 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:11:33.842275 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fe2e35-a38d-495b-9886-0b907d05a9b4 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:33.856236 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd881bd2-70a0-4c3b-b272-c1ace21c7f8f None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:33.891402 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8a957e-d971-4dee-93b7-2522d0a484a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:33.892059 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8a957e-d971-4dee-93b7-2522d0a484a1 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:33.892340 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8a957e-d971-4dee-93b7-2522d0a484a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:33.892619 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8a957e-d971-4dee-93b7-2522d0a484a1 None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:33.893294 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8a957e-d971-4dee-93b7-2522d0a484a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:33.894696 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8a957e-d971-4dee-93b7-2522d0a484a1 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:33.900774 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 486/980] 10.222.0.57 () {56 vars in 1087 bytes} [Thu May 19 14:11:33 2022] GET /identity/v3/services => generated 1497 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 14:11:33.901535 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8a957e-d971-4dee-93b7-2522d0a484a1 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:33.951530 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fe2e35-a38d-495b-9886-0b907d05a9b4 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:33.993333 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1401ab0-1083-40ad-8a61-af0f066e41ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:33.994202 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1401ab0-1083-40ad-8a61-af0f066e41ee None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:33.994484 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1401ab0-1083-40ad-8a61-af0f066e41ee None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:33.994770 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1401ab0-1083-40ad-8a61-af0f066e41ee None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:34.002893 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1401ab0-1083-40ad-8a61-af0f066e41ee None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:34.004383 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1401ab0-1083-40ad-8a61-af0f066e41ee None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:34.017208 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8a957e-d971-4dee-93b7-2522d0a484a1 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:34.021741 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-b1401ab0-1083-40ad-8a61-af0f066e41ee None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 14:11:34.021741 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:34.021741 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:11:34.021741 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:11:34.021741 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:11:34.021741 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:11:34.021741 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:11:34.021741 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:11:34.021741 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:11:34.021741 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:11:34.021741 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:11:34.021741 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:11:34.021741 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:11:34.021741 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:11:34.021741 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:11:34.021741 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:11:34.022754 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:11:34.022754 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:11:34.022754 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:11:34.022754 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:11:34.022754 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:11:34.022754 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:11:34.022754 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:11:34.022754 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:11:34.022754 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:11:34.022754 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:11:34.022754 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:11:34.022754 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:11:34.022754 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:11:34.022754 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:11:34.022754 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:11:34.022754 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:11:34.022754 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:11:34.022754 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:11:34.024372 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:11:34.024372 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:11:34.024372 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:11:34.024372 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:11:34.024372 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:11:34.024372 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:11:34.024372 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:11:34.024372 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:11:34.024372 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 14:11:34.024372 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:11:34.024372 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 495/981] 10.222.0.57 () {56 vars in 1117 bytes} [Thu May 19 14:11:33 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:11:34.028753 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1401ab0-1083-40ad-8a61-af0f066e41ee None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:34.049237 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:34.053984 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:34.061643 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67e4b9c-2bc6-4e2e-aded-699f8a58152c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:34.062429 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67e4b9c-2bc6-4e2e-aded-699f8a58152c None admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:34.062916 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67e4b9c-2bc6-4e2e-aded-699f8a58152c None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:34.063195 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67e4b9c-2bc6-4e2e-aded-699f8a58152c None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:34.063937 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67e4b9c-2bc6-4e2e-aded-699f8a58152c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:34.065639 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67e4b9c-2bc6-4e2e-aded-699f8a58152c None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:34.081007 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 487/982] 10.222.0.57 () {60 vars in 1141 bytes} [Thu May 19 14:11:33 2022] POST /identity/v3/services => generated 242 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:11:34.161623 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1401ab0-1083-40ad-8a61-af0f066e41ee None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:34.209446 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1b9873-60aa-4f0e-9c36-9b3554d31efe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:34.210500 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1b9873-60aa-4f0e-9c36-9b3554d31efe None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:34.210951 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1b9873-60aa-4f0e-9c36-9b3554d31efe None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:34.211418 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1b9873-60aa-4f0e-9c36-9b3554d31efe None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:34.212437 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1b9873-60aa-4f0e-9c36-9b3554d31efe None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:34.212914 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1b9873-60aa-4f0e-9c36-9b3554d31efe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:34.214565 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1b9873-60aa-4f0e-9c36-9b3554d31efe None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:34.235290 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 496/983] 10.222.0.57 () {56 vars in 1149 bytes} [Thu May 19 14:11:34 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:11:34.240989 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67e4b9c-2bc6-4e2e-aded-699f8a58152c None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:34.348869 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67e4b9c-2bc6-4e2e-aded-699f8a58152c None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:34.384291 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d022fc-f9f7-4d13-aaeb-e247bf4fbf95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:34.384945 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d022fc-f9f7-4d13-aaeb-e247bf4fbf95 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:34.385233 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d022fc-f9f7-4d13-aaeb-e247bf4fbf95 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:34.385499 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d022fc-f9f7-4d13-aaeb-e247bf4fbf95 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:34.446264 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:34.459240 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d022fc-f9f7-4d13-aaeb-e247bf4fbf95 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:34.460735 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d022fc-f9f7-4d13-aaeb-e247bf4fbf95 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:34.560735 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 488/984] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:11:34 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 19 14:11:34.566856 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1b9873-60aa-4f0e-9c36-9b3554d31efe None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:34.639280 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:34.670263 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1b9873-60aa-4f0e-9c36-9b3554d31efe None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:34.705516 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf57fb0-5ac6-4929-86b9-27d1154978bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:34.706355 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf57fb0-5ac6-4929-86b9-27d1154978bb None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:34.706642 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf57fb0-5ac6-4929-86b9-27d1154978bb None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:34.706915 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf57fb0-5ac6-4929-86b9-27d1154978bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:34.717132 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf57fb0-5ac6-4929-86b9-27d1154978bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:34.718615 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf57fb0-5ac6-4929-86b9-27d1154978bb None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:34.736799 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-abf57fb0-5ac6-4929-86b9-27d1154978bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:11:34.736799 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:34.736799 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:11:34.736799 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:11:34.736799 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:11:34.736799 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:11:34.736799 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:11:34.736799 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:11:34.736799 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:11:34.736799 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:11:34.736799 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:11:34.736799 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:11:34.736799 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:11:34.736799 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:11:34.736799 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:11:34.737765 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:11:34.737765 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:11:34.737765 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:11:34.737765 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:11:34.737765 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:11:34.737765 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:11:34.737765 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:11:34.737765 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:11:34.737765 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:11:34.737765 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:11:34.737765 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:11:34.737765 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:11:34.737765 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:11:34.737765 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:11:34.737765 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:11:34.737765 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:11:34.737765 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:11:34.739516 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:11:34.739516 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:11:34.739516 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:11:34.739516 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:11:34.742228 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 497/985] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:11:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:11:34.743792 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d022fc-f9f7-4d13-aaeb-e247bf4fbf95 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:34.835395 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:34.846509 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d022fc-f9f7-4d13-aaeb-e247bf4fbf95 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:34.883351 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7adc9a-d08a-47c8-9055-4e21758ae0d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:34.884009 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7adc9a-d08a-47c8-9055-4e21758ae0d7 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:34.884291 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7adc9a-d08a-47c8-9055-4e21758ae0d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:34.884564 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7adc9a-d08a-47c8-9055-4e21758ae0d7 None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:34.885274 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7adc9a-d08a-47c8-9055-4e21758ae0d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:34.885578 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7adc9a-d08a-47c8-9055-4e21758ae0d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:34.886970 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7adc9a-d08a-47c8-9055-4e21758ae0d7 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:34.898609 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 489/986] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:11:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:34.903829 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf57fb0-5ac6-4929-86b9-27d1154978bb None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:35.006959 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf57fb0-5ac6-4929-86b9-27d1154978bb None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:35.030443 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:35.042665 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acec314-dd17-4f87-af65-c28e8f3dced2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:35.043464 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acec314-dd17-4f87-af65-c28e8f3dced2 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:35.043751 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acec314-dd17-4f87-af65-c28e8f3dced2 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:35.044027 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acec314-dd17-4f87-af65-c28e8f3dced2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:35.051312 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acec314-dd17-4f87-af65-c28e8f3dced2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:35.052791 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acec314-dd17-4f87-af65-c28e8f3dced2 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:35.068031 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-7acec314-dd17-4f87-af65-c28e8f3dced2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:11:35.068031 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:35.068031 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:11:35.068031 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:11:35.068031 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:11:35.068031 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:11:35.068031 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:11:35.068031 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:11:35.068031 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:11:35.068031 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:11:35.068031 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:11:35.068031 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:11:35.068031 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:11:35.068031 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:11:35.068031 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:11:35.069034 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:11:35.069034 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:11:35.069034 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:11:35.069034 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:11:35.069034 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:11:35.069034 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:11:35.069034 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:11:35.069034 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:11:35.069034 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:11:35.069034 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:11:35.069034 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:11:35.069034 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:11:35.069034 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:11:35.069034 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:11:35.069034 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:11:35.069034 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:11:35.069034 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:11:35.070015 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:11:35.070015 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:11:35.070015 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:11:35.070015 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:11:35.070015 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:11:35.070015 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:11:35.070015 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:11:35.070015 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:11:35.075566 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 498/987] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:11:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:11:35.075707 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7adc9a-d08a-47c8-9055-4e21758ae0d7 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:35.185034 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7adc9a-d08a-47c8-9055-4e21758ae0d7 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:35.220406 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d00ae2-a796-42eb-8e22-da0b372d10e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:35.221070 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d00ae2-a796-42eb-8e22-da0b372d10e8 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:35.221350 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d00ae2-a796-42eb-8e22-da0b372d10e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:35.221625 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d00ae2-a796-42eb-8e22-da0b372d10e8 None admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:35.222438 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d00ae2-a796-42eb-8e22-da0b372d10e8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:35.222750 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d00ae2-a796-42eb-8e22-da0b372d10e8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:35.224208 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d00ae2-a796-42eb-8e22-da0b372d10e8 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:35.239093 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 490/988] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:11:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:35.244745 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acec314-dd17-4f87-af65-c28e8f3dced2 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:35.346466 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acec314-dd17-4f87-af65-c28e8f3dced2 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:35.379680 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f130be-7849-47da-971f-cee644d63976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:35.380335 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f130be-7849-47da-971f-cee644d63976 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:35.380619 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f130be-7849-47da-971f-cee644d63976 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:35.380896 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f130be-7849-47da-971f-cee644d63976 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:35.381722 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f130be-7849-47da-971f-cee644d63976 None admin] RBAC: Adding query filter params (role.id=82029ff85919459780b85031cdfc369f, scope.project.id=c6f995ffa0a94b1584f25dbe31e20524, user.id=1623987d87ab4cf19b923d254a68c2c6) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:35.382062 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f130be-7849-47da-971f-cee644d63976 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:35.383496 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f130be-7849-47da-971f-cee644d63976 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:35.397880 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 499/989] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:11:35 2022] GET /identity/v3/role_assignments?user.id=1623987d87ab4cf19b923d254a68c2c6&scope.project.id=c6f995ffa0a94b1584f25dbe31e20524&role.id=82029ff85919459780b85031cdfc369f => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:36.188093 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceda275-a007-4f53-9bd8-642407cdef1b None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:36.188654 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceda275-a007-4f53-9bd8-642407cdef1b None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:36.188654 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceda275-a007-4f53-9bd8-642407cdef1b None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:36.190109 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 491/990] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:11:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:11:36.195822 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f86de7-1634-408f-a8a6-98ae465dee73 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:36.196119 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f86de7-1634-408f-a8a6-98ae465dee73 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:36.196406 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f86de7-1634-408f-a8a6-98ae465dee73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:36.213229 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-87f86de7-1634-408f-a8a6-98ae465dee73 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:11:36.214444 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-87f86de7-1634-408f-a8a6-98ae465dee73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:11:36.261432 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:36.287420 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 500/991] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:11:36 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:11:36.310451 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eceda275-a007-4f53-9bd8-642407cdef1b None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:36.323700 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:36.352329 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f79bcee-e3d6-44bc-931c-62d44e5749de None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:36.352880 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f79bcee-e3d6-44bc-931c-62d44e5749de None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:36.353165 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f79bcee-e3d6-44bc-931c-62d44e5749de None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:36.355305 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 501/992] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:11:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:11:36.360383 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7401fbf7-73fc-4245-8877-72165958d157 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:36.360620 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7401fbf7-73fc-4245-8877-72165958d157 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:36.360882 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7401fbf7-73fc-4245-8877-72165958d157 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:36.377807 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-7401fbf7-73fc-4245-8877-72165958d157 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:11:36.379015 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-7401fbf7-73fc-4245-8877-72165958d157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:11:36.425391 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eceda275-a007-4f53-9bd8-642407cdef1b None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:36.450647 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 502/993] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:11:36 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:11:36.461311 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cbb6e8-189d-4962-bbcf-87946a5f06bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:36.462142 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cbb6e8-189d-4962-bbcf-87946a5f06bd None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:36.462427 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cbb6e8-189d-4962-bbcf-87946a5f06bd None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:36.462701 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cbb6e8-189d-4962-bbcf-87946a5f06bd None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:36.463352 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cbb6e8-189d-4962-bbcf-87946a5f06bd None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:36.464709 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cbb6e8-189d-4962-bbcf-87946a5f06bd None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:36.473565 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7401fbf7-73fc-4245-8877-72165958d157 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:36.477214 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-68cbb6e8-189d-4962-bbcf-87946a5f06bd None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 14:11:36.477214 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:36.477214 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:11:36.477214 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:11:36.477214 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:11:36.477214 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:11:36.477214 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:11:36.477214 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:11:36.477214 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 14:11:36.477214 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:11:36.477214 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:11:36.477214 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:11:36.477214 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:36.477214 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:11:36.477214 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:11:36.477214 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:11:36.477214 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:11:36.478420 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:11:36.478420 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:11:36.478420 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:11:36.478420 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:11:36.478420 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:11:36.478420 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:11:36.478420 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:11:36.478420 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:11:36.478420 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:11:36.478420 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:11:36.478420 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:11:36.478420 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:11:36.478420 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:11:36.478420 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:11:36.478420 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:11:36.478420 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:11:36.479311 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:11:36.479311 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:11:36.479311 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:11:36.479311 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:11:36.479311 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:11:36.479311 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:11:36.479311 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:11:36.479311 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:11:36.479311 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:11:36.479311 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:11:36.479311 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:11:36.479311 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:11:36.479311 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 14:11:36.479311 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:11:36.479311 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 492/994] 10.222.0.57 () {56 vars in 1126 bytes} [Thu May 19 14:11:36 2022] GET /identity/v3/services/object-store => generated 93 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:11:36.485072 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cbb6e8-189d-4962-bbcf-87946a5f06bd None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:36.570092 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:36.589698 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7401fbf7-73fc-4245-8877-72165958d157 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:36.598161 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cbb6e8-189d-4962-bbcf-87946a5f06bd None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:36.600084 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:36.629823 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160ab336-adbe-460f-8190-8893ec8009da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:36.630652 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160ab336-adbe-460f-8190-8893ec8009da None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:36.630934 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160ab336-adbe-460f-8190-8893ec8009da None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:36.631203 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160ab336-adbe-460f-8190-8893ec8009da None admin] PATH_INFO: `/v3/services/compute` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:36.631856 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160ab336-adbe-460f-8190-8893ec8009da None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:36.633242 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160ab336-adbe-460f-8190-8893ec8009da None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:36.634705 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dad75f-fd03-4ea2-933a-83b28d3c6caf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:36.635376 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dad75f-fd03-4ea2-933a-83b28d3c6caf None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:36.635665 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dad75f-fd03-4ea2-933a-83b28d3c6caf None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:36.635951 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dad75f-fd03-4ea2-933a-83b28d3c6caf None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:36.636671 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dad75f-fd03-4ea2-933a-83b28d3c6caf None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:36.636985 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dad75f-fd03-4ea2-933a-83b28d3c6caf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:36.638408 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dad75f-fd03-4ea2-933a-83b28d3c6caf None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:36.643844 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 493/995] 10.222.0.57 () {56 vars in 1122 bytes} [Thu May 19 14:11:36 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:36.646372 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-160ab336-adbe-460f-8190-8893ec8009da None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 19 14:11:36.646372 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:36.646372 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:11:36.646372 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:11:36.646372 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:11:36.646372 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:11:36.646372 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:11:36.646372 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:11:36.646372 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 14:11:36.646372 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:11:36.646372 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:11:36.646372 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:11:36.646372 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:36.646372 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:11:36.646372 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:11:36.646372 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:11:36.646372 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:11:36.647399 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:11:36.647399 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:11:36.647399 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:11:36.647399 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:11:36.647399 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:11:36.647399 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:11:36.647399 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:11:36.647399 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:11:36.647399 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:11:36.647399 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:11:36.647399 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:11:36.647399 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:11:36.647399 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:11:36.647399 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:11:36.647399 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:11:36.647399 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:11:36.648259 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:11:36.648259 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:11:36.648259 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:11:36.648259 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:11:36.648259 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:11:36.648259 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:11:36.648259 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:11:36.648259 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:11:36.648259 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:11:36.648259 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:11:36.648259 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:11:36.648259 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:11:36.648259 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 14:11:36.648259 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:11:36.649950 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dad75f-fd03-4ea2-933a-83b28d3c6caf None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:36.650164 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 503/996] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:11:36 2022] GET /identity/v3/services/compute => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:11:36.653571 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160ab336-adbe-460f-8190-8893ec8009da None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:36.759808 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dad75f-fd03-4ea2-933a-83b28d3c6caf None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:36.763182 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160ab336-adbe-460f-8190-8893ec8009da None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:36.795994 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883bc47b-173a-48fd-afcf-8c84e70cbacc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:36.796665 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883bc47b-173a-48fd-afcf-8c84e70cbacc None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:36.797544 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cf74ab-a14b-48ed-9618-7b076498fcfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:36.797708 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883bc47b-173a-48fd-afcf-8c84e70cbacc None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:36.797905 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883bc47b-173a-48fd-afcf-8c84e70cbacc None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:36.798235 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cf74ab-a14b-48ed-9618-7b076498fcfb None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:36.798520 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cf74ab-a14b-48ed-9618-7b076498fcfb None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:36.798686 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883bc47b-173a-48fd-afcf-8c84e70cbacc None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:36.798779 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cf74ab-a14b-48ed-9618-7b076498fcfb None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:36.799084 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883bc47b-173a-48fd-afcf-8c84e70cbacc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:36.799482 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cf74ab-a14b-48ed-9618-7b076498fcfb None admin] RBAC: Adding query filter params (name=compute) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:36.799790 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cf74ab-a14b-48ed-9618-7b076498fcfb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:36.800444 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883bc47b-173a-48fd-afcf-8c84e70cbacc None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:36.801156 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cf74ab-a14b-48ed-9618-7b076498fcfb None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:36.810372 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 494/997] 10.222.0.57 () {56 vars in 1122 bytes} [Thu May 19 14:11:36 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:36.810534 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 504/998] 10.222.0.57 () {56 vars in 1112 bytes} [Thu May 19 14:11:36 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:11:36.811525 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cf74ab-a14b-48ed-9618-7b076498fcfb None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:36.811668 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883bc47b-173a-48fd-afcf-8c84e70cbacc None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:36.814512 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:36.848692 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:36.920417 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883bc47b-173a-48fd-afcf-8c84e70cbacc None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:36.920613 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cf74ab-a14b-48ed-9618-7b076498fcfb None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:36.956898 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b44db0-3884-4235-81be-c288ca13d2ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:36.957555 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b44db0-3884-4235-81be-c288ca13d2ba None admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:36.957855 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b44db0-3884-4235-81be-c288ca13d2ba None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:36.958021 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f82698f-690f-4f6f-8d9f-d8ef17ffe2d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:36.958153 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b44db0-3884-4235-81be-c288ca13d2ba None admin] PATH_INFO: `/v3/endpoints` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:36.958684 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f82698f-690f-4f6f-8d9f-d8ef17ffe2d3 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:36.958848 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b44db0-3884-4235-81be-c288ca13d2ba None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:36.959001 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f82698f-690f-4f6f-8d9f-d8ef17ffe2d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:36.959245 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f82698f-690f-4f6f-8d9f-d8ef17ffe2d3 None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:36.959951 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f82698f-690f-4f6f-8d9f-d8ef17ffe2d3 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:36.960216 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b44db0-3884-4235-81be-c288ca13d2ba None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:36.960311 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f82698f-690f-4f6f-8d9f-d8ef17ffe2d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:36.961605 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f82698f-690f-4f6f-8d9f-d8ef17ffe2d3 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:36.967236 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 505/999] 10.222.0.57 () {56 vars in 1112 bytes} [Thu May 19 14:11:36 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:36.974435 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f82698f-690f-4f6f-8d9f-d8ef17ffe2d3 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:36.976913 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 495/1000] 10.222.0.57 () {60 vars in 1144 bytes} [Thu May 19 14:11:36 2022] POST /identity/v3/endpoints => generated 325 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 14:11:37.085748 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:37.089734 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f82698f-690f-4f6f-8d9f-d8ef17ffe2d3 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:37.143381 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6c78e4-672d-4403-a9cc-1e786e0dff68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:37.144060 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6c78e4-672d-4403-a9cc-1e786e0dff68 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:37.144346 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6c78e4-672d-4403-a9cc-1e786e0dff68 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:37.144623 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6c78e4-672d-4403-a9cc-1e786e0dff68 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:37.145423 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6c78e4-672d-4403-a9cc-1e786e0dff68 None admin] RBAC: Adding query filter params (interface=public, service_id=f4f1dc7ac435406888829d36a129ac19, region_id=RegionOne) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:37.145736 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6c78e4-672d-4403-a9cc-1e786e0dff68 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:37.147171 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6c78e4-672d-4403-a9cc-1e786e0dff68 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:37.153295 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 506/1001] 10.222.0.57 () {56 vars in 1251 bytes} [Thu May 19 14:11:36 2022] GET /identity/v3/endpoints?service_id=f4f1dc7ac435406888829d36a129ac19&interface=public®ion_id=RegionOne => generated 194 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:11:37.159268 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b44db0-3884-4235-81be-c288ca13d2ba None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:37.229072 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:37.273706 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b44db0-3884-4235-81be-c288ca13d2ba None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:37.307839 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb9ae4c-143d-428a-a175-27e20f9d0e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:37.308511 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb9ae4c-143d-428a-a175-27e20f9d0e55 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:37.308800 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb9ae4c-143d-428a-a175-27e20f9d0e55 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:37.309079 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb9ae4c-143d-428a-a175-27e20f9d0e55 None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:37.309768 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb9ae4c-143d-428a-a175-27e20f9d0e55 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:37.311187 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb9ae4c-143d-428a-a175-27e20f9d0e55 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:37.318455 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 496/1002] 10.222.0.57 () {56 vars in 1087 bytes} [Thu May 19 14:11:37 2022] GET /identity/v3/services => generated 1727 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 14:11:37.660986 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193b302e-892a-4fa6-9048-aa13ec2c1e74 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:37.661506 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193b302e-892a-4fa6-9048-aa13ec2c1e74 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:37.661609 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193b302e-892a-4fa6-9048-aa13ec2c1e74 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:37.662999 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 507/1003] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:11:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:11:37.668153 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bb6857-d452-4d3a-8709-270b2daef81e None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:37.668455 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bb6857-d452-4d3a-8709-270b2daef81e None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:37.668722 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bb6857-d452-4d3a-8709-270b2daef81e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:37.686284 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-82bb6857-d452-4d3a-8709-270b2daef81e None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:11:37.688035 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-82bb6857-d452-4d3a-8709-270b2daef81e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:11:37.756613 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 497/1004] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:11:37 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:11:37.778216 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193b302e-892a-4fa6-9048-aa13ec2c1e74 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:37.885992 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193b302e-892a-4fa6-9048-aa13ec2c1e74 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:37.919050 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5697c8a9-4efc-4c9f-9876-a43fe0dcaed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:37.919769 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5697c8a9-4efc-4c9f-9876-a43fe0dcaed1 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:37.920053 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5697c8a9-4efc-4c9f-9876-a43fe0dcaed1 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:37.920330 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5697c8a9-4efc-4c9f-9876-a43fe0dcaed1 None admin] PATH_INFO: `/v3/services/image` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:37.920979 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5697c8a9-4efc-4c9f-9876-a43fe0dcaed1 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:37.923097 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5697c8a9-4efc-4c9f-9876-a43fe0dcaed1 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:37.934748 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-5697c8a9-4efc-4c9f-9876-a43fe0dcaed1 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 19 14:11:37.934748 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:37.934748 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:11:37.934748 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:11:37.934748 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:11:37.934748 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:11:37.934748 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:11:37.934748 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:11:37.934748 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 14:11:37.934748 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:11:37.934748 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:11:37.934748 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:11:37.934748 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:37.934748 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:11:37.934748 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:11:37.934748 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:11:37.934748 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:11:37.935724 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:11:37.935724 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:11:37.935724 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:11:37.935724 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:11:37.935724 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:11:37.935724 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:11:37.935724 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:11:37.935724 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:11:37.935724 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:11:37.935724 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:11:37.935724 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:11:37.935724 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:11:37.935724 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:11:37.935724 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:11:37.935724 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:11:37.935724 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:11:37.936603 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:11:37.936603 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:11:37.936603 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:11:37.936603 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:11:37.936603 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:11:37.936603 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:11:37.936603 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:11:37.936603 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:11:37.936603 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:11:37.936603 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:11:37.936603 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:11:37.936603 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:11:37.936603 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 14:11:37.936603 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:11:37.944682 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 508/1005] 10.222.0.57 () {56 vars in 1105 bytes} [Thu May 19 14:11:37 2022] GET /identity/v3/services/image => generated 86 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:11:37.944909 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bb6857-d452-4d3a-8709-270b2daef81e None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:37.999989 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:38.047324 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bb6857-d452-4d3a-8709-270b2daef81e None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:38.081936 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5891fe-9674-40c3-8614-fc636ee65072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:38.082613 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5891fe-9674-40c3-8614-fc636ee65072 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:38.082901 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5891fe-9674-40c3-8614-fc636ee65072 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:38.083179 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5891fe-9674-40c3-8614-fc636ee65072 None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:38.083892 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5891fe-9674-40c3-8614-fc636ee65072 None admin] RBAC: Adding query filter params (name=image) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:38.084202 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5891fe-9674-40c3-8614-fc636ee65072 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:38.085575 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5891fe-9674-40c3-8614-fc636ee65072 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:38.095796 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 498/1006] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:11:37 2022] GET /identity/v3/services?name=image => generated 122 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:38.096775 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5697c8a9-4efc-4c9f-9876-a43fe0dcaed1 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:38.129430 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:38.208947 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5697c8a9-4efc-4c9f-9876-a43fe0dcaed1 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:38.244141 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294bc237-4b30-478c-9f45-631abba8e0af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:38.244807 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294bc237-4b30-478c-9f45-631abba8e0af None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:38.245095 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294bc237-4b30-478c-9f45-631abba8e0af None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:38.245374 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294bc237-4b30-478c-9f45-631abba8e0af None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:38.246113 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294bc237-4b30-478c-9f45-631abba8e0af None admin] RBAC: Adding query filter params (type=image) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:38.246424 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294bc237-4b30-478c-9f45-631abba8e0af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:38.247807 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294bc237-4b30-478c-9f45-631abba8e0af None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:38.258108 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 509/1007] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:11:38 2022] GET /identity/v3/services?type=image => generated 122 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:39.257013 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daf43cc-0018-46af-b483-a2dc766ebf07 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:39.257562 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daf43cc-0018-46af-b483-a2dc766ebf07 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:39.257562 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daf43cc-0018-46af-b483-a2dc766ebf07 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:39.258962 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 499/1008] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:11:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:11:39.264361 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9725d53-2e42-44d3-a0d0-586abccd5448 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:39.264763 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9725d53-2e42-44d3-a0d0-586abccd5448 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:39.264928 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9725d53-2e42-44d3-a0d0-586abccd5448 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:39.282571 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-f9725d53-2e42-44d3-a0d0-586abccd5448 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:11:39.283804 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-f9725d53-2e42-44d3-a0d0-586abccd5448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:11:39.353876 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 510/1009] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:11:39 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:11:39.380342 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daf43cc-0018-46af-b483-a2dc766ebf07 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:39.492730 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daf43cc-0018-46af-b483-a2dc766ebf07 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:39.514203 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:39.529546 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7d89dd-ddf4-4b85-990a-b087f10bd25f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:39.530396 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7d89dd-ddf4-4b85-990a-b087f10bd25f None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:39.530679 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7d89dd-ddf4-4b85-990a-b087f10bd25f None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:39.531484 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7d89dd-ddf4-4b85-990a-b087f10bd25f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:39.532814 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7d89dd-ddf4-4b85-990a-b087f10bd25f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:39.534291 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7d89dd-ddf4-4b85-990a-b087f10bd25f None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:39.536113 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 500/1010] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:11:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:39.541558 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9725d53-2e42-44d3-a0d0-586abccd5448 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:39.561516 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:39.571172 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f39e749-a1a7-4621-8b43-b76487663ff4 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:39.571465 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f39e749-a1a7-4621-8b43-b76487663ff4 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:39.571739 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f39e749-a1a7-4621-8b43-b76487663ff4 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:39.573243 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 501/1011] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:11:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:11:39.578518 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a22382-fdab-4635-99d8-1cbbac0cfd56 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:39.578816 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a22382-fdab-4635-99d8-1cbbac0cfd56 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:39.579162 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a22382-fdab-4635-99d8-1cbbac0cfd56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:39.596190 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-40a22382-fdab-4635-99d8-1cbbac0cfd56 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:11:39.597893 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-40a22382-fdab-4635-99d8-1cbbac0cfd56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:11:39.649463 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9725d53-2e42-44d3-a0d0-586abccd5448 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:39.676716 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 502/1012] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:11:39 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:11:39.685728 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083d038d-2581-4df5-b1d6-8d11df5593a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:39.686424 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083d038d-2581-4df5-b1d6-8d11df5593a6 None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:39.686712 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083d038d-2581-4df5-b1d6-8d11df5593a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:39.686992 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083d038d-2581-4df5-b1d6-8d11df5593a6 None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:39.687843 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083d038d-2581-4df5-b1d6-8d11df5593a6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:39.689283 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083d038d-2581-4df5-b1d6-8d11df5593a6 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:39.699836 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a22382-fdab-4635-99d8-1cbbac0cfd56 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:39.721878 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 511/1013] 10.222.0.57 () {60 vars in 1141 bytes} [Thu May 19 14:11:39 2022] POST /identity/v3/projects => generated 311 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:11:39.834046 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a22382-fdab-4635-99d8-1cbbac0cfd56 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:39.855773 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:39.878875 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2321f865-03ca-48f7-a5a2-1822adce878f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:39.879692 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2321f865-03ca-48f7-a5a2-1822adce878f None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:39.879966 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2321f865-03ca-48f7-a5a2-1822adce878f None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:39.880235 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2321f865-03ca-48f7-a5a2-1822adce878f None admin] PATH_INFO: `/v3/services/compute` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:39.881049 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2321f865-03ca-48f7-a5a2-1822adce878f None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:39.882681 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2321f865-03ca-48f7-a5a2-1822adce878f None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:39.897155 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-2321f865-03ca-48f7-a5a2-1822adce878f None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 19 14:11:39.897155 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:39.897155 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:11:39.897155 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:11:39.897155 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:11:39.897155 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:11:39.897155 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:11:39.897155 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:11:39.897155 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 14:11:39.897155 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:11:39.897155 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:11:39.897155 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:11:39.897155 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:39.897155 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:11:39.897155 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:11:39.897155 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:11:39.897155 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:11:39.898281 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:11:39.898281 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:11:39.898281 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:11:39.898281 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:11:39.898281 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:11:39.898281 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:11:39.898281 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:11:39.898281 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:11:39.898281 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:11:39.898281 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:11:39.898281 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:11:39.898281 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:11:39.898281 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:11:39.898281 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:11:39.898281 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:11:39.898281 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:11:39.899228 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:11:39.899228 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:11:39.899228 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:11:39.899228 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:11:39.899228 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:11:39.899228 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:11:39.899228 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:11:39.899228 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:11:39.899228 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:11:39.899228 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:11:39.899228 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:11:39.899228 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:11:39.899228 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 14:11:39.899228 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:11:39.903773 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 503/1014] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:11:39 2022] GET /identity/v3/services/compute => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:11:39.905247 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083d038d-2581-4df5-b1d6-8d11df5593a6 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:39.990474 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:40.028103 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083d038d-2581-4df5-b1d6-8d11df5593a6 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:40.067939 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed5aabd-9d85-401c-a516-6525e2d7d302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:40.069062 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed5aabd-9d85-401c-a516-6525e2d7d302 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:40.069574 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed5aabd-9d85-401c-a516-6525e2d7d302 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:40.070074 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed5aabd-9d85-401c-a516-6525e2d7d302 None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:40.070975 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed5aabd-9d85-401c-a516-6525e2d7d302 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:40.071465 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed5aabd-9d85-401c-a516-6525e2d7d302 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:40.073045 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed5aabd-9d85-401c-a516-6525e2d7d302 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:40.083722 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 512/1015] 10.222.0.57 () {56 vars in 1112 bytes} [Thu May 19 14:11:39 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:40.085400 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2321f865-03ca-48f7-a5a2-1822adce878f None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:40.202548 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2321f865-03ca-48f7-a5a2-1822adce878f None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:40.238758 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69863c1f-968f-44a5-9caa-8e0ded75ea4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:40.239505 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69863c1f-968f-44a5-9caa-8e0ded75ea4c None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:40.239783 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69863c1f-968f-44a5-9caa-8e0ded75ea4c None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:40.240104 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69863c1f-968f-44a5-9caa-8e0ded75ea4c None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:40.240802 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69863c1f-968f-44a5-9caa-8e0ded75ea4c None admin] RBAC: Adding query filter params (type=compute) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:40.241104 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69863c1f-968f-44a5-9caa-8e0ded75ea4c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:40.242658 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69863c1f-968f-44a5-9caa-8e0ded75ea4c None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:40.252554 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 504/1016] 10.222.0.57 () {56 vars in 1112 bytes} [Thu May 19 14:11:40 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:11:40.254108 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed5aabd-9d85-401c-a516-6525e2d7d302 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:40.365720 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed5aabd-9d85-401c-a516-6525e2d7d302 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:40.401764 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7417fa42-4c2a-4166-99c2-c92913af7be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:40.402458 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7417fa42-4c2a-4166-99c2-c92913af7be3 None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:40.402748 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7417fa42-4c2a-4166-99c2-c92913af7be3 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:40.403029 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7417fa42-4c2a-4166-99c2-c92913af7be3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:40.403772 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7417fa42-4c2a-4166-99c2-c92913af7be3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:40.405162 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7417fa42-4c2a-4166-99c2-c92913af7be3 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:40.408874 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:40.426262 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 513/1017] 10.222.0.57 () {60 vars in 1144 bytes} [Thu May 19 14:11:40 2022] POST /identity/v3/endpoints => generated 334 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:11:40.511341 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc763ec7-d069-4e18-8271-59450f005646 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:40.511847 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc763ec7-d069-4e18-8271-59450f005646 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:40.511847 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc763ec7-d069-4e18-8271-59450f005646 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:40.513368 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 505/1018] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:11:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:11:40.519621 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eeb8ce-0097-4b51-b435-392b80821e6d None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:40.519917 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eeb8ce-0097-4b51-b435-392b80821e6d None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:40.520192 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eeb8ce-0097-4b51-b435-392b80821e6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:40.538398 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-a8eeb8ce-0097-4b51-b435-392b80821e6d None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:11:40.539605 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-a8eeb8ce-0097-4b51-b435-392b80821e6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:11:40.621900 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 514/1019] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:11:40 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:11:40.644630 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc763ec7-d069-4e18-8271-59450f005646 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:40.696559 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:40.754836 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc763ec7-d069-4e18-8271-59450f005646 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:40.789242 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07114b12-86c7-443e-80e8-8a332f08b3d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:40.789834 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07114b12-86c7-443e-80e8-8a332f08b3d5 None admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:40.790146 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07114b12-86c7-443e-80e8-8a332f08b3d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:40.790425 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07114b12-86c7-443e-80e8-8a332f08b3d5 None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:40.791161 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07114b12-86c7-443e-80e8-8a332f08b3d5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:40.792530 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07114b12-86c7-443e-80e8-8a332f08b3d5 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:40.808207 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 506/1020] 10.222.0.57 () {60 vars in 1141 bytes} [Thu May 19 14:11:40 2022] POST /identity/v3/services => generated 242 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:11:42.067815 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980e4104-9504-41fc-b155-ef3429b497f1 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:42.068370 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980e4104-9504-41fc-b155-ef3429b497f1 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:42.068452 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980e4104-9504-41fc-b155-ef3429b497f1 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:42.069941 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 515/1021] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:11:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:11:42.075340 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11020561-76a0-44fb-9f1b-aafde18ada63 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:42.075755 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11020561-76a0-44fb-9f1b-aafde18ada63 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:42.075927 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11020561-76a0-44fb-9f1b-aafde18ada63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:42.093056 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-11020561-76a0-44fb-9f1b-aafde18ada63 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:11:42.094420 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-11020561-76a0-44fb-9f1b-aafde18ada63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:11:42.178253 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 507/1022] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:11:42 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:11:42.203125 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980e4104-9504-41fc-b155-ef3429b497f1 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:42.307749 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980e4104-9504-41fc-b155-ef3429b497f1 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:42.325438 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:42.345404 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f967a1a-06fe-4780-a8d7-d56be63d4b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:42.346229 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f967a1a-06fe-4780-a8d7-d56be63d4b3e None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:42.346518 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f967a1a-06fe-4780-a8d7-d56be63d4b3e None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:42.346797 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f967a1a-06fe-4780-a8d7-d56be63d4b3e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:42.347975 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f967a1a-06fe-4780-a8d7-d56be63d4b3e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:42.349421 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f967a1a-06fe-4780-a8d7-d56be63d4b3e None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:42.355722 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 516/1023] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:11:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:42.356874 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11020561-76a0-44fb-9f1b-aafde18ada63 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:42.388946 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:42.460559 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11020561-76a0-44fb-9f1b-aafde18ada63 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:42.496119 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8021f2eb-c072-4fe0-8e2b-7929367e39d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:42.496794 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8021f2eb-c072-4fe0-8e2b-7929367e39d5 None admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:42.497080 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8021f2eb-c072-4fe0-8e2b-7929367e39d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:42.497357 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8021f2eb-c072-4fe0-8e2b-7929367e39d5 None admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:42.498114 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8021f2eb-c072-4fe0-8e2b-7929367e39d5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:42.499601 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8021f2eb-c072-4fe0-8e2b-7929367e39d5 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:42.525876 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 508/1024] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 19 14:11:42 2022] POST /identity/v3/users => generated 285 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 14:11:42.784927 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e3cd90-ddef-4171-a74a-6ef8b9072da1 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:42.785225 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e3cd90-ddef-4171-a74a-6ef8b9072da1 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:42.785511 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e3cd90-ddef-4171-a74a-6ef8b9072da1 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:42.787064 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 517/1025] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:11:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:11:42.792515 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a38a694-485b-4352-b9b9-28ef667452e2 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:42.793075 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a38a694-485b-4352-b9b9-28ef667452e2 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:42.793558 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a38a694-485b-4352-b9b9-28ef667452e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:42.811322 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-8a38a694-485b-4352-b9b9-28ef667452e2 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:11:42.813087 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-8a38a694-485b-4352-b9b9-28ef667452e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:11:42.886543 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 509/1026] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:11:42 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:11:42.916437 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e3cd90-ddef-4171-a74a-6ef8b9072da1 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:43.023925 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e3cd90-ddef-4171-a74a-6ef8b9072da1 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:43.063766 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136c17be-2398-4c5e-8e20-88abab54f671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:43.064441 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136c17be-2398-4c5e-8e20-88abab54f671 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:43.064695 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136c17be-2398-4c5e-8e20-88abab54f671 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:43.064967 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136c17be-2398-4c5e-8e20-88abab54f671 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:43.075819 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136c17be-2398-4c5e-8e20-88abab54f671 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:11:43.085141 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-136c17be-2398-4c5e-8e20-88abab54f671 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 14:11:43.085141 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:43.085141 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:11:43.085141 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:11:43.085141 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:11:43.085141 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:11:43.085141 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:11:43.085141 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:11:43.085141 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:11:43.085141 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:11:43.085141 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:11:43.085141 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:11:43.085141 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:11:43.085141 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:11:43.085141 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:11:43.085141 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:11:43.086420 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:11:43.086420 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:11:43.086420 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:11:43.086420 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:11:43.086420 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:11:43.086420 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:11:43.086420 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:11:43.086420 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:11:43.086420 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:11:43.086420 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:11:43.086420 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:11:43.086420 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:11:43.086420 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:11:43.086420 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:11:43.086420 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:11:43.086420 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:11:43.087309 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:11:43.087309 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:11:43.087309 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:11:43.087309 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:11:43.087309 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:11:43.087309 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:11:43.087309 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:11:43.087309 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:11:43.087309 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:11:43.087309 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:11:43.087309 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 14:11:43.087309 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:11:43.087309 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 518/1027] 10.222.0.57 () {56 vars in 1120 bytes} [Thu May 19 14:11:42 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:11:43.092726 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf14fdd5-1f27-431c-9138-32a398eefcb4 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:43.092726 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf14fdd5-1f27-431c-9138-32a398eefcb4 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:43.092726 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf14fdd5-1f27-431c-9138-32a398eefcb4 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:43.092726 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 510/1028] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:11:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:11:43.094981 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136c17be-2398-4c5e-8e20-88abab54f671 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:43.096534 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eb444a-a8b6-427e-bc7e-ae0a0da9a039 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:43.096828 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eb444a-a8b6-427e-bc7e-ae0a0da9a039 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:43.097103 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eb444a-a8b6-427e-bc7e-ae0a0da9a039 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:43.114479 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-d4eb444a-a8b6-427e-bc7e-ae0a0da9a039 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:11:43.115725 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-d4eb444a-a8b6-427e-bc7e-ae0a0da9a039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:11:43.129924 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:43.151375 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:43.190522 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 511/1029] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:11:43 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:11:43.212915 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136c17be-2398-4c5e-8e20-88abab54f671 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:43.213461 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eb444a-a8b6-427e-bc7e-ae0a0da9a039 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:43.250920 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ba4fba-ddaf-402a-a5e5-adcd8a45e7fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:43.251594 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ba4fba-ddaf-402a-a5e5-adcd8a45e7fc None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:43.251875 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ba4fba-ddaf-402a-a5e5-adcd8a45e7fc None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:43.252151 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ba4fba-ddaf-402a-a5e5-adcd8a45e7fc None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:43.252893 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ba4fba-ddaf-402a-a5e5-adcd8a45e7fc None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:43.253198 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ba4fba-ddaf-402a-a5e5-adcd8a45e7fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:43.254598 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ba4fba-ddaf-402a-a5e5-adcd8a45e7fc None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:43.268079 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 519/1030] 10.222.0.57 () {56 vars in 1115 bytes} [Thu May 19 14:11:43 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:43.270030 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ba4fba-ddaf-402a-a5e5-adcd8a45e7fc None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:43.331556 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eb444a-a8b6-427e-bc7e-ae0a0da9a039 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:43.368031 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c89bd9-9fc5-4677-baa1-c15b19b33a41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:43.368823 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c89bd9-9fc5-4677-baa1-c15b19b33a41 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:43.369106 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c89bd9-9fc5-4677-baa1-c15b19b33a41 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:43.369380 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c89bd9-9fc5-4677-baa1-c15b19b33a41 None admin] PATH_INFO: `/v3/services/image` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:43.370060 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c89bd9-9fc5-4677-baa1-c15b19b33a41 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:43.371427 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c89bd9-9fc5-4677-baa1-c15b19b33a41 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:43.379209 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ba4fba-ddaf-402a-a5e5-adcd8a45e7fc None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:43.383725 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-54c89bd9-9fc5-4677-baa1-c15b19b33a41 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 19 14:11:43.383725 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:43.383725 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:11:43.383725 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:11:43.383725 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:11:43.383725 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:11:43.383725 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:11:43.383725 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:11:43.383725 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 14:11:43.383725 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:11:43.383725 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:11:43.383725 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:11:43.383725 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:43.383725 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:11:43.383725 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:11:43.383725 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:11:43.383725 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:11:43.384732 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:11:43.384732 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:11:43.384732 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:11:43.384732 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:11:43.384732 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:11:43.384732 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:11:43.384732 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:11:43.384732 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:11:43.384732 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:11:43.384732 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:11:43.384732 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:11:43.384732 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:11:43.384732 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:11:43.384732 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:11:43.384732 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:11:43.384732 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:11:43.385583 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:11:43.385583 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:11:43.385583 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:11:43.385583 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:11:43.385583 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:11:43.385583 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:11:43.385583 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:11:43.385583 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:11:43.385583 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:11:43.385583 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:11:43.385583 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:11:43.385583 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:11:43.385583 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 14:11:43.385583 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:11:43.387825 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:43.390699 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 512/1031] 10.222.0.57 () {56 vars in 1105 bytes} [Thu May 19 14:11:43 2022] GET /identity/v3/services/image => generated 86 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:11:43.391778 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c89bd9-9fc5-4677-baa1-c15b19b33a41 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:43.407373 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:43.420615 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb96ab7-9c30-4b5d-80c5-d6ad218b5844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:43.421647 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb96ab7-9c30-4b5d-80c5-d6ad218b5844 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:43.422027 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb96ab7-9c30-4b5d-80c5-d6ad218b5844 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:43.422672 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb96ab7-9c30-4b5d-80c5-d6ad218b5844 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:43.436606 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb96ab7-9c30-4b5d-80c5-d6ad218b5844 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:43.439364 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb96ab7-9c30-4b5d-80c5-d6ad218b5844 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:43.458928 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-4bb96ab7-9c30-4b5d-80c5-d6ad218b5844 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:11:43.458928 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:43.458928 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:11:43.458928 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:11:43.458928 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:11:43.458928 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:11:43.458928 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:11:43.458928 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:11:43.458928 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:11:43.458928 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:11:43.458928 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:11:43.458928 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:11:43.458928 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:11:43.458928 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:11:43.458928 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:11:43.459938 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:11:43.459938 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:11:43.459938 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:11:43.459938 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:11:43.459938 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:11:43.459938 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:11:43.459938 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:11:43.459938 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:11:43.459938 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:11:43.459938 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:11:43.459938 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:11:43.459938 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:11:43.459938 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:11:43.459938 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:11:43.459938 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:11:43.459938 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:11:43.459938 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:11:43.465629 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:11:43.465629 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:11:43.465629 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:11:43.465629 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:11:43.465629 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 520/1032] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:11:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:11:43.465946 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb96ab7-9c30-4b5d-80c5-d6ad218b5844 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:43.502273 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c89bd9-9fc5-4677-baa1-c15b19b33a41 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:43.537226 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d0c2f1-3bb0-4785-929d-ebc5c9a25a56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:43.537919 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d0c2f1-3bb0-4785-929d-ebc5c9a25a56 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:43.538479 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d0c2f1-3bb0-4785-929d-ebc5c9a25a56 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:43.538709 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d0c2f1-3bb0-4785-929d-ebc5c9a25a56 None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:43.539568 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d0c2f1-3bb0-4785-929d-ebc5c9a25a56 None admin] RBAC: Adding query filter params (name=image) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:43.539952 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d0c2f1-3bb0-4785-929d-ebc5c9a25a56 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:43.541262 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d0c2f1-3bb0-4785-929d-ebc5c9a25a56 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:43.553466 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 513/1033] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:11:43 2022] GET /identity/v3/services?name=image => generated 122 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:43.554878 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d0c2f1-3bb0-4785-929d-ebc5c9a25a56 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:43.578837 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb96ab7-9c30-4b5d-80c5-d6ad218b5844 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:43.617701 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38026c88-3f70-4f78-8b81-02d3e65837d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:43.618491 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38026c88-3f70-4f78-8b81-02d3e65837d4 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:43.618899 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38026c88-3f70-4f78-8b81-02d3e65837d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:43.619345 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38026c88-3f70-4f78-8b81-02d3e65837d4 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:43.620132 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38026c88-3f70-4f78-8b81-02d3e65837d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:43.620546 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38026c88-3f70-4f78-8b81-02d3e65837d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:43.622116 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38026c88-3f70-4f78-8b81-02d3e65837d4 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:43.641928 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 521/1034] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:11:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:43.643122 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38026c88-3f70-4f78-8b81-02d3e65837d4 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:43.645088 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:43.664616 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:43.676465 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d0c2f1-3bb0-4785-929d-ebc5c9a25a56 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:43.728326 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1811193-22f5-49b7-a99b-91dbeaa12a36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:43.728995 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1811193-22f5-49b7-a99b-91dbeaa12a36 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:43.729271 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1811193-22f5-49b7-a99b-91dbeaa12a36 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:43.729543 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1811193-22f5-49b7-a99b-91dbeaa12a36 None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:43.730290 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1811193-22f5-49b7-a99b-91dbeaa12a36 None admin] RBAC: Adding query filter params (type=image) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:43.730594 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1811193-22f5-49b7-a99b-91dbeaa12a36 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:43.731959 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1811193-22f5-49b7-a99b-91dbeaa12a36 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:43.738284 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 514/1035] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:11:43 2022] GET /identity/v3/services?type=image => generated 350 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:11:43.745058 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1811193-22f5-49b7-a99b-91dbeaa12a36 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:43.787984 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38026c88-3f70-4f78-8b81-02d3e65837d4 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:43.835833 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4390f929-1ed5-4471-b6dc-9c2aca1b1196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:43.836755 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4390f929-1ed5-4471-b6dc-9c2aca1b1196 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:43.837042 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4390f929-1ed5-4471-b6dc-9c2aca1b1196 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:43.837318 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4390f929-1ed5-4471-b6dc-9c2aca1b1196 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:43.848142 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4390f929-1ed5-4471-b6dc-9c2aca1b1196 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:43.850262 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4390f929-1ed5-4471-b6dc-9c2aca1b1196 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:43.867197 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-4390f929-1ed5-4471-b6dc-9c2aca1b1196 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 14:11:43.867197 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:43.867197 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:11:43.867197 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:11:43.867197 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:11:43.867197 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:11:43.867197 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:11:43.867197 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:11:43.867197 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:11:43.867197 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:11:43.867197 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:11:43.867197 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:11:43.867197 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:11:43.867197 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:11:43.867197 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:11:43.867197 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:11:43.869887 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:11:43.869887 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:11:43.869887 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:11:43.869887 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:11:43.869887 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:11:43.869887 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:11:43.869887 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:11:43.869887 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:11:43.869887 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:11:43.869887 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:11:43.869887 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:11:43.869887 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:11:43.869887 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:11:43.869887 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:11:43.869887 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:11:43.869887 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:11:43.869887 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:11:43.869887 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:11:43.872087 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:11:43.872087 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:11:43.872087 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:11:43.872087 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:11:43.872087 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:11:43.872087 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:11:43.872087 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:11:43.872087 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:11:43.872087 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 14:11:43.872087 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:11:43.874272 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 522/1036] 10.222.0.57 () {56 vars in 1093 bytes} [Thu May 19 14:11:43 2022] GET /identity/v3/users/nova => generated 82 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:11:43.876326 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4390f929-1ed5-4471-b6dc-9c2aca1b1196 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:43.881181 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1811193-22f5-49b7-a99b-91dbeaa12a36 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:43.918375 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d163f0f5-7c52-4e29-9724-a1c52cf8ccfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:43.919133 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d163f0f5-7c52-4e29-9724-a1c52cf8ccfc None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:43.919486 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d163f0f5-7c52-4e29-9724-a1c52cf8ccfc None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:43.919826 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d163f0f5-7c52-4e29-9724-a1c52cf8ccfc None admin] PATH_INFO: `/v3/endpoints` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:43.920707 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d163f0f5-7c52-4e29-9724-a1c52cf8ccfc None admin] RBAC: Adding query filter params (interface=public, service_id=10ab567b441e4a15b14822ffdd4cd830, region_id=RegionOne) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:43.921079 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d163f0f5-7c52-4e29-9724-a1c52cf8ccfc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:43.923863 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d163f0f5-7c52-4e29-9724-a1c52cf8ccfc None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:43.940705 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 515/1037] 10.222.0.57 () {56 vars in 1251 bytes} [Thu May 19 14:11:43 2022] GET /identity/v3/endpoints?service_id=10ab567b441e4a15b14822ffdd4cd830&interface=public®ion_id=RegionOne => generated 194 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:11:43.943699 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d163f0f5-7c52-4e29-9724-a1c52cf8ccfc None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:43.948268 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:43.982703 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:44.009902 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4390f929-1ed5-4471-b6dc-9c2aca1b1196 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:44.049442 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238821e6-0f20-4bc3-942d-13826f2efb2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:44.050165 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238821e6-0f20-4bc3-942d-13826f2efb2c None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:44.050449 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238821e6-0f20-4bc3-942d-13826f2efb2c None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:44.050729 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238821e6-0f20-4bc3-942d-13826f2efb2c None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:44.051584 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238821e6-0f20-4bc3-942d-13826f2efb2c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:44.051896 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238821e6-0f20-4bc3-942d-13826f2efb2c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:44.053364 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238821e6-0f20-4bc3-942d-13826f2efb2c None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:44.074570 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d163f0f5-7c52-4e29-9724-a1c52cf8ccfc None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:44.082258 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 523/1038] 10.222.0.57 () {56 vars in 1133 bytes} [Thu May 19 14:11:43 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:44.089073 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238821e6-0f20-4bc3-942d-13826f2efb2c None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:44.131138 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ac92f1-af96-4e02-a775-804c56bcc019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:44.131820 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ac92f1-af96-4e02-a775-804c56bcc019 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:44.132098 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ac92f1-af96-4e02-a775-804c56bcc019 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:44.132373 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ac92f1-af96-4e02-a775-804c56bcc019 None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:44.133303 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ac92f1-af96-4e02-a775-804c56bcc019 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:44.135653 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ac92f1-af96-4e02-a775-804c56bcc019 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:44.146021 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 516/1039] 10.222.0.57 () {56 vars in 1087 bytes} [Thu May 19 14:11:43 2022] GET /identity/v3/services => generated 1957 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 14:11:44.219630 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238821e6-0f20-4bc3-942d-13826f2efb2c None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:44.258347 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfc9d96-1113-4d5b-859a-00e29a8446bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:44.259009 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfc9d96-1113-4d5b-859a-00e29a8446bb None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:44.259290 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfc9d96-1113-4d5b-859a-00e29a8446bb None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:44.259562 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfc9d96-1113-4d5b-859a-00e29a8446bb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:44.263503 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:44.336675 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfc9d96-1113-4d5b-859a-00e29a8446bb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:44.338185 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfc9d96-1113-4d5b-859a-00e29a8446bb None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:44.445187 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 524/1040] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:11:44 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 19 14:11:44.450987 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ac92f1-af96-4e02-a775-804c56bcc019 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:44.524684 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:44.561729 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ac92f1-af96-4e02-a775-804c56bcc019 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:44.598818 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4891ee57-ac66-4d5a-9093-9d499292ef2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:44.599606 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4891ee57-ac66-4d5a-9093-9d499292ef2d None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:44.599884 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4891ee57-ac66-4d5a-9093-9d499292ef2d None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:44.600156 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4891ee57-ac66-4d5a-9093-9d499292ef2d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:44.611931 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4891ee57-ac66-4d5a-9093-9d499292ef2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:44.613608 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4891ee57-ac66-4d5a-9093-9d499292ef2d None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:44.631144 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-4891ee57-ac66-4d5a-9093-9d499292ef2d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:11:44.631144 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:44.631144 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:11:44.631144 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:11:44.631144 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:11:44.631144 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:11:44.631144 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:11:44.631144 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:11:44.631144 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:11:44.631144 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:11:44.631144 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:11:44.631144 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:11:44.631144 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:11:44.631144 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:11:44.631144 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:11:44.632218 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:11:44.632218 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:11:44.632218 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:11:44.632218 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:11:44.632218 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:11:44.632218 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:11:44.632218 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:11:44.632218 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:11:44.632218 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:11:44.632218 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:11:44.632218 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:11:44.632218 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:11:44.632218 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:11:44.632218 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:11:44.632218 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:11:44.632218 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:11:44.632218 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:11:44.633349 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:11:44.633349 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:11:44.633349 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:11:44.633349 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:11:44.636961 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 517/1041] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:11:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:11:44.637987 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfc9d96-1113-4d5b-859a-00e29a8446bb None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:44.680327 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:44.752157 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfc9d96-1113-4d5b-859a-00e29a8446bb None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:44.788346 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac426241-1b8d-4729-9e15-cc3f5042add5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:44.789045 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac426241-1b8d-4729-9e15-cc3f5042add5 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:44.789286 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac426241-1b8d-4729-9e15-cc3f5042add5 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:44.789558 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac426241-1b8d-4729-9e15-cc3f5042add5 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:44.790301 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac426241-1b8d-4729-9e15-cc3f5042add5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:44.790607 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac426241-1b8d-4729-9e15-cc3f5042add5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:44.791973 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac426241-1b8d-4729-9e15-cc3f5042add5 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:44.804211 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 525/1042] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:11:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:44.809488 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4891ee57-ac66-4d5a-9093-9d499292ef2d None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:44.838020 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee2bd8a-5020-4500-be78-a0c8698b795a None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:44.838325 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee2bd8a-5020-4500-be78-a0c8698b795a None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:44.838583 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee2bd8a-5020-4500-be78-a0c8698b795a None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:44.840069 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 526/1043] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:11:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:11:44.845608 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da884ee2-b6c1-4d53-8fe2-92d1243474c4 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:44.845956 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da884ee2-b6c1-4d53-8fe2-92d1243474c4 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:44.846240 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da884ee2-b6c1-4d53-8fe2-92d1243474c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:44.863650 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-da884ee2-b6c1-4d53-8fe2-92d1243474c4 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:11:44.864831 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-da884ee2-b6c1-4d53-8fe2-92d1243474c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:11:44.920605 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4891ee57-ac66-4d5a-9093-9d499292ef2d None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:44.942079 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:44.946209 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 527/1044] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:11:44 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:11:44.958430 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5433ca-c4c8-42a4-9b7b-ac2a73bfb7af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:44.959231 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5433ca-c4c8-42a4-9b7b-ac2a73bfb7af None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:44.959513 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5433ca-c4c8-42a4-9b7b-ac2a73bfb7af None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:44.959787 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5433ca-c4c8-42a4-9b7b-ac2a73bfb7af None admin] PATH_INFO: `/v3/projects/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:44.963877 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da884ee2-b6c1-4d53-8fe2-92d1243474c4 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:44.967139 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5433ca-c4c8-42a4-9b7b-ac2a73bfb7af None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:44.968958 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5433ca-c4c8-42a4-9b7b-ac2a73bfb7af None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:44.983158 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-db5433ca-c4c8-42a4-9b7b-ac2a73bfb7af None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:11:44.983158 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:44.983158 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:11:44.983158 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:11:44.983158 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:11:44.983158 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:11:44.983158 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:11:44.983158 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:11:44.983158 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:11:44.983158 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:11:44.983158 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:11:44.983158 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:11:44.983158 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:11:44.983158 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:11:44.983158 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:11:44.984843 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:11:44.984843 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:11:44.984843 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:11:44.984843 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:11:44.984843 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:11:44.984843 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:11:44.984843 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:11:44.984843 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:11:44.984843 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:11:44.984843 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:11:44.984843 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:11:44.984843 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:11:44.984843 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:11:44.984843 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:11:44.984843 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:11:44.984843 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:11:44.984843 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:11:44.985758 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:11:44.985758 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:11:44.985758 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:11:44.985758 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:11:44.985758 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:11:44.985758 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:11:44.985758 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:11:44.985758 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:11:44.985758 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 518/1045] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:11:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:11:44.990263 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5433ca-c4c8-42a4-9b7b-ac2a73bfb7af None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:45.002501 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:45.084051 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da884ee2-b6c1-4d53-8fe2-92d1243474c4 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:45.097430 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5433ca-c4c8-42a4-9b7b-ac2a73bfb7af None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:45.127693 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b099bc-4c7e-4a8d-9b97-91ac256f0d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:45.128498 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b099bc-4c7e-4a8d-9b97-91ac256f0d0b None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:45.128993 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b099bc-4c7e-4a8d-9b97-91ac256f0d0b None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:45.129124 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b099bc-4c7e-4a8d-9b97-91ac256f0d0b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:45.140179 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b099bc-4c7e-4a8d-9b97-91ac256f0d0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:11:45.140576 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af31acde-1875-44e6-bbc2-c94d4f596eef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:45.141282 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af31acde-1875-44e6-bbc2-c94d4f596eef None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:45.141507 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af31acde-1875-44e6-bbc2-c94d4f596eef None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:45.141777 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af31acde-1875-44e6-bbc2-c94d4f596eef None admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:45.142603 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af31acde-1875-44e6-bbc2-c94d4f596eef None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:45.142909 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af31acde-1875-44e6-bbc2-c94d4f596eef None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:45.144342 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af31acde-1875-44e6-bbc2-c94d4f596eef None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:45.147780 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-b8b099bc-4c7e-4a8d-9b97-91ac256f0d0b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:11:45.147780 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:45.147780 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:11:45.147780 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:11:45.147780 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:11:45.147780 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:11:45.147780 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:11:45.147780 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:11:45.147780 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:11:45.147780 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:11:45.147780 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:11:45.147780 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:11:45.147780 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:11:45.147780 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:11:45.147780 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:11:45.147780 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:11:45.148710 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:11:45.148710 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:11:45.148710 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:11:45.148710 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:11:45.148710 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:11:45.148710 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:11:45.148710 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:11:45.148710 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:11:45.148710 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:11:45.148710 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:11:45.148710 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:11:45.148710 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:11:45.148710 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:11:45.148710 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:11:45.148710 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:11:45.148710 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:11:45.155299 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:11:45.155299 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:11:45.155299 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:11:45.155299 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:11:45.155299 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:11:45.155299 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:11:45.155299 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:11:45.155299 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:11:45.155299 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:11:45.155299 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:11:45.155299 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:11:45.155299 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:11:45.155299 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 528/1046] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:11:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:11:45.156514 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b099bc-4c7e-4a8d-9b97-91ac256f0d0b None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:45.165025 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 519/1047] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:11:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:11:45.165999 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af31acde-1875-44e6-bbc2-c94d4f596eef None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:45.210688 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:45.251986 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:45.270212 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b099bc-4c7e-4a8d-9b97-91ac256f0d0b None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:45.281969 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af31acde-1875-44e6-bbc2-c94d4f596eef None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:45.317649 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27784181-ec46-4101-a99e-a0130016fd7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:45.318356 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27784181-ec46-4101-a99e-a0130016fd7f None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:45.318645 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27784181-ec46-4101-a99e-a0130016fd7f None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:45.318929 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27784181-ec46-4101-a99e-a0130016fd7f None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:45.319674 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27784181-ec46-4101-a99e-a0130016fd7f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:45.319984 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27784181-ec46-4101-a99e-a0130016fd7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:45.320094 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f20908-2367-4ee8-b3d4-61cd5061a1b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:45.320753 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f20908-2367-4ee8-b3d4-61cd5061a1b6 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:45.321034 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f20908-2367-4ee8-b3d4-61cd5061a1b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:45.321306 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f20908-2367-4ee8-b3d4-61cd5061a1b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:45.321401 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27784181-ec46-4101-a99e-a0130016fd7f None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:45.322175 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f20908-2367-4ee8-b3d4-61cd5061a1b6 None admin] RBAC: Adding query filter params (role.id=82029ff85919459780b85031cdfc369f, scope.project.id=c6f995ffa0a94b1584f25dbe31e20524, user.id=8c87f361a92d4834854ca07f652be2ff) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:45.322495 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f20908-2367-4ee8-b3d4-61cd5061a1b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:45.323913 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f20908-2367-4ee8-b3d4-61cd5061a1b6 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:45.330197 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 520/1048] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:11:45 2022] GET /identity/v3/role_assignments?user.id=8c87f361a92d4834854ca07f652be2ff&scope.project.id=c6f995ffa0a94b1584f25dbe31e20524&role.id=82029ff85919459780b85031cdfc369f => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:11:45.330991 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 529/1049] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:11:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:45.336646 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f20908-2367-4ee8-b3d4-61cd5061a1b6 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:45.451139 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f20908-2367-4ee8-b3d4-61cd5061a1b6 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:45.454073 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:45.491670 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4d5e26-3ba3-48ff-8d80-b536d00126f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:45.492428 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4d5e26-3ba3-48ff-8d80-b536d00126f6 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:45.492711 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4d5e26-3ba3-48ff-8d80-b536d00126f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:45.492984 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4d5e26-3ba3-48ff-8d80-b536d00126f6 None admin] PATH_INFO: `/v3/users/7db9f2236f9b4ee88fca9db84dff7818` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:45.508127 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4d5e26-3ba3-48ff-8d80-b536d00126f6 None admin] RBAC: Authorizing `identity:get_user(user_id=7db9f2236f9b4ee88fca9db84dff7818)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:45.510317 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4d5e26-3ba3-48ff-8d80-b536d00126f6 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:45.512223 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 521/1050] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:11:45 2022] GET /identity/v3/users/7db9f2236f9b4ee88fca9db84dff7818 => generated 285 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:45.518228 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27784181-ec46-4101-a99e-a0130016fd7f None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:45.623708 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27784181-ec46-4101-a99e-a0130016fd7f None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:45.658669 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0de100-a093-4516-ad88-7d495e692909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:45.659324 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0de100-a093-4516-ad88-7d495e692909 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:45.659613 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0de100-a093-4516-ad88-7d495e692909 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:45.659894 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0de100-a093-4516-ad88-7d495e692909 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:45.678631 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:45.729648 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0de100-a093-4516-ad88-7d495e692909 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:45.731148 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0de100-a093-4516-ad88-7d495e692909 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:45.833683 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 530/1051] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:11:45 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:11:45.839639 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4d5e26-3ba3-48ff-8d80-b536d00126f6 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:45.944408 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4d5e26-3ba3-48ff-8d80-b536d00126f6 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:45.980411 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12617a1e-46af-497c-ad57-0c81bccf421d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:45.981273 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12617a1e-46af-497c-ad57-0c81bccf421d None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:45.981556 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12617a1e-46af-497c-ad57-0c81bccf421d None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:45.981832 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12617a1e-46af-497c-ad57-0c81bccf421d None admin] PATH_INFO: `/v3/projects/83f70b997931496a926ba305064bcb65` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:45.983044 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12617a1e-46af-497c-ad57-0c81bccf421d None admin] RBAC: Authorizing `identity:get_project(project_id=83f70b997931496a926ba305064bcb65)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:45.984654 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12617a1e-46af-497c-ad57-0c81bccf421d None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:45.991644 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 522/1052] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:11:45 2022] GET /identity/v3/projects/83f70b997931496a926ba305064bcb65 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:45.993281 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0de100-a093-4516-ad88-7d495e692909 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:46.043803 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:46.099718 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0de100-a093-4516-ad88-7d495e692909 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:46.142069 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b49f0f9-eef2-4448-bcd2-0e2d255e1f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:46.142732 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b49f0f9-eef2-4448-bcd2-0e2d255e1f0d None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:46.143021 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b49f0f9-eef2-4448-bcd2-0e2d255e1f0d None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:46.143296 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b49f0f9-eef2-4448-bcd2-0e2d255e1f0d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:46.144125 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b49f0f9-eef2-4448-bcd2-0e2d255e1f0d None admin] RBAC: Adding query filter params (role.id=4946cf0ff7164307b36efeee7c787146, scope.project.id=83f70b997931496a926ba305064bcb65, user.id=7db9f2236f9b4ee88fca9db84dff7818) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:46.144434 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b49f0f9-eef2-4448-bcd2-0e2d255e1f0d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:46.145880 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b49f0f9-eef2-4448-bcd2-0e2d255e1f0d None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:46.157883 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 531/1053] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:11:45 2022] GET /identity/v3/role_assignments?user.id=7db9f2236f9b4ee88fca9db84dff7818&scope.project.id=83f70b997931496a926ba305064bcb65&role.id=4946cf0ff7164307b36efeee7c787146 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:46.447125 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f7c08f-1c2b-4822-a992-d66746374b99 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:46.447686 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f7c08f-1c2b-4822-a992-d66746374b99 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:46.447686 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f7c08f-1c2b-4822-a992-d66746374b99 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:46.449304 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 523/1054] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:11:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:11:46.455530 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58191bc2-b729-4d53-9906-d599d38b7cb2 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:46.455849 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58191bc2-b729-4d53-9906-d599d38b7cb2 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:46.456089 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58191bc2-b729-4d53-9906-d599d38b7cb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:46.472810 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-58191bc2-b729-4d53-9906-d599d38b7cb2 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:11:46.474014 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-58191bc2-b729-4d53-9906-d599d38b7cb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:11:46.542476 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 532/1055] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:11:46 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:11:46.565767 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f7c08f-1c2b-4822-a992-d66746374b99 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:46.596844 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:46.671010 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f7c08f-1c2b-4822-a992-d66746374b99 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:46.705776 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec88c2b9-761f-4409-ad69-ead2012b48c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:46.706511 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec88c2b9-761f-4409-ad69-ead2012b48c7 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:46.706796 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec88c2b9-761f-4409-ad69-ead2012b48c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:46.707067 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec88c2b9-761f-4409-ad69-ead2012b48c7 None admin] PATH_INFO: `/v3/services/image` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:46.707712 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec88c2b9-761f-4409-ad69-ead2012b48c7 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:46.709068 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec88c2b9-761f-4409-ad69-ead2012b48c7 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:46.720837 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-ec88c2b9-761f-4409-ad69-ead2012b48c7 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 19 14:11:46.720837 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:46.720837 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:11:46.720837 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:11:46.720837 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:11:46.720837 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:11:46.720837 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:11:46.720837 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:11:46.720837 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 14:11:46.720837 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:11:46.720837 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:11:46.720837 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:11:46.720837 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:46.720837 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:11:46.720837 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:11:46.720837 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:11:46.720837 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:11:46.721969 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:11:46.721969 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:11:46.721969 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:11:46.721969 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:11:46.721969 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:11:46.721969 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:11:46.721969 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:11:46.721969 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:11:46.721969 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:11:46.721969 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:11:46.721969 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:11:46.721969 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:11:46.721969 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:11:46.721969 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:11:46.721969 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:11:46.721969 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:11:46.722829 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:11:46.722829 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:11:46.722829 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:11:46.722829 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:11:46.722829 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:11:46.722829 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:11:46.722829 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:11:46.722829 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:11:46.722829 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:11:46.722829 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:11:46.722829 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:11:46.722829 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:11:46.722829 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 14:11:46.722829 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:11:46.729916 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 524/1056] 10.222.0.57 () {56 vars in 1105 bytes} [Thu May 19 14:11:46 2022] GET /identity/v3/services/image => generated 86 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:11:46.730063 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58191bc2-b729-4d53-9906-d599d38b7cb2 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:46.775346 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:46.835182 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58191bc2-b729-4d53-9906-d599d38b7cb2 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:46.870091 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784d16eb-281e-4972-a1fe-6c272bc3181d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:46.870763 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784d16eb-281e-4972-a1fe-6c272bc3181d None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:46.871043 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784d16eb-281e-4972-a1fe-6c272bc3181d None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:46.871318 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784d16eb-281e-4972-a1fe-6c272bc3181d None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:46.872016 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784d16eb-281e-4972-a1fe-6c272bc3181d None admin] RBAC: Adding query filter params (name=image) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:46.872320 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784d16eb-281e-4972-a1fe-6c272bc3181d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:46.873684 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784d16eb-281e-4972-a1fe-6c272bc3181d None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:46.884936 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 533/1057] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:11:46 2022] GET /identity/v3/services?name=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:46.886272 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec88c2b9-761f-4409-ad69-ead2012b48c7 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:46.985640 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:46.993411 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec88c2b9-761f-4409-ad69-ead2012b48c7 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:47.027070 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e0c577-9e60-48d3-bca2-9eb82fa87321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:47.027733 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e0c577-9e60-48d3-bca2-9eb82fa87321 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:47.028096 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e0c577-9e60-48d3-bca2-9eb82fa87321 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:47.028502 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e0c577-9e60-48d3-bca2-9eb82fa87321 None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:47.029082 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e0c577-9e60-48d3-bca2-9eb82fa87321 None admin] RBAC: Adding query filter params (type=image) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:47.029383 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e0c577-9e60-48d3-bca2-9eb82fa87321 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:47.031303 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e0c577-9e60-48d3-bca2-9eb82fa87321 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:47.036501 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 525/1058] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:11:46 2022] GET /identity/v3/services?type=image => generated 350 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:47.042037 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784d16eb-281e-4972-a1fe-6c272bc3181d None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:47.154518 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784d16eb-281e-4972-a1fe-6c272bc3181d None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:47.163225 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:47.192624 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794475c1-9eae-40f3-8170-363c183782aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:47.193290 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794475c1-9eae-40f3-8170-363c183782aa None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:47.193578 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794475c1-9eae-40f3-8170-363c183782aa None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:47.193872 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794475c1-9eae-40f3-8170-363c183782aa None admin] PATH_INFO: `/v3/endpoints` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:47.194694 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794475c1-9eae-40f3-8170-363c183782aa None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:47.196083 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794475c1-9eae-40f3-8170-363c183782aa None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:47.215429 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 534/1059] 10.222.0.57 () {60 vars in 1144 bytes} [Thu May 19 14:11:47 2022] POST /identity/v3/endpoints => generated 327 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 14:11:47.577154 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc174ea2-e73a-41c1-9018-60b2772ba675 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:47.578397 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc174ea2-e73a-41c1-9018-60b2772ba675 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:47.578661 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc174ea2-e73a-41c1-9018-60b2772ba675 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:47.580162 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 526/1060] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:11:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:11:47.585164 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b987b385-99ec-40b1-9c68-b60d3fff6b8d None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:47.585462 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b987b385-99ec-40b1-9c68-b60d3fff6b8d None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:47.585740 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b987b385-99ec-40b1-9c68-b60d3fff6b8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:47.603421 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-b987b385-99ec-40b1-9c68-b60d3fff6b8d None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:11:47.604592 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-b987b385-99ec-40b1-9c68-b60d3fff6b8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:11:47.680706 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 535/1061] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:11:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:11:47.704602 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc174ea2-e73a-41c1-9018-60b2772ba675 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:47.809858 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc174ea2-e73a-41c1-9018-60b2772ba675 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:47.845054 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb57d1f-81fa-4cc8-b379-12e8c0b344d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:47.845784 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb57d1f-81fa-4cc8-b379-12e8c0b344d9 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:47.846093 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb57d1f-81fa-4cc8-b379-12e8c0b344d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:47.846367 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb57d1f-81fa-4cc8-b379-12e8c0b344d9 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:47.856846 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb57d1f-81fa-4cc8-b379-12e8c0b344d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:11:47.865557 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-0fb57d1f-81fa-4cc8-b379-12e8c0b344d9 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 14:11:47.865557 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:47.865557 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:11:47.865557 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:11:47.865557 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:11:47.865557 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:11:47.865557 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:11:47.865557 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:11:47.865557 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:11:47.865557 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:11:47.865557 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:11:47.865557 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:11:47.865557 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:11:47.865557 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:11:47.865557 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:11:47.865557 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:11:47.866752 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:11:47.866752 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:11:47.866752 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:11:47.866752 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:11:47.866752 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:11:47.866752 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:11:47.866752 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:11:47.866752 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:11:47.866752 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:11:47.866752 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:11:47.866752 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:11:47.866752 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:11:47.866752 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:11:47.866752 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:11:47.866752 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:11:47.866752 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:11:47.868161 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:11:47.868161 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:11:47.868161 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:11:47.868161 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:11:47.868161 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:11:47.868161 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:11:47.868161 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:11:47.868161 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:11:47.868161 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:11:47.868161 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:11:47.868161 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 14:11:47.868161 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:11:47.869557 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 527/1062] 10.222.0.57 () {56 vars in 1120 bytes} [Thu May 19 14:11:47 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:11:47.872928 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b987b385-99ec-40b1-9c68-b60d3fff6b8d None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:47.961416 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:47.975092 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b987b385-99ec-40b1-9c68-b60d3fff6b8d None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:48.010660 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fa5453-23e5-4781-888a-282471073248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:48.011330 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fa5453-23e5-4781-888a-282471073248 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:48.011613 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fa5453-23e5-4781-888a-282471073248 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:48.011887 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fa5453-23e5-4781-888a-282471073248 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:48.012629 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fa5453-23e5-4781-888a-282471073248 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:48.012934 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fa5453-23e5-4781-888a-282471073248 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:48.014338 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fa5453-23e5-4781-888a-282471073248 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:48.027958 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 536/1063] 10.222.0.57 () {56 vars in 1115 bytes} [Thu May 19 14:11:47 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:48.029272 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb57d1f-81fa-4cc8-b379-12e8c0b344d9 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:48.049048 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:48.139450 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb57d1f-81fa-4cc8-b379-12e8c0b344d9 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:48.175332 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79e2add-b216-47a9-a4d2-4de63617184b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:48.176151 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79e2add-b216-47a9-a4d2-4de63617184b None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:48.176430 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79e2add-b216-47a9-a4d2-4de63617184b None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:48.176700 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79e2add-b216-47a9-a4d2-4de63617184b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:48.191051 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79e2add-b216-47a9-a4d2-4de63617184b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:48.192533 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79e2add-b216-47a9-a4d2-4de63617184b None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:48.211536 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-f79e2add-b216-47a9-a4d2-4de63617184b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:11:48.211536 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:48.211536 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:11:48.211536 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:11:48.211536 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:11:48.211536 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:11:48.211536 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:11:48.211536 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:11:48.211536 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:11:48.211536 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:11:48.211536 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:11:48.211536 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:11:48.211536 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:11:48.211536 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:11:48.211536 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:11:48.212518 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:11:48.212518 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:11:48.212518 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:11:48.212518 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:11:48.212518 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:11:48.212518 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:11:48.212518 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:11:48.212518 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:11:48.212518 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:11:48.212518 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:11:48.212518 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:11:48.212518 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:11:48.212518 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:11:48.212518 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:11:48.212518 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:11:48.212518 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:11:48.212518 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:11:48.213447 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:11:48.213447 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:11:48.213447 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:11:48.213447 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:11:48.214362 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 528/1064] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:11:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:11:48.218548 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fa5453-23e5-4781-888a-282471073248 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:48.319800 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fa5453-23e5-4781-888a-282471073248 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:48.353138 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc04e37-ec8c-4f08-99f5-1897d85d4102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:48.353791 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc04e37-ec8c-4f08-99f5-1897d85d4102 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:48.354099 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc04e37-ec8c-4f08-99f5-1897d85d4102 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:48.354370 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc04e37-ec8c-4f08-99f5-1897d85d4102 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:48.355077 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc04e37-ec8c-4f08-99f5-1897d85d4102 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:48.355380 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc04e37-ec8c-4f08-99f5-1897d85d4102 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:48.356737 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc04e37-ec8c-4f08-99f5-1897d85d4102 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:48.368693 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab98026-b977-49fd-aa70-5c5132d10751 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:48.368898 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab98026-b977-49fd-aa70-5c5132d10751 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:48.369168 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab98026-b977-49fd-aa70-5c5132d10751 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:48.370830 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 529/1065] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:11:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:11:48.374289 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 537/1066] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:11:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:48.378231 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce04aa1-47a1-4fb6-b2a2-a79ef6721677 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:48.378522 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce04aa1-47a1-4fb6-b2a2-a79ef6721677 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:48.378796 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce04aa1-47a1-4fb6-b2a2-a79ef6721677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:48.380631 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc04e37-ec8c-4f08-99f5-1897d85d4102 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:48.393748 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:48.398227 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-5ce04aa1-47a1-4fb6-b2a2-a79ef6721677 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:11:48.399806 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-5ce04aa1-47a1-4fb6-b2a2-a79ef6721677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:11:48.474123 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:48.480750 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 530/1067] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:11:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:11:48.495486 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc04e37-ec8c-4f08-99f5-1897d85d4102 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:48.503970 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce04aa1-47a1-4fb6-b2a2-a79ef6721677 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:48.534000 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1e1b5c-2455-4170-9206-06bbdda3b34e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:48.534837 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1e1b5c-2455-4170-9206-06bbdda3b34e None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:48.535123 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1e1b5c-2455-4170-9206-06bbdda3b34e None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:48.535404 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1e1b5c-2455-4170-9206-06bbdda3b34e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:48.542628 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1e1b5c-2455-4170-9206-06bbdda3b34e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:48.544125 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1e1b5c-2455-4170-9206-06bbdda3b34e None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:48.559808 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-2d1e1b5c-2455-4170-9206-06bbdda3b34e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 14:11:48.559808 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:48.559808 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:11:48.559808 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:11:48.559808 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:11:48.559808 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:11:48.559808 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:11:48.559808 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:11:48.559808 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:11:48.559808 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:11:48.559808 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:11:48.559808 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:11:48.559808 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:11:48.559808 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:11:48.559808 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:11:48.559808 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:11:48.561134 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:11:48.561134 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:11:48.561134 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:11:48.561134 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:11:48.561134 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:11:48.561134 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:11:48.561134 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:11:48.561134 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:11:48.561134 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:11:48.561134 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:11:48.561134 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:11:48.561134 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:11:48.561134 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:11:48.561134 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:11:48.561134 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:11:48.561134 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:11:48.561134 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:11:48.561134 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:11:48.562379 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:11:48.562379 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:11:48.562379 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:11:48.562379 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:11:48.562379 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:11:48.562379 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:11:48.562379 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:11:48.562379 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:11:48.562379 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 14:11:48.562379 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:11:48.562379 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 538/1068] 10.222.0.57 () {56 vars in 1093 bytes} [Thu May 19 14:11:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:11:48.566729 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1e1b5c-2455-4170-9206-06bbdda3b34e None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:48.618186 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce04aa1-47a1-4fb6-b2a2-a79ef6721677 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:48.645161 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:48.659479 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11572db0-6187-4734-85a6-794b51fd104d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:48.660281 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11572db0-6187-4734-85a6-794b51fd104d None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:48.660624 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11572db0-6187-4734-85a6-794b51fd104d None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:48.660905 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11572db0-6187-4734-85a6-794b51fd104d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:48.672870 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11572db0-6187-4734-85a6-794b51fd104d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:11:48.680379 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-11572db0-6187-4734-85a6-794b51fd104d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:11:48.680379 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:48.680379 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:11:48.680379 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:11:48.680379 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:11:48.680379 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:11:48.680379 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:11:48.680379 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:11:48.680379 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:11:48.680379 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:11:48.680379 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:11:48.680379 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:11:48.680379 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:11:48.680379 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:11:48.680379 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:11:48.680379 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:11:48.681351 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:11:48.681351 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:11:48.681351 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:11:48.681351 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:11:48.681351 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:11:48.681351 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:11:48.681351 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:11:48.681351 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:11:48.681351 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:11:48.681351 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:11:48.681351 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:11:48.681351 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:11:48.681351 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:11:48.681351 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:11:48.681351 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:11:48.681351 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:11:48.682259 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:11:48.682259 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:11:48.682259 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:11:48.682259 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:11:48.682259 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:11:48.682259 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:11:48.682259 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:11:48.682259 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:11:48.682259 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:11:48.682259 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:11:48.682259 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:11:48.682259 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:11:48.682259 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 531/1069] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:11:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:11:48.682997 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1e1b5c-2455-4170-9206-06bbdda3b34e None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:48.687521 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11572db0-6187-4734-85a6-794b51fd104d None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:48.719889 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319f7c6f-5e6c-4e18-ae77-05168ed56129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:48.720802 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319f7c6f-5e6c-4e18-ae77-05168ed56129 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:48.721185 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319f7c6f-5e6c-4e18-ae77-05168ed56129 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:48.721576 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319f7c6f-5e6c-4e18-ae77-05168ed56129 None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:48.722938 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319f7c6f-5e6c-4e18-ae77-05168ed56129 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:48.723416 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319f7c6f-5e6c-4e18-ae77-05168ed56129 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:48.725566 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319f7c6f-5e6c-4e18-ae77-05168ed56129 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:48.750170 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 539/1070] 10.222.0.57 () {56 vars in 1133 bytes} [Thu May 19 14:11:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:11:48.751482 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319f7c6f-5e6c-4e18-ae77-05168ed56129 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:48.751735 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:48.800646 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11572db0-6187-4734-85a6-794b51fd104d None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:48.838897 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c9df71-69fe-4b4e-93ab-ff83bb0b20d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:48.839653 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c9df71-69fe-4b4e-93ab-ff83bb0b20d3 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:48.839910 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c9df71-69fe-4b4e-93ab-ff83bb0b20d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:48.840120 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c9df71-69fe-4b4e-93ab-ff83bb0b20d3 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:48.840863 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c9df71-69fe-4b4e-93ab-ff83bb0b20d3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:48.841290 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c9df71-69fe-4b4e-93ab-ff83bb0b20d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:48.842794 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c9df71-69fe-4b4e-93ab-ff83bb0b20d3 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:48.857194 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 532/1071] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:11:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:11:48.858203 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c9df71-69fe-4b4e-93ab-ff83bb0b20d3 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:48.861627 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319f7c6f-5e6c-4e18-ae77-05168ed56129 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:48.889756 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:48.899475 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca892302-6205-4aa4-b633-f846c1b96422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:48.900210 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca892302-6205-4aa4-b633-f846c1b96422 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:48.900497 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca892302-6205-4aa4-b633-f846c1b96422 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:48.900774 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca892302-6205-4aa4-b633-f846c1b96422 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:48.911890 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca892302-6205-4aa4-b633-f846c1b96422 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:48.913362 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca892302-6205-4aa4-b633-f846c1b96422 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:48.933303 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-ca892302-6205-4aa4-b633-f846c1b96422 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:11:48.933303 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:48.933303 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:11:48.933303 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:11:48.933303 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:11:48.933303 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:11:48.933303 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:11:48.933303 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:11:48.933303 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:11:48.933303 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:11:48.933303 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:11:48.933303 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:11:48.933303 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:11:48.933303 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:11:48.933303 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:11:48.934559 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:11:48.934559 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:11:48.934559 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:11:48.934559 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:11:48.934559 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:11:48.934559 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:11:48.934559 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:11:48.934559 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:11:48.934559 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:11:48.934559 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:11:48.934559 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:11:48.934559 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:11:48.934559 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:11:48.934559 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:11:48.934559 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:11:48.934559 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:11:48.934559 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:11:48.935496 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:11:48.935496 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:11:48.935496 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:11:48.935496 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:11:48.935496 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 540/1072] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:11:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:11:48.940244 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca892302-6205-4aa4-b633-f846c1b96422 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:48.972410 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c9df71-69fe-4b4e-93ab-ff83bb0b20d3 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:48.997404 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:49.011516 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd73f1c-1597-4ff1-afcb-c3e3572aab07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:49.012418 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd73f1c-1597-4ff1-afcb-c3e3572aab07 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:49.012679 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd73f1c-1597-4ff1-afcb-c3e3572aab07 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:49.012956 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd73f1c-1597-4ff1-afcb-c3e3572aab07 None admin] PATH_INFO: `/v3/users/7db9f2236f9b4ee88fca9db84dff7818` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:49.014259 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd73f1c-1597-4ff1-afcb-c3e3572aab07 None admin] RBAC: Authorizing `identity:get_user(user_id=7db9f2236f9b4ee88fca9db84dff7818)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:49.016236 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd73f1c-1597-4ff1-afcb-c3e3572aab07 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:49.022556 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 533/1073] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:11:48 2022] GET /identity/v3/users/7db9f2236f9b4ee88fca9db84dff7818 => generated 285 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:49.023499 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd73f1c-1597-4ff1-afcb-c3e3572aab07 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:49.049685 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca892302-6205-4aa4-b633-f846c1b96422 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:49.087027 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37624b56-3a07-4a0d-a598-a7913255691d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:49.087698 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37624b56-3a07-4a0d-a598-a7913255691d None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:49.087987 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37624b56-3a07-4a0d-a598-a7913255691d None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:49.088503 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37624b56-3a07-4a0d-a598-a7913255691d None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:49.088983 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37624b56-3a07-4a0d-a598-a7913255691d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:49.089295 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37624b56-3a07-4a0d-a598-a7913255691d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:49.090706 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37624b56-3a07-4a0d-a598-a7913255691d None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:49.109227 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 541/1074] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:11:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:11:49.110552 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37624b56-3a07-4a0d-a598-a7913255691d None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:49.142315 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd73f1c-1597-4ff1-afcb-c3e3572aab07 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:49.159568 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:49.181062 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6132e7-4f56-4e00-af90-e3e09a3d9395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:49.182566 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6132e7-4f56-4e00-af90-e3e09a3d9395 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:49.183043 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6132e7-4f56-4e00-af90-e3e09a3d9395 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:49.183506 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6132e7-4f56-4e00-af90-e3e09a3d9395 None admin] PATH_INFO: `/v3/projects/83f70b997931496a926ba305064bcb65` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:49.185465 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6132e7-4f56-4e00-af90-e3e09a3d9395 None admin] RBAC: Authorizing `identity:get_project(project_id=83f70b997931496a926ba305064bcb65)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:49.188058 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6132e7-4f56-4e00-af90-e3e09a3d9395 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:49.189872 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 534/1075] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:11:49 2022] GET /identity/v3/projects/83f70b997931496a926ba305064bcb65 => generated 311 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:11:49.195894 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6132e7-4f56-4e00-af90-e3e09a3d9395 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:49.231448 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37624b56-3a07-4a0d-a598-a7913255691d None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:49.259187 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:49.270285 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9347a0-8dc2-4a6e-9ed7-8999ca87e2a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:49.271077 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9347a0-8dc2-4a6e-9ed7-8999ca87e2a4 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:49.271367 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9347a0-8dc2-4a6e-9ed7-8999ca87e2a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:49.271643 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9347a0-8dc2-4a6e-9ed7-8999ca87e2a4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:49.278854 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9347a0-8dc2-4a6e-9ed7-8999ca87e2a4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:49.280313 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9347a0-8dc2-4a6e-9ed7-8999ca87e2a4 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:49.295075 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-8b9347a0-8dc2-4a6e-9ed7-8999ca87e2a4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:11:49.295075 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:49.295075 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:11:49.295075 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:11:49.295075 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:11:49.295075 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:11:49.295075 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:11:49.295075 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:11:49.295075 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:11:49.295075 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:11:49.295075 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:11:49.295075 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:11:49.295075 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:11:49.295075 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:11:49.295075 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:11:49.296065 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:11:49.296065 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:11:49.296065 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:11:49.296065 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:11:49.296065 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:11:49.296065 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:11:49.296065 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:11:49.296065 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:11:49.296065 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:11:49.296065 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:11:49.296065 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:11:49.296065 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:11:49.296065 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:11:49.296065 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:11:49.296065 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:11:49.296065 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:11:49.296065 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:11:49.296981 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:11:49.296981 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:11:49.296981 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:11:49.296981 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:11:49.296981 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:11:49.296981 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:11:49.296981 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:11:49.296981 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:11:49.296981 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 542/1076] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:11:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:11:49.302043 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9347a0-8dc2-4a6e-9ed7-8999ca87e2a4 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:49.308337 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6132e7-4f56-4e00-af90-e3e09a3d9395 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:49.347046 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c64478d-d8d7-4970-957a-5e25f944da5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:49.347953 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c64478d-d8d7-4970-957a-5e25f944da5f None admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:49.348239 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c64478d-d8d7-4970-957a-5e25f944da5f None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:49.348516 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c64478d-d8d7-4970-957a-5e25f944da5f None admin] PATH_INFO: `/v3/projects/83f70b997931496a926ba305064bcb65/users/7db9f2236f9b4ee88fca9db84dff7818/roles/4946cf0ff7164307b36efeee7c787146` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:49.350553 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c64478d-d8d7-4970-957a-5e25f944da5f None admin] RBAC: Authorizing `identity:create_grant(project_id=83f70b997931496a926ba305064bcb65, user_id=7db9f2236f9b4ee88fca9db84dff7818, role_id=4946cf0ff7164307b36efeee7c787146)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:49.351995 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c64478d-d8d7-4970-957a-5e25f944da5f None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:49.367546 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 535/1077] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 14:11:49 2022] PUT /identity/v3/projects/83f70b997931496a926ba305064bcb65/users/7db9f2236f9b4ee88fca9db84dff7818/roles/4946cf0ff7164307b36efeee7c787146 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 14:11:49.417985 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:49.422230 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9347a0-8dc2-4a6e-9ed7-8999ca87e2a4 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:49.462123 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2cd582-789e-4c91-8b65-6515c3a56f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:49.462813 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2cd582-789e-4c91-8b65-6515c3a56f68 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:49.463103 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2cd582-789e-4c91-8b65-6515c3a56f68 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:49.463397 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2cd582-789e-4c91-8b65-6515c3a56f68 None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:49.464188 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2cd582-789e-4c91-8b65-6515c3a56f68 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:49.464502 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2cd582-789e-4c91-8b65-6515c3a56f68 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:49.466067 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2cd582-789e-4c91-8b65-6515c3a56f68 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:49.479718 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 543/1078] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:11:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:11:49.485758 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c64478d-d8d7-4970-957a-5e25f944da5f None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:49.512850 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1019bb-37ec-47af-9a7d-4be2f7767366 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:49.513169 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1019bb-37ec-47af-9a7d-4be2f7767366 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:49.513426 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1019bb-37ec-47af-9a7d-4be2f7767366 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:49.515543 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 544/1079] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:11:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:11:49.521134 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1242f26-679d-418d-a461-bbfb0e20da64 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:49.521454 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1242f26-679d-418d-a461-bbfb0e20da64 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:49.521707 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1242f26-679d-418d-a461-bbfb0e20da64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:49.540432 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-b1242f26-679d-418d-a461-bbfb0e20da64 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:11:49.541646 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-b1242f26-679d-418d-a461-bbfb0e20da64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:11:49.606768 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c64478d-d8d7-4970-957a-5e25f944da5f None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:49.623244 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 545/1080] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:11:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:11:49.627719 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:49.646664 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5512eb6e-30fb-4492-8453-b65cef827127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:49.646996 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1242f26-679d-418d-a461-bbfb0e20da64 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:49.647693 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5512eb6e-30fb-4492-8453-b65cef827127 None admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:49.647924 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5512eb6e-30fb-4492-8453-b65cef827127 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:49.648202 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5512eb6e-30fb-4492-8453-b65cef827127 None admin] PATH_INFO: `/v3/projects/c6f995ffa0a94b1584f25dbe31e20524/users/8c87f361a92d4834854ca07f652be2ff/roles/82029ff85919459780b85031cdfc369f` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:49.651675 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5512eb6e-30fb-4492-8453-b65cef827127 None admin] RBAC: Authorizing `identity:create_grant(project_id=c6f995ffa0a94b1584f25dbe31e20524, user_id=8c87f361a92d4834854ca07f652be2ff, role_id=82029ff85919459780b85031cdfc369f)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:49.653101 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5512eb6e-30fb-4492-8453-b65cef827127 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:49.670634 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 536/1081] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 14:11:49 2022] PUT /identity/v3/projects/c6f995ffa0a94b1584f25dbe31e20524/users/8c87f361a92d4834854ca07f652be2ff/roles/82029ff85919459780b85031cdfc369f => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:11:49.755078 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:49.768576 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1242f26-679d-418d-a461-bbfb0e20da64 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:49.808092 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353aa230-e078-4936-a9e8-efb5f6e56c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:49.808773 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353aa230-e078-4936-a9e8-efb5f6e56c1b None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:49.809061 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353aa230-e078-4936-a9e8-efb5f6e56c1b None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:49.809343 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353aa230-e078-4936-a9e8-efb5f6e56c1b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:49.888756 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353aa230-e078-4936-a9e8-efb5f6e56c1b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:49.890326 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353aa230-e078-4936-a9e8-efb5f6e56c1b None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:49.981629 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:49.999232 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 546/1082] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:11:49 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 19 14:11:50.000892 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5512eb6e-30fb-4492-8453-b65cef827127 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:50.111445 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5512eb6e-30fb-4492-8453-b65cef827127 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:50.152644 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606dd429-5119-41f2-a64f-519e70625ff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:50.153808 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606dd429-5119-41f2-a64f-519e70625ff5 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:50.154353 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606dd429-5119-41f2-a64f-519e70625ff5 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:50.154867 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606dd429-5119-41f2-a64f-519e70625ff5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:50.166872 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606dd429-5119-41f2-a64f-519e70625ff5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:50.168734 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606dd429-5119-41f2-a64f-519e70625ff5 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:50.187896 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-606dd429-5119-41f2-a64f-519e70625ff5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:11:50.187896 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:50.187896 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:11:50.187896 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:11:50.187896 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:11:50.187896 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:11:50.187896 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:11:50.187896 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:11:50.187896 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:11:50.187896 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:11:50.187896 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:11:50.187896 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:11:50.187896 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:11:50.187896 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:11:50.187896 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:11:50.188871 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:11:50.188871 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:11:50.188871 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:11:50.188871 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:11:50.188871 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:11:50.188871 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:11:50.188871 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:11:50.188871 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:11:50.188871 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:11:50.188871 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:11:50.188871 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:11:50.188871 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:11:50.188871 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:11:50.188871 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:11:50.188871 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:11:50.188871 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:11:50.188871 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:11:50.189780 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:11:50.189780 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:11:50.189780 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:11:50.189780 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:11:50.191381 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 537/1083] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:11:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:11:50.197208 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353aa230-e078-4936-a9e8-efb5f6e56c1b None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:50.302248 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353aa230-e078-4936-a9e8-efb5f6e56c1b None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:50.336191 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9104e9b5-6a22-469e-b19b-30af0c197637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:50.336855 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9104e9b5-6a22-469e-b19b-30af0c197637 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:50.337141 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9104e9b5-6a22-469e-b19b-30af0c197637 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:50.337416 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9104e9b5-6a22-469e-b19b-30af0c197637 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:50.338165 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9104e9b5-6a22-469e-b19b-30af0c197637 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:50.338473 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9104e9b5-6a22-469e-b19b-30af0c197637 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:50.339837 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9104e9b5-6a22-469e-b19b-30af0c197637 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:50.357871 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 547/1084] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:11:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:51.613067 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2e8fed-a1c8-4bb6-9bda-cf8fb0f6881d None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:51.613621 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2e8fed-a1c8-4bb6-9bda-cf8fb0f6881d None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:51.613721 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2e8fed-a1c8-4bb6-9bda-cf8fb0f6881d None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:51.615140 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 538/1085] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:11:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:11:51.620644 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bc9737-0d3f-411d-8f43-8fcd9828bd36 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:51.621054 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bc9737-0d3f-411d-8f43-8fcd9828bd36 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:51.621214 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bc9737-0d3f-411d-8f43-8fcd9828bd36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:51.638236 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-19bc9737-0d3f-411d-8f43-8fcd9828bd36 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:11:51.639431 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-19bc9737-0d3f-411d-8f43-8fcd9828bd36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:11:51.685343 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:51.708758 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 548/1086] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:11:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:11:51.732001 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2e8fed-a1c8-4bb6-9bda-cf8fb0f6881d None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:51.766688 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:51.839062 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2e8fed-a1c8-4bb6-9bda-cf8fb0f6881d None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:51.875897 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6ad899-0858-4260-8459-2a27a328d198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:51.876699 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6ad899-0858-4260-8459-2a27a328d198 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:51.876981 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6ad899-0858-4260-8459-2a27a328d198 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:51.877256 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6ad899-0858-4260-8459-2a27a328d198 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:51.887724 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6ad899-0858-4260-8459-2a27a328d198 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:11:51.895535 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-bc6ad899-0858-4260-8459-2a27a328d198 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:11:51.895535 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:51.895535 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:11:51.895535 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:11:51.895535 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:11:51.895535 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:11:51.895535 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:11:51.895535 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:11:51.895535 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:11:51.895535 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:11:51.895535 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:11:51.895535 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:11:51.895535 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:11:51.895535 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:11:51.895535 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:11:51.895535 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:11:51.896634 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:11:51.896634 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:11:51.896634 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:11:51.896634 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:11:51.896634 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:11:51.896634 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:11:51.896634 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:11:51.896634 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:11:51.896634 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:11:51.896634 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:11:51.896634 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:11:51.896634 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:11:51.896634 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:11:51.896634 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:11:51.896634 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:11:51.896634 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:11:51.897498 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:11:51.897498 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:11:51.897498 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:11:51.897498 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:11:51.897498 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:11:51.897498 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:11:51.897498 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:11:51.897498 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:11:51.897498 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:11:51.897498 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:11:51.897498 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:11:51.897498 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:11:51.897498 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 539/1087] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:11:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:11:51.902802 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bc9737-0d3f-411d-8f43-8fcd9828bd36 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:51.911706 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b642d7-68c4-469a-9bdf-eb3f78af2b83 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:51.911997 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b642d7-68c4-469a-9bdf-eb3f78af2b83 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:51.912273 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b642d7-68c4-469a-9bdf-eb3f78af2b83 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:51.913624 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 540/1088] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:11:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:11:51.918712 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ef0764-c71d-4b0f-b286-6e397277c759 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:51.919003 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ef0764-c71d-4b0f-b286-6e397277c759 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:51.919285 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ef0764-c71d-4b0f-b286-6e397277c759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:51.936248 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-c0ef0764-c71d-4b0f-b286-6e397277c759 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:11:51.938636 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-c0ef0764-c71d-4b0f-b286-6e397277c759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:11:52.003622 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bc9737-0d3f-411d-8f43-8fcd9828bd36 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:52.005048 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 541/1089] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:11:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:11:52.028674 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ef0764-c71d-4b0f-b286-6e397277c759 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:52.048649 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532c0a15-1686-44e0-ba7e-06077284134c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:52.049815 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532c0a15-1686-44e0-ba7e-06077284134c None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:52.050147 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532c0a15-1686-44e0-ba7e-06077284134c None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:52.050427 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532c0a15-1686-44e0-ba7e-06077284134c None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:52.051252 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532c0a15-1686-44e0-ba7e-06077284134c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:52.051376 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:52.051560 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532c0a15-1686-44e0-ba7e-06077284134c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:52.052964 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532c0a15-1686-44e0-ba7e-06077284134c None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:52.061522 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 549/1090] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:11:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:52.067823 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532c0a15-1686-44e0-ba7e-06077284134c None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:52.110424 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:52.136213 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ef0764-c71d-4b0f-b286-6e397277c759 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:52.173569 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b680ddf8-31d1-4356-92d4-974a7793d7fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:52.174584 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b680ddf8-31d1-4356-92d4-974a7793d7fa None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:52.174869 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b680ddf8-31d1-4356-92d4-974a7793d7fa None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:52.175145 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b680ddf8-31d1-4356-92d4-974a7793d7fa None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:52.179086 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532c0a15-1686-44e0-ba7e-06077284134c None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:52.197956 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b680ddf8-31d1-4356-92d4-974a7793d7fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:11:52.206273 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-b680ddf8-31d1-4356-92d4-974a7793d7fa None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 14:11:52.206273 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:52.206273 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:11:52.206273 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:11:52.206273 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:11:52.206273 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:11:52.206273 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:11:52.206273 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:11:52.206273 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:11:52.206273 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:11:52.206273 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:11:52.206273 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:11:52.206273 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:11:52.206273 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:11:52.206273 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:11:52.206273 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:11:52.207402 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:11:52.207402 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:11:52.207402 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:11:52.207402 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:11:52.207402 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:11:52.207402 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:11:52.207402 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:11:52.207402 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:11:52.207402 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:11:52.207402 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:11:52.207402 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:11:52.207402 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:11:52.207402 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:11:52.207402 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:11:52.207402 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:11:52.207402 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:11:52.208285 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:11:52.208285 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:11:52.208285 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:11:52.208285 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:11:52.208285 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:11:52.208285 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:11:52.208285 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:11:52.208285 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:11:52.208285 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:11:52.208285 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:11:52.208285 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 14:11:52.208285 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:11:52.208285 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 542/1091] 10.222.0.57 () {56 vars in 1120 bytes} [Thu May 19 14:11:52 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:11:52.213384 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b680ddf8-31d1-4356-92d4-974a7793d7fa None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:52.220657 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461313ea-bd82-469b-b83f-fbeff9685bb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:52.221539 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461313ea-bd82-469b-b83f-fbeff9685bb3 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:52.221823 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461313ea-bd82-469b-b83f-fbeff9685bb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:52.222135 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461313ea-bd82-469b-b83f-fbeff9685bb3 None admin] PATH_INFO: `/v3/users/7db9f2236f9b4ee88fca9db84dff7818` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:52.223352 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461313ea-bd82-469b-b83f-fbeff9685bb3 None admin] RBAC: Authorizing `identity:get_user(user_id=7db9f2236f9b4ee88fca9db84dff7818)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:52.224800 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461313ea-bd82-469b-b83f-fbeff9685bb3 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:52.226716 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 550/1092] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:11:52 2022] GET /identity/v3/users/7db9f2236f9b4ee88fca9db84dff7818 => generated 285 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:11:52.232278 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461313ea-bd82-469b-b83f-fbeff9685bb3 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:52.307742 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:52.330161 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b680ddf8-31d1-4356-92d4-974a7793d7fa None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:52.342445 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461313ea-bd82-469b-b83f-fbeff9685bb3 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:52.354694 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:52.369737 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5e6df5-93e6-41ef-81bc-dc123cb55afa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:52.370781 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5e6df5-93e6-41ef-81bc-dc123cb55afa None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:52.371157 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5e6df5-93e6-41ef-81bc-dc123cb55afa None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:52.371456 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5e6df5-93e6-41ef-81bc-dc123cb55afa None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:52.372358 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5e6df5-93e6-41ef-81bc-dc123cb55afa None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:52.372662 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5e6df5-93e6-41ef-81bc-dc123cb55afa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:52.374091 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5e6df5-93e6-41ef-81bc-dc123cb55afa None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:52.382680 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6757e1ca-ddc7-45d7-8e10-f954f5011158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:52.383618 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6757e1ca-ddc7-45d7-8e10-f954f5011158 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:52.383754 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 543/1093] 10.222.0.57 () {56 vars in 1115 bytes} [Thu May 19 14:11:52 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:11:52.383856 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6757e1ca-ddc7-45d7-8e10-f954f5011158 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:52.384137 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6757e1ca-ddc7-45d7-8e10-f954f5011158 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:52.389236 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5e6df5-93e6-41ef-81bc-dc123cb55afa None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:52.457985 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6757e1ca-ddc7-45d7-8e10-f954f5011158 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:52.459476 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6757e1ca-ddc7-45d7-8e10-f954f5011158 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:52.497322 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5e6df5-93e6-41ef-81bc-dc123cb55afa None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:52.535230 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7df3674-1eff-422a-a0ff-d3b97cd9d169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:52.537492 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7df3674-1eff-422a-a0ff-d3b97cd9d169 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:52.537785 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7df3674-1eff-422a-a0ff-d3b97cd9d169 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:52.538096 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7df3674-1eff-422a-a0ff-d3b97cd9d169 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:52.549555 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7df3674-1eff-422a-a0ff-d3b97cd9d169 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:52.551039 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7df3674-1eff-422a-a0ff-d3b97cd9d169 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:52.552800 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:52.570104 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-b7df3674-1eff-422a-a0ff-d3b97cd9d169 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:11:52.570104 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:52.570104 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:11:52.570104 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:11:52.570104 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:11:52.570104 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:11:52.570104 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:11:52.570104 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:11:52.570104 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:11:52.570104 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:11:52.570104 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:11:52.570104 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:11:52.570104 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:11:52.570104 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:11:52.570104 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:11:52.571080 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:11:52.571080 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:11:52.571080 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:11:52.571080 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:11:52.571080 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:11:52.571080 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:11:52.571080 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:11:52.571080 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:11:52.571080 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:11:52.571080 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:11:52.571080 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:11:52.571080 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:11:52.571080 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:11:52.571080 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:11:52.571080 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:11:52.571080 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:11:52.571080 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:11:52.572006 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:11:52.572006 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:11:52.572006 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:11:52.572006 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:11:52.572241 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 544/1094] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:11:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:11:52.579321 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7df3674-1eff-422a-a0ff-d3b97cd9d169 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:52.587634 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 551/1095] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:11:52 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 19 14:11:52.594035 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6757e1ca-ddc7-45d7-8e10-f954f5011158 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:52.606970 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:52.697042 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7df3674-1eff-422a-a0ff-d3b97cd9d169 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:52.705307 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6757e1ca-ddc7-45d7-8e10-f954f5011158 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:52.737950 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309d4499-e693-4a1e-9e26-b9376357fd36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:52.738614 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309d4499-e693-4a1e-9e26-b9376357fd36 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:52.738894 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309d4499-e693-4a1e-9e26-b9376357fd36 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:52.739165 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309d4499-e693-4a1e-9e26-b9376357fd36 None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:52.739917 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309d4499-e693-4a1e-9e26-b9376357fd36 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:52.740220 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309d4499-e693-4a1e-9e26-b9376357fd36 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:52.740964 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b642221-0283-4500-ac1c-8f8c74e47370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:52.741626 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309d4499-e693-4a1e-9e26-b9376357fd36 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:52.741823 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b642221-0283-4500-ac1c-8f8c74e47370 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:52.742151 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b642221-0283-4500-ac1c-8f8c74e47370 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:52.742429 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b642221-0283-4500-ac1c-8f8c74e47370 None admin] PATH_INFO: `/v3/projects/83f70b997931496a926ba305064bcb65` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:52.743630 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b642221-0283-4500-ac1c-8f8c74e47370 None admin] RBAC: Authorizing `identity:get_project(project_id=83f70b997931496a926ba305064bcb65)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:52.745073 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b642221-0283-4500-ac1c-8f8c74e47370 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:52.746820 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 552/1096] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:11:52 2022] GET /identity/v3/projects/83f70b997931496a926ba305064bcb65 => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:11:52.756160 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b642221-0283-4500-ac1c-8f8c74e47370 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:52.757567 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 545/1097] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:11:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:11:52.763244 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309d4499-e693-4a1e-9e26-b9376357fd36 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:52.824878 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:52.849999 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:52.874612 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b642221-0283-4500-ac1c-8f8c74e47370 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:52.876182 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309d4499-e693-4a1e-9e26-b9376357fd36 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:52.912521 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cf1747-19e0-4735-8b3d-09d38c701b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:52.913186 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cf1747-19e0-4735-8b3d-09d38c701b18 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:52.913473 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cf1747-19e0-4735-8b3d-09d38c701b18 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:52.913756 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cf1747-19e0-4735-8b3d-09d38c701b18 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:52.914632 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cf1747-19e0-4735-8b3d-09d38c701b18 None admin] RBAC: Adding query filter params (role.id=4946cf0ff7164307b36efeee7c787146, scope.project.id=83f70b997931496a926ba305064bcb65, user.id=7db9f2236f9b4ee88fca9db84dff7818) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:52.914946 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cf1747-19e0-4735-8b3d-09d38c701b18 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:52.915934 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddca1f4a-b1a8-44e2-99cd-5fc10983e565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:52.916385 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cf1747-19e0-4735-8b3d-09d38c701b18 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:52.916726 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddca1f4a-b1a8-44e2-99cd-5fc10983e565 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:52.917008 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddca1f4a-b1a8-44e2-99cd-5fc10983e565 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:52.917284 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddca1f4a-b1a8-44e2-99cd-5fc10983e565 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:52.922508 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 553/1098] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:11:52 2022] GET /identity/v3/role_assignments?user.id=7db9f2236f9b4ee88fca9db84dff7818&scope.project.id=83f70b997931496a926ba305064bcb65&role.id=4946cf0ff7164307b36efeee7c787146 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:11:52.924631 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddca1f4a-b1a8-44e2-99cd-5fc10983e565 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:52.926110 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddca1f4a-b1a8-44e2-99cd-5fc10983e565 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:52.941956 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-ddca1f4a-b1a8-44e2-99cd-5fc10983e565 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 14:11:52.941956 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:52.941956 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:11:52.941956 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:11:52.941956 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:11:52.941956 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:11:52.941956 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:11:52.941956 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:11:52.941956 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:11:52.941956 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:11:52.941956 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:11:52.941956 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:11:52.941956 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:11:52.941956 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:11:52.941956 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:11:52.941956 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:11:52.943183 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:11:52.943183 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:11:52.943183 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:11:52.943183 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:11:52.943183 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:11:52.943183 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:11:52.943183 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:11:52.943183 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:11:52.943183 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:11:52.943183 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:11:52.943183 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:11:52.943183 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:11:52.943183 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:11:52.943183 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:11:52.943183 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:11:52.943183 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:11:52.943183 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:11:52.943183 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:11:52.949597 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:11:52.949597 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:11:52.949597 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:11:52.949597 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:11:52.949597 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:11:52.949597 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:11:52.949597 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:11:52.949597 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:11:52.949597 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 14:11:52.949597 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:11:52.949597 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 546/1099] 10.222.0.57 () {56 vars in 1093 bytes} [Thu May 19 14:11:52 2022] GET /identity/v3/users/nova => generated 82 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:11:52.950442 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cf1747-19e0-4735-8b3d-09d38c701b18 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:53.059667 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cf1747-19e0-4735-8b3d-09d38c701b18 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:53.096330 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0ce7a2-e1b1-4431-820f-74b543b75ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:53.097007 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0ce7a2-e1b1-4431-820f-74b543b75ff4 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:53.097293 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0ce7a2-e1b1-4431-820f-74b543b75ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:53.097575 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0ce7a2-e1b1-4431-820f-74b543b75ff4 None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:53.098472 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0ce7a2-e1b1-4431-820f-74b543b75ff4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:53.098781 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0ce7a2-e1b1-4431-820f-74b543b75ff4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:53.100240 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0ce7a2-e1b1-4431-820f-74b543b75ff4 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:53.116554 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:53.121474 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 554/1100] 10.222.0.57 () {56 vars in 1133 bytes} [Thu May 19 14:11:52 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:53.127897 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddca1f4a-b1a8-44e2-99cd-5fc10983e565 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:53.237389 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddca1f4a-b1a8-44e2-99cd-5fc10983e565 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:53.252958 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:53.274376 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3e7942-c033-47e3-b6fc-994c25d13a76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:53.275034 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3e7942-c033-47e3-b6fc-994c25d13a76 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:53.275320 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3e7942-c033-47e3-b6fc-994c25d13a76 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:53.275597 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3e7942-c033-47e3-b6fc-994c25d13a76 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:53.343789 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3e7942-c033-47e3-b6fc-994c25d13a76 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:53.345238 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3e7942-c033-47e3-b6fc-994c25d13a76 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:53.443085 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 547/1101] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:11:53 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:11:53.448855 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0ce7a2-e1b1-4431-820f-74b543b75ff4 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:53.547707 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0ce7a2-e1b1-4431-820f-74b543b75ff4 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:53.582331 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef0d9af-6240-4719-9842-29db0a83d816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:53.583126 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef0d9af-6240-4719-9842-29db0a83d816 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:53.583410 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef0d9af-6240-4719-9842-29db0a83d816 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:53.583688 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef0d9af-6240-4719-9842-29db0a83d816 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:53.593554 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef0d9af-6240-4719-9842-29db0a83d816 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:53.595038 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef0d9af-6240-4719-9842-29db0a83d816 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:53.613177 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-7ef0d9af-6240-4719-9842-29db0a83d816 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:11:53.613177 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:53.613177 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:11:53.613177 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:11:53.613177 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:11:53.613177 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:11:53.613177 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:11:53.613177 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:11:53.613177 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:11:53.613177 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:11:53.613177 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:11:53.613177 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:11:53.613177 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:11:53.613177 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:11:53.613177 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:11:53.614212 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:11:53.614212 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:11:53.614212 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:11:53.614212 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:11:53.614212 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:11:53.614212 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:11:53.614212 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:11:53.614212 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:11:53.614212 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:11:53.614212 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:11:53.614212 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:11:53.614212 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:11:53.614212 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:11:53.614212 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:11:53.614212 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:11:53.614212 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:11:53.614212 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:11:53.615150 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:11:53.615150 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:11:53.615150 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:11:53.615150 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:11:53.615150 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 555/1102] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:11:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:11:53.620008 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3e7942-c033-47e3-b6fc-994c25d13a76 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:53.636274 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:53.723444 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3e7942-c033-47e3-b6fc-994c25d13a76 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:53.757655 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b347105c-d39d-45d7-8537-9391ebaa3516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:53.758345 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b347105c-d39d-45d7-8537-9391ebaa3516 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:53.758629 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b347105c-d39d-45d7-8537-9391ebaa3516 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:53.758902 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b347105c-d39d-45d7-8537-9391ebaa3516 None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:53.759609 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b347105c-d39d-45d7-8537-9391ebaa3516 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:53.759914 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b347105c-d39d-45d7-8537-9391ebaa3516 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:53.761274 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b347105c-d39d-45d7-8537-9391ebaa3516 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:53.781268 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 548/1103] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:11:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:53.786929 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef0d9af-6240-4719-9842-29db0a83d816 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:53.847475 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:53.894018 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef0d9af-6240-4719-9842-29db0a83d816 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:53.928288 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08075e9e-ae72-47b0-9773-16f112731d4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:53.929005 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08075e9e-ae72-47b0-9773-16f112731d4a None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:53.929292 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08075e9e-ae72-47b0-9773-16f112731d4a None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:53.929567 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08075e9e-ae72-47b0-9773-16f112731d4a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:53.936113 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08075e9e-ae72-47b0-9773-16f112731d4a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:53.937825 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08075e9e-ae72-47b0-9773-16f112731d4a None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:53.951287 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-08075e9e-ae72-47b0-9773-16f112731d4a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:11:53.951287 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:11:53.951287 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:11:53.951287 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:11:53.951287 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:11:53.951287 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:11:53.951287 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:11:53.951287 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:11:53.951287 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:11:53.951287 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:11:53.951287 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:11:53.951287 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:11:53.951287 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:11:53.951287 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:11:53.951287 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:11:53.952293 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:11:53.952293 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:11:53.952293 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:11:53.952293 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:11:53.952293 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:11:53.952293 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:11:53.952293 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:11:53.952293 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:11:53.952293 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:11:53.952293 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:11:53.952293 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:11:53.952293 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:11:53.952293 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:11:53.952293 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:11:53.952293 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:11:53.952293 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:11:53.952293 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:11:53.953237 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:11:53.953237 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:11:53.953237 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:11:53.953237 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:11:53.953237 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:11:53.953237 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:11:53.953237 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:11:53.953237 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:11:53.953237 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 556/1104] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:11:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:11:53.958641 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b347105c-d39d-45d7-8537-9391ebaa3516 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:54.035062 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:54.060362 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b347105c-d39d-45d7-8537-9391ebaa3516 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:54.095596 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403506a1-013c-4a16-973f-d7fc302446d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:54.096263 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403506a1-013c-4a16-973f-d7fc302446d5 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:54.096547 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403506a1-013c-4a16-973f-d7fc302446d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:54.096825 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403506a1-013c-4a16-973f-d7fc302446d5 None admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:54.097665 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403506a1-013c-4a16-973f-d7fc302446d5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:54.097954 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403506a1-013c-4a16-973f-d7fc302446d5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:54.099402 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403506a1-013c-4a16-973f-d7fc302446d5 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:54.112810 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 549/1105] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:11:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:54.118803 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08075e9e-ae72-47b0-9773-16f112731d4a None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:54.221044 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08075e9e-ae72-47b0-9773-16f112731d4a None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:54.233580 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:54.255037 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad42feb7-f083-4a19-af2a-cfaefc0cab24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:54.255692 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad42feb7-f083-4a19-af2a-cfaefc0cab24 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:54.255996 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad42feb7-f083-4a19-af2a-cfaefc0cab24 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:54.256264 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad42feb7-f083-4a19-af2a-cfaefc0cab24 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:54.257087 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad42feb7-f083-4a19-af2a-cfaefc0cab24 None admin] RBAC: Adding query filter params (role.id=82029ff85919459780b85031cdfc369f, scope.project.id=c6f995ffa0a94b1584f25dbe31e20524, user.id=8c87f361a92d4834854ca07f652be2ff) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:54.257398 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad42feb7-f083-4a19-af2a-cfaefc0cab24 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:54.258853 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad42feb7-f083-4a19-af2a-cfaefc0cab24 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:54.265124 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 557/1106] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:11:54 2022] GET /identity/v3/role_assignments?user.id=8c87f361a92d4834854ca07f652be2ff&scope.project.id=c6f995ffa0a94b1584f25dbe31e20524&role.id=82029ff85919459780b85031cdfc369f => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:55.192985 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc573d3-441e-495c-a7be-440c211c67cc None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:55.193527 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc573d3-441e-495c-a7be-440c211c67cc None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:55.193527 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc573d3-441e-495c-a7be-440c211c67cc None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:55.194940 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 550/1107] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:11:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:11:55.200325 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae283eba-695e-4a62-8b4e-406fdeff4c5f None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:55.200739 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae283eba-695e-4a62-8b4e-406fdeff4c5f None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:55.200882 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae283eba-695e-4a62-8b4e-406fdeff4c5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:55.217528 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-ae283eba-695e-4a62-8b4e-406fdeff4c5f None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:11:55.219510 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-ae283eba-695e-4a62-8b4e-406fdeff4c5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:11:55.283984 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 558/1108] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:11:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:11:55.309003 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc573d3-441e-495c-a7be-440c211c67cc None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:55.408921 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc573d3-441e-495c-a7be-440c211c67cc None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:55.441702 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98b2343-16b2-46d5-9e19-6ecf4b5db640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:55.442579 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98b2343-16b2-46d5-9e19-6ecf4b5db640 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:55.442862 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98b2343-16b2-46d5-9e19-6ecf4b5db640 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:55.443137 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98b2343-16b2-46d5-9e19-6ecf4b5db640 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:55.444309 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98b2343-16b2-46d5-9e19-6ecf4b5db640 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:55.445737 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98b2343-16b2-46d5-9e19-6ecf4b5db640 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:55.447372 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 551/1109] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:11:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:55.453079 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae283eba-695e-4a62-8b4e-406fdeff4c5f None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:55.549167 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae283eba-695e-4a62-8b4e-406fdeff4c5f None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:55.556235 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:55.582809 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f8bbf9-82c5-497c-857b-5f012d99ad47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:55.583463 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f8bbf9-82c5-497c-857b-5f012d99ad47 None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:55.583745 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f8bbf9-82c5-497c-857b-5f012d99ad47 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:55.584016 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f8bbf9-82c5-497c-857b-5f012d99ad47 None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:55.584727 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f8bbf9-82c5-497c-857b-5f012d99ad47 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:55.586180 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f8bbf9-82c5-497c-857b-5f012d99ad47 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:55.605933 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 559/1110] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 19 14:11:55 2022] POST /identity/v3/users => generated 286 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:11:57.856782 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f704a6-f79a-4685-ae2f-e4611db74a90 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:57.857316 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f704a6-f79a-4685-ae2f-e4611db74a90 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:57.857316 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f704a6-f79a-4685-ae2f-e4611db74a90 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:57.859390 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 552/1111] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:11:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:11:57.864843 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7d3669-5456-477b-9496-d508f578c577 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:57.865251 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7d3669-5456-477b-9496-d508f578c577 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:57.865396 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7d3669-5456-477b-9496-d508f578c577 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:57.885802 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-4e7d3669-5456-477b-9496-d508f578c577 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:11:57.887320 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-4e7d3669-5456-477b-9496-d508f578c577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:11:57.953627 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 560/1112] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:11:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:11:57.977305 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f704a6-f79a-4685-ae2f-e4611db74a90 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:57.980295 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:58.078533 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f704a6-f79a-4685-ae2f-e4611db74a90 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:58.113166 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462970e8-2ec3-46c2-8235-fc34083b477c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:58.114075 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462970e8-2ec3-46c2-8235-fc34083b477c None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:58.114354 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462970e8-2ec3-46c2-8235-fc34083b477c None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:58.114623 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462970e8-2ec3-46c2-8235-fc34083b477c None admin] PATH_INFO: `/v3/roles/23d9dd31632340bdba9cf3093c095dbb` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:58.116097 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462970e8-2ec3-46c2-8235-fc34083b477c None admin] RBAC: Authorizing `identity:get_role(role_id=23d9dd31632340bdba9cf3093c095dbb)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:58.117467 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462970e8-2ec3-46c2-8235-fc34083b477c None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:58.118862 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 553/1113] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:11:57 2022] GET /identity/v3/roles/23d9dd31632340bdba9cf3093c095dbb => generated 223 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:58.125810 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7d3669-5456-477b-9496-d508f578c577 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:58.221274 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:58.231304 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7d3669-5456-477b-9496-d508f578c577 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:58.265435 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4846b55-578d-43ac-a2ca-c69041ec4191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:58.266334 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4846b55-578d-43ac-a2ca-c69041ec4191 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:58.266619 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4846b55-578d-43ac-a2ca-c69041ec4191 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:58.266893 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4846b55-578d-43ac-a2ca-c69041ec4191 None admin] PATH_INFO: `/v3/users/6d5a7f68a74046fbaeef43027fe9d258` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:58.279589 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4846b55-578d-43ac-a2ca-c69041ec4191 None admin] RBAC: Authorizing `identity:get_user(user_id=6d5a7f68a74046fbaeef43027fe9d258)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:58.281619 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4846b55-578d-43ac-a2ca-c69041ec4191 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:58.283288 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 561/1114] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:11:58 2022] GET /identity/v3/users/6d5a7f68a74046fbaeef43027fe9d258 => generated 286 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:58.288981 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462970e8-2ec3-46c2-8235-fc34083b477c None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:58.355343 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:58.385368 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462970e8-2ec3-46c2-8235-fc34083b477c None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:58.418116 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e5fac5-0844-4e21-8f15-6e36254cb718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:58.418744 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e5fac5-0844-4e21-8f15-6e36254cb718 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:58.419024 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e5fac5-0844-4e21-8f15-6e36254cb718 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:58.419299 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e5fac5-0844-4e21-8f15-6e36254cb718 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:58.485450 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e5fac5-0844-4e21-8f15-6e36254cb718 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:58.486924 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e5fac5-0844-4e21-8f15-6e36254cb718 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:58.551331 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:11:58.583119 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 554/1115] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:11:58 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:11:58.588926 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4846b55-578d-43ac-a2ca-c69041ec4191 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:58.685130 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4846b55-578d-43ac-a2ca-c69041ec4191 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:58.719120 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45979dc2-b1e1-4634-8c24-6ffe90d52f86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:58.719961 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45979dc2-b1e1-4634-8c24-6ffe90d52f86 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:58.720242 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45979dc2-b1e1-4634-8c24-6ffe90d52f86 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:58.720514 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45979dc2-b1e1-4634-8c24-6ffe90d52f86 None admin] PATH_INFO: `/v3/projects/83f70b997931496a926ba305064bcb65` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:58.721672 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45979dc2-b1e1-4634-8c24-6ffe90d52f86 None admin] RBAC: Authorizing `identity:get_project(project_id=83f70b997931496a926ba305064bcb65)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:58.723125 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45979dc2-b1e1-4634-8c24-6ffe90d52f86 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:58.724796 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 562/1116] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:11:58 2022] GET /identity/v3/projects/83f70b997931496a926ba305064bcb65 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:11:58.730841 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e5fac5-0844-4e21-8f15-6e36254cb718 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:11:58.827381 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e5fac5-0844-4e21-8f15-6e36254cb718 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:11:58.860270 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc21fd3f-e2e0-47d8-89c0-189fd04700c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:11:58.860912 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc21fd3f-e2e0-47d8-89c0-189fd04700c2 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:11:58.861193 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc21fd3f-e2e0-47d8-89c0-189fd04700c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:11:58.861466 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc21fd3f-e2e0-47d8-89c0-189fd04700c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:11:58.862569 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc21fd3f-e2e0-47d8-89c0-189fd04700c2 None admin] RBAC: Adding query filter params (role.id=23d9dd31632340bdba9cf3093c095dbb, scope.project.id=83f70b997931496a926ba305064bcb65, user.id=6d5a7f68a74046fbaeef43027fe9d258) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:11:58.862880 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc21fd3f-e2e0-47d8-89c0-189fd04700c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:11:58.864296 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc21fd3f-e2e0-47d8-89c0-189fd04700c2 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:11:58.870132 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 555/1117] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:11:58 2022] GET /identity/v3/role_assignments?user.id=6d5a7f68a74046fbaeef43027fe9d258&scope.project.id=83f70b997931496a926ba305064bcb65&role.id=23d9dd31632340bdba9cf3093c095dbb => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:12:01.060116 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e1ff54-c0f7-4d89-83c0-95b8e4977e14 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:12:01.060653 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e1ff54-c0f7-4d89-83c0-95b8e4977e14 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:12:01.060653 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e1ff54-c0f7-4d89-83c0-95b8e4977e14 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:12:01.062231 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 563/1118] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:12:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:12:01.067318 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02c68d0-001f-416b-bc4b-43056a9ce539 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:12:01.067698 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02c68d0-001f-416b-bc4b-43056a9ce539 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:12:01.067886 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02c68d0-001f-416b-bc4b-43056a9ce539 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:12:01.085017 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-c02c68d0-001f-416b-bc4b-43056a9ce539 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:12:01.086258 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-c02c68d0-001f-416b-bc4b-43056a9ce539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:12:01.113041 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:12:01.153810 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 556/1119] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:12:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:12:01.176554 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e1ff54-c0f7-4d89-83c0-95b8e4977e14 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:12:01.195730 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:12:01.278214 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e1ff54-c0f7-4d89-83c0-95b8e4977e14 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:12:01.311978 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc99a443-8bb0-4f84-8085-6f0b0e1105f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:12:01.312759 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc99a443-8bb0-4f84-8085-6f0b0e1105f3 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:12:01.313041 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc99a443-8bb0-4f84-8085-6f0b0e1105f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:12:01.313317 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc99a443-8bb0-4f84-8085-6f0b0e1105f3 None admin] PATH_INFO: `/v3/roles/23d9dd31632340bdba9cf3093c095dbb` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:12:01.314813 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc99a443-8bb0-4f84-8085-6f0b0e1105f3 None admin] RBAC: Authorizing `identity:get_role(role_id=23d9dd31632340bdba9cf3093c095dbb)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:12:01.316192 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc99a443-8bb0-4f84-8085-6f0b0e1105f3 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:12:01.317571 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 564/1120] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:12:01 2022] GET /identity/v3/roles/23d9dd31632340bdba9cf3093c095dbb => generated 223 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:12:01.323788 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02c68d0-001f-416b-bc4b-43056a9ce539 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:12:01.418760 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02c68d0-001f-416b-bc4b-43056a9ce539 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:12:01.451803 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57d27a6-e31a-4fde-8739-99ccc81bdf51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:12:01.453159 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57d27a6-e31a-4fde-8739-99ccc81bdf51 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:12:01.453434 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57d27a6-e31a-4fde-8739-99ccc81bdf51 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:12:01.453703 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57d27a6-e31a-4fde-8739-99ccc81bdf51 None admin] PATH_INFO: `/v3/users/6d5a7f68a74046fbaeef43027fe9d258` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:12:01.454911 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57d27a6-e31a-4fde-8739-99ccc81bdf51 None admin] RBAC: Authorizing `identity:get_user(user_id=6d5a7f68a74046fbaeef43027fe9d258)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:12:01.456328 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57d27a6-e31a-4fde-8739-99ccc81bdf51 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:12:01.457958 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 557/1121] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:12:01 2022] GET /identity/v3/users/6d5a7f68a74046fbaeef43027fe9d258 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:12:01.463560 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc99a443-8bb0-4f84-8085-6f0b0e1105f3 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:12:01.545573 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:12:01.559997 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc99a443-8bb0-4f84-8085-6f0b0e1105f3 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:12:01.593693 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e37d2fe-b2d7-48d9-91bc-d18af0bec699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:12:01.594518 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e37d2fe-b2d7-48d9-91bc-d18af0bec699 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:12:01.594809 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e37d2fe-b2d7-48d9-91bc-d18af0bec699 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:12:01.595091 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e37d2fe-b2d7-48d9-91bc-d18af0bec699 None admin] PATH_INFO: `/v3/projects/83f70b997931496a926ba305064bcb65` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:12:01.596269 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e37d2fe-b2d7-48d9-91bc-d18af0bec699 None admin] RBAC: Authorizing `identity:get_project(project_id=83f70b997931496a926ba305064bcb65)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:12:01.597709 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e37d2fe-b2d7-48d9-91bc-d18af0bec699 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:12:01.599548 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 565/1122] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:12:01 2022] GET /identity/v3/projects/83f70b997931496a926ba305064bcb65 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:12:01.605315 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57d27a6-e31a-4fde-8739-99ccc81bdf51 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:12:01.637102 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:12:01.705920 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57d27a6-e31a-4fde-8739-99ccc81bdf51 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:12:01.740007 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9434e6a-50ed-4586-84c5-f6a77b185269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:12:01.740890 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9434e6a-50ed-4586-84c5-f6a77b185269 None admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:12:01.741175 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9434e6a-50ed-4586-84c5-f6a77b185269 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:12:01.741454 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9434e6a-50ed-4586-84c5-f6a77b185269 None admin] PATH_INFO: `/v3/projects/83f70b997931496a926ba305064bcb65/users/6d5a7f68a74046fbaeef43027fe9d258/roles/23d9dd31632340bdba9cf3093c095dbb` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:12:01.743679 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9434e6a-50ed-4586-84c5-f6a77b185269 None admin] RBAC: Authorizing `identity:create_grant(project_id=83f70b997931496a926ba305064bcb65, user_id=6d5a7f68a74046fbaeef43027fe9d258, role_id=23d9dd31632340bdba9cf3093c095dbb)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:12:01.745116 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9434e6a-50ed-4586-84c5-f6a77b185269 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:12:01.758446 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 558/1123] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 14:12:01 2022] PUT /identity/v3/projects/83f70b997931496a926ba305064bcb65/users/6d5a7f68a74046fbaeef43027fe9d258/roles/23d9dd31632340bdba9cf3093c095dbb => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:12:03.950477 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bc1972-a671-4f38-adb9-7c8a0bb1e35a None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:12:03.951013 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bc1972-a671-4f38-adb9-7c8a0bb1e35a None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:12:03.951013 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bc1972-a671-4f38-adb9-7c8a0bb1e35a None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:12:03.952899 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 566/1124] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:12:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:12:03.958319 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09b9e5c-6d58-481b-b8a6-d6c1cbfee09f None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:12:03.958734 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09b9e5c-6d58-481b-b8a6-d6c1cbfee09f None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:12:03.958877 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09b9e5c-6d58-481b-b8a6-d6c1cbfee09f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:12:03.976195 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-b09b9e5c-6d58-481b-b8a6-d6c1cbfee09f None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:12:03.977384 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-b09b9e5c-6d58-481b-b8a6-d6c1cbfee09f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:12:04.042558 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 559/1125] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:12:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:12:04.066150 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bc1972-a671-4f38-adb9-7c8a0bb1e35a None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:12:04.169252 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bc1972-a671-4f38-adb9-7c8a0bb1e35a None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:12:04.202958 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed43046-25b2-4fb5-8551-b3aca592d9db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:12:04.203734 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed43046-25b2-4fb5-8551-b3aca592d9db None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:12:04.204011 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed43046-25b2-4fb5-8551-b3aca592d9db None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:12:04.204281 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed43046-25b2-4fb5-8551-b3aca592d9db None admin] PATH_INFO: `/v3/roles/23d9dd31632340bdba9cf3093c095dbb` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:12:04.205732 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed43046-25b2-4fb5-8551-b3aca592d9db None admin] RBAC: Authorizing `identity:get_role(role_id=23d9dd31632340bdba9cf3093c095dbb)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:12:04.207126 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed43046-25b2-4fb5-8551-b3aca592d9db None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:12:04.208493 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 567/1126] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:12:04 2022] GET /identity/v3/roles/23d9dd31632340bdba9cf3093c095dbb => generated 223 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:12:04.214274 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09b9e5c-6d58-481b-b8a6-d6c1cbfee09f None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:12:04.238036 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:12:04.313379 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09b9e5c-6d58-481b-b8a6-d6c1cbfee09f None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:12:04.346144 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9913ffb-972c-4d93-a499-0a20634b74ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:12:04.347003 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9913ffb-972c-4d93-a499-0a20634b74ad None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:12:04.347284 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9913ffb-972c-4d93-a499-0a20634b74ad None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:12:04.347557 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9913ffb-972c-4d93-a499-0a20634b74ad None admin] PATH_INFO: `/v3/users/6d5a7f68a74046fbaeef43027fe9d258` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:12:04.348729 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9913ffb-972c-4d93-a499-0a20634b74ad None admin] RBAC: Authorizing `identity:get_user(user_id=6d5a7f68a74046fbaeef43027fe9d258)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:12:04.350199 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9913ffb-972c-4d93-a499-0a20634b74ad None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:12:04.351813 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 560/1127] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:12:04 2022] GET /identity/v3/users/6d5a7f68a74046fbaeef43027fe9d258 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:12:04.357480 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed43046-25b2-4fb5-8551-b3aca592d9db None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:12:04.381782 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:12:04.454544 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed43046-25b2-4fb5-8551-b3aca592d9db None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:12:04.487355 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4417d49e-5d59-402a-a58d-00fa74f97dd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:12:04.487986 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4417d49e-5d59-402a-a58d-00fa74f97dd5 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:12:04.488265 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4417d49e-5d59-402a-a58d-00fa74f97dd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:12:04.488535 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4417d49e-5d59-402a-a58d-00fa74f97dd5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:12:04.552502 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4417d49e-5d59-402a-a58d-00fa74f97dd5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:12:04.554049 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4417d49e-5d59-402a-a58d-00fa74f97dd5 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:12:04.576152 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:12:04.649512 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 568/1128] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:12:04 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:12:04.655491 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9913ffb-972c-4d93-a499-0a20634b74ad None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:12:04.743826 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:12:04.754376 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9913ffb-972c-4d93-a499-0a20634b74ad None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:12:04.787980 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0cfff0-f992-419b-928b-1fe9c9432152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:12:04.788836 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0cfff0-f992-419b-928b-1fe9c9432152 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:12:04.789123 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0cfff0-f992-419b-928b-1fe9c9432152 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:12:04.789401 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0cfff0-f992-419b-928b-1fe9c9432152 None admin] PATH_INFO: `/v3/projects/83f70b997931496a926ba305064bcb65` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:12:04.790599 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0cfff0-f992-419b-928b-1fe9c9432152 None admin] RBAC: Authorizing `identity:get_project(project_id=83f70b997931496a926ba305064bcb65)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:12:04.792035 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0cfff0-f992-419b-928b-1fe9c9432152 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:12:04.793729 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 561/1129] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:12:04 2022] GET /identity/v3/projects/83f70b997931496a926ba305064bcb65 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:12:04.800416 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4417d49e-5d59-402a-a58d-00fa74f97dd5 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:12:04.896729 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4417d49e-5d59-402a-a58d-00fa74f97dd5 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:12:04.918648 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:12:04.930795 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90608fa1-372e-41f3-a955-3b9418562fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:12:04.931440 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90608fa1-372e-41f3-a955-3b9418562fe2 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:12:04.931722 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90608fa1-372e-41f3-a955-3b9418562fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:12:04.931996 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90608fa1-372e-41f3-a955-3b9418562fe2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:12:04.932822 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90608fa1-372e-41f3-a955-3b9418562fe2 None admin] RBAC: Adding query filter params (role.id=23d9dd31632340bdba9cf3093c095dbb, scope.project.id=83f70b997931496a926ba305064bcb65, user.id=6d5a7f68a74046fbaeef43027fe9d258) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:12:04.933131 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90608fa1-372e-41f3-a955-3b9418562fe2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:12:04.934587 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90608fa1-372e-41f3-a955-3b9418562fe2 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:12:04.940955 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 569/1130] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:12:04 2022] GET /identity/v3/role_assignments?user.id=6d5a7f68a74046fbaeef43027fe9d258&scope.project.id=83f70b997931496a926ba305064bcb65&role.id=23d9dd31632340bdba9cf3093c095dbb => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:12:07.175574 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02856fc-ede9-4fa6-a9b3-da148fb47540 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:12:07.176113 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02856fc-ede9-4fa6-a9b3-da148fb47540 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:12:07.176113 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02856fc-ede9-4fa6-a9b3-da148fb47540 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:12:07.177482 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 562/1131] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:12:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:12:07.182735 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5486507-e1b3-42c9-9c46-255847b712b8 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:12:07.183131 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5486507-e1b3-42c9-9c46-255847b712b8 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:12:07.183287 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5486507-e1b3-42c9-9c46-255847b712b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:12:07.200352 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-d5486507-e1b3-42c9-9c46-255847b712b8 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:12:07.202274 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-d5486507-e1b3-42c9-9c46-255847b712b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:12:07.266812 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 570/1132] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:12:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:12:07.289906 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02856fc-ede9-4fa6-a9b3-da148fb47540 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:12:07.389612 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02856fc-ede9-4fa6-a9b3-da148fb47540 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:12:07.422493 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d2739a-b92b-41f5-8fdd-95c735ac31d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:12:07.423274 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d2739a-b92b-41f5-8fdd-95c735ac31d1 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:12:07.423556 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d2739a-b92b-41f5-8fdd-95c735ac31d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:12:07.423831 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d2739a-b92b-41f5-8fdd-95c735ac31d1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:12:07.424994 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d2739a-b92b-41f5-8fdd-95c735ac31d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:12:07.426458 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d2739a-b92b-41f5-8fdd-95c735ac31d1 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:12:07.428076 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 563/1133] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:12:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:12:07.433761 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5486507-e1b3-42c9-9c46-255847b712b8 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:12:07.529303 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5486507-e1b3-42c9-9c46-255847b712b8 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:12:07.545283 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:12:07.563183 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d2ea70-5507-4403-aa4e-4623676d119e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:12:07.563832 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d2ea70-5507-4403-aa4e-4623676d119e None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:12:07.564116 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d2ea70-5507-4403-aa4e-4623676d119e None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:12:07.564391 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d2ea70-5507-4403-aa4e-4623676d119e None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:12:07.565098 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d2ea70-5507-4403-aa4e-4623676d119e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:12:07.566565 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d2ea70-5507-4403-aa4e-4623676d119e None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:12:07.591883 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 571/1134] 10.222.0.57 () {60 vars in 1141 bytes} [Thu May 19 14:12:07 2022] POST /identity/v3/projects => generated 311 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:12:09.839734 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7d539c-77f4-4870-85ed-5f5225dc7dd4 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:12:09.840297 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7d539c-77f4-4870-85ed-5f5225dc7dd4 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:12:09.840297 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7d539c-77f4-4870-85ed-5f5225dc7dd4 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:12:09.841647 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 564/1135] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:12:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:12:09.847521 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001b7fca-8653-4078-8a0c-504e345f180b None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:12:09.847962 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001b7fca-8653-4078-8a0c-504e345f180b None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:12:09.848140 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001b7fca-8653-4078-8a0c-504e345f180b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:12:09.865206 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-001b7fca-8653-4078-8a0c-504e345f180b None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:12:09.866431 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-001b7fca-8653-4078-8a0c-504e345f180b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:12:09.931889 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 572/1136] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:12:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:12:09.956746 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7d539c-77f4-4870-85ed-5f5225dc7dd4 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:12:09.984647 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:12:10.058131 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7d539c-77f4-4870-85ed-5f5225dc7dd4 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:12:10.092719 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c84fea-e99f-4e8c-bc3a-c9c3a6fb7fea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:12:10.093415 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c84fea-e99f-4e8c-bc3a-c9c3a6fb7fea None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:12:10.093694 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c84fea-e99f-4e8c-bc3a-c9c3a6fb7fea None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:12:10.093996 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c84fea-e99f-4e8c-bc3a-c9c3a6fb7fea None admin] PATH_INFO: `/v3/domains/default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:12:10.095186 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c84fea-e99f-4e8c-bc3a-c9c3a6fb7fea None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:12:10.096622 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c84fea-e99f-4e8c-bc3a-c9c3a6fb7fea None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:12:10.098274 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 565/1137] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:12:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:12:10.104316 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001b7fca-8653-4078-8a0c-504e345f180b None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:12:10.202731 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:12:10.207039 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001b7fca-8653-4078-8a0c-504e345f180b None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:12:10.240328 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ce7bf9-7feb-47b1-9860-b4c87442f6da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:12:10.240990 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ce7bf9-7feb-47b1-9860-b4c87442f6da None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:12:10.241270 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ce7bf9-7feb-47b1-9860-b4c87442f6da None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:12:10.241543 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ce7bf9-7feb-47b1-9860-b4c87442f6da None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:12:10.242289 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ce7bf9-7feb-47b1-9860-b4c87442f6da None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:12:10.243729 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ce7bf9-7feb-47b1-9860-b4c87442f6da None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:12:10.263288 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 573/1138] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 19 14:12:10 2022] POST /identity/v3/users => generated 286 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:12:12.519038 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58269f1-01bd-447c-8577-4222da4ae5ac None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:12:12.519563 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58269f1-01bd-447c-8577-4222da4ae5ac None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:12:12.519649 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58269f1-01bd-447c-8577-4222da4ae5ac None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:12:12.520963 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 566/1139] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:12:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:12:12.526089 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6f948d-62ce-435e-ab66-21491a4e758f None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:12:12.526472 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6f948d-62ce-435e-ab66-21491a4e758f None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:12:12.526648 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6f948d-62ce-435e-ab66-21491a4e758f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:12:12.543663 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-4f6f948d-62ce-435e-ab66-21491a4e758f None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:12:12.545425 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-4f6f948d-62ce-435e-ab66-21491a4e758f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:12:12.610266 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 574/1140] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:12:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:12:12.633126 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58269f1-01bd-447c-8577-4222da4ae5ac None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:12:12.726395 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:12:12.735180 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58269f1-01bd-447c-8577-4222da4ae5ac None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:12:12.768308 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68e5a3c-c97a-448c-945a-914c812cab4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:12:12.769150 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68e5a3c-c97a-448c-945a-914c812cab4e None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:12:12.769378 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68e5a3c-c97a-448c-945a-914c812cab4e None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:12:12.769653 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68e5a3c-c97a-448c-945a-914c812cab4e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:12:12.780079 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68e5a3c-c97a-448c-945a-914c812cab4e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:12:12.787676 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-a68e5a3c-c97a-448c-945a-914c812cab4e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:12:12.787676 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:12:12.787676 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:12:12.787676 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:12:12.787676 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:12:12.787676 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:12:12.787676 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:12:12.787676 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:12:12.787676 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:12:12.787676 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:12:12.787676 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:12:12.787676 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:12:12.787676 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:12:12.787676 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:12:12.787676 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:12:12.787676 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:12:12.788790 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:12:12.788790 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:12:12.788790 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:12:12.788790 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:12:12.788790 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:12:12.788790 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:12:12.788790 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:12:12.788790 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:12:12.788790 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:12:12.788790 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:12:12.788790 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:12:12.788790 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:12:12.788790 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:12:12.788790 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:12:12.788790 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:12:12.788790 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:12:12.789887 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:12:12.789887 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:12:12.789887 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:12:12.789887 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:12:12.789887 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:12:12.789887 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:12:12.789887 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:12:12.789887 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:12:12.789887 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:12:12.789887 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:12:12.789887 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:12:12.789887 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:12:12.789887 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 567/1141] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:12:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:12:12.795552 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6f948d-62ce-435e-ab66-21491a4e758f None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:12:12.877247 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:12:12.894243 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6f948d-62ce-435e-ab66-21491a4e758f None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:12:12.927971 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd4a514-8bcd-46e3-aec1-ce590c293ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:12:12.928632 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd4a514-8bcd-46e3-aec1-ce590c293ba3 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:12:12.928920 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd4a514-8bcd-46e3-aec1-ce590c293ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:12:12.929200 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd4a514-8bcd-46e3-aec1-ce590c293ba3 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:12:12.929968 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd4a514-8bcd-46e3-aec1-ce590c293ba3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:12:12.930280 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd4a514-8bcd-46e3-aec1-ce590c293ba3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:12:12.931656 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd4a514-8bcd-46e3-aec1-ce590c293ba3 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:12:12.939543 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 575/1142] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:12:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:12:12.945255 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68e5a3c-c97a-448c-945a-914c812cab4e None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:12:13.043546 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68e5a3c-c97a-448c-945a-914c812cab4e None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:12:13.075390 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def2f60c-4653-49b2-a8ca-030d891d7fdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:12:13.076933 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def2f60c-4653-49b2-a8ca-030d891d7fdd None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:12:13.077387 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def2f60c-4653-49b2-a8ca-030d891d7fdd None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:12:13.077668 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def2f60c-4653-49b2-a8ca-030d891d7fdd None admin] PATH_INFO: `/v3/users/ab36ac04e37f4a4cb2146aa7229c294f` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:12:13.090511 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def2f60c-4653-49b2-a8ca-030d891d7fdd None admin] RBAC: Authorizing `identity:get_user(user_id=ab36ac04e37f4a4cb2146aa7229c294f)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:12:13.092736 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def2f60c-4653-49b2-a8ca-030d891d7fdd None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:12:13.094400 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 568/1143] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:12:12 2022] GET /identity/v3/users/ab36ac04e37f4a4cb2146aa7229c294f => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:12:13.100188 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd4a514-8bcd-46e3-aec1-ce590c293ba3 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:12:13.199251 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd4a514-8bcd-46e3-aec1-ce590c293ba3 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:12:13.232135 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b874988-cf1a-45a7-a367-639de8bbd389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:12:13.232805 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b874988-cf1a-45a7-a367-639de8bbd389 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:12:13.233089 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b874988-cf1a-45a7-a367-639de8bbd389 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:12:13.233365 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b874988-cf1a-45a7-a367-639de8bbd389 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:12:13.248747 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:12:13.299727 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b874988-cf1a-45a7-a367-639de8bbd389 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:12:13.301111 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b874988-cf1a-45a7-a367-639de8bbd389 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:12:13.396901 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 576/1144] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:12:13 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:12:13.402790 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def2f60c-4653-49b2-a8ca-030d891d7fdd None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:12:13.417486 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:12:13.501629 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def2f60c-4653-49b2-a8ca-030d891d7fdd None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:12:13.534983 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1471db-50c9-4dd5-9653-8a51d27efd70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:12:13.535830 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1471db-50c9-4dd5-9653-8a51d27efd70 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:12:13.536108 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1471db-50c9-4dd5-9653-8a51d27efd70 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:12:13.536377 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1471db-50c9-4dd5-9653-8a51d27efd70 None admin] PATH_INFO: `/v3/projects/f9ff614f02224c508733d388527f354e` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:12:13.537896 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1471db-50c9-4dd5-9653-8a51d27efd70 None admin] RBAC: Authorizing `identity:get_project(project_id=f9ff614f02224c508733d388527f354e)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:12:13.539328 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1471db-50c9-4dd5-9653-8a51d27efd70 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:12:13.540987 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 569/1145] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:12:13 2022] GET /identity/v3/projects/f9ff614f02224c508733d388527f354e => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:12:13.547098 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b874988-cf1a-45a7-a367-639de8bbd389 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:12:13.592992 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:12:13.645935 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b874988-cf1a-45a7-a367-639de8bbd389 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:12:13.679701 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47327a6f-e39f-4160-ae3b-95ff463a60c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:12:13.680348 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47327a6f-e39f-4160-ae3b-95ff463a60c6 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:12:13.680633 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47327a6f-e39f-4160-ae3b-95ff463a60c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:12:13.680911 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47327a6f-e39f-4160-ae3b-95ff463a60c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:12:13.681738 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47327a6f-e39f-4160-ae3b-95ff463a60c6 None admin] RBAC: Adding query filter params (role.id=4946cf0ff7164307b36efeee7c787146, scope.project.id=f9ff614f02224c508733d388527f354e, user.id=ab36ac04e37f4a4cb2146aa7229c294f) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:12:13.682088 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47327a6f-e39f-4160-ae3b-95ff463a60c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:12:13.683519 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47327a6f-e39f-4160-ae3b-95ff463a60c6 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:12:13.689564 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 577/1146] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:12:13 2022] GET /identity/v3/role_assignments?user.id=ab36ac04e37f4a4cb2146aa7229c294f&scope.project.id=f9ff614f02224c508733d388527f354e&role.id=4946cf0ff7164307b36efeee7c787146 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:12:15.878865 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1c9c74-32b5-4e9f-89a1-91fef04bd435 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:12:15.879414 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1c9c74-32b5-4e9f-89a1-91fef04bd435 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:12:15.879529 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1c9c74-32b5-4e9f-89a1-91fef04bd435 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:12:15.880878 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 570/1147] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:12:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:12:15.886176 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7266c37a-6aae-43d5-9242-fa3a0e8edead None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:12:15.886632 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7266c37a-6aae-43d5-9242-fa3a0e8edead None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:12:15.886727 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7266c37a-6aae-43d5-9242-fa3a0e8edead None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:12:15.904902 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-7266c37a-6aae-43d5-9242-fa3a0e8edead None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:12:15.906116 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-7266c37a-6aae-43d5-9242-fa3a0e8edead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:12:15.970465 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 578/1148] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:12:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:12:15.993095 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1c9c74-32b5-4e9f-89a1-91fef04bd435 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:12:16.073604 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:12:16.094718 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1c9c74-32b5-4e9f-89a1-91fef04bd435 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:12:16.129529 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3841fa-2c7e-487b-9bc6-b6216859873b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:12:16.130306 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3841fa-2c7e-487b-9bc6-b6216859873b None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:12:16.130587 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3841fa-2c7e-487b-9bc6-b6216859873b None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:12:16.130863 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3841fa-2c7e-487b-9bc6-b6216859873b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:12:16.141605 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3841fa-2c7e-487b-9bc6-b6216859873b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:12:16.149233 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-4f3841fa-2c7e-487b-9bc6-b6216859873b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:12:16.149233 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:12:16.149233 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:12:16.149233 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:12:16.149233 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:12:16.149233 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:12:16.149233 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:12:16.149233 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:12:16.149233 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:12:16.149233 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:12:16.149233 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:12:16.149233 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:12:16.149233 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:12:16.149233 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:12:16.149233 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:12:16.149233 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:12:16.150405 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:12:16.150405 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:12:16.150405 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:12:16.150405 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:12:16.150405 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:12:16.150405 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:12:16.150405 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:12:16.150405 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:12:16.150405 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:12:16.150405 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:12:16.150405 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:12:16.150405 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:12:16.150405 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:12:16.150405 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:12:16.150405 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:12:16.150405 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:12:16.151464 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:12:16.151464 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:12:16.151464 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:12:16.151464 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:12:16.151464 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:12:16.151464 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:12:16.151464 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:12:16.151464 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:12:16.151464 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:12:16.151464 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:12:16.151464 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:12:16.151464 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:12:16.151464 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 571/1149] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:12:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:12:16.157140 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7266c37a-6aae-43d5-9242-fa3a0e8edead None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:12:16.197240 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:12:16.254142 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7266c37a-6aae-43d5-9242-fa3a0e8edead None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:12:16.286952 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba0d186-4102-4f0e-bf68-9cfd1543ebbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:12:16.287613 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba0d186-4102-4f0e-bf68-9cfd1543ebbd None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:12:16.287897 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba0d186-4102-4f0e-bf68-9cfd1543ebbd None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:12:16.288170 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba0d186-4102-4f0e-bf68-9cfd1543ebbd None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:12:16.288904 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba0d186-4102-4f0e-bf68-9cfd1543ebbd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:12:16.289210 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba0d186-4102-4f0e-bf68-9cfd1543ebbd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:12:16.290605 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba0d186-4102-4f0e-bf68-9cfd1543ebbd None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:12:16.298510 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 579/1150] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:12:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:12:16.304402 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3841fa-2c7e-487b-9bc6-b6216859873b None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:12:16.403526 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3841fa-2c7e-487b-9bc6-b6216859873b None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:12:16.436004 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc541264-abe1-49df-8e17-d6f0ebe9f189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:12:16.436829 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc541264-abe1-49df-8e17-d6f0ebe9f189 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:12:16.437106 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc541264-abe1-49df-8e17-d6f0ebe9f189 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:12:16.437377 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc541264-abe1-49df-8e17-d6f0ebe9f189 None admin] PATH_INFO: `/v3/users/ab36ac04e37f4a4cb2146aa7229c294f` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:12:16.438513 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc541264-abe1-49df-8e17-d6f0ebe9f189 None admin] RBAC: Authorizing `identity:get_user(user_id=ab36ac04e37f4a4cb2146aa7229c294f)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:12:16.439927 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc541264-abe1-49df-8e17-d6f0ebe9f189 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:12:16.441516 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 572/1151] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:12:16 2022] GET /identity/v3/users/ab36ac04e37f4a4cb2146aa7229c294f => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:12:16.447197 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba0d186-4102-4f0e-bf68-9cfd1543ebbd None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:12:16.544747 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba0d186-4102-4f0e-bf68-9cfd1543ebbd None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:12:16.546276 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:12:16.579289 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da723d0-b656-421a-8ee0-5dd976a2bf53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:12:16.580139 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da723d0-b656-421a-8ee0-5dd976a2bf53 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:12:16.580424 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da723d0-b656-421a-8ee0-5dd976a2bf53 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:12:16.580702 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da723d0-b656-421a-8ee0-5dd976a2bf53 None admin] PATH_INFO: `/v3/projects/f9ff614f02224c508733d388527f354e` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:12:16.581899 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da723d0-b656-421a-8ee0-5dd976a2bf53 None admin] RBAC: Authorizing `identity:get_project(project_id=f9ff614f02224c508733d388527f354e)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:12:16.583343 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da723d0-b656-421a-8ee0-5dd976a2bf53 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:12:16.585028 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 580/1152] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:12:16 2022] GET /identity/v3/projects/f9ff614f02224c508733d388527f354e => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:12:16.590915 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc541264-abe1-49df-8e17-d6f0ebe9f189 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:12:16.602201 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:12:16.690397 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc541264-abe1-49df-8e17-d6f0ebe9f189 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:12:16.722429 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4a9301-ef57-4e87-878d-92c0b901692a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:12:16.723302 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4a9301-ef57-4e87-878d-92c0b901692a None admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:12:16.723582 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4a9301-ef57-4e87-878d-92c0b901692a None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:12:16.723854 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4a9301-ef57-4e87-878d-92c0b901692a None admin] PATH_INFO: `/v3/projects/f9ff614f02224c508733d388527f354e/users/ab36ac04e37f4a4cb2146aa7229c294f/roles/4946cf0ff7164307b36efeee7c787146` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:12:16.725770 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4a9301-ef57-4e87-878d-92c0b901692a None admin] RBAC: Authorizing `identity:create_grant(project_id=f9ff614f02224c508733d388527f354e, user_id=ab36ac04e37f4a4cb2146aa7229c294f, role_id=4946cf0ff7164307b36efeee7c787146)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:12:16.727230 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4a9301-ef57-4e87-878d-92c0b901692a None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:12:16.740933 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 573/1153] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 14:12:16 2022] PUT /identity/v3/projects/f9ff614f02224c508733d388527f354e/users/ab36ac04e37f4a4cb2146aa7229c294f/roles/4946cf0ff7164307b36efeee7c787146 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:12:18.942791 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485c25b5-6201-4b2c-9010-e32f41bf9770 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:12:18.943342 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485c25b5-6201-4b2c-9010-e32f41bf9770 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:12:18.943342 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485c25b5-6201-4b2c-9010-e32f41bf9770 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:12:18.944692 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 581/1154] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:12:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:12:18.949998 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f178cbf-d7c7-4328-a939-33946baa1983 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:12:18.950390 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f178cbf-d7c7-4328-a939-33946baa1983 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:12:18.950566 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f178cbf-d7c7-4328-a939-33946baa1983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:12:18.967645 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-5f178cbf-d7c7-4328-a939-33946baa1983 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:12:18.968845 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-5f178cbf-d7c7-4328-a939-33946baa1983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:12:19.029984 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:12:19.036420 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 574/1155] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:12:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:12:19.061009 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485c25b5-6201-4b2c-9010-e32f41bf9770 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:12:19.163993 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485c25b5-6201-4b2c-9010-e32f41bf9770 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:12:19.197799 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef20a953-f1e2-4a09-95b8-226f6d02e485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:12:19.198564 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef20a953-f1e2-4a09-95b8-226f6d02e485 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:12:19.198845 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef20a953-f1e2-4a09-95b8-226f6d02e485 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:12:19.199118 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef20a953-f1e2-4a09-95b8-226f6d02e485 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:12:19.209259 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef20a953-f1e2-4a09-95b8-226f6d02e485 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:12:19.217372 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-ef20a953-f1e2-4a09-95b8-226f6d02e485 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:12:19.217372 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:12:19.217372 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:12:19.217372 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:12:19.217372 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:12:19.217372 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:12:19.217372 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:12:19.217372 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:12:19.217372 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:12:19.217372 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:12:19.217372 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:12:19.217372 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:12:19.217372 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:12:19.217372 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:12:19.217372 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:12:19.217372 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:12:19.218686 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:12:19.218686 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:12:19.218686 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:12:19.218686 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:12:19.218686 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:12:19.218686 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:12:19.218686 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:12:19.218686 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:12:19.218686 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:12:19.218686 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:12:19.218686 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:12:19.218686 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:12:19.218686 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:12:19.218686 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:12:19.218686 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:12:19.218686 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:12:19.219779 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:12:19.219779 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:12:19.219779 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:12:19.219779 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:12:19.219779 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:12:19.219779 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:12:19.219779 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:12:19.219779 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:12:19.219779 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:12:19.219779 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:12:19.219779 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:12:19.219779 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:12:19.219779 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 582/1156] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:12:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:12:19.224713 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f178cbf-d7c7-4328-a939-33946baa1983 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:12:19.321310 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f178cbf-d7c7-4328-a939-33946baa1983 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:12:19.354482 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7fbd8f-dc52-4d41-8831-9b968ced6059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:12:19.355136 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7fbd8f-dc52-4d41-8831-9b968ced6059 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:12:19.355418 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7fbd8f-dc52-4d41-8831-9b968ced6059 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:12:19.355691 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7fbd8f-dc52-4d41-8831-9b968ced6059 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:12:19.356423 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7fbd8f-dc52-4d41-8831-9b968ced6059 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:12:19.356727 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7fbd8f-dc52-4d41-8831-9b968ced6059 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:12:19.358861 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7fbd8f-dc52-4d41-8831-9b968ced6059 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:12:19.367522 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 575/1157] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:12:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:12:19.373211 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef20a953-f1e2-4a09-95b8-226f6d02e485 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:12:19.403412 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:12:19.472072 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef20a953-f1e2-4a09-95b8-226f6d02e485 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:12:19.504985 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8709f07c-9c2d-46b1-a189-4202595b1981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:12:19.505937 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8709f07c-9c2d-46b1-a189-4202595b1981 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:12:19.506165 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8709f07c-9c2d-46b1-a189-4202595b1981 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:12:19.506443 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8709f07c-9c2d-46b1-a189-4202595b1981 None admin] PATH_INFO: `/v3/users/ab36ac04e37f4a4cb2146aa7229c294f` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:12:19.507627 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8709f07c-9c2d-46b1-a189-4202595b1981 None admin] RBAC: Authorizing `identity:get_user(user_id=ab36ac04e37f4a4cb2146aa7229c294f)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:12:19.509078 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8709f07c-9c2d-46b1-a189-4202595b1981 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:12:19.510748 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 583/1158] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:12:19 2022] GET /identity/v3/users/ab36ac04e37f4a4cb2146aa7229c294f => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:12:19.516479 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7fbd8f-dc52-4d41-8831-9b968ced6059 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:12:19.574493 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:12:19.615466 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7fbd8f-dc52-4d41-8831-9b968ced6059 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:12:19.648220 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dcee24-6b15-4e00-97d9-08be85c3f0c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:12:19.648853 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dcee24-6b15-4e00-97d9-08be85c3f0c7 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:12:19.649136 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dcee24-6b15-4e00-97d9-08be85c3f0c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:12:19.649408 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dcee24-6b15-4e00-97d9-08be85c3f0c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:12:19.714315 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dcee24-6b15-4e00-97d9-08be85c3f0c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:12:19.715750 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dcee24-6b15-4e00-97d9-08be85c3f0c7 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:12:19.773909 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:12:19.815619 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 576/1159] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:12:19 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:12:19.821646 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8709f07c-9c2d-46b1-a189-4202595b1981 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:12:19.923793 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:12:19.927697 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8709f07c-9c2d-46b1-a189-4202595b1981 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:12:19.960394 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc68e040-17e4-43b9-a1fc-4ccbd12e551c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:12:19.961243 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc68e040-17e4-43b9-a1fc-4ccbd12e551c None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:12:19.961523 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc68e040-17e4-43b9-a1fc-4ccbd12e551c None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:12:19.961795 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc68e040-17e4-43b9-a1fc-4ccbd12e551c None admin] PATH_INFO: `/v3/projects/f9ff614f02224c508733d388527f354e` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:12:19.962986 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc68e040-17e4-43b9-a1fc-4ccbd12e551c None admin] RBAC: Authorizing `identity:get_project(project_id=f9ff614f02224c508733d388527f354e)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:12:19.964412 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc68e040-17e4-43b9-a1fc-4ccbd12e551c None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:12:19.966118 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 584/1160] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:12:19 2022] GET /identity/v3/projects/f9ff614f02224c508733d388527f354e => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:12:19.973102 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dcee24-6b15-4e00-97d9-08be85c3f0c7 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:12:20.069593 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dcee24-6b15-4e00-97d9-08be85c3f0c7 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:12:20.102546 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce6e22c-d8b3-4b0d-be37-51fcb125240a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:12:20.103192 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce6e22c-d8b3-4b0d-be37-51fcb125240a None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:12:20.103475 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce6e22c-d8b3-4b0d-be37-51fcb125240a None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:12:20.103749 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce6e22c-d8b3-4b0d-be37-51fcb125240a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:12:20.104588 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce6e22c-d8b3-4b0d-be37-51fcb125240a None admin] RBAC: Adding query filter params (role.id=4946cf0ff7164307b36efeee7c787146, scope.project.id=f9ff614f02224c508733d388527f354e, user.id=ab36ac04e37f4a4cb2146aa7229c294f) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:12:20.104898 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce6e22c-d8b3-4b0d-be37-51fcb125240a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:12:20.106387 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce6e22c-d8b3-4b0d-be37-51fcb125240a None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:12:20.113327 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 577/1161] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:12:19 2022] GET /identity/v3/role_assignments?user.id=ab36ac04e37f4a4cb2146aa7229c294f&scope.project.id=f9ff614f02224c508733d388527f354e&role.id=4946cf0ff7164307b36efeee7c787146 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:12:22.326828 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440ec946-0175-45de-8c66-db5ca67447dc None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:12:22.327379 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440ec946-0175-45de-8c66-db5ca67447dc None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:12:22.327379 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440ec946-0175-45de-8c66-db5ca67447dc None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:12:22.328754 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 585/1162] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:12:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:12:22.334019 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac757e1-17aa-459d-9a42-c167c3b1ca4b None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:12:22.334452 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac757e1-17aa-459d-9a42-c167c3b1ca4b None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:12:22.334579 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac757e1-17aa-459d-9a42-c167c3b1ca4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:12:22.351905 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-9ac757e1-17aa-459d-9a42-c167c3b1ca4b None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:12:22.353095 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-9ac757e1-17aa-459d-9a42-c167c3b1ca4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:12:22.370612 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:12:22.420581 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 578/1163] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:12:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:12:22.443082 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440ec946-0175-45de-8c66-db5ca67447dc None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:12:22.544205 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440ec946-0175-45de-8c66-db5ca67447dc None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:12:22.577747 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13412304-a16c-452f-a960-6f895dc2e365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:12:22.578408 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13412304-a16c-452f-a960-6f895dc2e365 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:12:22.578696 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13412304-a16c-452f-a960-6f895dc2e365 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:12:22.578977 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13412304-a16c-452f-a960-6f895dc2e365 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:12:22.611440 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:12:22.645768 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13412304-a16c-452f-a960-6f895dc2e365 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:12:22.647255 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13412304-a16c-452f-a960-6f895dc2e365 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:12:22.744404 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 586/1164] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:12:22 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:12:22.751228 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac757e1-17aa-459d-9a42-c167c3b1ca4b None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:12:22.846761 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac757e1-17aa-459d-9a42-c167c3b1ca4b None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:12:22.880775 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831e4e17-f277-46bf-9940-d678eb8cead9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:12:22.881565 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831e4e17-f277-46bf-9940-d678eb8cead9 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:12:22.881861 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831e4e17-f277-46bf-9940-d678eb8cead9 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:12:22.882158 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831e4e17-f277-46bf-9940-d678eb8cead9 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:12:22.888782 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831e4e17-f277-46bf-9940-d678eb8cead9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:12:22.890256 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831e4e17-f277-46bf-9940-d678eb8cead9 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:12:22.904326 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-831e4e17-f277-46bf-9940-d678eb8cead9 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. May 19 14:12:22.904326 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:12:22.904326 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 19 14:12:22.904326 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 19 14:12:22.904326 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:12:22.904326 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:12:22.904326 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:12:22.904326 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:12:22.904326 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. May 19 14:12:22.904326 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:12:22.904326 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:12:22.904326 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:12:22.904326 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:12:22.904326 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:12:22.904326 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:12:22.904326 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:12:22.904326 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:12:22.905703 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:12:22.905703 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:12:22.905703 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:12:22.905703 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:12:22.905703 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:12:22.905703 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:12:22.905703 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:12:22.905703 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:12:22.905703 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:12:22.905703 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:12:22.905703 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:12:22.905703 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:12:22.905703 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:12:22.905703 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:12:22.905703 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:12:22.905703 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:12:22.906607 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:12:22.906607 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:12:22.906607 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:12:22.906607 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:12:22.906607 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:12:22.906607 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:12:22.906607 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:12:22.906607 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:12:22.906607 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:12:22.906607 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:12:22.906607 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 19 14:12:22.906607 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:12:22.906607 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. May 19 14:12:22.906607 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:12:22.906607 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 579/1165] 10.222.0.57 () {56 vars in 1117 bytes} [Thu May 19 14:12:22 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:12:22.911481 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13412304-a16c-452f-a960-6f895dc2e365 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:12:22.974808 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:12:23.009293 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13412304-a16c-452f-a960-6f895dc2e365 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:12:23.042263 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3068d59-6cd5-482f-a8d1-42b7dd3a22aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:12:23.042911 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3068d59-6cd5-482f-a8d1-42b7dd3a22aa None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:12:23.043193 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3068d59-6cd5-482f-a8d1-42b7dd3a22aa None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:12:23.043467 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3068d59-6cd5-482f-a8d1-42b7dd3a22aa None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:12:23.044169 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3068d59-6cd5-482f-a8d1-42b7dd3a22aa None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:12:23.044611 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3068d59-6cd5-482f-a8d1-42b7dd3a22aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:12:23.045970 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3068d59-6cd5-482f-a8d1-42b7dd3a22aa None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:12:23.052897 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 587/1166] 10.222.0.57 () {56 vars in 1115 bytes} [Thu May 19 14:12:22 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:12:25.232619 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9874117a-292e-4b3a-8fc0-1a8bf76c17ab None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:12:25.233143 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9874117a-292e-4b3a-8fc0-1a8bf76c17ab None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:12:25.233283 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9874117a-292e-4b3a-8fc0-1a8bf76c17ab None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:12:25.234683 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 580/1167] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:12:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:12:25.240127 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b23042-8661-43cd-b6b0-71a4fe5c375f None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:12:25.240563 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b23042-8661-43cd-b6b0-71a4fe5c375f None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:12:25.240698 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b23042-8661-43cd-b6b0-71a4fe5c375f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:12:25.257813 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-37b23042-8661-43cd-b6b0-71a4fe5c375f None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:12:25.259024 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-37b23042-8661-43cd-b6b0-71a4fe5c375f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:12:25.324052 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 588/1168] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:12:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:12:25.348048 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9874117a-292e-4b3a-8fc0-1a8bf76c17ab None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:12:25.364632 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:12:25.449374 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9874117a-292e-4b3a-8fc0-1a8bf76c17ab None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:12:25.483728 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b0741c-1f5c-4930-9065-8f7cc9d158ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:12:25.484377 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b0741c-1f5c-4930-9065-8f7cc9d158ba None admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:12:25.484663 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b0741c-1f5c-4930-9065-8f7cc9d158ba None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:12:25.484942 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b0741c-1f5c-4930-9065-8f7cc9d158ba None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:12:25.485675 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b0741c-1f5c-4930-9065-8f7cc9d158ba None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:12:25.487348 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b0741c-1f5c-4930-9065-8f7cc9d158ba None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:12:25.502783 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 581/1169] 10.222.0.57 () {60 vars in 1138 bytes} [Thu May 19 14:12:25 2022] POST /identity/v3/domains => generated 267 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:12:27.730160 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da947737-3fb0-4a05-bc7d-e292e4587192 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:12:27.730692 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da947737-3fb0-4a05-bc7d-e292e4587192 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:12:27.730801 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da947737-3fb0-4a05-bc7d-e292e4587192 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:12:27.732825 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 589/1170] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:12:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:12:27.738544 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c5e243-ae28-40c8-a5bc-3b317ac629a0 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:12:27.738951 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c5e243-ae28-40c8-a5bc-3b317ac629a0 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:12:27.739103 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c5e243-ae28-40c8-a5bc-3b317ac629a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:12:27.756139 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-b4c5e243-ae28-40c8-a5bc-3b317ac629a0 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:12:27.757327 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-b4c5e243-ae28-40c8-a5bc-3b317ac629a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:12:27.822579 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 582/1171] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:12:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:12:27.845987 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da947737-3fb0-4a05-bc7d-e292e4587192 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:12:27.906158 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:12:27.952671 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da947737-3fb0-4a05-bc7d-e292e4587192 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:12:27.985680 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeb4c1b-2b0a-4d69-8e69-012c7b47f67b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:12:27.986769 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeb4c1b-2b0a-4d69-8e69-012c7b47f67b None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:12:27.987049 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeb4c1b-2b0a-4d69-8e69-012c7b47f67b None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:12:27.987320 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeb4c1b-2b0a-4d69-8e69-012c7b47f67b None admin] PATH_INFO: `/v3/domains/f8184698f7ba447883512b83ace178b4` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:12:27.997943 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddeb4c1b-2b0a-4d69-8e69-012c7b47f67b None admin] RBAC: Authorizing `identity:get_domain(domain_id=f8184698f7ba447883512b83ace178b4)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:12:27.999400 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddeb4c1b-2b0a-4d69-8e69-012c7b47f67b None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:12:28.001042 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 590/1172] 10.222.0.57 () {56 vars in 1183 bytes} [Thu May 19 14:12:27 2022] GET /identity/v3/domains/f8184698f7ba447883512b83ace178b4 => generated 267 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:12:28.006844 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c5e243-ae28-40c8-a5bc-3b317ac629a0 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:12:28.009933 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:12:28.102357 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c5e243-ae28-40c8-a5bc-3b317ac629a0 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:12:28.135710 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7cf1a5-fc49-49cb-bb2f-7251d037fc6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:12:28.136373 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7cf1a5-fc49-49cb-bb2f-7251d037fc6c None admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:12:28.136655 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7cf1a5-fc49-49cb-bb2f-7251d037fc6c None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:12:28.136930 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7cf1a5-fc49-49cb-bb2f-7251d037fc6c None admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:12:28.137647 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7cf1a5-fc49-49cb-bb2f-7251d037fc6c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:12:28.139794 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7cf1a5-fc49-49cb-bb2f-7251d037fc6c None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:12:28.167214 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 583/1173] 10.222.0.57 () {60 vars in 1141 bytes} [Thu May 19 14:12:28 2022] POST /identity/v3/projects => generated 361 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 14:12:30.395025 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22916cb5-69d3-4e5a-924f-2a4cb3065a0f None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:12:30.395554 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22916cb5-69d3-4e5a-924f-2a4cb3065a0f None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:12:30.395661 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22916cb5-69d3-4e5a-924f-2a4cb3065a0f None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:12:30.396974 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 591/1174] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:12:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:12:30.402156 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4ee140-61d0-4b9e-a993-13e91e8b65d1 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:12:30.402525 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4ee140-61d0-4b9e-a993-13e91e8b65d1 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:12:30.402725 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4ee140-61d0-4b9e-a993-13e91e8b65d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:12:30.420007 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-ac4ee140-61d0-4b9e-a993-13e91e8b65d1 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:12:30.421192 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-ac4ee140-61d0-4b9e-a993-13e91e8b65d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:12:30.471703 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:12:30.489124 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 584/1175] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:12:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:12:30.517296 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22916cb5-69d3-4e5a-924f-2a4cb3065a0f None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:12:30.615295 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22916cb5-69d3-4e5a-924f-2a4cb3065a0f None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:12:30.634917 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:12:30.649313 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e603e84-b785-4954-87dd-959b047870d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:12:30.650123 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e603e84-b785-4954-87dd-959b047870d8 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:12:30.650407 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e603e84-b785-4954-87dd-959b047870d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:12:30.650674 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e603e84-b785-4954-87dd-959b047870d8 None admin] PATH_INFO: `/v3/domains/f8184698f7ba447883512b83ace178b4` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:12:30.651832 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e603e84-b785-4954-87dd-959b047870d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=f8184698f7ba447883512b83ace178b4)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:12:30.653256 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e603e84-b785-4954-87dd-959b047870d8 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:12:30.655084 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 592/1176] 10.222.0.57 () {56 vars in 1183 bytes} [Thu May 19 14:12:30 2022] GET /identity/v3/domains/f8184698f7ba447883512b83ace178b4 => generated 267 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:12:30.660727 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4ee140-61d0-4b9e-a993-13e91e8b65d1 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:12:30.758205 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4ee140-61d0-4b9e-a993-13e91e8b65d1 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:12:30.791971 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0317900-eeb6-43d3-aa34-e2d3d28fa92c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:12:30.792628 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0317900-eeb6-43d3-aa34-e2d3d28fa92c None admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:12:30.792909 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0317900-eeb6-43d3-aa34-e2d3d28fa92c None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:12:30.793185 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0317900-eeb6-43d3-aa34-e2d3d28fa92c None admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:12:30.793933 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0317900-eeb6-43d3-aa34-e2d3d28fa92c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:12:30.795381 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0317900-eeb6-43d3-aa34-e2d3d28fa92c None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:12:30.818145 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 585/1177] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 19 14:12:30 2022] POST /identity/v3/users => generated 311 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:12:33.068682 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539a7616-897b-4e48-b1ed-0766f667e8e3 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:12:33.069238 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539a7616-897b-4e48-b1ed-0766f667e8e3 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:12:33.069238 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539a7616-897b-4e48-b1ed-0766f667e8e3 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:12:33.070624 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 593/1178] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:12:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:12:33.076476 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a76cf52-d9d1-415c-88fc-7e2e7a7d8a76 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:12:33.076866 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a76cf52-d9d1-415c-88fc-7e2e7a7d8a76 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:12:33.077037 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a76cf52-d9d1-415c-88fc-7e2e7a7d8a76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:12:33.094090 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-5a76cf52-d9d1-415c-88fc-7e2e7a7d8a76 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:12:33.095288 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-5a76cf52-d9d1-415c-88fc-7e2e7a7d8a76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:12:33.129452 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:12:33.161994 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 586/1179] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:12:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:12:33.185362 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539a7616-897b-4e48-b1ed-0766f667e8e3 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:12:33.284132 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539a7616-897b-4e48-b1ed-0766f667e8e3 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:12:33.316327 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96790ac-2eb2-45b3-a636-f1b11ba1c88f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:12:33.317110 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96790ac-2eb2-45b3-a636-f1b11ba1c88f None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:12:33.317391 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96790ac-2eb2-45b3-a636-f1b11ba1c88f None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:12:33.317663 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96790ac-2eb2-45b3-a636-f1b11ba1c88f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:12:33.328068 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96790ac-2eb2-45b3-a636-f1b11ba1c88f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:12:33.335599 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-b96790ac-2eb2-45b3-a636-f1b11ba1c88f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:12:33.335599 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:12:33.335599 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:12:33.335599 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:12:33.335599 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:12:33.335599 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:12:33.335599 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:12:33.335599 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:12:33.335599 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:12:33.335599 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:12:33.335599 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:12:33.335599 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:12:33.335599 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:12:33.335599 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:12:33.335599 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:12:33.335599 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:12:33.336712 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:12:33.336712 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:12:33.336712 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:12:33.336712 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:12:33.336712 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:12:33.336712 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:12:33.336712 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:12:33.336712 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:12:33.336712 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:12:33.336712 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:12:33.336712 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:12:33.336712 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:12:33.336712 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:12:33.336712 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:12:33.336712 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:12:33.336712 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:12:33.337880 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:12:33.337880 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:12:33.337880 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:12:33.337880 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:12:33.337880 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:12:33.337880 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:12:33.337880 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:12:33.337880 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:12:33.337880 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:12:33.337880 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:12:33.337880 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:12:33.337880 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:12:33.337880 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 594/1180] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:12:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:12:33.343935 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a76cf52-d9d1-415c-88fc-7e2e7a7d8a76 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:12:33.441871 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a76cf52-d9d1-415c-88fc-7e2e7a7d8a76 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:12:33.475388 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97f6f3c-7030-4fdd-ae69-984ccc50ccff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:12:33.476043 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97f6f3c-7030-4fdd-ae69-984ccc50ccff None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:12:33.476330 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97f6f3c-7030-4fdd-ae69-984ccc50ccff None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:12:33.476610 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97f6f3c-7030-4fdd-ae69-984ccc50ccff None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:12:33.477354 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97f6f3c-7030-4fdd-ae69-984ccc50ccff None admin] RBAC: Adding query filter params (name=admin) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:12:33.477668 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97f6f3c-7030-4fdd-ae69-984ccc50ccff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:12:33.479056 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97f6f3c-7030-4fdd-ae69-984ccc50ccff None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:12:33.487886 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 587/1181] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:12:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:12:33.493677 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96790ac-2eb2-45b3-a636-f1b11ba1c88f None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:12:33.542324 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:12:33.593171 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96790ac-2eb2-45b3-a636-f1b11ba1c88f None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:12:33.626874 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6628c776-41e1-4539-adfe-06d6af468f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:12:33.627739 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6628c776-41e1-4539-adfe-06d6af468f2e None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:12:33.628027 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6628c776-41e1-4539-adfe-06d6af468f2e None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:12:33.628306 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6628c776-41e1-4539-adfe-06d6af468f2e None admin] PATH_INFO: `/v3/users/56e4fd04b97d43da8c62394ad2d24e66` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:12:33.641327 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6628c776-41e1-4539-adfe-06d6af468f2e None admin] RBAC: Authorizing `identity:get_user(user_id=56e4fd04b97d43da8c62394ad2d24e66)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:12:33.642810 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6628c776-41e1-4539-adfe-06d6af468f2e None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:12:33.644462 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 595/1182] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:12:33 2022] GET /identity/v3/users/56e4fd04b97d43da8c62394ad2d24e66 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:12:33.650232 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97f6f3c-7030-4fdd-ae69-984ccc50ccff None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:12:33.685163 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:12:33.749596 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97f6f3c-7030-4fdd-ae69-984ccc50ccff None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:12:33.782302 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b11c4ed-fe93-4e02-93e8-b654a2010419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:12:33.782940 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b11c4ed-fe93-4e02-93e8-b654a2010419 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:12:33.783228 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b11c4ed-fe93-4e02-93e8-b654a2010419 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:12:33.783509 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b11c4ed-fe93-4e02-93e8-b654a2010419 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:12:33.849986 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b11c4ed-fe93-4e02-93e8-b654a2010419 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:12:33.851367 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b11c4ed-fe93-4e02-93e8-b654a2010419 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:12:33.882667 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:12:33.950463 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 588/1183] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:12:33 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:12:33.956747 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6628c776-41e1-4539-adfe-06d6af468f2e None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:12:34.054957 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6628c776-41e1-4539-adfe-06d6af468f2e None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:12:34.062652 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:12:34.089381 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290d7184-04e0-4175-a57c-8b45c33c183a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:12:34.090269 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290d7184-04e0-4175-a57c-8b45c33c183a None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:12:34.090551 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290d7184-04e0-4175-a57c-8b45c33c183a None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:12:34.090929 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290d7184-04e0-4175-a57c-8b45c33c183a None admin] PATH_INFO: `/v3/projects/64ccf5f1dbb741fbb2a5d275c32dd929` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:12:34.092102 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290d7184-04e0-4175-a57c-8b45c33c183a None admin] RBAC: Authorizing `identity:get_project(project_id=64ccf5f1dbb741fbb2a5d275c32dd929)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:12:34.093543 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290d7184-04e0-4175-a57c-8b45c33c183a None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:12:34.095270 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 596/1184] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:12:33 2022] GET /identity/v3/projects/64ccf5f1dbb741fbb2a5d275c32dd929 => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:12:34.101387 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b11c4ed-fe93-4e02-93e8-b654a2010419 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:12:34.198279 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b11c4ed-fe93-4e02-93e8-b654a2010419 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:12:34.232400 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9c1ec5-45de-4d0a-8bce-1693aab812bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:12:34.233060 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9c1ec5-45de-4d0a-8bce-1693aab812bd None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:12:34.233335 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9c1ec5-45de-4d0a-8bce-1693aab812bd None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:12:34.233610 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9c1ec5-45de-4d0a-8bce-1693aab812bd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:12:34.234461 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9c1ec5-45de-4d0a-8bce-1693aab812bd None admin] RBAC: Adding query filter params (role.id=4946cf0ff7164307b36efeee7c787146, scope.project.id=64ccf5f1dbb741fbb2a5d275c32dd929, user.id=56e4fd04b97d43da8c62394ad2d24e66) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:12:34.234777 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9c1ec5-45de-4d0a-8bce-1693aab812bd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:12:34.236487 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9c1ec5-45de-4d0a-8bce-1693aab812bd None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:12:34.238272 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:12:34.242256 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 589/1185] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:12:34 2022] GET /identity/v3/role_assignments?user.id=56e4fd04b97d43da8c62394ad2d24e66&scope.project.id=64ccf5f1dbb741fbb2a5d275c32dd929&role.id=4946cf0ff7164307b36efeee7c787146 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:12:36.430181 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a92509-0159-4f48-b09f-58c4783f7970 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:12:36.430704 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a92509-0159-4f48-b09f-58c4783f7970 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:12:36.430798 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a92509-0159-4f48-b09f-58c4783f7970 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:12:36.432084 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 597/1186] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:12:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:12:36.437650 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87645e7-6f07-40c9-a931-71c9fc55c374 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:12:36.438114 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87645e7-6f07-40c9-a931-71c9fc55c374 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:12:36.438239 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87645e7-6f07-40c9-a931-71c9fc55c374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:12:36.455301 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-e87645e7-6f07-40c9-a931-71c9fc55c374 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:12:36.456496 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-e87645e7-6f07-40c9-a931-71c9fc55c374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:12:36.521660 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 590/1187] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:12:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:12:36.544121 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a92509-0159-4f48-b09f-58c4783f7970 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:12:36.643997 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a92509-0159-4f48-b09f-58c4783f7970 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:12:36.676607 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54699de9-fc74-4f91-8385-f8946d0135d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:12:36.677387 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54699de9-fc74-4f91-8385-f8946d0135d3 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:12:36.677673 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54699de9-fc74-4f91-8385-f8946d0135d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:12:36.677980 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54699de9-fc74-4f91-8385-f8946d0135d3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:12:36.688781 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54699de9-fc74-4f91-8385-f8946d0135d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:12:36.696332 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-54699de9-fc74-4f91-8385-f8946d0135d3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:12:36.696332 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:12:36.696332 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:12:36.696332 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:12:36.696332 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:12:36.696332 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:12:36.696332 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:12:36.696332 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:12:36.696332 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:12:36.696332 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:12:36.696332 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:12:36.696332 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:12:36.696332 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:12:36.696332 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:12:36.696332 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:12:36.696332 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:12:36.697480 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:12:36.697480 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:12:36.697480 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:12:36.697480 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:12:36.697480 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:12:36.697480 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:12:36.697480 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:12:36.697480 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:12:36.697480 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:12:36.697480 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:12:36.697480 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:12:36.697480 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:12:36.697480 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:12:36.697480 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:12:36.697480 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:12:36.697480 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:12:36.698558 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:12:36.698558 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:12:36.698558 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:12:36.698558 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:12:36.698558 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:12:36.698558 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:12:36.698558 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:12:36.698558 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:12:36.698558 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:12:36.698558 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:12:36.698558 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:12:36.698558 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:12:36.698558 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 598/1188] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:12:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:12:36.704044 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87645e7-6f07-40c9-a931-71c9fc55c374 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:12:36.799905 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87645e7-6f07-40c9-a931-71c9fc55c374 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:12:36.826949 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:12:36.833974 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a27002-c455-41c3-bca3-f541cd35fe70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:12:36.834637 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a27002-c455-41c3-bca3-f541cd35fe70 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:12:36.834915 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a27002-c455-41c3-bca3-f541cd35fe70 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:12:36.835213 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a27002-c455-41c3-bca3-f541cd35fe70 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:12:36.835968 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a27002-c455-41c3-bca3-f541cd35fe70 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:12:36.836269 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a27002-c455-41c3-bca3-f541cd35fe70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:12:36.837616 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a27002-c455-41c3-bca3-f541cd35fe70 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:12:36.845728 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 591/1189] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:12:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:12:36.851754 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54699de9-fc74-4f91-8385-f8946d0135d3 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:12:36.886591 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:12:36.951967 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54699de9-fc74-4f91-8385-f8946d0135d3 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:12:36.984638 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323c7217-f8f5-436f-be02-9203fd0f9e31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:12:36.985759 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323c7217-f8f5-436f-be02-9203fd0f9e31 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:12:36.986083 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323c7217-f8f5-436f-be02-9203fd0f9e31 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:12:36.986363 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323c7217-f8f5-436f-be02-9203fd0f9e31 None admin] PATH_INFO: `/v3/users/56e4fd04b97d43da8c62394ad2d24e66` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:12:36.987551 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323c7217-f8f5-436f-be02-9203fd0f9e31 None admin] RBAC: Authorizing `identity:get_user(user_id=56e4fd04b97d43da8c62394ad2d24e66)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:12:36.988987 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323c7217-f8f5-436f-be02-9203fd0f9e31 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:12:36.990661 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 599/1190] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:12:36 2022] GET /identity/v3/users/56e4fd04b97d43da8c62394ad2d24e66 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:12:36.996288 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a27002-c455-41c3-bca3-f541cd35fe70 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:12:37.093805 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a27002-c455-41c3-bca3-f541cd35fe70 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:12:37.127617 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f110c00a-20fb-4035-92c7-5776a47a17b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:12:37.128474 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f110c00a-20fb-4035-92c7-5776a47a17b3 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:12:37.128754 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f110c00a-20fb-4035-92c7-5776a47a17b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:12:37.129024 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f110c00a-20fb-4035-92c7-5776a47a17b3 None admin] PATH_INFO: `/v3/projects/64ccf5f1dbb741fbb2a5d275c32dd929` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:12:37.130213 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f110c00a-20fb-4035-92c7-5776a47a17b3 None admin] RBAC: Authorizing `identity:get_project(project_id=64ccf5f1dbb741fbb2a5d275c32dd929)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:12:37.131641 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f110c00a-20fb-4035-92c7-5776a47a17b3 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:12:37.133321 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 592/1191] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:12:36 2022] GET /identity/v3/projects/64ccf5f1dbb741fbb2a5d275c32dd929 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:12:37.139870 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323c7217-f8f5-436f-be02-9203fd0f9e31 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:12:37.238860 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323c7217-f8f5-436f-be02-9203fd0f9e31 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:12:37.240295 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:12:37.273736 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edc1b9a-fd8b-4b99-8729-283534b51619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:12:37.274637 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edc1b9a-fd8b-4b99-8729-283534b51619 None admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:12:37.274917 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edc1b9a-fd8b-4b99-8729-283534b51619 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:12:37.275183 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edc1b9a-fd8b-4b99-8729-283534b51619 None admin] PATH_INFO: `/v3/projects/64ccf5f1dbb741fbb2a5d275c32dd929/users/56e4fd04b97d43da8c62394ad2d24e66/roles/4946cf0ff7164307b36efeee7c787146` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:12:37.277104 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edc1b9a-fd8b-4b99-8729-283534b51619 None admin] RBAC: Authorizing `identity:create_grant(project_id=64ccf5f1dbb741fbb2a5d275c32dd929, user_id=56e4fd04b97d43da8c62394ad2d24e66, role_id=4946cf0ff7164307b36efeee7c787146)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:12:37.278729 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edc1b9a-fd8b-4b99-8729-283534b51619 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:12:37.290956 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 600/1192] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 14:12:37 2022] PUT /identity/v3/projects/64ccf5f1dbb741fbb2a5d275c32dd929/users/56e4fd04b97d43da8c62394ad2d24e66/roles/4946cf0ff7164307b36efeee7c787146 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:12:39.489696 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e9d48f-c30b-4622-a380-1ca5b66e4835 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:12:39.490284 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e9d48f-c30b-4622-a380-1ca5b66e4835 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:12:39.490284 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e9d48f-c30b-4622-a380-1ca5b66e4835 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:12:39.491644 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 593/1193] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:12:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:12:39.496843 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644e1c19-0499-48b5-ac60-b1aa039429c5 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:12:39.497353 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644e1c19-0499-48b5-ac60-b1aa039429c5 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:12:39.497702 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644e1c19-0499-48b5-ac60-b1aa039429c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:12:39.514171 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-644e1c19-0499-48b5-ac60-b1aa039429c5 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:12:39.515357 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-644e1c19-0499-48b5-ac60-b1aa039429c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:12:39.581054 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 601/1194] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:12:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:12:39.604523 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e9d48f-c30b-4622-a380-1ca5b66e4835 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:12:39.664967 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:12:39.707283 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e9d48f-c30b-4622-a380-1ca5b66e4835 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:12:39.740740 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba60dec-ccb8-4778-831f-80d9937a2c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:12:39.741477 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba60dec-ccb8-4778-831f-80d9937a2c61 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:12:39.741759 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba60dec-ccb8-4778-831f-80d9937a2c61 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:12:39.742060 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba60dec-ccb8-4778-831f-80d9937a2c61 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:12:39.752050 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba60dec-ccb8-4778-831f-80d9937a2c61 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:12:39.759564 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-4ba60dec-ccb8-4778-831f-80d9937a2c61 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:12:39.759564 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:12:39.759564 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:12:39.759564 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:12:39.759564 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:12:39.759564 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:12:39.759564 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:12:39.759564 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:12:39.759564 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:12:39.759564 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:12:39.759564 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:12:39.759564 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:12:39.759564 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:12:39.759564 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:12:39.759564 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:12:39.759564 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:12:39.760729 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:12:39.760729 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:12:39.760729 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:12:39.760729 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:12:39.760729 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:12:39.760729 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:12:39.760729 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:12:39.760729 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:12:39.760729 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:12:39.760729 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:12:39.760729 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:12:39.760729 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:12:39.760729 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:12:39.760729 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:12:39.760729 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:12:39.760729 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:12:39.761689 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:12:39.761689 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:12:39.761689 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:12:39.761689 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:12:39.761689 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:12:39.761689 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:12:39.761689 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:12:39.761689 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:12:39.761689 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:12:39.761689 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:12:39.761689 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:12:39.761689 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:12:39.761689 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 594/1195] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:12:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:12:39.766792 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644e1c19-0499-48b5-ac60-b1aa039429c5 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:12:39.854739 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:12:39.864263 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644e1c19-0499-48b5-ac60-b1aa039429c5 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:12:39.897642 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed83070-4353-4914-bbd9-104d13ecf3f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:12:39.898329 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed83070-4353-4914-bbd9-104d13ecf3f5 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:12:39.898611 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed83070-4353-4914-bbd9-104d13ecf3f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:12:39.898886 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed83070-4353-4914-bbd9-104d13ecf3f5 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:12:39.899643 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed83070-4353-4914-bbd9-104d13ecf3f5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:12:39.899948 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed83070-4353-4914-bbd9-104d13ecf3f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:12:39.901318 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed83070-4353-4914-bbd9-104d13ecf3f5 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:12:39.909866 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 602/1196] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:12:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:12:39.915576 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba60dec-ccb8-4778-831f-80d9937a2c61 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:12:40.018724 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba60dec-ccb8-4778-831f-80d9937a2c61 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:12:40.035506 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:12:40.051805 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac62752-c499-4c58-b890-c6244263f699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:12:40.052670 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac62752-c499-4c58-b890-c6244263f699 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:12:40.052949 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac62752-c499-4c58-b890-c6244263f699 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:12:40.053220 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac62752-c499-4c58-b890-c6244263f699 None admin] PATH_INFO: `/v3/users/56e4fd04b97d43da8c62394ad2d24e66` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:12:40.054422 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac62752-c499-4c58-b890-c6244263f699 None admin] RBAC: Authorizing `identity:get_user(user_id=56e4fd04b97d43da8c62394ad2d24e66)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:12:40.055859 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac62752-c499-4c58-b890-c6244263f699 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:12:40.057471 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 595/1197] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:12:39 2022] GET /identity/v3/users/56e4fd04b97d43da8c62394ad2d24e66 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:12:40.063343 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed83070-4353-4914-bbd9-104d13ecf3f5 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:12:40.163120 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed83070-4353-4914-bbd9-104d13ecf3f5 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:12:40.196138 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98253a0-bffd-477d-85fd-04335ef3de17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:12:40.196782 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98253a0-bffd-477d-85fd-04335ef3de17 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:12:40.197069 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98253a0-bffd-477d-85fd-04335ef3de17 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:12:40.197346 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98253a0-bffd-477d-85fd-04335ef3de17 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:12:40.221663 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:12:40.270491 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98253a0-bffd-477d-85fd-04335ef3de17 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:12:40.271877 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98253a0-bffd-477d-85fd-04335ef3de17 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:12:40.373887 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 603/1198] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:12:40 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:12:40.377272 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac62752-c499-4c58-b890-c6244263f699 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:12:40.478868 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac62752-c499-4c58-b890-c6244263f699 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:12:40.513737 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4604c7ef-a5c1-4b9e-9e27-7decc766d6ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:12:40.514631 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4604c7ef-a5c1-4b9e-9e27-7decc766d6ca None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:12:40.514914 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4604c7ef-a5c1-4b9e-9e27-7decc766d6ca None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:12:40.515298 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4604c7ef-a5c1-4b9e-9e27-7decc766d6ca None admin] PATH_INFO: `/v3/projects/64ccf5f1dbb741fbb2a5d275c32dd929` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:12:40.516763 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4604c7ef-a5c1-4b9e-9e27-7decc766d6ca None admin] RBAC: Authorizing `identity:get_project(project_id=64ccf5f1dbb741fbb2a5d275c32dd929)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:12:40.518252 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4604c7ef-a5c1-4b9e-9e27-7decc766d6ca None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:12:40.519965 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 596/1199] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:12:40 2022] GET /identity/v3/projects/64ccf5f1dbb741fbb2a5d275c32dd929 => generated 361 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:12:40.526084 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98253a0-bffd-477d-85fd-04335ef3de17 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:12:40.583027 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:12:40.627186 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98253a0-bffd-477d-85fd-04335ef3de17 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:12:40.660787 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292fa8f9-9b77-4e63-bd82-d0730706b422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:12:40.661430 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292fa8f9-9b77-4e63-bd82-d0730706b422 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:12:40.661714 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292fa8f9-9b77-4e63-bd82-d0730706b422 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:12:40.662021 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292fa8f9-9b77-4e63-bd82-d0730706b422 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:12:40.662853 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292fa8f9-9b77-4e63-bd82-d0730706b422 None admin] RBAC: Adding query filter params (role.id=4946cf0ff7164307b36efeee7c787146, scope.project.id=64ccf5f1dbb741fbb2a5d275c32dd929, user.id=56e4fd04b97d43da8c62394ad2d24e66) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:12:40.663164 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292fa8f9-9b77-4e63-bd82-d0730706b422 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:12:40.664592 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292fa8f9-9b77-4e63-bd82-d0730706b422 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:12:40.670402 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 604/1200] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:12:40 2022] GET /identity/v3/role_assignments?user.id=56e4fd04b97d43da8c62394ad2d24e66&scope.project.id=64ccf5f1dbb741fbb2a5d275c32dd929&role.id=4946cf0ff7164307b36efeee7c787146 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:12:50.509244 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=73 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:12:50.668580 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=41 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:13:41.456476 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deccbb6c-6ebf-4ec2-a7e5-4412c44e2cd1 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:13:41.457077 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deccbb6c-6ebf-4ec2-a7e5-4412c44e2cd1 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:13:41.457077 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deccbb6c-6ebf-4ec2-a7e5-4412c44e2cd1 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:13:41.458442 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 597/1201] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:13:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:13:41.464239 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa63093-999f-4b8c-b290-0932d3387819 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:13:41.464700 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa63093-999f-4b8c-b290-0932d3387819 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:13:41.464833 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa63093-999f-4b8c-b290-0932d3387819 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:13:41.483918 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-9aa63093-999f-4b8c-b290-0932d3387819 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:13:41.485187 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-9aa63093-999f-4b8c-b290-0932d3387819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:13:41.565781 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 605/1202] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:13:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:13:41.589177 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deccbb6c-6ebf-4ec2-a7e5-4412c44e2cd1 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:13:41.706884 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deccbb6c-6ebf-4ec2-a7e5-4412c44e2cd1 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:13:41.750823 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b58640-3728-45f9-b1ac-bb1c4c19a37f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:13:41.751655 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b58640-3728-45f9-b1ac-bb1c4c19a37f None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:13:41.751979 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b58640-3728-45f9-b1ac-bb1c4c19a37f None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:13:41.752274 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b58640-3728-45f9-b1ac-bb1c4c19a37f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:13:41.769341 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b58640-3728-45f9-b1ac-bb1c4c19a37f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:13:41.770894 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b58640-3728-45f9-b1ac-bb1c4c19a37f None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:13:41.791075 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-d0b58640-3728-45f9-b1ac-bb1c4c19a37f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:13:41.791075 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:13:41.791075 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:13:41.791075 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:13:41.791075 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:13:41.791075 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:13:41.791075 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:13:41.791075 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:13:41.791075 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:13:41.791075 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:13:41.791075 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:13:41.791075 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:13:41.791075 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:13:41.791075 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:13:41.791075 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:13:41.792358 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:13:41.792358 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:13:41.792358 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:13:41.792358 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:13:41.792358 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:13:41.792358 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:13:41.792358 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:13:41.792358 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:13:41.792358 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:13:41.792358 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:13:41.792358 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:13:41.792358 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:13:41.792358 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:13:41.792358 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:13:41.792358 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:13:41.792358 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:13:41.792358 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:13:41.793480 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:13:41.793480 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:13:41.793480 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:13:41.793480 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:13:41.793480 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 598/1203] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:13:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:13:41.798319 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa63093-999f-4b8c-b290-0932d3387819 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:13:41.921126 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa63093-999f-4b8c-b290-0932d3387819 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:13:41.956327 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:13:41.963995 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e2c96d-9319-4d66-9510-0b54d4f4df79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:13:41.964404 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e2c96d-9319-4d66-9510-0b54d4f4df79 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:13:41.964607 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e2c96d-9319-4d66-9510-0b54d4f4df79 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:13:41.964923 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e2c96d-9319-4d66-9510-0b54d4f4df79 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:13:41.965610 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e2c96d-9319-4d66-9510-0b54d4f4df79 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:13:41.965954 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e2c96d-9319-4d66-9510-0b54d4f4df79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:13:41.967336 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e2c96d-9319-4d66-9510-0b54d4f4df79 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:13:41.982605 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 606/1204] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:13:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:13:41.988413 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b58640-3728-45f9-b1ac-bb1c4c19a37f None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:13:42.052520 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:13:42.111835 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b58640-3728-45f9-b1ac-bb1c4c19a37f None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:13:42.165068 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf562f6-e3af-490e-9970-4d43b56eade6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:13:42.166641 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf562f6-e3af-490e-9970-4d43b56eade6 None admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:13:42.166641 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf562f6-e3af-490e-9970-4d43b56eade6 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:13:42.166641 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf562f6-e3af-490e-9970-4d43b56eade6 None admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:13:42.167376 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf562f6-e3af-490e-9970-4d43b56eade6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:13:42.168960 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf562f6-e3af-490e-9970-4d43b56eade6 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:13:42.192460 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 599/1205] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 19 14:13:41 2022] POST /identity/v3/users => generated 251 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:13:44.565269 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444bbc27-2809-4f06-a581-44f104b603ee None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:13:44.565825 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444bbc27-2809-4f06-a581-44f104b603ee None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:13:44.566027 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444bbc27-2809-4f06-a581-44f104b603ee None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:13:44.567522 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 607/1206] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:13:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:13:44.574471 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7c0242-9de0-4e29-be52-aa0f69f7b237 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:13:44.574967 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7c0242-9de0-4e29-be52-aa0f69f7b237 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:13:44.575081 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7c0242-9de0-4e29-be52-aa0f69f7b237 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:13:44.594372 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-ef7c0242-9de0-4e29-be52-aa0f69f7b237 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:13:44.598406 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-ef7c0242-9de0-4e29-be52-aa0f69f7b237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:13:44.683768 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 600/1207] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:13:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:13:44.709798 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444bbc27-2809-4f06-a581-44f104b603ee None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:13:44.856962 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444bbc27-2809-4f06-a581-44f104b603ee None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:13:44.914731 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7bbbd1-720b-4274-ac1d-7182aa195649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:13:44.915531 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7bbbd1-720b-4274-ac1d-7182aa195649 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:13:44.915823 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7bbbd1-720b-4274-ac1d-7182aa195649 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:13:44.916415 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7bbbd1-720b-4274-ac1d-7182aa195649 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:13:44.928807 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7bbbd1-720b-4274-ac1d-7182aa195649 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:13:44.937505 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-ff7bbbd1-720b-4274-ac1d-7182aa195649 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:13:44.937505 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:13:44.937505 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:13:44.937505 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:13:44.937505 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:13:44.937505 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:13:44.937505 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:13:44.937505 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:13:44.937505 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:13:44.937505 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:13:44.937505 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:13:44.937505 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:13:44.937505 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:13:44.937505 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:13:44.937505 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:13:44.937505 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:13:44.938749 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:13:44.938749 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:13:44.938749 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:13:44.938749 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:13:44.938749 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:13:44.938749 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:13:44.938749 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:13:44.938749 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:13:44.938749 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:13:44.938749 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:13:44.938749 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:13:44.938749 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:13:44.938749 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:13:44.938749 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:13:44.938749 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:13:44.938749 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:13:44.939643 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:13:44.939643 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:13:44.939643 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:13:44.939643 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:13:44.939643 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:13:44.939643 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:13:44.939643 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:13:44.939643 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:13:44.939643 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:13:44.939643 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:13:44.939643 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:13:44.939643 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:13:44.939643 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 608/1208] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 14:13:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:13:44.944977 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7c0242-9de0-4e29-be52-aa0f69f7b237 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:13:44.992116 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:13:45.064785 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7c0242-9de0-4e29-be52-aa0f69f7b237 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:13:45.109058 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7bc13a-df62-423d-881a-47252bfd0560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:13:45.110183 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7bc13a-df62-423d-881a-47252bfd0560 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:13:45.110463 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7bc13a-df62-423d-881a-47252bfd0560 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:13:45.111083 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7bc13a-df62-423d-881a-47252bfd0560 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:13:45.111850 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7bc13a-df62-423d-881a-47252bfd0560 None admin] RBAC: Adding query filter params (name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:13:45.112213 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7bc13a-df62-423d-881a-47252bfd0560 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:13:45.113638 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7bc13a-df62-423d-881a-47252bfd0560 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:13:45.128679 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 601/1209] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 19 14:13:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:13:45.129798 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7bbbd1-720b-4274-ac1d-7182aa195649 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:13:45.193705 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:13:45.260586 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7bbbd1-720b-4274-ac1d-7182aa195649 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:13:45.313036 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d5ace1-dfa5-428c-9944-35ecd53c9863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:13:45.314214 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d5ace1-dfa5-428c-9944-35ecd53c9863 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:13:45.314739 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d5ace1-dfa5-428c-9944-35ecd53c9863 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:13:45.315156 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d5ace1-dfa5-428c-9944-35ecd53c9863 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:13:45.332792 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d5ace1-dfa5-428c-9944-35ecd53c9863 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:13:45.334874 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d5ace1-dfa5-428c-9944-35ecd53c9863 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:13:45.367611 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-38d5ace1-dfa5-428c-9944-35ecd53c9863 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:13:45.367611 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:13:45.367611 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:13:45.367611 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:13:45.367611 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:13:45.367611 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:13:45.367611 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:13:45.367611 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:13:45.367611 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:13:45.367611 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:13:45.367611 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:13:45.367611 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:13:45.367611 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:13:45.367611 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:13:45.367611 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:13:45.374023 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:13:45.374023 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:13:45.374023 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:13:45.374023 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:13:45.374023 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:13:45.374023 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:13:45.374023 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:13:45.374023 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:13:45.374023 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:13:45.374023 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:13:45.374023 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:13:45.374023 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:13:45.374023 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:13:45.374023 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:13:45.374023 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:13:45.374023 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:13:45.374023 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:13:45.375181 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:13:45.375181 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:13:45.375181 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:13:45.375181 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:13:45.375181 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 609/1210] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:13:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:13:45.379296 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7bc13a-df62-423d-881a-47252bfd0560 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:13:45.514202 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:13:45.517592 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7bc13a-df62-423d-881a-47252bfd0560 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:13:45.560110 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4732e230-ca11-438c-9b40-a68397226e9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:13:45.560890 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4732e230-ca11-438c-9b40-a68397226e9c None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:13:45.561212 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4732e230-ca11-438c-9b40-a68397226e9c None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:13:45.561539 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4732e230-ca11-438c-9b40-a68397226e9c None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:13:45.562298 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4732e230-ca11-438c-9b40-a68397226e9c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:13:45.562675 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4732e230-ca11-438c-9b40-a68397226e9c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:13:45.564212 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4732e230-ca11-438c-9b40-a68397226e9c None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:13:45.585394 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 602/1211] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:13:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:13:45.586050 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d5ace1-dfa5-428c-9944-35ecd53c9863 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:13:45.711213 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d5ace1-dfa5-428c-9944-35ecd53c9863 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:13:45.713502 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:13:45.752280 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d6dea4-9751-4714-8fe5-cf0c0a04fa16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:13:45.753256 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d6dea4-9751-4714-8fe5-cf0c0a04fa16 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:13:45.753584 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d6dea4-9751-4714-8fe5-cf0c0a04fa16 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:13:45.753936 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d6dea4-9751-4714-8fe5-cf0c0a04fa16 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:13:45.763245 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d6dea4-9751-4714-8fe5-cf0c0a04fa16 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:13:45.764867 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d6dea4-9751-4714-8fe5-cf0c0a04fa16 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:13:45.782428 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-b0d6dea4-9751-4714-8fe5-cf0c0a04fa16 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 14:13:45.782428 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:13:45.782428 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:13:45.782428 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:13:45.782428 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:13:45.782428 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:13:45.782428 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:13:45.782428 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:13:45.782428 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:13:45.782428 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:13:45.782428 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:13:45.782428 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:13:45.782428 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:13:45.782428 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:13:45.782428 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:13:45.782428 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:13:45.783484 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:13:45.783484 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:13:45.783484 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:13:45.783484 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:13:45.783484 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:13:45.783484 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:13:45.783484 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:13:45.783484 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:13:45.783484 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:13:45.783484 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:13:45.783484 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:13:45.783484 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:13:45.783484 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:13:45.783484 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:13:45.783484 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:13:45.783484 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:13:45.783484 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:13:45.783484 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:13:45.784939 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:13:45.784939 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:13:45.784939 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:13:45.784939 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:13:45.784939 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:13:45.784939 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:13:45.784939 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:13:45.784939 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:13:45.784939 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 14:13:45.784939 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:13:45.784939 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 610/1212] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:13:45 2022] GET /identity/v3/users/placement => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:13:45.789734 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4732e230-ca11-438c-9b40-a68397226e9c None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:13:45.914548 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4732e230-ca11-438c-9b40-a68397226e9c None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:13:45.959765 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d617adb-347c-4000-a63a-121fc1d17d4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:13:45.960495 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d617adb-347c-4000-a63a-121fc1d17d4a None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:13:45.961887 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d617adb-347c-4000-a63a-121fc1d17d4a None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:13:45.961887 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d617adb-347c-4000-a63a-121fc1d17d4a None admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:13:45.961887 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d617adb-347c-4000-a63a-121fc1d17d4a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:13:45.962342 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d617adb-347c-4000-a63a-121fc1d17d4a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:13:45.964202 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d617adb-347c-4000-a63a-121fc1d17d4a None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:13:45.987709 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 603/1213] 10.222.0.57 () {56 vars in 1143 bytes} [Thu May 19 14:13:45 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:13:45.989577 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d6dea4-9751-4714-8fe5-cf0c0a04fa16 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:13:46.118531 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d6dea4-9751-4714-8fe5-cf0c0a04fa16 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:13:46.165499 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84149ffc-a71b-4816-89b1-88771d7d6792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:13:46.166756 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84149ffc-a71b-4816-89b1-88771d7d6792 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:13:46.167245 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84149ffc-a71b-4816-89b1-88771d7d6792 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:13:46.167775 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84149ffc-a71b-4816-89b1-88771d7d6792 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:13:46.214136 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:13:46.266292 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84149ffc-a71b-4816-89b1-88771d7d6792 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:13:46.267920 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84149ffc-a71b-4816-89b1-88771d7d6792 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:13:46.399749 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 611/1214] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:13:45 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:13:46.401251 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d617adb-347c-4000-a63a-121fc1d17d4a None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:13:46.421546 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:13:46.518080 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d617adb-347c-4000-a63a-121fc1d17d4a None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:13:46.560173 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eb49f8-2a0a-4f06-83cf-0ac7db497f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:13:46.560411 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eb49f8-2a0a-4f06-83cf-0ac7db497f10 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:13:46.562336 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eb49f8-2a0a-4f06-83cf-0ac7db497f10 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:13:46.562336 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eb49f8-2a0a-4f06-83cf-0ac7db497f10 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:13:46.575738 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9eb49f8-2a0a-4f06-83cf-0ac7db497f10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:13:46.577566 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9eb49f8-2a0a-4f06-83cf-0ac7db497f10 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:13:46.598308 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-c9eb49f8-2a0a-4f06-83cf-0ac7db497f10 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:13:46.598308 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:13:46.598308 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:13:46.598308 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:13:46.598308 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:13:46.598308 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:13:46.598308 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:13:46.598308 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:13:46.598308 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:13:46.598308 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:13:46.598308 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:13:46.598308 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:13:46.598308 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:13:46.598308 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:13:46.598308 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:13:46.599787 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:13:46.599787 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:13:46.599787 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:13:46.599787 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:13:46.599787 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:13:46.599787 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:13:46.599787 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:13:46.599787 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:13:46.599787 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:13:46.599787 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:13:46.599787 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:13:46.599787 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:13:46.599787 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:13:46.599787 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:13:46.599787 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:13:46.599787 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:13:46.599787 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:13:46.601145 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:13:46.601145 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:13:46.601145 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:13:46.601145 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:13:46.604504 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 604/1215] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:13:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:13:46.606551 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84149ffc-a71b-4816-89b1-88771d7d6792 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:13:46.677795 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:13:46.724057 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84149ffc-a71b-4816-89b1-88771d7d6792 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:13:46.765724 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bb29f2-6765-4338-8b95-3424873993b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:13:46.766456 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bb29f2-6765-4338-8b95-3424873993b5 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:13:46.766737 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bb29f2-6765-4338-8b95-3424873993b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:13:46.767012 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bb29f2-6765-4338-8b95-3424873993b5 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:13:46.767765 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bb29f2-6765-4338-8b95-3424873993b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:13:46.768075 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bb29f2-6765-4338-8b95-3424873993b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:13:46.769538 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bb29f2-6765-4338-8b95-3424873993b5 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:13:46.786161 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 612/1216] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:13:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:13:46.787442 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eb49f8-2a0a-4f06-83cf-0ac7db497f10 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:13:46.872004 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:13:46.908725 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eb49f8-2a0a-4f06-83cf-0ac7db497f10 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:13:46.944684 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318da8c1-8040-4a9c-a40a-22621d8cab7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:13:46.945501 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318da8c1-8040-4a9c-a40a-22621d8cab7e None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:13:46.945803 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318da8c1-8040-4a9c-a40a-22621d8cab7e None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:13:46.946108 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318da8c1-8040-4a9c-a40a-22621d8cab7e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:13:46.953696 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318da8c1-8040-4a9c-a40a-22621d8cab7e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:13:46.955743 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318da8c1-8040-4a9c-a40a-22621d8cab7e None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:13:46.971165 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-318da8c1-8040-4a9c-a40a-22621d8cab7e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:13:46.971165 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:13:46.971165 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:13:46.971165 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:13:46.971165 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:13:46.971165 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:13:46.971165 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:13:46.971165 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:13:46.971165 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:13:46.971165 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:13:46.971165 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:13:46.971165 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:13:46.971165 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:13:46.971165 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:13:46.971165 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:13:46.972263 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:13:46.972263 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:13:46.972263 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:13:46.972263 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:13:46.972263 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:13:46.972263 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:13:46.972263 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:13:46.972263 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:13:46.972263 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:13:46.972263 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:13:46.972263 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:13:46.972263 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:13:46.972263 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:13:46.972263 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:13:46.972263 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:13:46.972263 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:13:46.972263 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:13:46.973202 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:13:46.973202 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:13:46.973202 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:13:46.973202 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:13:46.973202 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:13:46.973202 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:13:46.973202 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:13:46.973202 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:13:46.973202 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 605/1217] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:13:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:13:46.979415 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bb29f2-6765-4338-8b95-3424873993b5 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:13:47.107236 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bb29f2-6765-4338-8b95-3424873993b5 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:13:47.136699 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:13:47.149370 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8a12c9-ebf9-458b-8fd3-2691fce5fd26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:13:47.150588 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8a12c9-ebf9-458b-8fd3-2691fce5fd26 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:13:47.150993 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8a12c9-ebf9-458b-8fd3-2691fce5fd26 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:13:47.151350 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8a12c9-ebf9-458b-8fd3-2691fce5fd26 None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:13:47.152151 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8a12c9-ebf9-458b-8fd3-2691fce5fd26 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:13:47.152468 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8a12c9-ebf9-458b-8fd3-2691fce5fd26 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:13:47.153951 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8a12c9-ebf9-458b-8fd3-2691fce5fd26 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:13:47.167538 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 613/1218] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:13:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:13:47.174288 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318da8c1-8040-4a9c-a40a-22621d8cab7e None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:13:47.304467 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318da8c1-8040-4a9c-a40a-22621d8cab7e None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:13:47.338525 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:13:47.349109 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7e1c32-866f-4dd6-9257-2ec65309fae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:13:47.350099 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7e1c32-866f-4dd6-9257-2ec65309fae0 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:13:47.350636 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7e1c32-866f-4dd6-9257-2ec65309fae0 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:13:47.351110 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7e1c32-866f-4dd6-9257-2ec65309fae0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:13:47.352210 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7e1c32-866f-4dd6-9257-2ec65309fae0 None admin] RBAC: Adding query filter params (role.id=9c7d4d848dd549adb9f3465d4201f26d, scope.project.id=c6f995ffa0a94b1584f25dbe31e20524, user.id=dbe346b6d22e429e8e460b4cd9a0f613) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:13:47.352810 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7e1c32-866f-4dd6-9257-2ec65309fae0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:13:47.355015 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7e1c32-866f-4dd6-9257-2ec65309fae0 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:13:47.361418 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 606/1219] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:13:47 2022] GET /identity/v3/role_assignments?user.id=dbe346b6d22e429e8e460b4cd9a0f613&scope.project.id=c6f995ffa0a94b1584f25dbe31e20524&role.id=9c7d4d848dd549adb9f3465d4201f26d => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:13:49.761355 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaf9d4d-1a53-435d-8eae-6f4deffa2da5 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:13:49.762099 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaf9d4d-1a53-435d-8eae-6f4deffa2da5 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:13:49.762099 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaf9d4d-1a53-435d-8eae-6f4deffa2da5 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:13:49.763677 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 614/1220] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:13:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:13:49.769878 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a5050d-eff6-4b99-a906-e6e532103e5e None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:13:49.770324 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a5050d-eff6-4b99-a906-e6e532103e5e None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:13:49.770439 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a5050d-eff6-4b99-a906-e6e532103e5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:13:49.789430 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-59a5050d-eff6-4b99-a906-e6e532103e5e None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:13:49.791206 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-59a5050d-eff6-4b99-a906-e6e532103e5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:13:49.877830 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 607/1221] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:13:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:13:49.903302 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaf9d4d-1a53-435d-8eae-6f4deffa2da5 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:13:50.032381 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaf9d4d-1a53-435d-8eae-6f4deffa2da5 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:13:50.068714 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e581528c-e2af-415f-bca2-102fb9ff2238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:13:50.069567 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e581528c-e2af-415f-bca2-102fb9ff2238 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:13:50.069896 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e581528c-e2af-415f-bca2-102fb9ff2238 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:13:50.070162 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e581528c-e2af-415f-bca2-102fb9ff2238 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:13:50.082088 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e581528c-e2af-415f-bca2-102fb9ff2238 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:13:50.090523 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-e581528c-e2af-415f-bca2-102fb9ff2238 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:13:50.090523 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:13:50.090523 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:13:50.090523 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:13:50.090523 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:13:50.090523 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:13:50.090523 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:13:50.090523 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:13:50.090523 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:13:50.090523 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:13:50.090523 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:13:50.090523 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:13:50.090523 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:13:50.090523 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:13:50.090523 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:13:50.090523 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:13:50.091502 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:13:50.091502 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:13:50.091502 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:13:50.091502 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:13:50.091502 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:13:50.091502 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:13:50.091502 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:13:50.091502 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:13:50.091502 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:13:50.091502 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:13:50.091502 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:13:50.091502 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:13:50.091502 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:13:50.091502 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:13:50.091502 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:13:50.091502 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:13:50.092398 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:13:50.092398 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:13:50.092398 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:13:50.092398 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:13:50.092398 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:13:50.092398 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:13:50.092398 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:13:50.092398 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:13:50.092398 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:13:50.092398 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:13:50.092398 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:13:50.092398 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:13:50.092398 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 615/1222] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 14:13:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:13:50.097748 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a5050d-eff6-4b99-a906-e6e532103e5e None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:13:50.216875 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a5050d-eff6-4b99-a906-e6e532103e5e None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:13:50.247328 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:13:50.260934 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735db5eb-e986-4295-a6e4-4842ccd09948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:13:50.261734 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735db5eb-e986-4295-a6e4-4842ccd09948 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:13:50.262144 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735db5eb-e986-4295-a6e4-4842ccd09948 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:13:50.262465 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735db5eb-e986-4295-a6e4-4842ccd09948 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:13:50.263179 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735db5eb-e986-4295-a6e4-4842ccd09948 None admin] RBAC: Adding query filter params (name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:13:50.263481 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735db5eb-e986-4295-a6e4-4842ccd09948 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:13:50.264867 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735db5eb-e986-4295-a6e4-4842ccd09948 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:13:50.275303 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 608/1223] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 19 14:13:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:13:50.281678 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e581528c-e2af-415f-bca2-102fb9ff2238 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:13:50.340280 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:13:50.409537 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e581528c-e2af-415f-bca2-102fb9ff2238 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:13:50.447609 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e7f494-b500-4328-8ac1-3d4001768d20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:13:50.448400 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e7f494-b500-4328-8ac1-3d4001768d20 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:13:50.448681 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e7f494-b500-4328-8ac1-3d4001768d20 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:13:50.448962 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e7f494-b500-4328-8ac1-3d4001768d20 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:13:50.460018 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e7f494-b500-4328-8ac1-3d4001768d20 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:13:50.461470 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e7f494-b500-4328-8ac1-3d4001768d20 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:13:50.480534 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-b8e7f494-b500-4328-8ac1-3d4001768d20 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:13:50.480534 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:13:50.480534 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:13:50.480534 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:13:50.480534 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:13:50.480534 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:13:50.480534 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:13:50.480534 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:13:50.480534 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:13:50.480534 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:13:50.480534 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:13:50.480534 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:13:50.480534 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:13:50.480534 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:13:50.480534 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:13:50.481730 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:13:50.481730 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:13:50.481730 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:13:50.481730 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:13:50.481730 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:13:50.481730 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:13:50.481730 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:13:50.481730 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:13:50.481730 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:13:50.481730 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:13:50.481730 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:13:50.481730 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:13:50.481730 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:13:50.481730 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:13:50.481730 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:13:50.481730 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:13:50.481730 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:13:50.483117 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:13:50.483117 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:13:50.483117 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:13:50.483117 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:13:50.483117 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 616/1224] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:13:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:13:50.487893 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735db5eb-e986-4295-a6e4-4842ccd09948 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:13:50.599993 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735db5eb-e986-4295-a6e4-4842ccd09948 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:13:50.637005 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6892fcf-898a-467f-b622-7d4fc7fcad95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:13:50.637963 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6892fcf-898a-467f-b622-7d4fc7fcad95 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:13:50.638340 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6892fcf-898a-467f-b622-7d4fc7fcad95 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:13:50.638713 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6892fcf-898a-467f-b622-7d4fc7fcad95 None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:13:50.639877 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6892fcf-898a-467f-b622-7d4fc7fcad95 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:13:50.640519 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6892fcf-898a-467f-b622-7d4fc7fcad95 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:13:50.642121 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6892fcf-898a-467f-b622-7d4fc7fcad95 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:13:50.656022 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 609/1225] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:13:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:13:50.661709 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e7f494-b500-4328-8ac1-3d4001768d20 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:13:50.776882 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:13:50.785761 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e7f494-b500-4328-8ac1-3d4001768d20 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:13:50.823147 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a666d819-38bd-46be-b509-b5ce7f231daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:13:50.823988 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a666d819-38bd-46be-b509-b5ce7f231daa None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:13:50.824249 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a666d819-38bd-46be-b509-b5ce7f231daa None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:13:50.824526 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a666d819-38bd-46be-b509-b5ce7f231daa None admin] PATH_INFO: `/v3/users/placement` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:13:50.833895 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a666d819-38bd-46be-b509-b5ce7f231daa None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:13:50.835745 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a666d819-38bd-46be-b509-b5ce7f231daa None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:13:50.855025 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-a666d819-38bd-46be-b509-b5ce7f231daa None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 14:13:50.855025 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:13:50.855025 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:13:50.855025 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:13:50.855025 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:13:50.855025 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:13:50.855025 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:13:50.855025 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:13:50.855025 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:13:50.855025 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:13:50.855025 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:13:50.855025 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:13:50.855025 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:13:50.855025 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:13:50.855025 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:13:50.855025 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:13:50.856495 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:13:50.856495 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:13:50.856495 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:13:50.856495 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:13:50.856495 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:13:50.856495 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:13:50.856495 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:13:50.856495 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:13:50.856495 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:13:50.856495 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:13:50.856495 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:13:50.856495 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:13:50.856495 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:13:50.856495 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:13:50.856495 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:13:50.856495 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:13:50.856495 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:13:50.856495 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:13:50.857974 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:13:50.857974 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:13:50.857974 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:13:50.857974 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:13:50.857974 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:13:50.857974 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:13:50.857974 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:13:50.857974 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:13:50.857974 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 14:13:50.857974 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:13:50.857974 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 617/1226] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:13:50 2022] GET /identity/v3/users/placement => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:13:50.863077 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6892fcf-898a-467f-b622-7d4fc7fcad95 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:13:50.930006 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:13:50.995299 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6892fcf-898a-467f-b622-7d4fc7fcad95 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:13:51.038338 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4732257-d65c-4005-a995-2ee1f18257af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:13:51.039111 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4732257-d65c-4005-a995-2ee1f18257af None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:13:51.039399 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4732257-d65c-4005-a995-2ee1f18257af None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:13:51.039801 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4732257-d65c-4005-a995-2ee1f18257af None admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:13:51.041669 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4732257-d65c-4005-a995-2ee1f18257af None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:13:51.042127 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4732257-d65c-4005-a995-2ee1f18257af None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:13:51.043577 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4732257-d65c-4005-a995-2ee1f18257af None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:13:51.064089 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 610/1227] 10.222.0.57 () {56 vars in 1143 bytes} [Thu May 19 14:13:50 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:13:51.070789 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a666d819-38bd-46be-b509-b5ce7f231daa None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:13:51.207345 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a666d819-38bd-46be-b509-b5ce7f231daa None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:13:51.247186 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856156f1-991b-4d58-9d6a-13c747c732b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:13:51.248110 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856156f1-991b-4d58-9d6a-13c747c732b8 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:13:51.248402 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856156f1-991b-4d58-9d6a-13c747c732b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:13:51.248682 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856156f1-991b-4d58-9d6a-13c747c732b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:13:51.262574 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856156f1-991b-4d58-9d6a-13c747c732b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:13:51.264121 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856156f1-991b-4d58-9d6a-13c747c732b8 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:13:51.281781 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-856156f1-991b-4d58-9d6a-13c747c732b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:13:51.281781 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:13:51.281781 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:13:51.281781 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:13:51.281781 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:13:51.281781 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:13:51.281781 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:13:51.281781 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:13:51.281781 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:13:51.281781 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:13:51.281781 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:13:51.281781 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:13:51.281781 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:13:51.281781 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:13:51.281781 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:13:51.282781 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:13:51.282781 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:13:51.282781 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:13:51.282781 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:13:51.282781 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:13:51.282781 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:13:51.282781 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:13:51.282781 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:13:51.282781 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:13:51.282781 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:13:51.282781 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:13:51.282781 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:13:51.282781 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:13:51.282781 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:13:51.282781 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:13:51.282781 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:13:51.282781 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:13:51.284285 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:13:51.284285 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:13:51.284285 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:13:51.284285 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:13:51.284285 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 618/1228] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:13:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:13:51.288625 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4732257-d65c-4005-a995-2ee1f18257af None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:13:51.408988 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4732257-d65c-4005-a995-2ee1f18257af None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:13:51.411827 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:13:51.453729 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c0df39-905f-4eb0-9903-fb5895bfd2a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:13:51.454553 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c0df39-905f-4eb0-9903-fb5895bfd2a6 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:13:51.454888 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c0df39-905f-4eb0-9903-fb5895bfd2a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:13:51.455225 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c0df39-905f-4eb0-9903-fb5895bfd2a6 None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:13:51.456068 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c0df39-905f-4eb0-9903-fb5895bfd2a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:13:51.456459 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c0df39-905f-4eb0-9903-fb5895bfd2a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:13:51.458003 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c0df39-905f-4eb0-9903-fb5895bfd2a6 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:13:51.476225 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 611/1229] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:13:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:13:51.477455 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856156f1-991b-4d58-9d6a-13c747c732b8 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:13:51.487223 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:13:51.585077 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856156f1-991b-4d58-9d6a-13c747c732b8 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:13:51.624230 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86675c69-8358-4408-8e98-7298d3a5386b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:13:51.625115 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86675c69-8358-4408-8e98-7298d3a5386b None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:13:51.625422 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86675c69-8358-4408-8e98-7298d3a5386b None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:13:51.625722 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86675c69-8358-4408-8e98-7298d3a5386b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:13:51.632922 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86675c69-8358-4408-8e98-7298d3a5386b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:13:51.634990 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86675c69-8358-4408-8e98-7298d3a5386b None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:13:51.650537 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-86675c69-8358-4408-8e98-7298d3a5386b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:13:51.650537 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:13:51.650537 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:13:51.650537 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:13:51.650537 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:13:51.650537 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:13:51.650537 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:13:51.650537 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:13:51.650537 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:13:51.650537 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:13:51.650537 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:13:51.650537 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:13:51.650537 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:13:51.650537 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:13:51.650537 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:13:51.651722 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:13:51.651722 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:13:51.651722 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:13:51.651722 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:13:51.651722 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:13:51.651722 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:13:51.651722 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:13:51.651722 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:13:51.651722 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:13:51.651722 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:13:51.651722 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:13:51.651722 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:13:51.651722 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:13:51.651722 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:13:51.651722 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:13:51.651722 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:13:51.651722 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:13:51.652676 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:13:51.652676 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:13:51.652676 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:13:51.652676 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:13:51.652676 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:13:51.652676 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:13:51.652676 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:13:51.652676 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:13:51.652676 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 619/1230] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:13:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:13:51.657577 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c0df39-905f-4eb0-9903-fb5895bfd2a6 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:13:51.767425 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c0df39-905f-4eb0-9903-fb5895bfd2a6 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:13:51.804599 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f50837-cff3-481c-b1ad-7c1926f5d67e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:13:51.805323 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f50837-cff3-481c-b1ad-7c1926f5d67e None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:13:51.805617 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f50837-cff3-481c-b1ad-7c1926f5d67e None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:13:51.806583 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f50837-cff3-481c-b1ad-7c1926f5d67e None admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:13:51.807417 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f50837-cff3-481c-b1ad-7c1926f5d67e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:13:51.807796 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f50837-cff3-481c-b1ad-7c1926f5d67e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:13:51.809473 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f50837-cff3-481c-b1ad-7c1926f5d67e None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:13:51.822343 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 612/1231] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:13:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:13:51.828851 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86675c69-8358-4408-8e98-7298d3a5386b None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:13:51.901888 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:13:51.944446 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86675c69-8358-4408-8e98-7298d3a5386b None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:13:51.982904 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3375e9d9-2db9-4247-812a-ded2a0aa8090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:13:51.983803 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3375e9d9-2db9-4247-812a-ded2a0aa8090 None admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:13:51.984105 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3375e9d9-2db9-4247-812a-ded2a0aa8090 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:13:51.984421 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3375e9d9-2db9-4247-812a-ded2a0aa8090 None admin] PATH_INFO: `/v3/projects/c6f995ffa0a94b1584f25dbe31e20524/users/dbe346b6d22e429e8e460b4cd9a0f613/roles/9c7d4d848dd549adb9f3465d4201f26d` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:13:52.001162 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3375e9d9-2db9-4247-812a-ded2a0aa8090 None admin] RBAC: Authorizing `identity:create_grant(project_id=c6f995ffa0a94b1584f25dbe31e20524, user_id=dbe346b6d22e429e8e460b4cd9a0f613, role_id=9c7d4d848dd549adb9f3465d4201f26d)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:13:52.002544 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3375e9d9-2db9-4247-812a-ded2a0aa8090 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:13:52.016421 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 620/1232] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 14:13:51 2022] PUT /identity/v3/projects/c6f995ffa0a94b1584f25dbe31e20524/users/dbe346b6d22e429e8e460b4cd9a0f613/roles/9c7d4d848dd549adb9f3465d4201f26d => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 14:13:54.328439 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78280167-db5f-42f4-b3a8-4d11d75f0ac3 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:13:54.329002 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78280167-db5f-42f4-b3a8-4d11d75f0ac3 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:13:54.329120 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78280167-db5f-42f4-b3a8-4d11d75f0ac3 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:13:54.330739 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 613/1233] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:13:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:13:54.336019 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b185b86c-cf40-424a-bc29-8bd0367d5b48 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:13:54.336452 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b185b86c-cf40-424a-bc29-8bd0367d5b48 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:13:54.336620 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b185b86c-cf40-424a-bc29-8bd0367d5b48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:13:54.356628 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-b185b86c-cf40-424a-bc29-8bd0367d5b48 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:13:54.358708 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-b185b86c-cf40-424a-bc29-8bd0367d5b48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:13:54.448303 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 621/1234] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:13:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:13:54.483397 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78280167-db5f-42f4-b3a8-4d11d75f0ac3 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:13:54.516780 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:13:54.605127 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78280167-db5f-42f4-b3a8-4d11d75f0ac3 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:13:54.641806 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63621fa-e6c2-41d0-a741-e353d12cf156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:13:54.642651 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63621fa-e6c2-41d0-a741-e353d12cf156 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:13:54.642919 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63621fa-e6c2-41d0-a741-e353d12cf156 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:13:54.643200 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63621fa-e6c2-41d0-a741-e353d12cf156 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:13:54.659330 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63621fa-e6c2-41d0-a741-e353d12cf156 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:13:54.671969 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-a63621fa-e6c2-41d0-a741-e353d12cf156 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:13:54.671969 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:13:54.671969 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:13:54.671969 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:13:54.671969 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:13:54.671969 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:13:54.671969 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:13:54.671969 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:13:54.671969 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:13:54.671969 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:13:54.671969 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:13:54.671969 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:13:54.671969 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:13:54.671969 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:13:54.671969 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:13:54.671969 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:13:54.673132 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:13:54.673132 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:13:54.673132 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:13:54.673132 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:13:54.673132 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:13:54.673132 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:13:54.673132 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:13:54.673132 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:13:54.673132 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:13:54.673132 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:13:54.673132 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:13:54.673132 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:13:54.673132 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:13:54.673132 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:13:54.673132 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:13:54.673132 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:13:54.674370 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:13:54.674370 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:13:54.674370 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:13:54.674370 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:13:54.674370 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:13:54.674370 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:13:54.674370 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:13:54.674370 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:13:54.674370 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:13:54.674370 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:13:54.674370 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:13:54.674370 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:13:54.674370 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 614/1235] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 14:13:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:13:54.679566 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b185b86c-cf40-424a-bc29-8bd0367d5b48 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:13:54.740102 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:13:54.800855 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b185b86c-cf40-424a-bc29-8bd0367d5b48 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:13:54.842440 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95488c2-b724-46a2-b87c-dbd73f0e9354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:13:54.843114 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95488c2-b724-46a2-b87c-dbd73f0e9354 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:13:54.843394 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95488c2-b724-46a2-b87c-dbd73f0e9354 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:13:54.843668 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95488c2-b724-46a2-b87c-dbd73f0e9354 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:13:54.844407 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95488c2-b724-46a2-b87c-dbd73f0e9354 None admin] RBAC: Adding query filter params (name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:13:54.844713 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95488c2-b724-46a2-b87c-dbd73f0e9354 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:13:54.846132 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95488c2-b724-46a2-b87c-dbd73f0e9354 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:13:54.856187 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 622/1236] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 19 14:13:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:13:54.863360 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63621fa-e6c2-41d0-a741-e353d12cf156 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:13:54.972869 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:13:54.990664 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63621fa-e6c2-41d0-a741-e353d12cf156 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:13:55.032221 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f64de6-e541-44c1-ba38-c61a908346d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:13:55.033054 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f64de6-e541-44c1-ba38-c61a908346d5 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:13:55.033469 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f64de6-e541-44c1-ba38-c61a908346d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:13:55.033752 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f64de6-e541-44c1-ba38-c61a908346d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:13:55.047990 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f64de6-e541-44c1-ba38-c61a908346d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:13:55.049718 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f64de6-e541-44c1-ba38-c61a908346d5 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:13:55.069498 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-88f64de6-e541-44c1-ba38-c61a908346d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:13:55.069498 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:13:55.069498 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:13:55.069498 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:13:55.069498 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:13:55.069498 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:13:55.069498 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:13:55.069498 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:13:55.069498 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:13:55.069498 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:13:55.069498 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:13:55.069498 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:13:55.069498 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:13:55.069498 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:13:55.069498 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:13:55.070549 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:13:55.070549 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:13:55.070549 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:13:55.070549 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:13:55.070549 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:13:55.070549 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:13:55.070549 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:13:55.070549 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:13:55.070549 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:13:55.070549 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:13:55.070549 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:13:55.070549 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:13:55.070549 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:13:55.070549 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:13:55.070549 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:13:55.070549 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:13:55.070549 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:13:55.072144 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:13:55.072144 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:13:55.072144 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:13:55.072144 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:13:55.072144 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 615/1237] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:13:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:13:55.077694 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95488c2-b724-46a2-b87c-dbd73f0e9354 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:13:55.207394 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95488c2-b724-46a2-b87c-dbd73f0e9354 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:13:55.218510 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:13:55.247196 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371172d3-8a47-42f2-8670-8d3aaac08f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:13:55.247748 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371172d3-8a47-42f2-8670-8d3aaac08f87 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:13:55.248029 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371172d3-8a47-42f2-8670-8d3aaac08f87 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:13:55.248316 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371172d3-8a47-42f2-8670-8d3aaac08f87 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:13:55.249354 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371172d3-8a47-42f2-8670-8d3aaac08f87 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:13:55.249665 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371172d3-8a47-42f2-8670-8d3aaac08f87 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:13:55.251256 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371172d3-8a47-42f2-8670-8d3aaac08f87 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:13:55.264969 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 623/1238] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:13:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:13:55.270891 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f64de6-e541-44c1-ba38-c61a908346d5 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:13:55.398798 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f64de6-e541-44c1-ba38-c61a908346d5 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:13:55.447741 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f14a722-cb66-4c30-b19d-bed339a50b2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:13:55.448825 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f14a722-cb66-4c30-b19d-bed339a50b2f None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:13:55.449114 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f14a722-cb66-4c30-b19d-bed339a50b2f None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:13:55.449502 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f14a722-cb66-4c30-b19d-bed339a50b2f None admin] PATH_INFO: `/v3/users/placement` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:13:55.453361 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:13:55.460806 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f14a722-cb66-4c30-b19d-bed339a50b2f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:13:55.463517 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f14a722-cb66-4c30-b19d-bed339a50b2f None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:13:55.483217 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-5f14a722-cb66-4c30-b19d-bed339a50b2f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 14:13:55.483217 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:13:55.483217 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:13:55.483217 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:13:55.483217 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:13:55.483217 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:13:55.483217 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:13:55.483217 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:13:55.483217 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:13:55.483217 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:13:55.483217 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:13:55.483217 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:13:55.483217 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:13:55.483217 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:13:55.483217 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:13:55.483217 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:13:55.484716 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:13:55.484716 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:13:55.484716 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:13:55.484716 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:13:55.484716 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:13:55.484716 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:13:55.484716 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:13:55.484716 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:13:55.484716 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:13:55.484716 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:13:55.484716 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:13:55.484716 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:13:55.484716 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:13:55.484716 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:13:55.484716 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:13:55.484716 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:13:55.484716 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:13:55.484716 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:13:55.486477 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:13:55.486477 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:13:55.486477 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:13:55.486477 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:13:55.486477 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:13:55.486477 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:13:55.486477 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:13:55.486477 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:13:55.486477 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 14:13:55.486477 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:13:55.486477 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 616/1239] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:13:55 2022] GET /identity/v3/users/placement => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:13:55.491073 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371172d3-8a47-42f2-8670-8d3aaac08f87 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:13:55.623917 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371172d3-8a47-42f2-8670-8d3aaac08f87 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:13:55.668288 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7d3adc-c3f6-485f-9f14-5b578677c8a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:13:55.669130 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7d3adc-c3f6-485f-9f14-5b578677c8a3 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:13:55.669426 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7d3adc-c3f6-485f-9f14-5b578677c8a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:13:55.669896 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7d3adc-c3f6-485f-9f14-5b578677c8a3 None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:13:55.670763 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7d3adc-c3f6-485f-9f14-5b578677c8a3 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:13:55.671210 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7d3adc-c3f6-485f-9f14-5b578677c8a3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:13:55.672889 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7d3adc-c3f6-485f-9f14-5b578677c8a3 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:13:55.695744 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 624/1240] 10.222.0.57 () {56 vars in 1143 bytes} [Thu May 19 14:13:55 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:13:55.702794 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f14a722-cb66-4c30-b19d-bed339a50b2f None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:13:55.836813 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f14a722-cb66-4c30-b19d-bed339a50b2f None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:13:55.877669 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b949d4-b033-411e-aa2a-5fa9fe58582f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:13:55.877669 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b949d4-b033-411e-aa2a-5fa9fe58582f None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:13:55.878038 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b949d4-b033-411e-aa2a-5fa9fe58582f None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:13:55.878498 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b949d4-b033-411e-aa2a-5fa9fe58582f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:13:55.956256 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:13:55.964367 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b949d4-b033-411e-aa2a-5fa9fe58582f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:13:55.965819 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b949d4-b033-411e-aa2a-5fa9fe58582f None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:13:56.086168 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 617/1241] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:13:55 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 19 14:13:56.088589 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7d3adc-c3f6-485f-9f14-5b578677c8a3 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:13:56.118068 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:13:56.199394 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7d3adc-c3f6-485f-9f14-5b578677c8a3 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:13:56.236255 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccc829b-cbd0-4426-bb68-6d7046a08ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:13:56.237033 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccc829b-cbd0-4426-bb68-6d7046a08ba4 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:13:56.237316 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccc829b-cbd0-4426-bb68-6d7046a08ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:13:56.237594 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccc829b-cbd0-4426-bb68-6d7046a08ba4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:13:56.248159 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccc829b-cbd0-4426-bb68-6d7046a08ba4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:13:56.249623 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccc829b-cbd0-4426-bb68-6d7046a08ba4 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:13:56.267821 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-9ccc829b-cbd0-4426-bb68-6d7046a08ba4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:13:56.267821 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:13:56.267821 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:13:56.267821 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:13:56.267821 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:13:56.267821 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:13:56.267821 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:13:56.267821 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:13:56.267821 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:13:56.267821 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:13:56.267821 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:13:56.267821 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:13:56.267821 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:13:56.267821 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:13:56.267821 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:13:56.269065 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:13:56.269065 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:13:56.269065 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:13:56.269065 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:13:56.269065 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:13:56.269065 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:13:56.269065 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:13:56.269065 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:13:56.269065 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:13:56.269065 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:13:56.269065 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:13:56.269065 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:13:56.269065 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:13:56.269065 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:13:56.269065 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:13:56.269065 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:13:56.269065 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:13:56.270284 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:13:56.270284 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:13:56.270284 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:13:56.270284 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:13:56.270284 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 625/1242] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:13:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:13:56.275872 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b949d4-b033-411e-aa2a-5fa9fe58582f None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:13:56.382989 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:13:56.390826 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b949d4-b033-411e-aa2a-5fa9fe58582f None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:13:56.438431 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79673077-5d64-4dd9-92b3-2db8b26221ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:13:56.439218 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79673077-5d64-4dd9-92b3-2db8b26221ad None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:13:56.439562 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79673077-5d64-4dd9-92b3-2db8b26221ad None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:13:56.439838 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79673077-5d64-4dd9-92b3-2db8b26221ad None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:13:56.440697 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79673077-5d64-4dd9-92b3-2db8b26221ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:13:56.441040 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79673077-5d64-4dd9-92b3-2db8b26221ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:13:56.442617 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79673077-5d64-4dd9-92b3-2db8b26221ad None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:13:56.457302 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 618/1243] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:13:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:13:56.464231 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccc829b-cbd0-4426-bb68-6d7046a08ba4 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:13:56.556799 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:13:56.589158 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccc829b-cbd0-4426-bb68-6d7046a08ba4 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:13:56.636174 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8dd6ea-9af4-4ed8-84a7-87aa637d7199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:13:56.636978 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8dd6ea-9af4-4ed8-84a7-87aa637d7199 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:13:56.637267 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8dd6ea-9af4-4ed8-84a7-87aa637d7199 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:13:56.637603 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8dd6ea-9af4-4ed8-84a7-87aa637d7199 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:13:56.645059 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8dd6ea-9af4-4ed8-84a7-87aa637d7199 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:13:56.646645 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8dd6ea-9af4-4ed8-84a7-87aa637d7199 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:13:56.661737 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-5b8dd6ea-9af4-4ed8-84a7-87aa637d7199 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:13:56.661737 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:13:56.661737 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:13:56.661737 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:13:56.661737 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:13:56.661737 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:13:56.661737 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:13:56.661737 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:13:56.661737 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:13:56.661737 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:13:56.661737 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:13:56.661737 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:13:56.661737 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:13:56.661737 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:13:56.661737 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:13:56.663168 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:13:56.663168 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:13:56.663168 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:13:56.663168 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:13:56.663168 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:13:56.663168 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:13:56.663168 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:13:56.663168 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:13:56.663168 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:13:56.663168 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:13:56.663168 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:13:56.663168 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:13:56.663168 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:13:56.663168 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:13:56.663168 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:13:56.663168 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:13:56.663168 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:13:56.664311 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:13:56.664311 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:13:56.664311 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:13:56.664311 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:13:56.664311 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:13:56.664311 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:13:56.664311 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:13:56.664311 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:13:56.664311 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 626/1244] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:13:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:13:56.669338 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79673077-5d64-4dd9-92b3-2db8b26221ad None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:13:56.783963 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79673077-5d64-4dd9-92b3-2db8b26221ad None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:13:56.827196 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf355b0-48da-4846-9456-e6d8befc3eaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:13:56.828419 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf355b0-48da-4846-9456-e6d8befc3eaf None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:13:56.828919 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf355b0-48da-4846-9456-e6d8befc3eaf None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:13:56.829406 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf355b0-48da-4846-9456-e6d8befc3eaf None admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:13:56.830678 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf355b0-48da-4846-9456-e6d8befc3eaf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:13:56.831289 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf355b0-48da-4846-9456-e6d8befc3eaf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:13:56.832745 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf355b0-48da-4846-9456-e6d8befc3eaf None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:13:56.849076 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 619/1245] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:13:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:13:56.856260 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8dd6ea-9af4-4ed8-84a7-87aa637d7199 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:13:56.973415 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8dd6ea-9af4-4ed8-84a7-87aa637d7199 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:13:57.012205 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a95845-bea3-491e-8c67-3a4e07336426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:13:57.012937 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a95845-bea3-491e-8c67-3a4e07336426 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:13:57.013222 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a95845-bea3-491e-8c67-3a4e07336426 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:13:57.013503 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a95845-bea3-491e-8c67-3a4e07336426 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:13:57.014416 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a95845-bea3-491e-8c67-3a4e07336426 None admin] RBAC: Adding query filter params (role.id=9c7d4d848dd549adb9f3465d4201f26d, scope.project.id=c6f995ffa0a94b1584f25dbe31e20524, user.id=dbe346b6d22e429e8e460b4cd9a0f613) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:13:57.014780 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a95845-bea3-491e-8c67-3a4e07336426 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:13:57.016296 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a95845-bea3-491e-8c67-3a4e07336426 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:13:57.022750 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 627/1246] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:13:56 2022] GET /identity/v3/role_assignments?user.id=dbe346b6d22e429e8e460b4cd9a0f613&scope.project.id=c6f995ffa0a94b1584f25dbe31e20524&role.id=9c7d4d848dd549adb9f3465d4201f26d => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:13:59.415992 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2897b4bd-df68-4fa5-86b5-3512d2a604a3 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:13:59.416653 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2897b4bd-df68-4fa5-86b5-3512d2a604a3 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:13:59.416653 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2897b4bd-df68-4fa5-86b5-3512d2a604a3 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:13:59.418124 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 620/1247] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:13:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:13:59.423942 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5388bd-ea43-4af4-8b41-a76cac4f762a None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:13:59.424470 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5388bd-ea43-4af4-8b41-a76cac4f762a None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:13:59.424764 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5388bd-ea43-4af4-8b41-a76cac4f762a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:13:59.431189 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:13:59.444987 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-3d5388bd-ea43-4af4-8b41-a76cac4f762a None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:13:59.446924 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-3d5388bd-ea43-4af4-8b41-a76cac4f762a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:13:59.527074 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 628/1248] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:13:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:13:59.551491 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2897b4bd-df68-4fa5-86b5-3512d2a604a3 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:13:59.572795 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:13:59.672756 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2897b4bd-df68-4fa5-86b5-3512d2a604a3 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:13:59.711709 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894abdc0-4e44-4236-aae9-11b91da40937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:13:59.712501 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894abdc0-4e44-4236-aae9-11b91da40937 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:13:59.712788 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894abdc0-4e44-4236-aae9-11b91da40937 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:13:59.713096 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894abdc0-4e44-4236-aae9-11b91da40937 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:13:59.724238 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894abdc0-4e44-4236-aae9-11b91da40937 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:13:59.732846 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-894abdc0-4e44-4236-aae9-11b91da40937 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:13:59.732846 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:13:59.732846 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:13:59.732846 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:13:59.732846 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:13:59.732846 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:13:59.732846 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:13:59.732846 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:13:59.732846 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:13:59.732846 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:13:59.732846 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:13:59.732846 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:13:59.732846 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:13:59.732846 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:13:59.732846 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:13:59.732846 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:13:59.733927 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:13:59.733927 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:13:59.733927 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:13:59.733927 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:13:59.733927 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:13:59.733927 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:13:59.733927 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:13:59.733927 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:13:59.733927 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:13:59.733927 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:13:59.733927 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:13:59.733927 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:13:59.733927 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:13:59.733927 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:13:59.733927 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:13:59.733927 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:13:59.735305 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:13:59.735305 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:13:59.735305 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:13:59.735305 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:13:59.735305 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:13:59.735305 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:13:59.735305 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:13:59.735305 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:13:59.735305 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:13:59.735305 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:13:59.735305 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:13:59.735305 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:13:59.735305 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 621/1249] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:13:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:13:59.740170 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5388bd-ea43-4af4-8b41-a76cac4f762a None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:13:59.844709 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5388bd-ea43-4af4-8b41-a76cac4f762a None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:13:59.875630 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:13:59.881595 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1882c39-d25b-4bf2-857f-d62ef1c3777c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:13:59.882538 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1882c39-d25b-4bf2-857f-d62ef1c3777c None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:13:59.882919 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1882c39-d25b-4bf2-857f-d62ef1c3777c None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:13:59.883218 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1882c39-d25b-4bf2-857f-d62ef1c3777c None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:13:59.884749 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1882c39-d25b-4bf2-857f-d62ef1c3777c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:13:59.885263 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1882c39-d25b-4bf2-857f-d62ef1c3777c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:13:59.887284 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1882c39-d25b-4bf2-857f-d62ef1c3777c None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:13:59.896949 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 629/1250] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:13:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:13:59.904225 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894abdc0-4e44-4236-aae9-11b91da40937 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:14:00.003010 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:14:00.028373 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894abdc0-4e44-4236-aae9-11b91da40937 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:14:00.066871 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e6c6a8-2eec-44b4-9005-1311ca4614f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:14:00.067835 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e6c6a8-2eec-44b4-9005-1311ca4614f2 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:14:00.068219 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e6c6a8-2eec-44b4-9005-1311ca4614f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:14:00.068505 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e6c6a8-2eec-44b4-9005-1311ca4614f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:14:00.082811 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e6c6a8-2eec-44b4-9005-1311ca4614f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:14:00.084629 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e6c6a8-2eec-44b4-9005-1311ca4614f2 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:14:00.104305 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-08e6c6a8-2eec-44b4-9005-1311ca4614f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:14:00.104305 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:14:00.104305 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:14:00.104305 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:14:00.104305 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:14:00.104305 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:14:00.104305 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:14:00.104305 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:14:00.104305 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:14:00.104305 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:14:00.104305 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:14:00.104305 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:14:00.104305 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:14:00.104305 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:14:00.104305 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:14:00.105318 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:14:00.105318 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:14:00.105318 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:14:00.105318 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:14:00.105318 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:14:00.105318 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:14:00.105318 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:14:00.105318 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:14:00.105318 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:14:00.105318 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:14:00.105318 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:14:00.105318 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:14:00.105318 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:14:00.105318 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:14:00.105318 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:14:00.105318 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:14:00.105318 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:14:00.106297 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:14:00.106297 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:14:00.106297 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:14:00.106297 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:14:00.106297 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 622/1251] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:13:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:14:00.111542 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1882c39-d25b-4bf2-857f-d62ef1c3777c None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:14:00.220113 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1882c39-d25b-4bf2-857f-d62ef1c3777c None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:14:00.261903 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5802dcf-a448-49e2-8420-cbfe5444ab9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:14:00.262497 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5802dcf-a448-49e2-8420-cbfe5444ab9b None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:14:00.262785 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5802dcf-a448-49e2-8420-cbfe5444ab9b None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:14:00.263065 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5802dcf-a448-49e2-8420-cbfe5444ab9b None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:14:00.263782 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5802dcf-a448-49e2-8420-cbfe5444ab9b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:14:00.264089 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5802dcf-a448-49e2-8420-cbfe5444ab9b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:14:00.265552 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5802dcf-a448-49e2-8420-cbfe5444ab9b None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:14:00.279476 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 630/1252] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:14:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:14:00.285205 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e6c6a8-2eec-44b4-9005-1311ca4614f2 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:14:00.400835 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e6c6a8-2eec-44b4-9005-1311ca4614f2 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:14:00.431802 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:14:00.440395 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc0e0ad-8544-4888-8b46-6ba546b5cea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:14:00.441202 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc0e0ad-8544-4888-8b46-6ba546b5cea9 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:14:00.441482 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc0e0ad-8544-4888-8b46-6ba546b5cea9 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:14:00.441753 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc0e0ad-8544-4888-8b46-6ba546b5cea9 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:14:00.450730 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc0e0ad-8544-4888-8b46-6ba546b5cea9 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:14:00.452200 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc0e0ad-8544-4888-8b46-6ba546b5cea9 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:14:00.468945 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-cbc0e0ad-8544-4888-8b46-6ba546b5cea9 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 14:14:00.468945 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:14:00.468945 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:14:00.468945 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:14:00.468945 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:14:00.468945 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:14:00.468945 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:14:00.468945 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:14:00.468945 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:14:00.468945 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:14:00.468945 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:14:00.468945 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:14:00.468945 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:14:00.468945 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:14:00.468945 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:14:00.468945 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:14:00.470040 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:14:00.470040 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:14:00.470040 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:14:00.470040 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:14:00.470040 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:14:00.470040 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:14:00.470040 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:14:00.470040 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:14:00.470040 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:14:00.470040 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:14:00.470040 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:14:00.470040 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:14:00.470040 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:14:00.470040 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:14:00.470040 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:14:00.470040 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:14:00.470040 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:14:00.470040 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:14:00.471032 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:14:00.471032 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:14:00.471032 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:14:00.471032 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:14:00.471032 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:14:00.471032 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:14:00.471032 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:14:00.471032 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:14:00.471032 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 14:14:00.471032 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:14:00.471032 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 623/1253] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:14:00 2022] GET /identity/v3/users/placement => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:14:00.476317 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5802dcf-a448-49e2-8420-cbfe5444ab9b None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:14:00.538551 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:14:00.588327 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5802dcf-a448-49e2-8420-cbfe5444ab9b None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:14:00.625826 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9c0adc-db36-42c2-9362-4878ae0c87ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:14:00.626542 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9c0adc-db36-42c2-9362-4878ae0c87ed None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:14:00.626822 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9c0adc-db36-42c2-9362-4878ae0c87ed None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:14:00.627114 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9c0adc-db36-42c2-9362-4878ae0c87ed None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:14:00.628037 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9c0adc-db36-42c2-9362-4878ae0c87ed None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:14:00.628346 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9c0adc-db36-42c2-9362-4878ae0c87ed None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:14:00.629878 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9c0adc-db36-42c2-9362-4878ae0c87ed None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:14:00.653913 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 631/1254] 10.222.0.57 () {56 vars in 1143 bytes} [Thu May 19 14:14:00 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:14:00.655181 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc0e0ad-8544-4888-8b46-6ba546b5cea9 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:14:00.760950 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc0e0ad-8544-4888-8b46-6ba546b5cea9 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:14:00.795991 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094c004a-38af-4fab-9aef-afcca7796b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:14:00.796662 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094c004a-38af-4fab-9aef-afcca7796b3c None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:14:00.796938 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094c004a-38af-4fab-9aef-afcca7796b3c None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:14:00.797204 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094c004a-38af-4fab-9aef-afcca7796b3c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:14:00.854775 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:14:00.868530 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094c004a-38af-4fab-9aef-afcca7796b3c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:14:00.870019 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094c004a-38af-4fab-9aef-afcca7796b3c None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:14:00.987882 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 624/1255] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:14:00 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:14:00.994032 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9c0adc-db36-42c2-9362-4878ae0c87ed None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:14:01.100839 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9c0adc-db36-42c2-9362-4878ae0c87ed None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:14:01.114214 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:14:01.136569 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e8aade-91d4-4929-903c-f90b8d1759f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:14:01.137355 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e8aade-91d4-4929-903c-f90b8d1759f5 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:14:01.137645 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e8aade-91d4-4929-903c-f90b8d1759f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:14:01.138014 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e8aade-91d4-4929-903c-f90b8d1759f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:14:01.147807 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e8aade-91d4-4929-903c-f90b8d1759f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:14:01.150065 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e8aade-91d4-4929-903c-f90b8d1759f5 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:14:01.167327 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-66e8aade-91d4-4929-903c-f90b8d1759f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:14:01.167327 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:14:01.167327 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:14:01.167327 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:14:01.167327 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:14:01.167327 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:14:01.167327 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:14:01.167327 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:14:01.167327 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:14:01.167327 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:14:01.167327 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:14:01.167327 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:14:01.167327 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:14:01.167327 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:14:01.167327 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:14:01.168401 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:14:01.168401 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:14:01.168401 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:14:01.168401 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:14:01.168401 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:14:01.168401 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:14:01.168401 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:14:01.168401 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:14:01.168401 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:14:01.168401 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:14:01.168401 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:14:01.168401 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:14:01.168401 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:14:01.168401 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:14:01.168401 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:14:01.168401 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:14:01.168401 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:14:01.169471 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:14:01.169471 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:14:01.169471 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:14:01.169471 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:14:01.169471 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 632/1256] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:14:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:14:01.174143 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094c004a-38af-4fab-9aef-afcca7796b3c None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:14:01.260717 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:14:01.272686 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094c004a-38af-4fab-9aef-afcca7796b3c None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:14:01.306631 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7cc484-4c17-4251-90fd-70f5cb57e65d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:14:01.307289 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7cc484-4c17-4251-90fd-70f5cb57e65d None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:14:01.307572 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7cc484-4c17-4251-90fd-70f5cb57e65d None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:14:01.307848 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7cc484-4c17-4251-90fd-70f5cb57e65d None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:14:01.308556 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7cc484-4c17-4251-90fd-70f5cb57e65d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:14:01.308862 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7cc484-4c17-4251-90fd-70f5cb57e65d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:14:01.310292 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7cc484-4c17-4251-90fd-70f5cb57e65d None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:14:01.322165 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 625/1257] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:14:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:14:01.327751 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e8aade-91d4-4929-903c-f90b8d1759f5 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:14:01.438301 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e8aade-91d4-4929-903c-f90b8d1759f5 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:14:01.472464 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa730cc-ff7c-4da1-82a2-d9003c0f93a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:14:01.473241 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa730cc-ff7c-4da1-82a2-d9003c0f93a6 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:14:01.473520 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa730cc-ff7c-4da1-82a2-d9003c0f93a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:14:01.473792 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa730cc-ff7c-4da1-82a2-d9003c0f93a6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:14:01.480378 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa730cc-ff7c-4da1-82a2-d9003c0f93a6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:14:01.481811 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa730cc-ff7c-4da1-82a2-d9003c0f93a6 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:14:01.496079 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-bfa730cc-ff7c-4da1-82a2-d9003c0f93a6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:14:01.496079 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:14:01.496079 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:14:01.496079 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:14:01.496079 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:14:01.496079 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:14:01.496079 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:14:01.496079 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:14:01.496079 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:14:01.496079 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:14:01.496079 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:14:01.496079 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:14:01.496079 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:14:01.496079 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:14:01.496079 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:14:01.496998 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:14:01.496998 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:14:01.496998 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:14:01.496998 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:14:01.496998 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:14:01.496998 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:14:01.496998 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:14:01.496998 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:14:01.496998 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:14:01.496998 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:14:01.496998 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:14:01.496998 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:14:01.496998 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:14:01.496998 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:14:01.496998 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:14:01.496998 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:14:01.496998 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:14:01.497983 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:14:01.497983 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:14:01.497983 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:14:01.497983 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:14:01.497983 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:14:01.497983 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:14:01.497983 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:14:01.497983 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:14:01.497983 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 633/1258] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:14:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:14:01.503029 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7cc484-4c17-4251-90fd-70f5cb57e65d None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:14:01.603346 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7cc484-4c17-4251-90fd-70f5cb57e65d None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:14:01.636634 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe5db0a-a870-455d-a3f3-ef821a3790f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:14:01.637286 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe5db0a-a870-455d-a3f3-ef821a3790f6 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:14:01.637569 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe5db0a-a870-455d-a3f3-ef821a3790f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:14:01.637874 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe5db0a-a870-455d-a3f3-ef821a3790f6 None admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:14:01.638654 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe5db0a-a870-455d-a3f3-ef821a3790f6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:14:01.638964 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe5db0a-a870-455d-a3f3-ef821a3790f6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:14:01.640387 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe5db0a-a870-455d-a3f3-ef821a3790f6 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:14:01.652870 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 626/1259] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:14:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:14:01.658846 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa730cc-ff7c-4da1-82a2-d9003c0f93a6 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:14:01.683823 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:14:01.757303 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa730cc-ff7c-4da1-82a2-d9003c0f93a6 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:14:01.790795 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8116ddc4-753a-4282-91b2-66598d3ff517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:14:01.791442 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8116ddc4-753a-4282-91b2-66598d3ff517 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:14:01.791724 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8116ddc4-753a-4282-91b2-66598d3ff517 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:14:01.791996 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8116ddc4-753a-4282-91b2-66598d3ff517 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:14:01.792820 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8116ddc4-753a-4282-91b2-66598d3ff517 None admin] RBAC: Adding query filter params (role.id=4946cf0ff7164307b36efeee7c787146, scope.project.id=c6f995ffa0a94b1584f25dbe31e20524, user.id=dbe346b6d22e429e8e460b4cd9a0f613) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:14:01.793129 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8116ddc4-753a-4282-91b2-66598d3ff517 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:14:01.794583 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8116ddc4-753a-4282-91b2-66598d3ff517 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:14:01.799881 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 634/1260] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:14:01 2022] GET /identity/v3/role_assignments?user.id=dbe346b6d22e429e8e460b4cd9a0f613&scope.project.id=c6f995ffa0a94b1584f25dbe31e20524&role.id=4946cf0ff7164307b36efeee7c787146 => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:14:04.028023 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c0e89d-1af6-4b29-b4c6-da3c3b285c62 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:14:04.028558 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c0e89d-1af6-4b29-b4c6-da3c3b285c62 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:14:04.028558 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c0e89d-1af6-4b29-b4c6-da3c3b285c62 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:14:04.029969 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 627/1261] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:14:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:14:04.035226 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa587815-26a0-47bb-a6df-30a56dde99b9 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:14:04.035637 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa587815-26a0-47bb-a6df-30a56dde99b9 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:14:04.035789 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa587815-26a0-47bb-a6df-30a56dde99b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:14:04.052358 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-aa587815-26a0-47bb-a6df-30a56dde99b9 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:14:04.053531 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-aa587815-26a0-47bb-a6df-30a56dde99b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:14:04.119541 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 635/1262] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:14:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:14:04.142438 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c0e89d-1af6-4b29-b4c6-da3c3b285c62 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:14:04.153990 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:14:04.242065 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c0e89d-1af6-4b29-b4c6-da3c3b285c62 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:14:04.275702 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438ae156-a653-4055-b2ad-5e7922ebf2a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:14:04.276489 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438ae156-a653-4055-b2ad-5e7922ebf2a5 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:14:04.276770 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438ae156-a653-4055-b2ad-5e7922ebf2a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:14:04.277050 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438ae156-a653-4055-b2ad-5e7922ebf2a5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:14:04.287227 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438ae156-a653-4055-b2ad-5e7922ebf2a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:14:04.294949 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-438ae156-a653-4055-b2ad-5e7922ebf2a5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:14:04.294949 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:14:04.294949 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:14:04.294949 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:14:04.294949 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:14:04.294949 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:14:04.294949 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:14:04.294949 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:14:04.294949 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:14:04.294949 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:14:04.294949 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:14:04.294949 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:14:04.294949 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:14:04.294949 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:14:04.294949 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:14:04.294949 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:14:04.296142 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:14:04.296142 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:14:04.296142 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:14:04.296142 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:14:04.296142 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:14:04.296142 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:14:04.296142 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:14:04.296142 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:14:04.296142 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:14:04.296142 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:14:04.296142 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:14:04.296142 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:14:04.296142 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:14:04.296142 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:14:04.296142 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:14:04.296142 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:14:04.297035 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:14:04.297035 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:14:04.297035 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:14:04.297035 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:14:04.297035 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:14:04.297035 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:14:04.297035 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:14:04.297035 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:14:04.297035 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:14:04.297035 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:14:04.297035 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:14:04.297035 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:14:04.297035 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 628/1263] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:14:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:14:04.302367 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa587815-26a0-47bb-a6df-30a56dde99b9 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:14:04.320542 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:14:04.401286 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa587815-26a0-47bb-a6df-30a56dde99b9 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:14:04.434957 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25d6983-b791-4641-bf23-aca526e955a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:14:04.435620 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25d6983-b791-4641-bf23-aca526e955a2 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:14:04.435905 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25d6983-b791-4641-bf23-aca526e955a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:14:04.436179 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25d6983-b791-4641-bf23-aca526e955a2 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:14:04.436926 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25d6983-b791-4641-bf23-aca526e955a2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:14:04.437226 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25d6983-b791-4641-bf23-aca526e955a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:14:04.438621 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25d6983-b791-4641-bf23-aca526e955a2 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:14:04.447128 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 636/1264] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:14:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:14:04.452969 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438ae156-a653-4055-b2ad-5e7922ebf2a5 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:14:04.524077 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:14:04.558076 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438ae156-a653-4055-b2ad-5e7922ebf2a5 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:14:04.590350 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5e0067-7c1b-475f-b7d7-858c3e9d7806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:14:04.591136 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5e0067-7c1b-475f-b7d7-858c3e9d7806 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:14:04.591417 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5e0067-7c1b-475f-b7d7-858c3e9d7806 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:14:04.591692 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5e0067-7c1b-475f-b7d7-858c3e9d7806 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:14:04.602584 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5e0067-7c1b-475f-b7d7-858c3e9d7806 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:14:04.604026 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5e0067-7c1b-475f-b7d7-858c3e9d7806 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:14:04.621360 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-2c5e0067-7c1b-475f-b7d7-858c3e9d7806 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:14:04.621360 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:14:04.621360 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:14:04.621360 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:14:04.621360 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:14:04.621360 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:14:04.621360 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:14:04.621360 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:14:04.621360 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:14:04.621360 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:14:04.621360 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:14:04.621360 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:14:04.621360 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:14:04.621360 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:14:04.621360 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:14:04.622448 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:14:04.622448 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:14:04.622448 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:14:04.622448 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:14:04.622448 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:14:04.622448 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:14:04.622448 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:14:04.622448 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:14:04.622448 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:14:04.622448 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:14:04.622448 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:14:04.622448 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:14:04.622448 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:14:04.622448 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:14:04.622448 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:14:04.622448 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:14:04.622448 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:14:04.623526 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:14:04.623526 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:14:04.623526 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:14:04.623526 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:14:04.623526 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 629/1265] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:14:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:14:04.628277 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25d6983-b791-4641-bf23-aca526e955a2 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:14:04.709836 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:14:04.729012 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25d6983-b791-4641-bf23-aca526e955a2 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:14:04.761901 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7dd3b1-b9f5-49dd-b76d-8d58ad511aba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:14:04.762551 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7dd3b1-b9f5-49dd-b76d-8d58ad511aba None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:14:04.762831 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7dd3b1-b9f5-49dd-b76d-8d58ad511aba None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:14:04.763104 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7dd3b1-b9f5-49dd-b76d-8d58ad511aba None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:14:04.763810 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7dd3b1-b9f5-49dd-b76d-8d58ad511aba None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:14:04.764115 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7dd3b1-b9f5-49dd-b76d-8d58ad511aba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:14:04.765478 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7dd3b1-b9f5-49dd-b76d-8d58ad511aba None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:14:04.776750 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 637/1266] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:14:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:14:04.782452 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5e0067-7c1b-475f-b7d7-858c3e9d7806 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:14:04.880724 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5e0067-7c1b-475f-b7d7-858c3e9d7806 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:14:04.900489 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:14:04.916917 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aaac7d-4323-42a8-8e87-7f3090238e79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:14:04.917761 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aaac7d-4323-42a8-8e87-7f3090238e79 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:14:04.918062 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aaac7d-4323-42a8-8e87-7f3090238e79 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:14:04.918300 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aaac7d-4323-42a8-8e87-7f3090238e79 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:14:04.929113 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4aaac7d-4323-42a8-8e87-7f3090238e79 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:14:04.931720 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4aaac7d-4323-42a8-8e87-7f3090238e79 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:14:04.948111 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-f4aaac7d-4323-42a8-8e87-7f3090238e79 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 14:14:04.948111 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:14:04.948111 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:14:04.948111 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:14:04.948111 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:14:04.948111 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:14:04.948111 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:14:04.948111 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:14:04.948111 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:14:04.948111 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:14:04.948111 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:14:04.948111 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:14:04.948111 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:14:04.948111 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:14:04.948111 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:14:04.948111 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:14:04.949553 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:14:04.949553 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:14:04.949553 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:14:04.949553 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:14:04.949553 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:14:04.949553 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:14:04.949553 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:14:04.949553 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:14:04.949553 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:14:04.949553 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:14:04.949553 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:14:04.949553 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:14:04.949553 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:14:04.949553 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:14:04.949553 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:14:04.949553 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:14:04.949553 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:14:04.949553 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:14:04.950882 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:14:04.950882 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:14:04.950882 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:14:04.950882 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:14:04.950882 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:14:04.950882 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:14:04.950882 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:14:04.950882 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:14:04.950882 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 14:14:04.950882 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:14:04.950882 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 630/1267] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:14:04 2022] GET /identity/v3/users/placement => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:14:04.955655 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7dd3b1-b9f5-49dd-b76d-8d58ad511aba None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:14:05.056352 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7dd3b1-b9f5-49dd-b76d-8d58ad511aba None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:14:05.089094 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d878c99-798b-4c3d-8e59-d78eb08574da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:14:05.089752 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d878c99-798b-4c3d-8e59-d78eb08574da None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:14:05.090063 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d878c99-798b-4c3d-8e59-d78eb08574da None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:14:05.090339 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d878c99-798b-4c3d-8e59-d78eb08574da None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:14:05.091171 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d878c99-798b-4c3d-8e59-d78eb08574da None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:14:05.091481 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d878c99-798b-4c3d-8e59-d78eb08574da None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:14:05.092911 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d878c99-798b-4c3d-8e59-d78eb08574da None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:14:05.101211 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:14:05.110978 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 638/1268] 10.222.0.57 () {56 vars in 1143 bytes} [Thu May 19 14:14:04 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:14:05.116461 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aaac7d-4323-42a8-8e87-7f3090238e79 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:14:05.214917 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aaac7d-4323-42a8-8e87-7f3090238e79 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:14:05.247769 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc72af7-09d7-4e6c-83d6-7ca55fa7836d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:14:05.248552 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc72af7-09d7-4e6c-83d6-7ca55fa7836d None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:14:05.248838 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc72af7-09d7-4e6c-83d6-7ca55fa7836d None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:14:05.249116 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc72af7-09d7-4e6c-83d6-7ca55fa7836d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:14:05.260460 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc72af7-09d7-4e6c-83d6-7ca55fa7836d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:14:05.261919 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc72af7-09d7-4e6c-83d6-7ca55fa7836d None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:14:05.278794 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-4bc72af7-09d7-4e6c-83d6-7ca55fa7836d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:14:05.278794 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:14:05.278794 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:14:05.278794 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:14:05.278794 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:14:05.278794 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:14:05.278794 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:14:05.278794 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:14:05.278794 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:14:05.278794 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:14:05.278794 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:14:05.278794 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:14:05.278794 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:14:05.278794 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:14:05.278794 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:14:05.279861 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:14:05.279861 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:14:05.279861 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:14:05.279861 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:14:05.279861 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:14:05.279861 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:14:05.279861 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:14:05.279861 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:14:05.279861 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:14:05.279861 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:14:05.279861 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:14:05.279861 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:14:05.279861 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:14:05.279861 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:14:05.279861 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:14:05.279861 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:14:05.279861 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:14:05.280856 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:14:05.280856 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:14:05.280856 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:14:05.280856 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:14:05.280856 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 631/1269] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:14:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:14:05.285612 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d878c99-798b-4c3d-8e59-d78eb08574da None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:14:05.383680 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d878c99-798b-4c3d-8e59-d78eb08574da None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:14:05.418074 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bbd2e8-b4c5-474c-8f42-af8837f3abe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:14:05.418731 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bbd2e8-b4c5-474c-8f42-af8837f3abe5 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:14:05.419013 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bbd2e8-b4c5-474c-8f42-af8837f3abe5 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:14:05.419287 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bbd2e8-b4c5-474c-8f42-af8837f3abe5 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:14:05.420029 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bbd2e8-b4c5-474c-8f42-af8837f3abe5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:14:05.420333 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bbd2e8-b4c5-474c-8f42-af8837f3abe5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:14:05.421705 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bbd2e8-b4c5-474c-8f42-af8837f3abe5 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:14:05.433190 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 639/1270] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:14:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:14:05.438813 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc72af7-09d7-4e6c-83d6-7ca55fa7836d None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:14:05.470672 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:14:05.539270 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc72af7-09d7-4e6c-83d6-7ca55fa7836d None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:14:05.572299 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c261d4a4-7c62-4441-a7a1-173b583771bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:14:05.573080 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c261d4a4-7c62-4441-a7a1-173b583771bf None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:14:05.573367 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c261d4a4-7c62-4441-a7a1-173b583771bf None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:14:05.573646 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c261d4a4-7c62-4441-a7a1-173b583771bf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:14:05.580170 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c261d4a4-7c62-4441-a7a1-173b583771bf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:14:05.581606 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c261d4a4-7c62-4441-a7a1-173b583771bf None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:14:05.596131 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-c261d4a4-7c62-4441-a7a1-173b583771bf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:14:05.596131 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:14:05.596131 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:14:05.596131 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:14:05.596131 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:14:05.596131 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:14:05.596131 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:14:05.596131 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:14:05.596131 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:14:05.596131 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:14:05.596131 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:14:05.596131 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:14:05.596131 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:14:05.596131 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:14:05.596131 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:14:05.597187 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:14:05.597187 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:14:05.597187 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:14:05.597187 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:14:05.597187 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:14:05.597187 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:14:05.597187 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:14:05.597187 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:14:05.597187 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:14:05.597187 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:14:05.597187 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:14:05.597187 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:14:05.597187 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:14:05.597187 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:14:05.597187 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:14:05.597187 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:14:05.597187 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:14:05.598215 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:14:05.598215 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:14:05.598215 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:14:05.598215 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:14:05.598215 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:14:05.598215 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:14:05.598215 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:14:05.598215 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:14:05.598215 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 632/1271] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:14:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:14:05.603176 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bbd2e8-b4c5-474c-8f42-af8837f3abe5 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:14:05.660393 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:14:05.701646 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bbd2e8-b4c5-474c-8f42-af8837f3abe5 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:14:05.734014 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec909c9-78c9-4515-a00d-aa5445b943aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:14:05.734615 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec909c9-78c9-4515-a00d-aa5445b943aa None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:14:05.734821 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec909c9-78c9-4515-a00d-aa5445b943aa None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:14:05.735105 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec909c9-78c9-4515-a00d-aa5445b943aa None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:14:05.735879 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec909c9-78c9-4515-a00d-aa5445b943aa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:14:05.736185 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec909c9-78c9-4515-a00d-aa5445b943aa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:14:05.737626 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec909c9-78c9-4515-a00d-aa5445b943aa None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:14:05.750013 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 640/1272] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:14:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:14:05.755907 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c261d4a4-7c62-4441-a7a1-173b583771bf None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:14:05.844876 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:14:05.854235 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c261d4a4-7c62-4441-a7a1-173b583771bf None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:14:05.888478 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85cba8d-517d-450c-b949-524cb45e9f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:14:05.889350 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85cba8d-517d-450c-b949-524cb45e9f5f None admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:14:05.889634 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85cba8d-517d-450c-b949-524cb45e9f5f None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:14:05.889937 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85cba8d-517d-450c-b949-524cb45e9f5f None admin] PATH_INFO: `/v3/projects/c6f995ffa0a94b1584f25dbe31e20524/users/dbe346b6d22e429e8e460b4cd9a0f613/roles/4946cf0ff7164307b36efeee7c787146` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:14:05.892208 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85cba8d-517d-450c-b949-524cb45e9f5f None admin] RBAC: Authorizing `identity:create_grant(project_id=c6f995ffa0a94b1584f25dbe31e20524, user_id=dbe346b6d22e429e8e460b4cd9a0f613, role_id=4946cf0ff7164307b36efeee7c787146)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:14:05.894075 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85cba8d-517d-450c-b949-524cb45e9f5f None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:14:05.909721 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 633/1273] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 14:14:05 2022] PUT /identity/v3/projects/c6f995ffa0a94b1584f25dbe31e20524/users/dbe346b6d22e429e8e460b4cd9a0f613/roles/4946cf0ff7164307b36efeee7c787146 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:14:08.092912 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12822901-4a3d-4d37-a711-5c405b54b412 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:14:08.093448 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12822901-4a3d-4d37-a711-5c405b54b412 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:14:08.093448 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12822901-4a3d-4d37-a711-5c405b54b412 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:14:08.094897 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 641/1274] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:14:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:14:08.099994 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5fce42-354b-4998-b573-e1744c720704 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:14:08.100370 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5fce42-354b-4998-b573-e1744c720704 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:14:08.100551 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5fce42-354b-4998-b573-e1744c720704 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:14:08.118927 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-cd5fce42-354b-4998-b573-e1744c720704 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:14:08.120116 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-cd5fce42-354b-4998-b573-e1744c720704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:14:08.184986 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 634/1275] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:14:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:14:08.208951 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12822901-4a3d-4d37-a711-5c405b54b412 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:14:08.310203 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12822901-4a3d-4d37-a711-5c405b54b412 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:14:08.326925 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:14:08.344578 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c572ba99-a138-4fd0-a4b9-184172a64d93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:14:08.345295 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c572ba99-a138-4fd0-a4b9-184172a64d93 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:14:08.345581 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c572ba99-a138-4fd0-a4b9-184172a64d93 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:14:08.345880 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c572ba99-a138-4fd0-a4b9-184172a64d93 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:14:08.356295 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c572ba99-a138-4fd0-a4b9-184172a64d93 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:14:08.364092 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-c572ba99-a138-4fd0-a4b9-184172a64d93 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:14:08.364092 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:14:08.364092 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:14:08.364092 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:14:08.364092 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:14:08.364092 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:14:08.364092 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:14:08.364092 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:14:08.364092 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:14:08.364092 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:14:08.364092 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:14:08.364092 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:14:08.364092 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:14:08.364092 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:14:08.364092 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:14:08.364092 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:14:08.365218 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:14:08.365218 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:14:08.365218 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:14:08.365218 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:14:08.365218 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:14:08.365218 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:14:08.365218 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:14:08.365218 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:14:08.365218 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:14:08.365218 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:14:08.365218 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:14:08.365218 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:14:08.365218 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:14:08.365218 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:14:08.365218 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:14:08.365218 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:14:08.366306 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:14:08.366306 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:14:08.366306 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:14:08.366306 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:14:08.366306 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:14:08.366306 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:14:08.366306 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:14:08.366306 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:14:08.366306 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:14:08.366306 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:14:08.366306 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:14:08.366306 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:14:08.366306 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 642/1276] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:14:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:14:08.371784 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5fce42-354b-4998-b573-e1744c720704 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:14:08.451638 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:14:08.472610 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5fce42-354b-4998-b573-e1744c720704 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:14:08.508457 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f3b075-ba33-4439-91b3-439acc09e21e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:14:08.509119 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f3b075-ba33-4439-91b3-439acc09e21e None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:14:08.509404 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f3b075-ba33-4439-91b3-439acc09e21e None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:14:08.509684 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f3b075-ba33-4439-91b3-439acc09e21e None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:14:08.510454 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f3b075-ba33-4439-91b3-439acc09e21e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:14:08.510763 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f3b075-ba33-4439-91b3-439acc09e21e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:14:08.512130 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f3b075-ba33-4439-91b3-439acc09e21e None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:14:08.519952 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 635/1277] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:14:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:14:08.525645 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c572ba99-a138-4fd0-a4b9-184172a64d93 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:14:08.622935 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c572ba99-a138-4fd0-a4b9-184172a64d93 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:14:08.654705 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ebd7e9-17a4-4cfb-8933-91f7fb52765a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:14:08.655477 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ebd7e9-17a4-4cfb-8933-91f7fb52765a None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:14:08.655758 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ebd7e9-17a4-4cfb-8933-91f7fb52765a None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:14:08.656031 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ebd7e9-17a4-4cfb-8933-91f7fb52765a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:14:08.669228 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ebd7e9-17a4-4cfb-8933-91f7fb52765a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:14:08.670543 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ebd7e9-17a4-4cfb-8933-91f7fb52765a None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:14:08.688480 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-34ebd7e9-17a4-4cfb-8933-91f7fb52765a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:14:08.688480 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:14:08.688480 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:14:08.688480 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:14:08.688480 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:14:08.688480 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:14:08.688480 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:14:08.688480 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:14:08.688480 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:14:08.688480 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:14:08.688480 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:14:08.688480 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:14:08.688480 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:14:08.688480 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:14:08.688480 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:14:08.689621 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:14:08.689621 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:14:08.689621 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:14:08.689621 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:14:08.689621 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:14:08.689621 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:14:08.689621 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:14:08.689621 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:14:08.689621 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:14:08.689621 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:14:08.689621 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:14:08.689621 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:14:08.689621 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:14:08.689621 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:14:08.689621 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:14:08.689621 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:14:08.689621 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:14:08.690649 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:14:08.690649 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:14:08.690649 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:14:08.690649 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:14:08.690649 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 643/1278] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:14:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:14:08.695427 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f3b075-ba33-4439-91b3-439acc09e21e None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:14:08.793745 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f3b075-ba33-4439-91b3-439acc09e21e None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:14:08.825979 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db355725-5520-4774-891d-95e9168990a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:14:08.826644 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db355725-5520-4774-891d-95e9168990a0 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:14:08.826929 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db355725-5520-4774-891d-95e9168990a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:14:08.827205 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db355725-5520-4774-891d-95e9168990a0 None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:14:08.827931 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db355725-5520-4774-891d-95e9168990a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:14:08.828261 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db355725-5520-4774-891d-95e9168990a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:14:08.829626 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db355725-5520-4774-891d-95e9168990a0 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:14:08.842062 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 636/1279] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:14:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:14:08.847597 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ebd7e9-17a4-4cfb-8933-91f7fb52765a None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:14:08.878522 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:14:08.966113 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ebd7e9-17a4-4cfb-8933-91f7fb52765a None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:14:09.000536 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9f6249-cee7-487d-9a28-6f1912e1e3fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:14:09.001329 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9f6249-cee7-487d-9a28-6f1912e1e3fe None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:14:09.001611 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9f6249-cee7-487d-9a28-6f1912e1e3fe None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:14:09.001946 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9f6249-cee7-487d-9a28-6f1912e1e3fe None admin] PATH_INFO: `/v3/users/placement` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:14:09.009323 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9f6249-cee7-487d-9a28-6f1912e1e3fe None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:14:09.010794 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9f6249-cee7-487d-9a28-6f1912e1e3fe None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:14:09.025362 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-0a9f6249-cee7-487d-9a28-6f1912e1e3fe None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 14:14:09.025362 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:14:09.025362 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:14:09.025362 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:14:09.025362 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:14:09.025362 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:14:09.025362 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:14:09.025362 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:14:09.025362 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:14:09.025362 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:14:09.025362 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:14:09.025362 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:14:09.025362 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:14:09.025362 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:14:09.025362 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:14:09.025362 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:14:09.026546 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:14:09.026546 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:14:09.026546 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:14:09.026546 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:14:09.026546 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:14:09.026546 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:14:09.026546 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:14:09.026546 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:14:09.026546 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:14:09.026546 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:14:09.026546 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:14:09.026546 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:14:09.026546 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:14:09.026546 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:14:09.026546 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:14:09.026546 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:14:09.026546 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:14:09.026546 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:14:09.027863 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:14:09.027863 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:14:09.027863 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:14:09.027863 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:14:09.027863 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:14:09.027863 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:14:09.027863 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:14:09.027863 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:14:09.027863 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 14:14:09.027863 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:14:09.027863 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 644/1280] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:14:08 2022] GET /identity/v3/users/placement => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:14:09.032466 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db355725-5520-4774-891d-95e9168990a0 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:14:09.036323 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:14:09.129596 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db355725-5520-4774-891d-95e9168990a0 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:14:09.162962 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16d874e-b1d4-4ba9-9583-ea269f1b3ec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:14:09.163618 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16d874e-b1d4-4ba9-9583-ea269f1b3ec3 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:14:09.163897 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16d874e-b1d4-4ba9-9583-ea269f1b3ec3 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:14:09.164308 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16d874e-b1d4-4ba9-9583-ea269f1b3ec3 None admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:14:09.165138 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16d874e-b1d4-4ba9-9583-ea269f1b3ec3 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:14:09.165444 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16d874e-b1d4-4ba9-9583-ea269f1b3ec3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:14:09.166889 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16d874e-b1d4-4ba9-9583-ea269f1b3ec3 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:14:09.183214 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 637/1281] 10.222.0.57 () {56 vars in 1143 bytes} [Thu May 19 14:14:09 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:14:09.188864 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9f6249-cee7-487d-9a28-6f1912e1e3fe None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:14:09.277056 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:14:09.286292 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9f6249-cee7-487d-9a28-6f1912e1e3fe None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:14:09.319082 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb48d23-90b7-4ca3-acb2-b5b3275e751b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:14:09.319720 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb48d23-90b7-4ca3-acb2-b5b3275e751b None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:14:09.320005 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb48d23-90b7-4ca3-acb2-b5b3275e751b None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:14:09.320777 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb48d23-90b7-4ca3-acb2-b5b3275e751b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:14:09.386170 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb48d23-90b7-4ca3-acb2-b5b3275e751b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:14:09.387552 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb48d23-90b7-4ca3-acb2-b5b3275e751b None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:14:09.473393 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:14:09.483792 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 645/1282] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:14:09 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:14:09.489552 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16d874e-b1d4-4ba9-9583-ea269f1b3ec3 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:14:09.549416 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:14:09.587865 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16d874e-b1d4-4ba9-9583-ea269f1b3ec3 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:14:09.621189 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790cacb7-2a92-4790-916c-8c086e418148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:14:09.621997 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790cacb7-2a92-4790-916c-8c086e418148 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:14:09.622283 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790cacb7-2a92-4790-916c-8c086e418148 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:14:09.622559 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790cacb7-2a92-4790-916c-8c086e418148 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:14:09.632394 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790cacb7-2a92-4790-916c-8c086e418148 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:14:09.634454 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790cacb7-2a92-4790-916c-8c086e418148 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:14:09.651378 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-790cacb7-2a92-4790-916c-8c086e418148 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:14:09.651378 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:14:09.651378 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:14:09.651378 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:14:09.651378 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:14:09.651378 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:14:09.651378 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:14:09.651378 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:14:09.651378 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:14:09.651378 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:14:09.651378 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:14:09.651378 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:14:09.651378 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:14:09.651378 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:14:09.651378 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:14:09.652406 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:14:09.652406 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:14:09.652406 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:14:09.652406 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:14:09.652406 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:14:09.652406 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:14:09.652406 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:14:09.652406 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:14:09.652406 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:14:09.652406 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:14:09.652406 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:14:09.652406 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:14:09.652406 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:14:09.652406 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:14:09.652406 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:14:09.652406 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:14:09.652406 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:14:09.653360 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:14:09.653360 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:14:09.653360 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:14:09.653360 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:14:09.653360 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 638/1283] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:14:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:14:09.658103 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb48d23-90b7-4ca3-acb2-b5b3275e751b None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:14:09.754657 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb48d23-90b7-4ca3-acb2-b5b3275e751b None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:14:09.788325 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8814115-a43d-4368-b61a-78b98dcbf418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:14:09.789574 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8814115-a43d-4368-b61a-78b98dcbf418 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:14:09.789947 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8814115-a43d-4368-b61a-78b98dcbf418 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:14:09.790227 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8814115-a43d-4368-b61a-78b98dcbf418 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:14:09.790941 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8814115-a43d-4368-b61a-78b98dcbf418 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:14:09.791245 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8814115-a43d-4368-b61a-78b98dcbf418 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:14:09.792607 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8814115-a43d-4368-b61a-78b98dcbf418 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:14:09.803451 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 646/1284] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:14:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:14:09.809739 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790cacb7-2a92-4790-916c-8c086e418148 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:14:09.907578 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790cacb7-2a92-4790-916c-8c086e418148 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:14:09.921913 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:14:09.942494 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b1b805-a02d-40c2-b63c-a848fa98c5a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:14:09.943276 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b1b805-a02d-40c2-b63c-a848fa98c5a4 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:14:09.943562 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b1b805-a02d-40c2-b63c-a848fa98c5a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:14:09.943838 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b1b805-a02d-40c2-b63c-a848fa98c5a4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:14:09.950477 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b1b805-a02d-40c2-b63c-a848fa98c5a4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:14:09.951932 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b1b805-a02d-40c2-b63c-a848fa98c5a4 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:14:09.965559 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-57b1b805-a02d-40c2-b63c-a848fa98c5a4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:14:09.965559 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:14:09.965559 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:14:09.965559 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:14:09.965559 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:14:09.965559 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:14:09.965559 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:14:09.965559 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:14:09.965559 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:14:09.965559 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:14:09.965559 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:14:09.965559 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:14:09.965559 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:14:09.965559 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:14:09.965559 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:14:09.966637 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:14:09.966637 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:14:09.966637 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:14:09.966637 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:14:09.966637 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:14:09.966637 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:14:09.966637 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:14:09.966637 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:14:09.966637 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( May 19 14:14:09.966637 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:14:09.966637 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() May 19 14:14:09.966637 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:14:09.966637 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:14:09.966637 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:14:09.966637 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() May 19 14:14:09.966637 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:14:09.966637 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( May 19 14:14:09.967928 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:14:09.967928 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:14:09.967928 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:14:09.967928 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:14:09.967928 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:14:09.967928 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:14:09.967928 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:14:09.967928 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application May 19 14:14:09.967928 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 639/1285] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:14:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:14:09.972591 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8814115-a43d-4368-b61a-78b98dcbf418 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:14:10.023081 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:14:10.073979 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8814115-a43d-4368-b61a-78b98dcbf418 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:14:10.107740 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eabc743-8aab-4927-ac6e-eb5d3400f300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:14:10.108487 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eabc743-8aab-4927-ac6e-eb5d3400f300 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:14:10.108766 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eabc743-8aab-4927-ac6e-eb5d3400f300 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:14:10.109373 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eabc743-8aab-4927-ac6e-eb5d3400f300 None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:14:10.109804 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eabc743-8aab-4927-ac6e-eb5d3400f300 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:14:10.110135 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eabc743-8aab-4927-ac6e-eb5d3400f300 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:14:10.111568 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eabc743-8aab-4927-ac6e-eb5d3400f300 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:14:10.124551 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 647/1286] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:14:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:14:10.130551 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b1b805-a02d-40c2-b63c-a848fa98c5a4 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:14:10.230286 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b1b805-a02d-40c2-b63c-a848fa98c5a4 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:14:10.262910 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dab7895-3f76-4591-a7c5-6b4a3d7176ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:14:10.263547 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dab7895-3f76-4591-a7c5-6b4a3d7176ba None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:14:10.263826 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dab7895-3f76-4591-a7c5-6b4a3d7176ba None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:14:10.264097 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dab7895-3f76-4591-a7c5-6b4a3d7176ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:14:10.265002 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dab7895-3f76-4591-a7c5-6b4a3d7176ba None admin] RBAC: Adding query filter params (role.id=4946cf0ff7164307b36efeee7c787146, scope.project.id=c6f995ffa0a94b1584f25dbe31e20524, user.id=dbe346b6d22e429e8e460b4cd9a0f613) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:14:10.265307 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dab7895-3f76-4591-a7c5-6b4a3d7176ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:14:10.266760 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dab7895-3f76-4591-a7c5-6b4a3d7176ba None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:14:10.272498 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 640/1287] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:14:10 2022] GET /identity/v3/role_assignments?user.id=dbe346b6d22e429e8e460b4cd9a0f613&scope.project.id=c6f995ffa0a94b1584f25dbe31e20524&role.id=4946cf0ff7164307b36efeee7c787146 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:14:12.513177 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc6feb9-37dd-4263-907c-52e3cac25a2d None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:14:12.513759 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc6feb9-37dd-4263-907c-52e3cac25a2d None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:14:12.513759 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc6feb9-37dd-4263-907c-52e3cac25a2d None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:14:12.515122 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 648/1288] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:14:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:14:12.520205 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9491bf92-a35f-4e53-bf21-ebe454c14eee None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:14:12.520635 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9491bf92-a35f-4e53-bf21-ebe454c14eee None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:14:12.520768 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9491bf92-a35f-4e53-bf21-ebe454c14eee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:14:12.537474 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-9491bf92-a35f-4e53-bf21-ebe454c14eee None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:14:12.538693 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-9491bf92-a35f-4e53-bf21-ebe454c14eee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:14:12.567315 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:14:12.604145 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 641/1289] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:14:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:14:12.626840 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc6feb9-37dd-4263-907c-52e3cac25a2d None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:14:12.726353 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc6feb9-37dd-4263-907c-52e3cac25a2d None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:14:12.737148 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:14:12.761049 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3f70a9-eda9-4d20-8a40-ce03744210fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:14:12.761829 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3f70a9-eda9-4d20-8a40-ce03744210fa None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:14:12.762142 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3f70a9-eda9-4d20-8a40-ce03744210fa None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:14:12.762414 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3f70a9-eda9-4d20-8a40-ce03744210fa None admin] PATH_INFO: `/v3/services/placement` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:14:12.763056 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3f70a9-eda9-4d20-8a40-ce03744210fa None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:14:12.764482 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3f70a9-eda9-4d20-8a40-ce03744210fa None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:14:12.776005 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-ec3f70a9-eda9-4d20-8a40-ce03744210fa None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 19 14:14:12.776005 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:14:12.776005 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:14:12.776005 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:14:12.776005 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:14:12.776005 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:14:12.776005 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:14:12.776005 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:14:12.776005 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 14:14:12.776005 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:14:12.776005 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:14:12.776005 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:14:12.776005 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:14:12.776005 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:14:12.776005 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:14:12.776005 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:14:12.776005 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:14:12.777309 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:14:12.777309 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:14:12.777309 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:14:12.777309 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:14:12.777309 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:14:12.777309 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:14:12.777309 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:14:12.777309 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:14:12.777309 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:14:12.777309 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:14:12.777309 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:14:12.777309 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:14:12.777309 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:14:12.777309 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:14:12.777309 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:14:12.777309 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:14:12.778483 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:14:12.778483 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:14:12.778483 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:14:12.778483 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:14:12.778483 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:14:12.778483 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:14:12.778483 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:14:12.778483 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:14:12.778483 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:14:12.778483 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:14:12.778483 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:14:12.778483 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:14:12.778483 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 14:14:12.778483 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:14:12.778483 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 649/1290] 10.222.0.57 () {56 vars in 1117 bytes} [Thu May 19 14:14:12 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:14:12.783193 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9491bf92-a35f-4e53-bf21-ebe454c14eee None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:14:12.881433 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9491bf92-a35f-4e53-bf21-ebe454c14eee None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:14:12.920386 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3394af-fee5-4270-807f-3ff1455f137a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:14:12.921024 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3394af-fee5-4270-807f-3ff1455f137a None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:14:12.921314 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3394af-fee5-4270-807f-3ff1455f137a None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:14:12.921592 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3394af-fee5-4270-807f-3ff1455f137a None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:14:12.922339 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3394af-fee5-4270-807f-3ff1455f137a None admin] RBAC: Adding query filter params (name=placement) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:14:12.922645 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3394af-fee5-4270-807f-3ff1455f137a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:14:12.924059 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3394af-fee5-4270-807f-3ff1455f137a None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:14:12.929579 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 642/1291] 10.222.0.57 () {56 vars in 1116 bytes} [Thu May 19 14:14:12 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:14:12.935559 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3f70a9-eda9-4d20-8a40-ce03744210fa None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:14:13.040533 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3f70a9-eda9-4d20-8a40-ce03744210fa None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:14:13.077998 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1049085d-2c13-4969-9901-1d0b2c50ac60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:14:13.078657 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1049085d-2c13-4969-9901-1d0b2c50ac60 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:14:13.078944 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1049085d-2c13-4969-9901-1d0b2c50ac60 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:14:13.079222 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1049085d-2c13-4969-9901-1d0b2c50ac60 None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:14:13.079927 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1049085d-2c13-4969-9901-1d0b2c50ac60 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:14:13.080233 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1049085d-2c13-4969-9901-1d0b2c50ac60 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:14:13.081603 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1049085d-2c13-4969-9901-1d0b2c50ac60 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:14:13.086597 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 650/1292] 10.222.0.57 () {56 vars in 1116 bytes} [Thu May 19 14:14:12 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:14:15.287635 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6929b6e5-12dd-459f-ba5b-cb768aad883f None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:14:15.288184 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6929b6e5-12dd-459f-ba5b-cb768aad883f None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:14:15.288184 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6929b6e5-12dd-459f-ba5b-cb768aad883f None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:14:15.290129 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 643/1293] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:14:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:14:15.295262 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e505b1a3-ba3b-465e-9d2b-28a96c4a94f3 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:14:15.295662 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e505b1a3-ba3b-465e-9d2b-28a96c4a94f3 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:14:15.295829 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e505b1a3-ba3b-465e-9d2b-28a96c4a94f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:14:15.312921 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-e505b1a3-ba3b-465e-9d2b-28a96c4a94f3 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:14:15.314126 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-e505b1a3-ba3b-465e-9d2b-28a96c4a94f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:14:15.342827 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:14:15.380657 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 651/1294] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:14:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:14:15.403148 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6929b6e5-12dd-459f-ba5b-cb768aad883f None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:14:15.434446 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:14:15.503840 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6929b6e5-12dd-459f-ba5b-cb768aad883f None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:14:15.536952 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cd2faa-6502-43b0-84ab-26575bf65770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:14:15.537608 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cd2faa-6502-43b0-84ab-26575bf65770 None admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:14:15.537913 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cd2faa-6502-43b0-84ab-26575bf65770 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:14:15.538192 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cd2faa-6502-43b0-84ab-26575bf65770 None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:14:15.539501 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cd2faa-6502-43b0-84ab-26575bf65770 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:14:15.541037 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cd2faa-6502-43b0-84ab-26575bf65770 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:14:15.553726 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 644/1295] 10.222.0.57 () {60 vars in 1141 bytes} [Thu May 19 14:14:15 2022] POST /identity/v3/services => generated 246 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:14:17.770832 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892b9af4-72ff-4a62-8a70-c2dab0d8ef96 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:14:17.771400 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892b9af4-72ff-4a62-8a70-c2dab0d8ef96 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:14:17.771400 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892b9af4-72ff-4a62-8a70-c2dab0d8ef96 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:14:17.772750 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 652/1296] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:14:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:14:17.778117 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79e2537-8610-4bab-8dd4-068561cfdf8e None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:14:17.778535 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79e2537-8610-4bab-8dd4-068561cfdf8e None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:14:17.778686 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79e2537-8610-4bab-8dd4-068561cfdf8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:14:17.795390 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-c79e2537-8610-4bab-8dd4-068561cfdf8e None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:14:17.796576 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-c79e2537-8610-4bab-8dd4-068561cfdf8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:14:17.869091 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 645/1297] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:14:17 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:14:17.892249 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892b9af4-72ff-4a62-8a70-c2dab0d8ef96 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:14:18.005516 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892b9af4-72ff-4a62-8a70-c2dab0d8ef96 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:14:18.039828 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3316cd33-c247-4afd-8363-5470dd3db586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:14:18.040553 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3316cd33-c247-4afd-8363-5470dd3db586 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:14:18.040836 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3316cd33-c247-4afd-8363-5470dd3db586 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:14:18.041108 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3316cd33-c247-4afd-8363-5470dd3db586 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:14:18.041756 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3316cd33-c247-4afd-8363-5470dd3db586 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:14:18.043155 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3316cd33-c247-4afd-8363-5470dd3db586 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:14:18.055429 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-3316cd33-c247-4afd-8363-5470dd3db586 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 19 14:14:18.055429 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:14:18.055429 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:14:18.055429 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:14:18.055429 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:14:18.055429 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:14:18.055429 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:14:18.055429 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:14:18.055429 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 14:14:18.055429 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:14:18.055429 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:14:18.055429 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:14:18.055429 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:14:18.055429 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:14:18.055429 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:14:18.055429 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:14:18.055429 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:14:18.056606 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:14:18.056606 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:14:18.056606 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:14:18.056606 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:14:18.056606 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:14:18.056606 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:14:18.056606 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:14:18.056606 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:14:18.056606 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:14:18.056606 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:14:18.056606 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:14:18.056606 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:14:18.056606 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:14:18.056606 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:14:18.056606 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:14:18.056606 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:14:18.057675 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:14:18.057675 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:14:18.057675 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:14:18.057675 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:14:18.057675 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:14:18.057675 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:14:18.057675 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:14:18.057675 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:14:18.057675 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:14:18.057675 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:14:18.057675 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:14:18.057675 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:14:18.057675 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 14:14:18.057675 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:14:18.057675 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 653/1298] 10.222.0.57 () {56 vars in 1117 bytes} [Thu May 19 14:14:17 2022] GET /identity/v3/services/placement => generated 90 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:14:18.062698 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79e2537-8610-4bab-8dd4-068561cfdf8e None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:14:18.080775 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:14:18.162682 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79e2537-8610-4bab-8dd4-068561cfdf8e None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:14:18.196167 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac44f52-818c-4035-a003-b8ef50057845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:14:18.196814 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac44f52-818c-4035-a003-b8ef50057845 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:14:18.197095 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac44f52-818c-4035-a003-b8ef50057845 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:14:18.197368 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac44f52-818c-4035-a003-b8ef50057845 None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:14:18.198085 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac44f52-818c-4035-a003-b8ef50057845 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:14:18.198391 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac44f52-818c-4035-a003-b8ef50057845 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:14:18.199764 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac44f52-818c-4035-a003-b8ef50057845 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:14:18.204727 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 646/1299] 10.222.0.57 () {56 vars in 1116 bytes} [Thu May 19 14:14:18 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:14:18.210692 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3316cd33-c247-4afd-8363-5470dd3db586 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:14:18.243673 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:14:18.310314 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3316cd33-c247-4afd-8363-5470dd3db586 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:14:18.342034 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6193d5-f5c9-4bcd-bb7a-cf3929a6c75d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:14:18.342677 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6193d5-f5c9-4bcd-bb7a-cf3929a6c75d None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:14:18.342958 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6193d5-f5c9-4bcd-bb7a-cf3929a6c75d None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:14:18.343231 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6193d5-f5c9-4bcd-bb7a-cf3929a6c75d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:14:18.344013 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6193d5-f5c9-4bcd-bb7a-cf3929a6c75d None admin] RBAC: Adding query filter params (interface=public, service_id=923c0222d0604c6a8d9ebae513831c8a, region_id=RegionOne) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:14:18.344324 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6193d5-f5c9-4bcd-bb7a-cf3929a6c75d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:14:18.345693 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6193d5-f5c9-4bcd-bb7a-cf3929a6c75d None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:14:18.350612 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 654/1300] 10.222.0.57 () {56 vars in 1251 bytes} [Thu May 19 14:14:18 2022] GET /identity/v3/endpoints?service_id=923c0222d0604c6a8d9ebae513831c8a&interface=public®ion_id=RegionOne => generated 194 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:14:18.357124 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac44f52-818c-4035-a003-b8ef50057845 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:14:18.437691 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:14:18.456299 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac44f52-818c-4035-a003-b8ef50057845 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:14:18.489809 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dabe58f-85e7-4519-898a-05da90430a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:14:18.490494 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dabe58f-85e7-4519-898a-05da90430a31 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:14:18.490773 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dabe58f-85e7-4519-898a-05da90430a31 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:14:18.491044 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dabe58f-85e7-4519-898a-05da90430a31 None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:14:18.492278 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dabe58f-85e7-4519-898a-05da90430a31 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:14:18.493725 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dabe58f-85e7-4519-898a-05da90430a31 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:14:18.500720 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 647/1301] 10.222.0.57 () {56 vars in 1087 bytes} [Thu May 19 14:14:18 2022] GET /identity/v3/services => generated 2191 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 14:14:20.687730 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afec4a7-346f-4092-a819-b73b2f55a73e None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:14:20.688288 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afec4a7-346f-4092-a819-b73b2f55a73e None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:14:20.688288 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afec4a7-346f-4092-a819-b73b2f55a73e None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:14:20.689789 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 655/1302] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:14:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:14:20.695736 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4075831d-ade8-4c2a-bdbb-3cc976e8bc55 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:14:20.696158 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4075831d-ade8-4c2a-bdbb-3cc976e8bc55 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:14:20.696291 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4075831d-ade8-4c2a-bdbb-3cc976e8bc55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:14:20.712698 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-4075831d-ade8-4c2a-bdbb-3cc976e8bc55 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:14:20.713921 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-4075831d-ade8-4c2a-bdbb-3cc976e8bc55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:14:20.779742 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 648/1303] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:14:20 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:14:20.802657 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afec4a7-346f-4092-a819-b73b2f55a73e None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:14:20.897549 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:14:20.905918 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afec4a7-346f-4092-a819-b73b2f55a73e None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:14:20.939206 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c294ad78-368a-4cac-a6b8-be73ac6d80b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:14:20.939930 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c294ad78-368a-4cac-a6b8-be73ac6d80b8 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:14:20.940213 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c294ad78-368a-4cac-a6b8-be73ac6d80b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:14:20.940488 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c294ad78-368a-4cac-a6b8-be73ac6d80b8 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:14:20.941133 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c294ad78-368a-4cac-a6b8-be73ac6d80b8 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:14:20.942532 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c294ad78-368a-4cac-a6b8-be73ac6d80b8 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:14:20.954893 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-c294ad78-368a-4cac-a6b8-be73ac6d80b8 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 19 14:14:20.954893 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:14:20.954893 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:14:20.954893 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:14:20.954893 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:14:20.954893 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:14:20.954893 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:14:20.954893 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:14:20.954893 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 14:14:20.954893 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:14:20.954893 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:14:20.954893 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:14:20.954893 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:14:20.954893 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:14:20.954893 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:14:20.954893 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:14:20.954893 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:14:20.956008 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:14:20.956008 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:14:20.956008 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:14:20.956008 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:14:20.956008 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:14:20.956008 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:14:20.956008 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:14:20.956008 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:14:20.956008 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:14:20.956008 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:14:20.956008 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:14:20.956008 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:14:20.956008 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:14:20.956008 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:14:20.956008 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:14:20.956008 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:14:20.956972 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:14:20.956972 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( May 19 14:14:20.956972 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:14:20.956972 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() May 19 14:14:20.956972 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:14:20.956972 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:14:20.956972 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:14:20.956972 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() May 19 14:14:20.956972 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:14:20.956972 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( May 19 14:14:20.956972 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:14:20.956972 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:14:20.956972 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 14:14:20.956972 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application May 19 14:14:20.956972 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 656/1304] 10.222.0.57 () {56 vars in 1117 bytes} [Thu May 19 14:14:20 2022] GET /identity/v3/services/placement => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:14:20.962469 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4075831d-ade8-4c2a-bdbb-3cc976e8bc55 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:14:21.037133 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:14:21.061156 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4075831d-ade8-4c2a-bdbb-3cc976e8bc55 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:14:21.094344 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43648987-1d54-4e67-b830-175472868042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:14:21.095001 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43648987-1d54-4e67-b830-175472868042 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:14:21.095280 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43648987-1d54-4e67-b830-175472868042 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:14:21.095553 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43648987-1d54-4e67-b830-175472868042 None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:14:21.096247 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43648987-1d54-4e67-b830-175472868042 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:14:21.096550 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43648987-1d54-4e67-b830-175472868042 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:14:21.098054 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43648987-1d54-4e67-b830-175472868042 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:14:21.103797 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 649/1305] 10.222.0.57 () {56 vars in 1116 bytes} [Thu May 19 14:14:20 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:14:21.109749 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c294ad78-368a-4cac-a6b8-be73ac6d80b8 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:14:21.206765 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c294ad78-368a-4cac-a6b8-be73ac6d80b8 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:14:21.239297 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7659f483-01a1-4bcb-a4b6-4e6f42621e98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4608701c57147b39ea0df421d2e5029', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:14:21.239945 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7659f483-01a1-4bcb-a4b6-4e6f42621e98 None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:14:21.240231 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7659f483-01a1-4bcb-a4b6-4e6f42621e98 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:14:21.240510 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7659f483-01a1-4bcb-a4b6-4e6f42621e98 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:14:21.241240 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7659f483-01a1-4bcb-a4b6-4e6f42621e98 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:14:21.243291 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7659f483-01a1-4bcb-a4b6-4e6f42621e98 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:14:21.260945 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 657/1306] 10.222.0.57 () {60 vars in 1144 bytes} [Thu May 19 14:14:21 2022] POST /identity/v3/endpoints => generated 331 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:14:31.100606 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=30 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:14:31.256466 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:15:25.571766 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2842f8e6-d0d0-4f3d-b121-f8185c1580f8 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:15:25.572781 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2842f8e6-d0d0-4f3d-b121-f8185c1580f8 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:15:25.573292 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2842f8e6-d0d0-4f3d-b121-f8185c1580f8 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:15:25.578853 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 650/1307] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:15:25 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:15:25.580149 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d1ad5d-a816-480e-b694-56c7a071d13d None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:15:25.581534 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d1ad5d-a816-480e-b694-56c7a071d13d None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:15:25.582489 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d1ad5d-a816-480e-b694-56c7a071d13d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:15:25.741256 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-49d1ad5d-a816-480e-b694-56c7a071d13d None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:15:25.743819 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-49d1ad5d-a816-480e-b694-56c7a071d13d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:15:25.924011 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 658/1308] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:15:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 345 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 14:15:25.974767 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21013c5f-d8a4-4b0b-ad55-03e272a0df3e None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:15:25.975157 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21013c5f-d8a4-4b0b-ad55-03e272a0df3e None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:15:25.975507 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21013c5f-d8a4-4b0b-ad55-03e272a0df3e None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:15:25.976934 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 651/1309] 10.222.0.57 () {54 vars in 964 bytes} [Thu May 19 14:15:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:15:25.983298 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f701bd-d29b-4623-ae56-8bc9f01ed54b None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:15:25.983603 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f701bd-d29b-4623-ae56-8bc9f01ed54b None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:15:25.983888 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f701bd-d29b-4623-ae56-8bc9f01ed54b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:15:26.066148 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-f7f701bd-d29b-4623-ae56-8bc9f01ed54b None None] MFA Rules not processed for user `56ce23600bfe42e6b98eeebfe7470560`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:15:26.067405 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-f7f701bd-d29b-4623-ae56-8bc9f01ed54b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:15:26.090326 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:15:26.155711 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 659/1310] 10.222.0.57 () {58 vars in 1061 bytes} [Thu May 19 14:15:25 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:15:26.164764 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21013c5f-d8a4-4b0b-ad55-03e272a0df3e None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:15:26.166710 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-21013c5f-d8a4-4b0b-ad55-03e272a0df3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:15:26.293481 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21013c5f-d8a4-4b0b-ad55-03e272a0df3e None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:15:26.374679 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d10c969-d1c7-449a-b39a-0f658f471197 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56ce23600bfe42e6b98eeebfe7470560', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6f995ffa0a94b1584f25dbe31e20524', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:15:26.375463 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d10c969-d1c7-449a-b39a-0f658f471197 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:15:26.375810 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d10c969-d1c7-449a-b39a-0f658f471197 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:15:26.376156 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d10c969-d1c7-449a-b39a-0f658f471197 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:15:26.378023 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-9d10c969-d1c7-449a-b39a-0f658f471197 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:15:26.439925 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:15:26.500568 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d10c969-d1c7-449a-b39a-0f658f471197 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:15:26.502050 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d10c969-d1c7-449a-b39a-0f658f471197 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:15:26.523827 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 652/1311] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 14:15:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:15:29.088929 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d08524-0f32-4f0b-831b-7e7473994fd1 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:15:29.089328 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d08524-0f32-4f0b-831b-7e7473994fd1 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:15:29.089501 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d08524-0f32-4f0b-831b-7e7473994fd1 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:15:29.091043 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 660/1312] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:15:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:15:29.096142 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fcb944-55b1-4e8b-adf4-9a260951f173 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:15:29.096566 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fcb944-55b1-4e8b-adf4-9a260951f173 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:15:29.096757 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fcb944-55b1-4e8b-adf4-9a260951f173 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:15:29.183989 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-57fcb944-55b1-4e8b-adf4-9a260951f173 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:15:29.185210 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-57fcb944-55b1-4e8b-adf4-9a260951f173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:15:29.283746 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 653/1313] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:15:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:15:29.322291 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d08524-0f32-4f0b-831b-7e7473994fd1 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:15:29.323885 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-33d08524-0f32-4f0b-831b-7e7473994fd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:15:29.433553 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d08524-0f32-4f0b-831b-7e7473994fd1 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:15:29.513585 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:15:29.516812 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d73d48d-1148-4c0b-97f0-30ee39c1bcce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56ce23600bfe42e6b98eeebfe7470560', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6f995ffa0a94b1584f25dbe31e20524', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:15:29.517469 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d73d48d-1148-4c0b-97f0-30ee39c1bcce service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:15:29.517751 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d73d48d-1148-4c0b-97f0-30ee39c1bcce service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:15:29.518117 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d73d48d-1148-4c0b-97f0-30ee39c1bcce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:15:29.520376 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-6d73d48d-1148-4c0b-97f0-30ee39c1bcce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:15:29.640896 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d73d48d-1148-4c0b-97f0-30ee39c1bcce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:15:29.642417 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d73d48d-1148-4c0b-97f0-30ee39c1bcce service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:15:29.666047 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 661/1314] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 14:15:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 14:15:32.262511 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38700137-c232-4764-8ca0-230ff9a9e7d4 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:15:32.263045 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38700137-c232-4764-8ca0-230ff9a9e7d4 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:15:32.263045 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38700137-c232-4764-8ca0-230ff9a9e7d4 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:15:32.264471 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 654/1315] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:15:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:15:32.270074 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b872260-3961-4661-a249-62fba6e204d0 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:15:32.270517 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b872260-3961-4661-a249-62fba6e204d0 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:15:32.270662 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b872260-3961-4661-a249-62fba6e204d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:15:32.338475 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-6b872260-3961-4661-a249-62fba6e204d0 None None] MFA Rules not processed for user `47b5316234e94937b78cedeab15c486e`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:15:32.339599 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-6b872260-3961-4661-a249-62fba6e204d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:15:32.396069 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:15:32.441531 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 662/1316] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:15:32 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:15:32.497120 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2926277a-e53c-42d6-9dae-87671ed884d9 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:15:32.497401 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2926277a-e53c-42d6-9dae-87671ed884d9 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:15:32.497676 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2926277a-e53c-42d6-9dae-87671ed884d9 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:15:32.499074 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 655/1317] 10.222.0.57 () {54 vars in 964 bytes} [Thu May 19 14:15:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:15:32.505098 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adaae0f-0e00-480e-9960-e5b23a6c69ac None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:15:32.505385 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adaae0f-0e00-480e-9960-e5b23a6c69ac None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:15:32.505661 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adaae0f-0e00-480e-9960-e5b23a6c69ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:15:32.578217 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-9adaae0f-0e00-480e-9960-e5b23a6c69ac None None] MFA Rules not processed for user `56ce23600bfe42e6b98eeebfe7470560`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:15:32.579478 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-9adaae0f-0e00-480e-9960-e5b23a6c69ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:15:32.653710 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 663/1318] 10.222.0.57 () {58 vars in 1061 bytes} [Thu May 19 14:15:32 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:15:32.663884 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2926277a-e53c-42d6-9dae-87671ed884d9 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:15:32.665417 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-2926277a-e53c-42d6-9dae-87671ed884d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:15:32.675724 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:15:32.767215 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2926277a-e53c-42d6-9dae-87671ed884d9 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:15:32.838898 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97f199c-a4cd-4f12-a1f9-875858e5bff2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56ce23600bfe42e6b98eeebfe7470560', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6f995ffa0a94b1584f25dbe31e20524', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:15:32.839538 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97f199c-a4cd-4f12-a1f9-875858e5bff2 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:15:32.839822 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97f199c-a4cd-4f12-a1f9-875858e5bff2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:15:32.840085 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97f199c-a4cd-4f12-a1f9-875858e5bff2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:15:32.841701 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-e97f199c-a4cd-4f12-a1f9-875858e5bff2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:15:32.905787 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:15:32.953875 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97f199c-a4cd-4f12-a1f9-875858e5bff2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:15:32.955347 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97f199c-a4cd-4f12-a1f9-875858e5bff2 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:15:32.975546 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 656/1319] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 14:15:32 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:15:36.129594 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4706925c-5e38-4ecf-91ae-3419b1b98311 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:15:36.130165 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4706925c-5e38-4ecf-91ae-3419b1b98311 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:15:36.130165 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4706925c-5e38-4ecf-91ae-3419b1b98311 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:15:36.131609 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 664/1320] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:15:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:15:36.137679 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83d3bfc-0446-4833-937e-5f5e90267b73 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:15:36.138265 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83d3bfc-0446-4833-937e-5f5e90267b73 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:15:36.138265 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83d3bfc-0446-4833-937e-5f5e90267b73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:15:36.216660 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-e83d3bfc-0446-4833-937e-5f5e90267b73 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:15:36.217917 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-e83d3bfc-0446-4833-937e-5f5e90267b73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:15:36.313765 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 657/1321] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:15:36 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:15:36.353959 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4706925c-5e38-4ecf-91ae-3419b1b98311 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:15:36.355487 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-4706925c-5e38-4ecf-91ae-3419b1b98311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:15:36.405746 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:15:36.459597 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4706925c-5e38-4ecf-91ae-3419b1b98311 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:15:36.530997 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e635f61e-2399-4ea8-a5f0-b49744370aa7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56ce23600bfe42e6b98eeebfe7470560', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6f995ffa0a94b1584f25dbe31e20524', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:15:36.531644 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e635f61e-2399-4ea8-a5f0-b49744370aa7 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:15:36.531927 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e635f61e-2399-4ea8-a5f0-b49744370aa7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:15:36.532199 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e635f61e-2399-4ea8-a5f0-b49744370aa7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:15:36.533870 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-e635f61e-2399-4ea8-a5f0-b49744370aa7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:15:36.630032 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:15:36.643643 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e635f61e-2399-4ea8-a5f0-b49744370aa7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:15:36.645083 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e635f61e-2399-4ea8-a5f0-b49744370aa7 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:15:36.663561 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 665/1322] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 14:15:36 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:15:39.651933 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b91b735-5693-4135-82d6-671e0466d262 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:15:39.652464 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b91b735-5693-4135-82d6-671e0466d262 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:15:39.652555 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b91b735-5693-4135-82d6-671e0466d262 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:15:39.653913 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 658/1323] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:15:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:15:39.659792 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965cf060-b60e-46f4-b049-b5882759c89b None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:15:39.660247 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965cf060-b60e-46f4-b049-b5882759c89b None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:15:39.660454 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965cf060-b60e-46f4-b049-b5882759c89b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:15:39.724144 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-965cf060-b60e-46f4-b049-b5882759c89b None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:15:39.725343 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-965cf060-b60e-46f4-b049-b5882759c89b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:15:39.819990 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 666/1324] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:15:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 14:15:39.855980 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b91b735-5693-4135-82d6-671e0466d262 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:15:39.857626 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-4b91b735-5693-4135-82d6-671e0466d262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:15:39.876595 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:15:39.965782 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b91b735-5693-4135-82d6-671e0466d262 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:15:40.042609 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8258ff5b-b624-4a2e-bf92-5855f9476911 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56ce23600bfe42e6b98eeebfe7470560', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6f995ffa0a94b1584f25dbe31e20524', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:15:40.043253 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8258ff5b-b624-4a2e-bf92-5855f9476911 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:15:40.043537 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8258ff5b-b624-4a2e-bf92-5855f9476911 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:15:40.043816 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8258ff5b-b624-4a2e-bf92-5855f9476911 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:15:40.045456 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-8258ff5b-b624-4a2e-bf92-5855f9476911 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:15:40.122679 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:15:40.161048 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8258ff5b-b624-4a2e-bf92-5855f9476911 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:15:40.162517 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8258ff5b-b624-4a2e-bf92-5855f9476911 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:15:40.183817 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 659/1325] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 14:15:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:15:42.477075 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea8949d-3cf5-4a34-b319-6cb637f0f213 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:15:42.477636 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea8949d-3cf5-4a34-b319-6cb637f0f213 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:15:42.477636 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea8949d-3cf5-4a34-b319-6cb637f0f213 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:15:42.479323 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 667/1326] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:15:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:15:42.484555 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e0528f-eebb-4036-ae1b-e6bbe16e6476 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:15:42.484960 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e0528f-eebb-4036-ae1b-e6bbe16e6476 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:15:42.485119 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e0528f-eebb-4036-ae1b-e6bbe16e6476 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:15:42.551717 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-60e0528f-eebb-4036-ae1b-e6bbe16e6476 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:15:42.552944 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-60e0528f-eebb-4036-ae1b-e6bbe16e6476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:15:42.649621 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 660/1327] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:15:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:15:42.685545 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea8949d-3cf5-4a34-b319-6cb637f0f213 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:15:42.687116 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-cea8949d-3cf5-4a34-b319-6cb637f0f213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:15:42.749429 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:15:42.792695 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea8949d-3cf5-4a34-b319-6cb637f0f213 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:15:42.866603 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3ae75c-7dd1-46c2-9093-d384b5ae272d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56ce23600bfe42e6b98eeebfe7470560', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6f995ffa0a94b1584f25dbe31e20524', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:15:42.867251 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3ae75c-7dd1-46c2-9093-d384b5ae272d service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:15:42.867534 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3ae75c-7dd1-46c2-9093-d384b5ae272d service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:15:42.867809 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3ae75c-7dd1-46c2-9093-d384b5ae272d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:15:42.869431 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-8a3ae75c-7dd1-46c2-9093-d384b5ae272d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:15:42.977995 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:15:42.981008 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3ae75c-7dd1-46c2-9093-d384b5ae272d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:15:42.982941 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3ae75c-7dd1-46c2-9093-d384b5ae272d service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:15:43.001749 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 668/1328] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 14:15:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:15:45.298939 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de17192-9582-467b-881a-4c3dbf35b47f None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:15:45.299641 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de17192-9582-467b-881a-4c3dbf35b47f None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:15:45.299641 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de17192-9582-467b-881a-4c3dbf35b47f None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:15:45.300908 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 661/1329] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:15:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:15:45.306295 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78969fcf-e5df-411c-81d9-21d1d45dcd6b None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:15:45.306750 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78969fcf-e5df-411c-81d9-21d1d45dcd6b None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:15:45.306878 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78969fcf-e5df-411c-81d9-21d1d45dcd6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:15:45.375138 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-78969fcf-e5df-411c-81d9-21d1d45dcd6b None None] MFA Rules not processed for user `47b5316234e94937b78cedeab15c486e`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:15:45.376346 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-78969fcf-e5df-411c-81d9-21d1d45dcd6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:15:45.478828 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 669/1330] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:15:45 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:15:45.515770 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de17192-9582-467b-881a-4c3dbf35b47f None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:15:45.517305 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-2de17192-9582-467b-881a-4c3dbf35b47f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:15:45.549257 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:15:45.625741 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de17192-9582-467b-881a-4c3dbf35b47f None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:15:45.698387 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e812c227-5779-450f-9020-5bbc8377237a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56ce23600bfe42e6b98eeebfe7470560', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6f995ffa0a94b1584f25dbe31e20524', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:15:45.698893 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e812c227-5779-450f-9020-5bbc8377237a service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:15:45.699176 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e812c227-5779-450f-9020-5bbc8377237a service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:15:45.699451 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e812c227-5779-450f-9020-5bbc8377237a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:15:45.701090 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-e812c227-5779-450f-9020-5bbc8377237a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:15:45.779750 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:15:45.813367 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e812c227-5779-450f-9020-5bbc8377237a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:15:45.814843 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e812c227-5779-450f-9020-5bbc8377237a service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:15:45.834258 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 662/1331] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 14:15:45 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:15:48.310104 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7337672c-7890-41b6-8eea-b0fc754b461b None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:15:48.310569 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7337672c-7890-41b6-8eea-b0fc754b461b None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:15:48.310684 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7337672c-7890-41b6-8eea-b0fc754b461b None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:15:48.312030 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 670/1332] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:15:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:15:48.317774 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df78bf7-f6b4-4b57-8052-d605e221c61a None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:15:48.318215 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df78bf7-f6b4-4b57-8052-d605e221c61a None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:15:48.318360 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df78bf7-f6b4-4b57-8052-d605e221c61a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:15:48.391092 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-6df78bf7-f6b4-4b57-8052-d605e221c61a None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:15:48.392335 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-6df78bf7-f6b4-4b57-8052-d605e221c61a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:15:48.492886 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 663/1333] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:15:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:15:48.530219 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7337672c-7890-41b6-8eea-b0fc754b461b None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:15:48.531675 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-7337672c-7890-41b6-8eea-b0fc754b461b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:15:48.601915 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:15:48.637107 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7337672c-7890-41b6-8eea-b0fc754b461b None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:15:48.714318 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70388ce1-37ae-4327-b47d-b404502886e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56ce23600bfe42e6b98eeebfe7470560', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6f995ffa0a94b1584f25dbe31e20524', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:15:48.714976 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70388ce1-37ae-4327-b47d-b404502886e6 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:15:48.715255 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70388ce1-37ae-4327-b47d-b404502886e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:15:48.715528 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70388ce1-37ae-4327-b47d-b404502886e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:15:48.717164 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-70388ce1-37ae-4327-b47d-b404502886e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:15:48.826196 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70388ce1-37ae-4327-b47d-b404502886e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:15:48.827647 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70388ce1-37ae-4327-b47d-b404502886e6 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:15:48.837729 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:15:48.846874 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 671/1334] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 14:15:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:15:51.154694 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17fe054-c0ff-4e1a-8f7a-c60b494fc388 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:15:51.155255 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17fe054-c0ff-4e1a-8f7a-c60b494fc388 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:15:51.155255 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17fe054-c0ff-4e1a-8f7a-c60b494fc388 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:15:51.156686 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 664/1335] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:15:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:15:51.162299 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc7e666-6c38-4b04-b75c-d33c7e736d8d None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:15:51.162688 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc7e666-6c38-4b04-b75c-d33c7e736d8d None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:15:51.162870 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc7e666-6c38-4b04-b75c-d33c7e736d8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:15:51.230101 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-8dc7e666-6c38-4b04-b75c-d33c7e736d8d None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:15:51.231315 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-8dc7e666-6c38-4b04-b75c-d33c7e736d8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:15:51.329479 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 672/1336] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:15:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:15:51.370327 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17fe054-c0ff-4e1a-8f7a-c60b494fc388 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:15:51.371851 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-a17fe054-c0ff-4e1a-8f7a-c60b494fc388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:15:51.412534 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:15:51.477709 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17fe054-c0ff-4e1a-8f7a-c60b494fc388 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:15:51.555129 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59696a52-9730-4d3e-8e32-237b008891a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56ce23600bfe42e6b98eeebfe7470560', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6f995ffa0a94b1584f25dbe31e20524', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:15:51.555755 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59696a52-9730-4d3e-8e32-237b008891a2 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:15:51.556036 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59696a52-9730-4d3e-8e32-237b008891a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:15:51.556310 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59696a52-9730-4d3e-8e32-237b008891a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:15:51.557985 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-59696a52-9730-4d3e-8e32-237b008891a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:15:51.645925 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:15:51.674255 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59696a52-9730-4d3e-8e32-237b008891a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:15:51.675906 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59696a52-9730-4d3e-8e32-237b008891a2 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:15:51.695006 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 665/1337] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 14:15:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:15:54.324571 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3ee106-d8e6-41b2-a22b-695579ee3f1c None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:15:54.325131 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3ee106-d8e6-41b2-a22b-695579ee3f1c None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:15:54.325131 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3ee106-d8e6-41b2-a22b-695579ee3f1c None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:15:54.326661 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 673/1338] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:15:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:15:54.332522 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb6ebbd-e93f-491d-ab99-b529b5327ead None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:15:54.332938 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb6ebbd-e93f-491d-ab99-b529b5327ead None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:15:54.333086 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb6ebbd-e93f-491d-ab99-b529b5327ead None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:15:54.399225 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-cbb6ebbd-e93f-491d-ab99-b529b5327ead None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:15:54.400426 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-cbb6ebbd-e93f-491d-ab99-b529b5327ead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:15:54.502103 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 666/1339] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:15:54 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:15:54.538064 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3ee106-d8e6-41b2-a22b-695579ee3f1c None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:15:54.539594 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-cc3ee106-d8e6-41b2-a22b-695579ee3f1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:15:54.614166 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:15:54.641602 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3ee106-d8e6-41b2-a22b-695579ee3f1c None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:15:54.713815 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d751e9-c60d-48a6-b74d-858b44eacfc8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56ce23600bfe42e6b98eeebfe7470560', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6f995ffa0a94b1584f25dbe31e20524', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:15:54.714407 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d751e9-c60d-48a6-b74d-858b44eacfc8 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:15:54.714687 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d751e9-c60d-48a6-b74d-858b44eacfc8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:15:54.714956 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d751e9-c60d-48a6-b74d-858b44eacfc8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:15:54.716562 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-b6d751e9-c60d-48a6-b74d-858b44eacfc8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:15:54.823655 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d751e9-c60d-48a6-b74d-858b44eacfc8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:15:54.825083 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d751e9-c60d-48a6-b74d-858b44eacfc8 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:15:54.843077 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 674/1340] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 14:15:54 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:15:59.165355 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5375639b-e2f8-4120-9f6d-d876202b986e None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:15:59.165898 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5375639b-e2f8-4120-9f6d-d876202b986e None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:15:59.165898 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5375639b-e2f8-4120-9f6d-d876202b986e None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:15:59.167463 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 667/1341] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:15:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:15:59.172743 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa646bf0-d29d-4819-b109-741a476cf711 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:15:59.173213 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa646bf0-d29d-4819-b109-741a476cf711 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:15:59.173311 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa646bf0-d29d-4819-b109-741a476cf711 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:15:59.180595 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:15:59.247505 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-fa646bf0-d29d-4819-b109-741a476cf711 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:15:59.248704 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-fa646bf0-d29d-4819-b109-741a476cf711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:15:59.356059 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 675/1342] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:15:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:15:59.395712 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5375639b-e2f8-4120-9f6d-d876202b986e None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:15:59.397936 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-5375639b-e2f8-4120-9f6d-d876202b986e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:15:59.450287 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:15:59.497507 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5375639b-e2f8-4120-9f6d-d876202b986e None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:15:59.501091 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5e148a-f1b0-468e-9189-91f4f31f16b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56ce23600bfe42e6b98eeebfe7470560', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6f995ffa0a94b1584f25dbe31e20524', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:15:59.501739 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5e148a-f1b0-468e-9189-91f4f31f16b1 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:15:59.502074 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5e148a-f1b0-468e-9189-91f4f31f16b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:15:59.502335 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5e148a-f1b0-468e-9189-91f4f31f16b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:15:59.511712 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5e148a-f1b0-468e-9189-91f4f31f16b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:15:59.513157 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5e148a-f1b0-468e-9189-91f4f31f16b1 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:15:59.522661 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 668/1343] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 14:15:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:16:02.313587 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e84c11b-6bdc-4fa0-8aa2-9cab7e1bbe9a None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:16:02.314149 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e84c11b-6bdc-4fa0-8aa2-9cab7e1bbe9a None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:16:02.314149 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e84c11b-6bdc-4fa0-8aa2-9cab7e1bbe9a None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:16:02.316051 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 676/1344] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:16:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:16:02.321286 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab87abe8-0c00-440a-8c06-1862b2e475f4 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:16:02.321695 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab87abe8-0c00-440a-8c06-1862b2e475f4 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:16:02.321930 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab87abe8-0c00-440a-8c06-1862b2e475f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:16:02.349366 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-ab87abe8-0c00-440a-8c06-1862b2e475f4 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:16:02.350509 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-ab87abe8-0c00-440a-8c06-1862b2e475f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:16:02.426307 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 669/1345] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:16:02 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:16:02.464330 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e84c11b-6bdc-4fa0-8aa2-9cab7e1bbe9a None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:16:02.475124 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e84c11b-6bdc-4fa0-8aa2-9cab7e1bbe9a None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:16:02.478903 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8840e26a-d952-447e-8b49-6f9482a6c85c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56ce23600bfe42e6b98eeebfe7470560', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6f995ffa0a94b1584f25dbe31e20524', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:16:02.479538 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8840e26a-d952-447e-8b49-6f9482a6c85c service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:16:02.479824 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8840e26a-d952-447e-8b49-6f9482a6c85c service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:16:02.480102 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8840e26a-d952-447e-8b49-6f9482a6c85c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:16:02.489089 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8840e26a-d952-447e-8b49-6f9482a6c85c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:16:02.490568 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8840e26a-d952-447e-8b49-6f9482a6c85c service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:16:02.499969 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 677/1346] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 14:16:02 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:16:06.759790 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff14ad3a-4ce0-48b0-a42c-926517f943d4 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:16:06.760348 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff14ad3a-4ce0-48b0-a42c-926517f943d4 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:16:06.760348 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff14ad3a-4ce0-48b0-a42c-926517f943d4 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:16:06.761726 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 670/1347] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:16:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:16:06.767134 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7794db1-a306-4acb-8eae-159fb0827375 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:16:06.767540 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7794db1-a306-4acb-8eae-159fb0827375 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:16:06.767730 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7794db1-a306-4acb-8eae-159fb0827375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:16:06.786694 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-f7794db1-a306-4acb-8eae-159fb0827375 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:16:06.787924 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-f7794db1-a306-4acb-8eae-159fb0827375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:16:06.796428 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 678/1348] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:16:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:16:06.834878 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff14ad3a-4ce0-48b0-a42c-926517f943d4 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:16:06.845450 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff14ad3a-4ce0-48b0-a42c-926517f943d4 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:16:06.849928 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d811a2e9-95a4-4db5-9d53-c792478b4f2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56ce23600bfe42e6b98eeebfe7470560', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6f995ffa0a94b1584f25dbe31e20524', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:16:06.850576 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d811a2e9-95a4-4db5-9d53-c792478b4f2c service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:16:06.850862 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d811a2e9-95a4-4db5-9d53-c792478b4f2c service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:16:06.851191 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d811a2e9-95a4-4db5-9d53-c792478b4f2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:16:06.861160 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d811a2e9-95a4-4db5-9d53-c792478b4f2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:16:06.863885 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d811a2e9-95a4-4db5-9d53-c792478b4f2c service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:16:06.874493 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 671/1349] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 14:16:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:16:12.311981 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958fe640-9a27-4fd1-99f2-81bb26dceade None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:16:12.312508 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958fe640-9a27-4fd1-99f2-81bb26dceade None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:16:12.312508 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958fe640-9a27-4fd1-99f2-81bb26dceade None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:16:12.314270 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 679/1350] 10.222.0.57 () {54 vars in 921 bytes} [Thu May 19 14:16:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:16:12.319843 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa7ce98-4df1-494a-af5f-51af6bf6d115 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:16:12.320656 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa7ce98-4df1-494a-af5f-51af6bf6d115 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:16:12.321146 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa7ce98-4df1-494a-af5f-51af6bf6d115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:16:12.400746 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-2aa7ce98-4df1-494a-af5f-51af6bf6d115 None None] MFA Rules not processed for user `dbe346b6d22e429e8e460b4cd9a0f613`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:16:12.403089 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-2aa7ce98-4df1-494a-af5f-51af6bf6d115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:16:12.431325 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:16:12.490709 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 672/1351] 10.222.0.57 () {58 vars in 1018 bytes} [Thu May 19 14:16:12 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:16:16.780421 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=72 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:16:16.842601 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be19e7b-9f22-42af-9b31-9b3bba9005fa None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:16:16.843168 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be19e7b-9f22-42af-9b31-9b3bba9005fa None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:16:16.843433 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be19e7b-9f22-42af-9b31-9b3bba9005fa None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:16:16.844964 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 680/1352] 10.222.0.57 () {54 vars in 921 bytes} [Thu May 19 14:16:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:16:16.850961 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d43ae0-f113-4e40-a4c3-3ea037161c44 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:16:16.851451 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d43ae0-f113-4e40-a4c3-3ea037161c44 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:16:16.851536 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d43ae0-f113-4e40-a4c3-3ea037161c44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:16:16.872293 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-e7d43ae0-f113-4e40-a4c3-3ea037161c44 None None] MFA Rules not processed for user `dbe346b6d22e429e8e460b4cd9a0f613`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:16:16.873507 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-e7d43ae0-f113-4e40-a4c3-3ea037161c44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:16:16.882857 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 673/1353] 10.222.0.57 () {58 vars in 1018 bytes} [Thu May 19 14:16:16 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:16:18.567777 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58dd6ee-3ddb-4801-bc8b-ab5ae25f7511 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:16:18.568305 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58dd6ee-3ddb-4801-bc8b-ab5ae25f7511 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:16:18.568305 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58dd6ee-3ddb-4801-bc8b-ab5ae25f7511 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:16:18.570038 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 681/1354] 10.222.0.57 () {54 vars in 921 bytes} [Thu May 19 14:16:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:16:18.575129 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2256e5d-fcb2-4e55-be06-d77ff85e81fb None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:16:18.575623 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2256e5d-fcb2-4e55-be06-d77ff85e81fb None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:16:18.575718 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2256e5d-fcb2-4e55-be06-d77ff85e81fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:16:18.594425 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-d2256e5d-fcb2-4e55-be06-d77ff85e81fb None None] MFA Rules not processed for user `dbe346b6d22e429e8e460b4cd9a0f613`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:16:18.595610 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-d2256e5d-fcb2-4e55-be06-d77ff85e81fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:16:18.604887 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 674/1355] 10.222.0.57 () {58 vars in 1018 bytes} [Thu May 19 14:16:18 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:16:22.452574 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a47cc5-2431-49cc-a562-d4eb60d46dfb None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:16:22.453168 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a47cc5-2431-49cc-a562-d4eb60d46dfb None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:16:22.453324 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a47cc5-2431-49cc-a562-d4eb60d46dfb None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:16:22.454797 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 682/1356] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:16:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:16:22.460732 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e84c82-b996-4576-9bff-1b34f265063e None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:16:22.461202 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e84c82-b996-4576-9bff-1b34f265063e None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:16:22.461494 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e84c82-b996-4576-9bff-1b34f265063e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:16:22.480433 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-30e84c82-b996-4576-9bff-1b34f265063e None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:16:22.481750 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-30e84c82-b996-4576-9bff-1b34f265063e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:16:22.502362 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 675/1357] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:16:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:16:22.548914 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7933fedb-8d42-49ed-9774-668f89695acb None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:16:22.549495 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7933fedb-8d42-49ed-9774-668f89695acb None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:16:22.550727 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7933fedb-8d42-49ed-9774-668f89695acb None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:16:22.557353 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 683/1358] 10.222.0.57 () {54 vars in 961 bytes} [Thu May 19 14:16:22 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:16:22.572083 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9825fb45-dc50-4dad-b4eb-3b894dacbf2b None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:16:22.572733 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9825fb45-dc50-4dad-b4eb-3b894dacbf2b None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:16:22.573407 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9825fb45-dc50-4dad-b4eb-3b894dacbf2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:16:22.633612 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-9825fb45-dc50-4dad-b4eb-3b894dacbf2b None None] MFA Rules not processed for user `8c87f361a92d4834854ca07f652be2ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:16:22.634970 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-9825fb45-dc50-4dad-b4eb-3b894dacbf2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:16:22.725338 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 676/1359] 10.222.0.57 () {58 vars in 1058 bytes} [Thu May 19 14:16:22 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:16:22.733359 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7933fedb-8d42-49ed-9774-668f89695acb None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:16:22.748963 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7933fedb-8d42-49ed-9774-668f89695acb None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:16:22.754765 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1926a475-c378-4886-9c05-7ffd312cdad8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c87f361a92d4834854ca07f652be2ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6f995ffa0a94b1584f25dbe31e20524', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:16:22.755481 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1926a475-c378-4886-9c05-7ffd312cdad8 service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:16:22.755832 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1926a475-c378-4886-9c05-7ffd312cdad8 service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:16:22.756175 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1926a475-c378-4886-9c05-7ffd312cdad8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:16:22.766285 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1926a475-c378-4886-9c05-7ffd312cdad8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:16:22.767911 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1926a475-c378-4886-9c05-7ffd312cdad8 service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:16:22.778570 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 684/1360] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 14:16:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:16:25.261060 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfcea55-178b-4371-ba09-47a80a4b053a None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:16:25.261522 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfcea55-178b-4371-ba09-47a80a4b053a None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:16:25.261613 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfcea55-178b-4371-ba09-47a80a4b053a None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:16:25.263044 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 677/1361] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:16:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:16:25.269513 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca515da-9508-4e42-ba4e-a2b8e1fb2d1b None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:16:25.270282 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca515da-9508-4e42-ba4e-a2b8e1fb2d1b None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:16:25.270563 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca515da-9508-4e42-ba4e-a2b8e1fb2d1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:16:25.289892 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-3ca515da-9508-4e42-ba4e-a2b8e1fb2d1b None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:16:25.291101 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-3ca515da-9508-4e42-ba4e-a2b8e1fb2d1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:16:25.299074 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 685/1362] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:16:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:16:25.343989 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfcea55-178b-4371-ba09-47a80a4b053a None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:16:25.360175 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfcea55-178b-4371-ba09-47a80a4b053a None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:16:25.365809 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1baf869-a65a-44e3-a6a2-cede3fec556b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c87f361a92d4834854ca07f652be2ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6f995ffa0a94b1584f25dbe31e20524', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:16:25.366487 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1baf869-a65a-44e3-a6a2-cede3fec556b service nova] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:16:25.366772 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1baf869-a65a-44e3-a6a2-cede3fec556b service nova] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:16:25.367049 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1baf869-a65a-44e3-a6a2-cede3fec556b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:16:25.376076 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1baf869-a65a-44e3-a6a2-cede3fec556b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:16:25.377781 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1baf869-a65a-44e3-a6a2-cede3fec556b service nova] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:16:25.388593 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 678/1363] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 14:16:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:16:27.949376 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069b648a-6b2a-43e8-a207-613b17879a00 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:16:27.950012 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069b648a-6b2a-43e8-a207-613b17879a00 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:16:27.950012 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069b648a-6b2a-43e8-a207-613b17879a00 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:16:27.951299 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 686/1364] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:16:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:16:27.956533 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad91da74-f1c8-45e8-bea0-c845e00405d1 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:16:27.957019 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad91da74-f1c8-45e8-bea0-c845e00405d1 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:16:27.957133 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad91da74-f1c8-45e8-bea0-c845e00405d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:16:27.975569 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-ad91da74-f1c8-45e8-bea0-c845e00405d1 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:16:27.977002 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-ad91da74-f1c8-45e8-bea0-c845e00405d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:16:27.985307 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 679/1365] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:16:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:16:28.040946 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e510af51-97a2-42f9-a0c3-e5cde748cdc0 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:16:28.041520 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e510af51-97a2-42f9-a0c3-e5cde748cdc0 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:16:28.042040 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e510af51-97a2-42f9-a0c3-e5cde748cdc0 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:16:28.043836 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 687/1366] 10.222.0.57 () {54 vars in 961 bytes} [Thu May 19 14:16:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:16:28.050990 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d961d1b-80d9-492c-8c32-2389fc4e87a0 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:16:28.053527 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d961d1b-80d9-492c-8c32-2389fc4e87a0 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:16:28.053821 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d961d1b-80d9-492c-8c32-2389fc4e87a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:16:28.076124 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-6d961d1b-80d9-492c-8c32-2389fc4e87a0 None None] MFA Rules not processed for user `8c87f361a92d4834854ca07f652be2ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:16:28.077344 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-6d961d1b-80d9-492c-8c32-2389fc4e87a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:16:28.088843 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 680/1367] 10.222.0.57 () {58 vars in 1058 bytes} [Thu May 19 14:16:28 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:16:28.098044 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e510af51-97a2-42f9-a0c3-e5cde748cdc0 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:16:28.123026 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e510af51-97a2-42f9-a0c3-e5cde748cdc0 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:16:28.129559 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a97ee9b-ec96-4b0c-b6e7-0670d2ec1062 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c87f361a92d4834854ca07f652be2ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6f995ffa0a94b1584f25dbe31e20524', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:16:28.130248 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a97ee9b-ec96-4b0c-b6e7-0670d2ec1062 service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:16:28.130536 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a97ee9b-ec96-4b0c-b6e7-0670d2ec1062 service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:16:28.130815 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a97ee9b-ec96-4b0c-b6e7-0670d2ec1062 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:16:28.139879 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a97ee9b-ec96-4b0c-b6e7-0670d2ec1062 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:16:28.141509 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a97ee9b-ec96-4b0c-b6e7-0670d2ec1062 service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:16:28.151525 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 688/1368] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 14:16:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:16:28.847454 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705f6d68-f5ef-4442-9879-06c8aa6e5126 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:16:28.847454 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705f6d68-f5ef-4442-9879-06c8aa6e5126 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:16:28.847713 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705f6d68-f5ef-4442-9879-06c8aa6e5126 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:16:28.849288 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 681/1369] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:16:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:16:28.855240 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd02573-28cd-490c-a3f2-0beef496b02a None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:16:28.855639 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd02573-28cd-490c-a3f2-0beef496b02a None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:16:28.856377 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd02573-28cd-490c-a3f2-0beef496b02a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:16:28.877744 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-5fd02573-28cd-490c-a3f2-0beef496b02a None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:16:28.879065 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-5fd02573-28cd-490c-a3f2-0beef496b02a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:16:28.898221 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 689/1370] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:16:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:16:30.454625 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5372b0a-762e-44b1-a69f-6aafb9af21b2 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:16:30.455216 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5372b0a-762e-44b1-a69f-6aafb9af21b2 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:16:30.455216 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5372b0a-762e-44b1-a69f-6aafb9af21b2 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:16:30.457122 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 682/1371] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:16:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:16:30.462557 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc54279-3e2d-4b22-b588-6bf01b7ec741 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:16:30.463077 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc54279-3e2d-4b22-b588-6bf01b7ec741 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:16:30.463175 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc54279-3e2d-4b22-b588-6bf01b7ec741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:16:30.480827 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-fcc54279-3e2d-4b22-b588-6bf01b7ec741 None None] MFA Rules not processed for user `d4608701c57147b39ea0df421d2e5029`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:16:30.482058 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-fcc54279-3e2d-4b22-b588-6bf01b7ec741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:16:30.489828 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 690/1372] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:16:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:16:30.534101 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5372b0a-762e-44b1-a69f-6aafb9af21b2 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:16:30.548638 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5372b0a-762e-44b1-a69f-6aafb9af21b2 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:16:30.554696 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e568ad35-34ab-47ec-a497-65cdb35bccb3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c87f361a92d4834854ca07f652be2ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6f995ffa0a94b1584f25dbe31e20524', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:16:30.555333 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e568ad35-34ab-47ec-a497-65cdb35bccb3 service nova] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:16:30.555617 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e568ad35-34ab-47ec-a497-65cdb35bccb3 service nova] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:16:30.555893 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e568ad35-34ab-47ec-a497-65cdb35bccb3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:16:30.564125 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e568ad35-34ab-47ec-a497-65cdb35bccb3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:16:30.566376 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e568ad35-34ab-47ec-a497-65cdb35bccb3 service nova] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:16:30.576120 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 683/1373] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 14:16:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:16:40.475471 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=21 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:16:40.571661 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=95 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:16:52.613835 dv-cinder-842541-1-iscsi devstack@keystone.service[80189]: Thu May 19 14:16:52 2022 - graceful shutdown triggered... May 19 14:16:52.614327 dv-cinder-842541-1-iscsi devstack@keystone.service[80202]: Gracefully killing worker 1 (pid: 80202)... May 19 14:16:52.614679 dv-cinder-842541-1-iscsi devstack@keystone.service[80203]: Gracefully killing worker 2 (pid: 80203)... May 19 14:16:52.616207 dv-cinder-842541-1-iscsi systemd[1]: Stopping Devstack devstack@keystone.service... May 19 14:16:53.614704 dv-cinder-842541-1-iscsi devstack@keystone.service[80189]: worker 1 buried after 1 seconds May 19 14:16:53.614704 dv-cinder-842541-1-iscsi devstack@keystone.service[80189]: worker 2 buried after 1 seconds May 19 14:16:53.614704 dv-cinder-842541-1-iscsi devstack@keystone.service[80189]: goodbye to uWSGI. May 19 14:16:53.616288 dv-cinder-842541-1-iscsi systemd[1]: devstack@keystone.service: Succeeded. May 19 14:16:53.616837 dv-cinder-842541-1-iscsi systemd[1]: Stopped Devstack devstack@keystone.service. May 19 14:22:47.360797 dv-cinder-842541-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... May 19 14:22:47.368336 dv-cinder-842541-1-iscsi devstack@keystone.service[143576]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini May 19 14:22:47.372084 dv-cinder-842541-1-iscsi devstack@keystone.service[143576]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu May 19 14:22:47 2022] *** May 19 14:22:47.372084 dv-cinder-842541-1-iscsi devstack@keystone.service[143576]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 May 19 14:22:47.372084 dv-cinder-842541-1-iscsi devstack@keystone.service[143576]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 May 19 14:22:47.372084 dv-cinder-842541-1-iscsi devstack@keystone.service[143576]: nodename: dv-cinder-842541-1-iscsi May 19 14:22:47.372084 dv-cinder-842541-1-iscsi devstack@keystone.service[143576]: machine: x86_64 May 19 14:22:47.372084 dv-cinder-842541-1-iscsi devstack@keystone.service[143576]: clock source: unix May 19 14:22:47.372084 dv-cinder-842541-1-iscsi devstack@keystone.service[143576]: pcre jit disabled May 19 14:22:47.372084 dv-cinder-842541-1-iscsi devstack@keystone.service[143576]: detected number of CPU cores: 4 May 19 14:22:47.372084 dv-cinder-842541-1-iscsi devstack@keystone.service[143576]: current working directory: / May 19 14:22:47.372084 dv-cinder-842541-1-iscsi devstack@keystone.service[143576]: detected binary path: /usr/bin/uwsgi-core May 19 14:22:47.372084 dv-cinder-842541-1-iscsi devstack@keystone.service[143576]: your processes number limit is 47680 May 19 14:22:47.372084 dv-cinder-842541-1-iscsi devstack@keystone.service[143576]: your memory page size is 4096 bytes May 19 14:22:47.372084 dv-cinder-842541-1-iscsi devstack@keystone.service[143576]: detected max file descriptor number: 2048 May 19 14:22:47.372084 dv-cinder-842541-1-iscsi devstack@keystone.service[143576]: lock engine: pthread robust mutexes May 19 14:22:47.372084 dv-cinder-842541-1-iscsi devstack@keystone.service[143576]: thunder lock: enabled May 19 14:22:47.372084 dv-cinder-842541-1-iscsi devstack@keystone.service[143576]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 May 19 14:22:47.372084 dv-cinder-842541-1-iscsi devstack@keystone.service[143576]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] May 19 14:22:47.418115 dv-cinder-842541-1-iscsi devstack@keystone.service[143576]: Python main interpreter initialized at 0x562ee60949c0 May 19 14:22:47.418115 dv-cinder-842541-1-iscsi devstack@keystone.service[143576]: python threads support enabled May 19 14:22:47.418115 dv-cinder-842541-1-iscsi devstack@keystone.service[143576]: your server socket listen backlog is limited to 100 connections May 19 14:22:47.418115 dv-cinder-842541-1-iscsi devstack@keystone.service[143576]: your mercy for graceful operations on workers is 90 seconds May 19 14:22:47.418517 dv-cinder-842541-1-iscsi devstack@keystone.service[143576]: mapped 403077 bytes (393 KB) for 2 cores May 19 14:22:47.418602 dv-cinder-842541-1-iscsi devstack@keystone.service[143576]: *** Operational MODE: preforking *** May 19 14:22:47.418808 dv-cinder-842541-1-iscsi devstack@keystone.service[143576]: *** uWSGI is running in multiple interpreter mode *** May 19 14:22:47.418808 dv-cinder-842541-1-iscsi devstack@keystone.service[143576]: spawned uWSGI master process (pid: 143576) May 19 14:22:47.419016 dv-cinder-842541-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. May 19 14:22:47.420085 dv-cinder-842541-1-iscsi devstack@keystone.service[143576]: spawned uWSGI worker 1 (pid: 143586, cores: 1) May 19 14:22:47.420085 dv-cinder-842541-1-iscsi devstack@keystone.service[143576]: spawned uWSGI worker 2 (pid: 143587, cores: 1) May 19 14:22:47.420085 dv-cinder-842541-1-iscsi devstack@keystone.service[143576]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... May 19 14:22:53.193335 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 14:22:53.193335 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: from cryptography.utils import int_from_bytes May 19 14:22:53.194246 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 14:22:53.194246 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: from cryptography.utils import int_from_bytes May 19 14:22:53.194413 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 14:22:53.194413 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: from cryptography.utils import int_from_bytes May 19 14:22:53.194920 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 14:22:53.194920 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: from cryptography.utils import int_from_bytes May 19 14:22:53.244371 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 19 14:22:53.244576 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 19 14:22:53.244736 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] command line args: [] {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 19 14:22:53.245004 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 19 14:22:53.245146 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 19 14:22:53.245249 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] ================================================================================ {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 19 14:22:53.245349 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 19 14:22:53.245440 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] admin_token = **** {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:22:53.245530 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] command line args: [] {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 19 14:22:53.245647 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] config_dir = [] {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:22:53.245793 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 19 14:22:53.245916 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:22:53.246025 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] ================================================================================ {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 19 14:22:53.246115 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] config_source = [] {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:22:53.246213 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] debug = True {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:22:53.246302 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] admin_token = **** {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:22:53.246480 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:22:53.246737 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] config_dir = [] {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:22:53.246737 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:22:53.246976 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:22:53.247099 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] config_source = [] {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:22:53.247218 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] insecure_debug = False {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:22:53.247358 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] debug = True {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:22:53.247476 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:22:53.247593 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:22:53.247749 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:22:53.247866 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:22:53.247866 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] insecure_debug = False {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:22:53.248637 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] list_limit = None {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:22:53.248637 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] log_config_append = None {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:22:53.248981 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:22:53.249146 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:22:53.249278 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:22:53.249399 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] log_dir = None {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:22:53.249495 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] list_limit = None {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:22:53.249603 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] log_file = None {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:22:53.249690 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] log_config_append = None {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:22:53.249782 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:22:53.249900 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:22:53.249900 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] log_dir = None {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:22:53.250078 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:22:53.250168 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] log_file = None {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:22:53.250258 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:22:53.250366 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:22:53.250451 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:22:53.250559 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:22:53.250657 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:22:53.250657 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:22:53.250842 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:22:53.250934 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:22:53.251039 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:22:53.251039 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:22:53.251201 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(user_domain)s %(project_domain)s {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:22:53.251351 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:22:53.251455 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:22:53.251599 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:22:53.251695 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:22:53.251780 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(user_domain)s %(project_domain)s {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:22:53.251927 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:22:53.252022 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:22:53.252113 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:22:53.252218 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:22:53.252358 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:22:53.252457 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:22:53.252560 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] notification_format = cadf {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:22:53.252662 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:22:53.252753 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:22:53.252853 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:22:53.252853 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] notification_format = cadf {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:22:53.253031 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.57/identity {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:22:53.253125 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:22:53.253210 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] publish_errors = False {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:22:53.253210 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:22:53.253391 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:22:53.253485 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.57/identity {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:22:53.253571 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:22:53.253665 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] publish_errors = False {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:22:53.253770 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:22:53.253884 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:22:53.254129 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:22:53.254228 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:22:53.254334 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] standard_threads = False {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:22:53.254436 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:22:53.254532 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] strict_password_check = False {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:22:53.254635 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:22:53.254734 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:22:53.254837 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:22:53.254930 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] use_eventlog = False {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:22:53.255032 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] standard_threads = False {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:22:53.255125 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] use_journal = False {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:22:53.255216 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] strict_password_check = False {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:22:53.255315 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] use_json = False {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:22:53.255315 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] use_stderr = False {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:22:53.255494 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:22:53.255594 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] use_syslog = False {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:22:53.255702 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] use_eventlog = False {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:22:53.255799 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] watch_log_file = False {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:22:53.255884 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] use_journal = False {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:22:53.255884 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] use_json = False {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:22:53.256063 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.256149 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] use_stderr = False {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:22:53.256252 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.256337 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] use_syslog = False {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:22:53.256431 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.256526 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] watch_log_file = False {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:22:53.256609 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.256713 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.256813 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.256911 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.257007 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.257091 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.257181 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.257277 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.257424 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.257529 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.257670 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.257772 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.257940 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.258044 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.258191 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.258289 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.258425 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.258522 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.258670 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.258768 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.258906 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.259005 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.259151 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.259242 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.259394 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.259486 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.259630 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.259728 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.259879 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.259994 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.260128 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.260218 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.260367 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.260472 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.260598 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.260705 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.260873 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.260998 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.261085 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.261167 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.261330 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.261428 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.261587 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.261695 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.261821 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.261945 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.262115 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.262231 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.262319 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.262411 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.262524 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.262622 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.262769 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.262866 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.263013 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.263123 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.263243 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.263347 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.263471 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.263676 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.264285 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.264428 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.264538 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.264792 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.264899 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.264991 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.265092 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.265191 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.265283 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.265373 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.265472 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.265573 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.265658 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.266058 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.266151 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.266247 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.266349 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.266446 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.266542 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.266636 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.266720 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.266803 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.266901 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.266998 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.267083 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.267176 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.267261 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.267383 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.267482 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.267619 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.267711 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.267893 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.267980 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.268079 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.268170 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.268288 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.268378 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.268541 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.268645 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.268739 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.268823 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.268970 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.269062 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.269204 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.269303 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.269439 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.269538 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.269672 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.269786 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.269935 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.270041 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.270163 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.270253 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.270410 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.270515 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.270650 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.270748 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.270885 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.270976 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.271116 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.271217 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.271371 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.271461 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.271604 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.271702 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.271839 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.271954 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.272074 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.272222 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.272324 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.272424 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.272540 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.272645 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.272780 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.272884 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.273025 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.273123 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.273270 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.273374 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.273506 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.273611 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.273751 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.273908 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.274009 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.274114 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.274247 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.274348 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] cache.enabled = True {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.274479 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.274586 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.274714 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.274815 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] cache.enabled = True {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.274955 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.275053 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.275189 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.275281 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.275424 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.275525 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.275659 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.275750 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.275893 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.275983 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.276126 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.276238 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.276363 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.276454 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.276603 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.276707 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.276838 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.276943 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.277072 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.277163 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.277309 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.277400 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.277543 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.277653 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.277778 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.277887 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.278038 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.278128 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.278267 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.278374 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.278481 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.278566 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.278711 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.278826 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.278946 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.279047 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.279174 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.279334 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.279632 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.279736 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.279837 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.279932 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.280026 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.280122 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.280207 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.280291 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.280482 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.280586 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.280727 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.280818 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.280989 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.281082 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.281279 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] auth.external = None {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.281382 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.281551 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] auth.mapped = None {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.281651 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.281827 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] auth.external = None {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.281969 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.282069 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] auth.mapped = None {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.282175 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.282360 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] auth.password = None {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.282451 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.282593 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] auth.token = None {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.282685 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.282830 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] auth.password = None {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.282934 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.283086 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] catalog.caching = True {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.283176 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] auth.token = None {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.283298 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.283389 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.283530 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.283622 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] catalog.caching = True {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.283777 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.283870 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.284012 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.284109 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.284239 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.284331 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.284476 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] credential.caching = True {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.284568 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.284724 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] credential.driver = sql {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.284821 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.284965 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] credential.caching = True {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.285054 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.285209 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.285299 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] credential.driver = sql {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.285445 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.285538 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.285689 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.285788 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.285951 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.286043 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.286202 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.286293 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.286443 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.286549 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.286681 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.286772 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.286934 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.287042 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.287189 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.287289 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.287446 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.287552 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.287708 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.287807 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.287948 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.288130 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.288225 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.288310 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.288411 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] federation.caching = True {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.288496 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.288644 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.288745 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.288887 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] federation.driver = sql {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.288992 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] federation.caching = True {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.289130 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.289230 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.289358 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.289471 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] federation.driver = sql {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.289603 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.289694 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.289836 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.289995 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.290106 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.290207 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.290340 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.290432 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.290591 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.290684 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.290824 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.290926 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.291067 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.291158 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.291302 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] identity.caching = True {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.291399 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.291551 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.291650 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.291793 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] identity.caching = True {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.291899 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.292026 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.292123 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.292260 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.292362 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.292505 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] identity.driver = sql {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.292597 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.292734 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.292896 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.292999 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.293098 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] identity.driver = sql {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.293184 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.293274 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.293435 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.293524 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.293660 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.293766 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.293913 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.294021 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.294144 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.294249 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.294402 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.294501 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.294618 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.294709 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.294890 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.294986 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.295145 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.295239 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.295396 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.295550 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.295648 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.295830 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.295935 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.296070 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.296162 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.296307 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.296410 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.296539 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.296640 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.296774 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.296865 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.297004 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.297107 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.297238 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.297342 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.297479 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.297570 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.297714 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.297804 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.297994 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.298101 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.298225 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.298328 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.298467 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.298559 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.298708 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.298817 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.298942 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.299034 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.299183 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.299281 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.299423 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.299532 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.299651 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.299743 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.299885 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.299995 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.300112 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] ldap.password = **** {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.300217 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.300343 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] ldap.password = **** {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.300442 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.300576 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.300673 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.300814 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.300914 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.301047 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.301136 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.301281 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.301371 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.301520 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.301626 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.301768 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.301879 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.302023 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.302116 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.302252 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.302357 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.302484 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.302575 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.302729 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.302827 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.302964 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.303080 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.303198 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.303288 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.303431 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.303523 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.303660 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] ldap.user = None {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.303773 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.303890 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] ldap.user = None {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.303980 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.304134 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.304225 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.304362 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.304462 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.304608 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.304707 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.304851 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.304950 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.305094 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.305187 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.305328 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.305418 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.305555 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.305658 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.305784 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.305935 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.306035 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.306119 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.306265 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.306356 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.306491 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.306591 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.306720 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.306828 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.306962 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.307067 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.307205 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.307297 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.307448 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.307539 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.307690 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.307789 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.307918 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.308022 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.308156 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.308259 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.308387 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.308492 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.308621 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.308757 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.308841 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.308920 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.309071 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.309164 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.309309 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.309407 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.309550 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.309640 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.309783 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.309916 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.310047 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.310153 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] policy.driver = sql {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.310288 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.310386 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] policy.driver = sql {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.311048 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.311154 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.311362 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.311804 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.311895 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.311977 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] receipt.caching = True {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.311977 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.312107 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] receipt.caching = True {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.312207 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.312345 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.312425 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.312497 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.312583 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.312680 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.312794 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.312926 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.313012 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] resource.caching = True {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.313160 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.313249 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.313407 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] resource.caching = True {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.313499 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] resource.driver = sql {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.313663 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.313751 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.314081 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.314186 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] resource.driver = sql {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.314274 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.314380 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.314469 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] revoke.caching = True {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.314552 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.314635 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.314719 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.314857 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.314960 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] revoke.caching = True {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.315096 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] role.cache_time = None {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.315187 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.315331 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] role.caching = True {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.315430 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.315575 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] role.driver = sql {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.315685 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] role.cache_time = None {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.315804 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] role.list_limit = None {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.315896 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] role.caching = True {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.316051 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.316144 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] role.driver = sql {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.316285 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] role.list_limit = None {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.316385 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.316530 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.316620 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.316775 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.316877 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.317018 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.317119 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.317261 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.317360 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.317503 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.317595 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.317753 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.317897 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.317999 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.318098 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.318229 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.318321 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.318471 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.318563 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.318713 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.318804 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.318958 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.319056 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.319206 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.319358 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.319450 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.319594 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.319843 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.319942 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.320085 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.320178 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.320329 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.320421 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.320578 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.320678 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.320811 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.320913 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.321060 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.321159 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.321297 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.321388 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.321532 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.321631 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.321761 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.321879 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.322020 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.322112 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.322251 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.322345 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.322488 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.322591 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.322723 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.322824 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.322971 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.323061 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.323209 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.323306 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.323443 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.323545 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.323674 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] token.cache_time = None {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.323778 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.323919 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.324009 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] token.caching = True {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.324154 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] token.cache_time = None {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.324260 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.324397 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] token.provider = fernet {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.324487 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] token.caching = True {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.324632 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.324732 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.324882 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] token.provider = fernet {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.324987 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.325125 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.325223 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.325361 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.325461 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.325602 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.325715 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.325896 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.326004 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.326175 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.326771 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] trust.driver = sql {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.326935 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.327112 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.327219 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.327370 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] trust.driver = sql {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.327452 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.327612 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.327704 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.327857 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.327957 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.328085 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.328176 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.328327 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.328419 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.328577 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.328672 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.328811 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.328903 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] database.connection = **** {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.329059 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.329159 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.329327 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.329424 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.329564 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.329664 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] database.connection = **** {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.329805 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.329938 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.330073 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.330184 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.330309 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.330401 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.330554 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.330645 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.330792 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.330892 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.331033 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.331125 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.331266 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.331356 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.331508 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.331609 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.331743 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.331836 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.331980 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.332071 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.332215 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.332316 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.332456 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.332555 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.332682 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.332773 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.332920 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.333023 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.333156 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.333254 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.333339 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=143587) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 19 14:22:53.333431 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.333618 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.333932 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:22:53.334038 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=143586) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 19 14:22:53.364286 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=143587) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:22:53.364514 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=143587) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:22:53.364734 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=143587) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:22:53.364877 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=143586) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:22:53.365117 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=143586) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:22:53.365344 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=143586) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:22:53.374005 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=143587) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:22:53.374786 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=143586) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:22:53.402737 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=143587) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:22:53.402965 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=143587) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:22:53.403769 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=143586) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:22:53.404006 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=143586) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:22:53.455947 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=143587) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:22:53.456644 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=143586) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:22:53.507244 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=143587) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:22:53.507469 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=143587) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:22:53.507743 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=143586) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:22:53.507983 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=143586) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:22:53.572891 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=143587) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:22:53.573138 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=143587) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:22:53.573313 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=143587) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:22:53.573422 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=143586) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:22:53.573516 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=143587) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:22:53.573602 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=143586) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:22:53.573733 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=143587) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:22:53.573876 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=143586) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:22:53.573986 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=143587) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:22:53.574093 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=143586) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:22:53.574226 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=143586) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:22:53.574443 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=143586) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:22:53.597682 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=143587) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:22:53.597945 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=143587) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:22:53.598113 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=143586) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:22:53.598267 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=143587) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:22:53.598374 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=143586) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:22:53.598494 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=143587) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:22:53.598596 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=143586) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:22:53.598710 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=143587) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:22:53.598803 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=143586) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:22:53.598924 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=143587) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:22:53.599101 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=143586) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:22:53.599190 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=143587) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:22:53.599275 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=143586) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:22:53.599359 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=143586) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:22:53.611191 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=143587) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:22:53.611537 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=143586) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:22:53.626304 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=143587) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:22:53.626527 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=143587) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:22:53.626621 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=143586) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:22:53.626750 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=143587) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:22:53.626860 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=143586) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:22:53.626965 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=143587) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:22:53.627059 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=143586) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:22:53.627175 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=143587) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:22:53.627275 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=143586) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:22:53.627374 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=143587) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:22:53.627470 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=143586) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:22:53.627588 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=143587) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:22:53.627682 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=143586) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:22:53.627827 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=143586) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:22:53.644794 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=143587) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.644997 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=143587) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 14:22:53.645324 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=143586) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.645440 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=143587) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.645533 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=143587) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 14:22:53.645619 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=143586) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 14:22:53.645771 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=143587) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.645898 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=143586) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.646055 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=143587) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.646149 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=143586) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 14:22:53.646312 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=143587) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.646413 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=143586) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.646563 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=143587) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.646672 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=143586) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.646821 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=143586) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.647078 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=143586) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.657791 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=143587) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.658104 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=143587) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.658238 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=143586) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.658372 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=143587) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.658490 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=143586) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.658631 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=143587) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.658755 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=143586) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.659009 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=143586) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.666753 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=143587) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:22:53.667159 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=143586) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:22:53.670563 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=143587) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:22:53.670889 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=143587) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.671059 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=143586) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:22:53.671167 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=143587) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.671339 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=143586) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.671532 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=143587) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.671636 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=143586) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.671814 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=143587) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.672008 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=143586) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.672103 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=143587) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.672297 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=143586) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.672390 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=143587) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.672562 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=143586) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.672657 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=143587) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.672837 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=143587) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.672928 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=143586) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.673110 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=143587) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.673204 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=143586) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.673378 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=143587) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.673480 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=143586) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.673654 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=143586) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.673953 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=143586) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.700803 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=143587) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.700982 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=143586) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.702604 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=143587) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:22:53.702765 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=143586) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:22:53.702887 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=143587) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.703045 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=143586) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.708209 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=143587) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:22:53.708345 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=143586) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:22:53.708440 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=143587) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.708624 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=143586) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.708716 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=143587) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.708906 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=143586) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.716021 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=143587) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:22:53.716225 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=143586) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:22:53.716320 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=143587) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.716512 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=143586) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.721261 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=143587) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:22:53.721401 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=143586) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:22:53.721543 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=143587) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.721626 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=143586) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.721824 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=143587) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.721967 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=143586) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.722131 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=143587) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.722227 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=143586) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.722412 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=143586) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.722514 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=143587) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.722686 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=143586) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.722794 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=143587) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.722955 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=143586) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.723049 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=143587) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.723221 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=143586) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.723315 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=143587) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.742664 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=143587) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.742816 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=143586) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.742923 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=143587) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.743014 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=143586) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.747424 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=143587) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:22:53.747528 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=143586) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:22:53.751698 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=143586) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.751814 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=143587) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.751987 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=143586) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.752107 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=143587) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.756901 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=143587) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:22:53.757013 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=143586) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:22:53.760469 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=143587) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:22:53.760585 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=143586) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:22:53.764264 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=143587) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.764421 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=143586) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.764562 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=143587) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.764724 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=143586) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.764842 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=143587) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.765013 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=143586) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.765121 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=143587) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.765298 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=143586) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.765399 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=143587) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.765578 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=143586) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.765694 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=143587) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.765911 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=143586) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.784908 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=143587) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:22:53.785101 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=143586) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:22:53.785286 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=143587) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.785480 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=143586) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.785575 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=143587) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.785752 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=143586) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.785877 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=143587) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.786047 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=143586) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.793691 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=143587) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.794003 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=143586) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.796185 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=143587) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.796310 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=143586) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.796457 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=143587) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.796569 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=143586) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.799537 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=143587) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:22:53.799636 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=143586) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:22:53.799811 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=143587) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.799919 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=143586) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.800096 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=143587) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.800189 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=143586) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.800382 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=143586) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.800483 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=143587) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.800655 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=143587) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.800754 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=143586) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.814701 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=143586) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:22:53.814909 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=143587) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:22:53.815006 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=143586) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.815186 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=143587) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.815287 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=143586) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.815460 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=143587) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.815553 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=143586) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.815735 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=143587) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.815849 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=143586) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.816011 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=143587) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.816105 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=143586) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.816292 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=143587) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.816393 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=143586) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.816573 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=143587) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.834460 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=143586) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:22:53.834894 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=143587) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:22:53.837717 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=143586) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:22:53.838173 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=143587) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:22:53.841071 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=143586) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.841463 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=143587) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.842928 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=143586) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.843110 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=143587) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.844579 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=143586) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:22:53.844739 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=143587) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:22:53.844856 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=143586) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.845023 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=143587) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.845119 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=143586) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.845308 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=143587) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.852855 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=143586) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.852964 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=143587) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.854511 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=143586) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:22:53.854622 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=143587) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:22:53.858295 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=143587) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.858444 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=143586) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.858580 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=143587) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.858738 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=143586) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.858862 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=143587) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.859023 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=143586) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.859129 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=143587) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.859302 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=143586) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.868269 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=143587) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:22:53.868577 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=143586) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:22:53.868683 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=143587) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.868865 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=143586) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.868958 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=143587) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.869154 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=143586) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.877381 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=143587) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:22:53.877622 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=143586) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:22:53.877733 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=143587) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.877933 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=143586) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.878030 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=143587) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.878218 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=143587) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.878319 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=143586) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.878487 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=143587) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.878583 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=143586) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.878765 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=143587) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.878874 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=143586) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.879043 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=143587) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.879138 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=143586) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.879307 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=143587) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.879409 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=143586) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.879573 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=143586) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.879744 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=143587) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.879847 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=143586) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.879937 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=143587) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.880094 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=143586) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.880189 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=143587) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.880354 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=143587) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.880455 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=143586) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.880620 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=143587) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.880713 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=143586) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.880875 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=143587) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.880977 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=143586) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.881142 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=143586) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:22:53.919304 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x562ee60949c0 pid: 143587 (default app) May 19 14:22:53.920598 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x562ee60949c0 pid: 143586 (default app) May 19 14:22:53.923273 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446ab111-5fbb-4d29-b3eb-afee6361b472 None None] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:53.923563 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446ab111-5fbb-4d29-b3eb-afee6361b472 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:53.923841 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446ab111-5fbb-4d29-b3eb-afee6361b472 None None] PATH_INFO: `/v3` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:53.928033 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 1/1] 10.222.0.57 () {50 vars in 791 bytes} [Thu May 19 14:22:53 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:01.950052 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869a5805-79b4-4b06-b31c-8ab2a1191383 None None] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:01.950623 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869a5805-79b4-4b06-b31c-8ab2a1191383 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:01.950623 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869a5805-79b4-4b06-b31c-8ab2a1191383 None None] PATH_INFO: `/` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:01.955402 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 1/2] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:23:01 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:23:01.960738 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a740d723-ecba-4e53-83c5-3c0513f7bc7f None None] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:01.961165 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a740d723-ecba-4e53-83c5-3c0513f7bc7f None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:01.961301 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a740d723-ecba-4e53-83c5-3c0513f7bc7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:02.032519 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: INFO dbcounter [None req-a740d723-ecba-4e53-83c5-3c0513f7bc7f None None] Registered counter for database keystone May 19 14:23:02.054432 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG oslo_db.sqlalchemy.engines [None req-a740d723-ecba-4e53-83c5-3c0513f7bc7f None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=143587) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 19 14:23:02.060655 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writer thread running {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 19 14:23:02.240230 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG passlib.handlers.bcrypt [None req-a740d723-ecba-4e53-83c5-3c0513f7bc7f None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=143587) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 19 14:23:02.240562 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG passlib.handlers.bcrypt [None req-a740d723-ecba-4e53-83c5-3c0513f7bc7f None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=143587) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 19 14:23:02.295613 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.auth.core [None req-a740d723-ecba-4e53-83c5-3c0513f7bc7f None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143587) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:23:02.297946 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-a740d723-ecba-4e53-83c5-3c0513f7bc7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:23:02.427346 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 2/3] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:23:01 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 468 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 19 14:23:02.450249 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869a5805-79b4-4b06-b31c-8ab2a1191383 None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:02.520784 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: INFO dbcounter [None req-869a5805-79b4-4b06-b31c-8ab2a1191383 None None] Registered counter for database keystone May 19 14:23:02.540724 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG oslo_db.sqlalchemy.engines [None req-869a5805-79b4-4b06-b31c-8ab2a1191383 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=143586) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 19 14:23:02.547692 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writer thread running {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 19 14:23:02.668157 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869a5805-79b4-4b06-b31c-8ab2a1191383 None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:02.672691 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f836560-16f3-4fd4-8543-419626d569ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9471b834f91741fbaf409539b1810552', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:02.673353 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f836560-16f3-4fd4-8543-419626d569ab admin admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:02.673638 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f836560-16f3-4fd4-8543-419626d569ab admin admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:02.673975 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f836560-16f3-4fd4-8543-419626d569ab admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:02.682160 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f836560-16f3-4fd4-8543-419626d569ab admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:02.735737 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: WARNING keystone.common.rbac_enforcer.enforcer [None req-8f836560-16f3-4fd4-8543-419626d569ab admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 19 14:23:02.736077 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f836560-16f3-4fd4-8543-419626d569ab admin admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:02.745773 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 2/4] 10.222.0.57 () {58 vars in 1324 bytes} [Thu May 19 14:23:02 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 14:23:02.751969 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a740d723-ecba-4e53-83c5-3c0513f7bc7f None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:02.767206 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a740d723-ecba-4e53-83c5-3c0513f7bc7f None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:02.771671 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78329f7f-2ff4-4420-b954-01c862ae87fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9471b834f91741fbaf409539b1810552', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:02.772718 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78329f7f-2ff4-4420-b954-01c862ae87fb admin admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:02.773002 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78329f7f-2ff4-4420-b954-01c862ae87fb admin admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:02.773278 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78329f7f-2ff4-4420-b954-01c862ae87fb admin admin] PATH_INFO: `/v3/projects/9471b834f91741fbaf409539b1810552` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:02.774835 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78329f7f-2ff4-4420-b954-01c862ae87fb admin admin] RBAC: Authorizing `identity:get_project(project_id=9471b834f91741fbaf409539b1810552)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:02.935833 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: WARNING keystone.common.rbac_enforcer.enforcer [None req-78329f7f-2ff4-4420-b954-01c862ae87fb admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 19 14:23:02.936176 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78329f7f-2ff4-4420-b954-01c862ae87fb admin admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:02.941397 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 3/5] 10.222.0.57 () {56 vars in 1207 bytes} [Thu May 19 14:23:02 2022] GET /identity/v3/projects/9471b834f91741fbaf409539b1810552 => generated 344 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:05.216623 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b7c17d-fe9e-4cc9-9284-1351b304127a None None] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:05.217186 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b7c17d-fe9e-4cc9-9284-1351b304127a None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:05.217186 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b7c17d-fe9e-4cc9-9284-1351b304127a None None] PATH_INFO: `/` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:05.222274 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 3/6] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:23:05 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:23:05.228118 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bdcf73-7efd-462f-8aa9-2a5393e15f0f None None] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:05.228540 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bdcf73-7efd-462f-8aa9-2a5393e15f0f None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:05.228687 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bdcf73-7efd-462f-8aa9-2a5393e15f0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:05.248457 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.auth.core [None req-15bdcf73-7efd-462f-8aa9-2a5393e15f0f None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143587) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:23:05.250138 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-15bdcf73-7efd-462f-8aa9-2a5393e15f0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:23:05.258032 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 4/7] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:23:05 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 19 14:23:05.279860 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b7c17d-fe9e-4cc9-9284-1351b304127a None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:05.292492 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b7c17d-fe9e-4cc9-9284-1351b304127a None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:05.297473 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b30a56b-702f-4eb6-b0c3-2cfdd3dfb522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9471b834f91741fbaf409539b1810552', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:05.298189 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b30a56b-702f-4eb6-b0c3-2cfdd3dfb522 admin admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:05.298475 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b30a56b-702f-4eb6-b0c3-2cfdd3dfb522 admin admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:05.298754 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b30a56b-702f-4eb6-b0c3-2cfdd3dfb522 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:05.306456 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b30a56b-702f-4eb6-b0c3-2cfdd3dfb522 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:05.308238 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b30a56b-702f-4eb6-b0c3-2cfdd3dfb522 admin admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:05.317292 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 4/8] 10.222.0.57 () {58 vars in 1324 bytes} [Thu May 19 14:23:05 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 14:23:05.323008 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bdcf73-7efd-462f-8aa9-2a5393e15f0f None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:05.332109 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bdcf73-7efd-462f-8aa9-2a5393e15f0f None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:05.336495 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6ec808-6bab-4272-9bc5-d39abdc533b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9471b834f91741fbaf409539b1810552', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:05.337366 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6ec808-6bab-4272-9bc5-d39abdc533b7 admin admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:05.337650 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6ec808-6bab-4272-9bc5-d39abdc533b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:05.337962 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6ec808-6bab-4272-9bc5-d39abdc533b7 admin admin] PATH_INFO: `/v3/users/4f2f0290e1b04ce48a90ce03cb93d060` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:05.339348 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6ec808-6bab-4272-9bc5-d39abdc533b7 admin admin] RBAC: Authorizing `identity:get_user(user_id=4f2f0290e1b04ce48a90ce03cb93d060)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:05.341179 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6ec808-6bab-4272-9bc5-d39abdc533b7 admin admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:05.342777 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 5/9] 10.222.0.57 () {56 vars in 1198 bytes} [Thu May 19 14:23:05 2022] GET /identity/v3/users/4f2f0290e1b04ce48a90ce03cb93d060 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:07.663684 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613aa511-0cc2-4e42-bede-7ad1e1e2be09 None None] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:07.664135 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613aa511-0cc2-4e42-bede-7ad1e1e2be09 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:07.664240 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613aa511-0cc2-4e42-bede-7ad1e1e2be09 None None] PATH_INFO: `/` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:07.665602 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 5/10] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:23:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:23:07.670878 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ba8627-794c-41de-864d-a9aaa73c178d None None] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:07.671305 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ba8627-794c-41de-864d-a9aaa73c178d None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:07.671444 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ba8627-794c-41de-864d-a9aaa73c178d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:07.682094 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93482bb7-67e5-4963-acde-b31c5ee7fa25 None None] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:07.682383 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93482bb7-67e5-4963-acde-b31c5ee7fa25 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:07.682659 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93482bb7-67e5-4963-acde-b31c5ee7fa25 None None] PATH_INFO: `/` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:07.683976 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 6/11] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:23:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:23:07.689128 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.auth.core [None req-b2ba8627-794c-41de-864d-a9aaa73c178d None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143587) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:23:07.689334 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8085825f-28f0-4676-a138-26cb0debcd6c None None] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:07.689622 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8085825f-28f0-4676-a138-26cb0debcd6c None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:07.689997 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8085825f-28f0-4676-a138-26cb0debcd6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:07.690357 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-b2ba8627-794c-41de-864d-a9aaa73c178d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:23:07.729451 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG passlib.handlers.bcrypt [None req-8085825f-28f0-4676-a138-26cb0debcd6c None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=143586) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 19 14:23:07.729958 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG passlib.handlers.bcrypt [None req-8085825f-28f0-4676-a138-26cb0debcd6c None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=143586) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 19 14:23:07.769850 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 6/12] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:23:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:23:07.776901 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.auth.core [None req-8085825f-28f0-4676-a138-26cb0debcd6c None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143586) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:23:07.778511 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.fernet_utils [None req-8085825f-28f0-4676-a138-26cb0debcd6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143586) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:23:07.793872 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ba8627-794c-41de-864d-a9aaa73c178d None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:07.818424 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:07.883240 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 7/13] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:23:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:23:07.901903 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ba8627-794c-41de-864d-a9aaa73c178d None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:07.905876 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8085825f-28f0-4676-a138-26cb0debcd6c None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:07.939578 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf530aad-a80f-42ec-982e-05e5ee4e7e4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:07.940316 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf530aad-a80f-42ec-982e-05e5ee4e7e4c None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:07.940604 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf530aad-a80f-42ec-982e-05e5ee4e7e4c None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:07.940887 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf530aad-a80f-42ec-982e-05e5ee4e7e4c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:07.954464 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf530aad-a80f-42ec-982e-05e5ee4e7e4c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:23:07.966197 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-cf530aad-a80f-42ec-982e-05e5ee4e7e4c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:23:07.966197 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:23:07.966197 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:23:07.966197 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:23:07.966197 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:23:07.966197 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:23:07.966197 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:23:07.966197 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:23:07.966197 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:23:07.966197 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:23:07.966197 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:23:07.966197 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:23:07.966197 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:23:07.966197 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:23:07.966197 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:23:07.966197 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:23:07.967573 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:23:07.967573 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:23:07.967573 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:23:07.967573 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:23:07.967573 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:23:07.967573 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:23:07.967573 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:23:07.967573 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:23:07.967573 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:23:07.967573 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:23:07.967573 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:23:07.967573 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:23:07.967573 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:23:07.967573 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:23:07.967573 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:23:07.967573 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:23:07.969452 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:23:07.969452 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:23:07.969452 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:23:07.969452 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:23:07.969452 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:23:07.969452 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:23:07.969452 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:23:07.969452 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:23:07.969452 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:23:07.969452 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:23:07.969452 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:23:07.969452 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:23:07.969452 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 7/14] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:23:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:23:07.974817 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf530aad-a80f-42ec-982e-05e5ee4e7e4c None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:08.061577 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:08.082519 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf530aad-a80f-42ec-982e-05e5ee4e7e4c None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:08.116787 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e4250d-ee67-49e2-9ae1-ad26e43a6626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:08.117452 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e4250d-ee67-49e2-9ae1-ad26e43a6626 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:08.117731 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e4250d-ee67-49e2-9ae1-ad26e43a6626 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:08.118038 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e4250d-ee67-49e2-9ae1-ad26e43a6626 None admin] PATH_INFO: `/v3/roles` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:08.118791 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e4250d-ee67-49e2-9ae1-ad26e43a6626 None admin] RBAC: Adding query filter params (name=admin) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:23:08.118921 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8085825f-28f0-4676-a138-26cb0debcd6c None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:08.119098 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e4250d-ee67-49e2-9ae1-ad26e43a6626 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:08.120510 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e4250d-ee67-49e2-9ae1-ad26e43a6626 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:08.121260 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:08.133441 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 8/15] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:23:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:23:08.139160 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e4250d-ee67-49e2-9ae1-ad26e43a6626 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:08.158339 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d93794-8088-4563-a166-29df0974f62b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:08.159250 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d93794-8088-4563-a166-29df0974f62b None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:08.159540 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d93794-8088-4563-a166-29df0974f62b None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:08.159828 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d93794-8088-4563-a166-29df0974f62b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:08.229452 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d93794-8088-4563-a166-29df0974f62b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:08.230943 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d93794-8088-4563-a166-29df0974f62b None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:08.240557 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e4250d-ee67-49e2-9ae1-ad26e43a6626 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:08.275299 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3150d27-c1b8-4c00-9c87-995abd551b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:08.276172 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3150d27-c1b8-4c00-9c87-995abd551b70 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:08.276455 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3150d27-c1b8-4c00-9c87-995abd551b70 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:08.276727 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3150d27-c1b8-4c00-9c87-995abd551b70 None admin] PATH_INFO: `/v3/users/4f2f0290e1b04ce48a90ce03cb93d060` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:08.277949 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3150d27-c1b8-4c00-9c87-995abd551b70 None admin] RBAC: Authorizing `identity:get_user(user_id=4f2f0290e1b04ce48a90ce03cb93d060)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:08.279415 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3150d27-c1b8-4c00-9c87-995abd551b70 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:08.281084 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 9/16] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:23:08 2022] GET /identity/v3/users/4f2f0290e1b04ce48a90ce03cb93d060 => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:23:08.286755 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3150d27-c1b8-4c00-9c87-995abd551b70 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:08.301066 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:08.336610 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 8/17] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:23:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 14:23:08.342233 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d93794-8088-4563-a166-29df0974f62b None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:08.345662 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:08.398503 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3150d27-c1b8-4c00-9c87-995abd551b70 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:08.431922 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7cf208-ce65-43ae-ac36-1cb2b2319938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:08.432692 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7cf208-ce65-43ae-ac36-1cb2b2319938 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:08.432976 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7cf208-ce65-43ae-ac36-1cb2b2319938 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:08.433252 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7cf208-ce65-43ae-ac36-1cb2b2319938 None admin] PATH_INFO: `/v3/domains/default` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:08.434470 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7cf208-ce65-43ae-ac36-1cb2b2319938 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:08.435953 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7cf208-ce65-43ae-ac36-1cb2b2319938 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:08.438658 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 10/18] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:23:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:08.444313 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7cf208-ce65-43ae-ac36-1cb2b2319938 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:08.444979 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d93794-8088-4563-a166-29df0974f62b None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:08.479098 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014ccb24-210c-4352-b7b8-2a37b3ace846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:08.479930 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014ccb24-210c-4352-b7b8-2a37b3ace846 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:08.480173 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014ccb24-210c-4352-b7b8-2a37b3ace846 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:08.480448 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014ccb24-210c-4352-b7b8-2a37b3ace846 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:08.499441 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014ccb24-210c-4352-b7b8-2a37b3ace846 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:08.501710 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014ccb24-210c-4352-b7b8-2a37b3ace846 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:08.523132 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-014ccb24-210c-4352-b7b8-2a37b3ace846 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:23:08.523132 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:23:08.523132 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:23:08.523132 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:23:08.523132 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:23:08.523132 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:23:08.523132 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:23:08.523132 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:23:08.523132 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:23:08.523132 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:23:08.523132 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:23:08.523132 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:23:08.523132 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:23:08.523132 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:23:08.523132 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:23:08.524300 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:23:08.524300 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:23:08.524300 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:23:08.524300 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:23:08.524300 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:23:08.524300 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:23:08.524300 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:23:08.524300 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:23:08.524300 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:23:08.524300 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:23:08.524300 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:23:08.524300 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:23:08.524300 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:23:08.524300 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:23:08.524300 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:23:08.524300 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:23:08.524300 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:23:08.525463 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:23:08.525463 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:23:08.525463 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:23:08.525463 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:23:08.525463 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 9/19] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:23:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:23:08.525885 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:08.532606 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014ccb24-210c-4352-b7b8-2a37b3ace846 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:08.546122 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7cf208-ce65-43ae-ac36-1cb2b2319938 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:08.582345 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ab35cd-ae43-410a-b61f-48b11a55280e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:08.583003 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ab35cd-ae43-410a-b61f-48b11a55280e None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:08.583283 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ab35cd-ae43-410a-b61f-48b11a55280e None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:08.583559 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ab35cd-ae43-410a-b61f-48b11a55280e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:08.584393 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ab35cd-ae43-410a-b61f-48b11a55280e None admin] RBAC: Adding query filter params (role.id=a3b793754e7b40b0bb21afe74eec3c18, scope.domain.id=default, user.id=4f2f0290e1b04ce48a90ce03cb93d060) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:23:08.584703 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ab35cd-ae43-410a-b61f-48b11a55280e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:08.586173 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ab35cd-ae43-410a-b61f-48b11a55280e None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:08.590736 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:08.593075 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 11/20] 10.222.0.57 () {56 vars in 1322 bytes} [Thu May 19 14:23:08 2022] GET /identity/v3/role_assignments?user.id=4f2f0290e1b04ce48a90ce03cb93d060&scope.domain.id=default&role.id=a3b793754e7b40b0bb21afe74eec3c18 => generated 233 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:23:08.640182 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014ccb24-210c-4352-b7b8-2a37b3ace846 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:08.682957 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00913fc9-d6b6-4bb5-8822-f1513a763ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:08.683669 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00913fc9-d6b6-4bb5-8822-f1513a763ee4 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:08.683911 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00913fc9-d6b6-4bb5-8822-f1513a763ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:08.684189 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00913fc9-d6b6-4bb5-8822-f1513a763ee4 None admin] PATH_INFO: `/v3/domains` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:08.685131 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00913fc9-d6b6-4bb5-8822-f1513a763ee4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:23:08.685450 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00913fc9-d6b6-4bb5-8822-f1513a763ee4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:08.686913 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00913fc9-d6b6-4bb5-8822-f1513a763ee4 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:08.707163 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 10/21] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:23:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:23:11.234695 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bd2bef-e988-4108-bf03-bd4cab06e922 None None] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:11.235966 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bd2bef-e988-4108-bf03-bd4cab06e922 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:11.236469 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bd2bef-e988-4108-bf03-bd4cab06e922 None None] PATH_INFO: `/` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:11.238105 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 12/22] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:23:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:23:11.243673 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894499d1-2ca1-4e4a-a0aa-24496ca4c5bd None None] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:11.244254 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894499d1-2ca1-4e4a-a0aa-24496ca4c5bd None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:11.244544 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894499d1-2ca1-4e4a-a0aa-24496ca4c5bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:11.267279 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.auth.core [None req-894499d1-2ca1-4e4a-a0aa-24496ca4c5bd None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143586) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:23:11.268629 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.fernet_utils [None req-894499d1-2ca1-4e4a-a0aa-24496ca4c5bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143586) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:23:11.356934 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 11/23] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:23:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:23:11.388501 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bd2bef-e988-4108-bf03-bd4cab06e922 None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:11.509590 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b9541e-6ceb-40d0-92d2-14bd337745ad None None] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:11.509787 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b9541e-6ceb-40d0-92d2-14bd337745ad None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:11.510107 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b9541e-6ceb-40d0-92d2-14bd337745ad None None] PATH_INFO: `/` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:11.511488 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 12/24] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:23:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:23:11.517352 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b92f443-8662-4271-b2c3-ba4cd08c6cd4 None None] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:11.517962 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b92f443-8662-4271-b2c3-ba4cd08c6cd4 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:11.518576 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b92f443-8662-4271-b2c3-ba4cd08c6cd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:11.549575 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.auth.core [None req-0b92f443-8662-4271-b2c3-ba4cd08c6cd4 None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143586) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:23:11.550549 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bd2bef-e988-4108-bf03-bd4cab06e922 None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:11.551497 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.fernet_utils [None req-0b92f443-8662-4271-b2c3-ba4cd08c6cd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143586) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:23:11.624376 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:11.638557 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2610d1b8-2f06-4c06-bee4-18e591d9201b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:11.639660 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2610d1b8-2f06-4c06-bee4-18e591d9201b None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:11.640153 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2610d1b8-2f06-4c06-bee4-18e591d9201b None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:11.640590 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2610d1b8-2f06-4c06-bee4-18e591d9201b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:11.679074 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2610d1b8-2f06-4c06-bee4-18e591d9201b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:23:11.695620 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-2610d1b8-2f06-4c06-bee4-18e591d9201b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:23:11.695620 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:23:11.695620 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:23:11.695620 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:23:11.695620 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:23:11.695620 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:23:11.695620 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:23:11.695620 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:23:11.695620 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:23:11.695620 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:23:11.695620 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:23:11.695620 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:23:11.695620 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:23:11.695620 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:23:11.695620 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:23:11.695620 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:23:11.696700 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:23:11.696700 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:23:11.696700 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:23:11.696700 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:23:11.696700 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:23:11.696700 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:23:11.696700 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:23:11.696700 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:23:11.696700 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:23:11.696700 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:23:11.696700 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:23:11.696700 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:23:11.696700 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:23:11.696700 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:23:11.696700 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:23:11.696700 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:23:11.697664 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:23:11.697664 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:23:11.697664 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:23:11.697664 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:23:11.697664 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:23:11.697664 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:23:11.697664 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:23:11.697664 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:23:11.697664 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:23:11.697664 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:23:11.697664 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:23:11.697664 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:23:11.697664 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 13/25] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:23:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:23:11.703309 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2610d1b8-2f06-4c06-bee4-18e591d9201b None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:11.714581 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 13/26] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:23:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:23:11.717016 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:11.760680 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b92f443-8662-4271-b2c3-ba4cd08c6cd4 None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:11.972011 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b92f443-8662-4271-b2c3-ba4cd08c6cd4 None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:11.979421 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2610d1b8-2f06-4c06-bee4-18e591d9201b None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:12.024584 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac955b3-3481-4d6f-9c47-ccda78046458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:12.025274 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac955b3-3481-4d6f-9c47-ccda78046458 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:12.025570 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac955b3-3481-4d6f-9c47-ccda78046458 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:12.025873 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac955b3-3481-4d6f-9c47-ccda78046458 None admin] PATH_INFO: `/v3/roles` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:12.026656 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac955b3-3481-4d6f-9c47-ccda78046458 None admin] RBAC: Adding query filter params (name=admin) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:23:12.027048 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac955b3-3481-4d6f-9c47-ccda78046458 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:12.028804 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac955b3-3481-4d6f-9c47-ccda78046458 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:12.033666 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b266e39b-4496-410f-920a-6cb14af3eee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:12.034637 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b266e39b-4496-410f-920a-6cb14af3eee9 None admin] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:12.035135 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b266e39b-4496-410f-920a-6cb14af3eee9 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:12.036198 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b266e39b-4496-410f-920a-6cb14af3eee9 None admin] PATH_INFO: `/v3/roles` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:12.037311 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b266e39b-4496-410f-920a-6cb14af3eee9 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:12.039188 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b266e39b-4496-410f-920a-6cb14af3eee9 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:12.039554 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 14/27] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:23:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:12.041068 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c7c336-4c24-4ac2-bee1-bfd7ba29c15b None None] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:12.041462 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c7c336-4c24-4ac2-bee1-bfd7ba29c15b None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:12.041862 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c7c336-4c24-4ac2-bee1-bfd7ba29c15b None None] PATH_INFO: `/` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:12.043347 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 15/28] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:23:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:23:12.044532 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8651d8d-bbed-499b-9363-464e0d61742f None None] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:12.045171 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8651d8d-bbed-499b-9363-464e0d61742f None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:12.045644 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8651d8d-bbed-499b-9363-464e0d61742f None None] PATH_INFO: `/` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:12.047736 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 16/29] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:23:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:23:12.053669 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8651d8d-bbed-499b-9363-464e0d61742f None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:12.130567 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:12.172651 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8651d8d-bbed-499b-9363-464e0d61742f None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:12.209503 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 14/30] 10.222.0.57 () {60 vars in 1131 bytes} [Thu May 19 14:23:11 2022] POST /identity/v3/roles => generated 219 bytes in 450 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 14:23:12.210101 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e984c959-fd8e-4b67-80bb-2c1d64f6d104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:12.211273 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e984c959-fd8e-4b67-80bb-2c1d64f6d104 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:12.211775 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e984c959-fd8e-4b67-80bb-2c1d64f6d104 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:12.212377 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e984c959-fd8e-4b67-80bb-2c1d64f6d104 None admin] PATH_INFO: `/v3/users/4f2f0290e1b04ce48a90ce03cb93d060` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:12.214112 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e984c959-fd8e-4b67-80bb-2c1d64f6d104 None admin] RBAC: Authorizing `identity:get_user(user_id=4f2f0290e1b04ce48a90ce03cb93d060)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:12.215263 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4207b771-e483-4383-8b46-d16464531998 None None] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:12.215566 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4207b771-e483-4383-8b46-d16464531998 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:12.215860 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4207b771-e483-4383-8b46-d16464531998 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:12.216586 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e984c959-fd8e-4b67-80bb-2c1d64f6d104 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:12.219626 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 17/31] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:23:12 2022] GET /identity/v3/users/4f2f0290e1b04ce48a90ce03cb93d060 => generated 247 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:12.221496 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9529777d-8b89-4e8c-9c54-3cb13ae7f96e None None] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:12.222079 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9529777d-8b89-4e8c-9c54-3cb13ae7f96e None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:12.222583 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9529777d-8b89-4e8c-9c54-3cb13ae7f96e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:12.246325 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.auth.core [None req-4207b771-e483-4383-8b46-d16464531998 None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143586) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:23:12.248694 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.fernet_utils [None req-4207b771-e483-4383-8b46-d16464531998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143586) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:23:12.252292 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.auth.core [None req-9529777d-8b89-4e8c-9c54-3cb13ae7f96e None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143587) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:23:12.254437 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-9529777d-8b89-4e8c-9c54-3cb13ae7f96e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:23:12.297661 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:12.388944 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 15/32] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:23:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:23:12.389625 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4207b771-e483-4383-8b46-d16464531998 None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:12.419780 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 18/33] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:23:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:23:12.421247 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9529777d-8b89-4e8c-9c54-3cb13ae7f96e None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:12.536862 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:12.564765 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4207b771-e483-4383-8b46-d16464531998 None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:12.586140 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9529777d-8b89-4e8c-9c54-3cb13ae7f96e None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:12.617833 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6e7dab-7948-48c0-b0c4-d12dc31ed7de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:12.619094 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6e7dab-7948-48c0-b0c4-d12dc31ed7de None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:12.619617 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6e7dab-7948-48c0-b0c4-d12dc31ed7de None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:12.620231 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6e7dab-7948-48c0-b0c4-d12dc31ed7de None admin] PATH_INFO: `/v3/domains/default` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:12.622270 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6e7dab-7948-48c0-b0c4-d12dc31ed7de None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:12.624646 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6e7dab-7948-48c0-b0c4-d12dc31ed7de None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:12.626130 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaea1c5-e11d-421e-a127-d206913a27c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:12.627051 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaea1c5-e11d-421e-a127-d206913a27c7 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:12.627262 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 16/34] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:23:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:12.628234 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6e7dab-7948-48c0-b0c4-d12dc31ed7de None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:12.631157 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaea1c5-e11d-421e-a127-d206913a27c7 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:12.632465 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaea1c5-e11d-421e-a127-d206913a27c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:12.645775 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaea1c5-e11d-421e-a127-d206913a27c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:12.647324 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaea1c5-e11d-421e-a127-d206913a27c7 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:12.674292 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-ecaea1c5-e11d-421e-a127-d206913a27c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:23:12.674292 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:23:12.674292 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:23:12.674292 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:23:12.674292 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:23:12.674292 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:23:12.674292 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:23:12.674292 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:23:12.674292 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:23:12.674292 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:23:12.674292 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:23:12.674292 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:23:12.674292 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:23:12.674292 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:23:12.674292 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:23:12.675343 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:23:12.675343 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:23:12.675343 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:23:12.675343 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:23:12.675343 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:23:12.675343 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:23:12.675343 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:23:12.675343 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:23:12.675343 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:23:12.675343 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:23:12.675343 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:23:12.675343 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:23:12.675343 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:23:12.675343 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:23:12.675343 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:23:12.675343 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:23:12.675343 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:23:12.676951 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:23:12.676951 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:23:12.676951 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:23:12.676951 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:23:12.677663 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:12.680691 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 19/35] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:23:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:23:12.681416 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaea1c5-e11d-421e-a127-d206913a27c7 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:12.771377 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6e7dab-7948-48c0-b0c4-d12dc31ed7de None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:12.791505 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaea1c5-e11d-421e-a127-d206913a27c7 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:12.806939 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850ff664-a9b3-4c82-a596-1aba5fc0aca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:12.807623 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850ff664-a9b3-4c82-a596-1aba5fc0aca3 None admin] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:12.807909 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850ff664-a9b3-4c82-a596-1aba5fc0aca3 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:12.808188 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850ff664-a9b3-4c82-a596-1aba5fc0aca3 None admin] PATH_INFO: `/v3/roles` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:12.808945 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850ff664-a9b3-4c82-a596-1aba5fc0aca3 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:12.810466 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850ff664-a9b3-4c82-a596-1aba5fc0aca3 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:12.811233 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:12.825693 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 17/36] 10.222.0.57 () {60 vars in 1131 bytes} [Thu May 19 14:23:12 2022] POST /identity/v3/roles => generated 225 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:23:12.826643 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850ff664-a9b3-4c82-a596-1aba5fc0aca3 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:12.836696 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e20ef0c-ed95-4980-858c-e676900af357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:12.838009 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e20ef0c-ed95-4980-858c-e676900af357 None admin] REQUEST_METHOD: `PUT` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:12.838547 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e20ef0c-ed95-4980-858c-e676900af357 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:12.839015 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e20ef0c-ed95-4980-858c-e676900af357 None admin] PATH_INFO: `/v3/domains/default/users/4f2f0290e1b04ce48a90ce03cb93d060/roles/a3b793754e7b40b0bb21afe74eec3c18` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:12.841526 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e20ef0c-ed95-4980-858c-e676900af357 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=4f2f0290e1b04ce48a90ce03cb93d060, role_id=a3b793754e7b40b0bb21afe74eec3c18)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:12.843453 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e20ef0c-ed95-4980-858c-e676900af357 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:12.953998 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:12.961033 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850ff664-a9b3-4c82-a596-1aba5fc0aca3 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:12.990222 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 20/37] 10.222.0.57 () {58 vars in 1361 bytes} [Thu May 19 14:23:12 2022] PUT /identity/v3/domains/default/users/4f2f0290e1b04ce48a90ce03cb93d060/roles/a3b793754e7b40b0bb21afe74eec3c18 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:23:12.992723 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f686283-34cd-41e4-9a3c-c2a4e6f2e905 None None] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:12.993253 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f686283-34cd-41e4-9a3c-c2a4e6f2e905 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:12.993738 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f686283-34cd-41e4-9a3c-c2a4e6f2e905 None None] PATH_INFO: `/` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:13.005334 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 21/38] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:23:12 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:23:13.007143 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826d2354-afb2-448c-9efd-8690cdfb01c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:13.007459 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca769d64-50ac-4530-b537-bbf49c3de2ab None None] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:13.007838 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826d2354-afb2-448c-9efd-8690cdfb01c9 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:13.008131 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826d2354-afb2-448c-9efd-8690cdfb01c9 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:13.008329 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca769d64-50ac-4530-b537-bbf49c3de2ab None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:13.008438 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826d2354-afb2-448c-9efd-8690cdfb01c9 None admin] PATH_INFO: `/v3/domains` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:13.008914 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca769d64-50ac-4530-b537-bbf49c3de2ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:13.009349 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826d2354-afb2-448c-9efd-8690cdfb01c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:23:13.009673 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826d2354-afb2-448c-9efd-8690cdfb01c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:13.017920 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826d2354-afb2-448c-9efd-8690cdfb01c9 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:13.026399 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 18/39] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:23:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:13.032293 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826d2354-afb2-448c-9efd-8690cdfb01c9 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:13.049779 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.auth.core [None req-ca769d64-50ac-4530-b537-bbf49c3de2ab None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143587) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:23:13.051479 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-ca769d64-50ac-4530-b537-bbf49c3de2ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:23:13.161712 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 22/40] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:23:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:23:13.164146 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4e7920-8fab-4add-85c1-80dba2a86919 None None] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:13.164446 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4e7920-8fab-4add-85c1-80dba2a86919 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:13.164741 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4e7920-8fab-4add-85c1-80dba2a86919 None None] PATH_INFO: `/` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:13.166586 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 23/41] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:23:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:23:13.173061 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b429311e-7f8e-47ed-992d-7bef49d9a7c4 None None] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:13.173380 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b429311e-7f8e-47ed-992d-7bef49d9a7c4 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:13.173642 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b429311e-7f8e-47ed-992d-7bef49d9a7c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:13.179166 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826d2354-afb2-448c-9efd-8690cdfb01c9 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:13.198413 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.auth.core [None req-b429311e-7f8e-47ed-992d-7bef49d9a7c4 None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143587) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:23:13.199659 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-b429311e-7f8e-47ed-992d-7bef49d9a7c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:23:13.227128 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b335b14c-5049-4c71-9476-c44ca89328a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:13.227805 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b335b14c-5049-4c71-9476-c44ca89328a7 None admin] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:13.228208 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b335b14c-5049-4c71-9476-c44ca89328a7 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:13.228488 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b335b14c-5049-4c71-9476-c44ca89328a7 None admin] PATH_INFO: `/v3/projects` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:13.229202 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b335b14c-5049-4c71-9476-c44ca89328a7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:13.230864 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b335b14c-5049-4c71-9476-c44ca89328a7 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:13.263319 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 19/42] 10.222.0.57 () {60 vars in 1141 bytes} [Thu May 19 14:23:13 2022] POST /identity/v3/projects => generated 301 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 14:23:13.264258 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b335b14c-5049-4c71-9476-c44ca89328a7 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:13.275206 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:13.292725 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 24/43] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:23:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:23:13.323453 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b429311e-7f8e-47ed-992d-7bef49d9a7c4 None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:13.375241 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:13.387268 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b335b14c-5049-4c71-9476-c44ca89328a7 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:13.430755 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653f526d-065e-4604-b66f-53e694a7425e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:13.431984 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653f526d-065e-4604-b66f-53e694a7425e None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:13.432487 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653f526d-065e-4604-b66f-53e694a7425e None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:13.432996 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653f526d-065e-4604-b66f-53e694a7425e None admin] PATH_INFO: `/v3/domains/default` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:13.434606 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653f526d-065e-4604-b66f-53e694a7425e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:13.436439 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653f526d-065e-4604-b66f-53e694a7425e None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:13.439636 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 20/44] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:23:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:13.446836 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653f526d-065e-4604-b66f-53e694a7425e None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:13.468584 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b429311e-7f8e-47ed-992d-7bef49d9a7c4 None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:13.512728 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f7f634-b9da-410d-8156-5c9014dddfd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:13.513406 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f7f634-b9da-410d-8156-5c9014dddfd4 None admin] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:13.513692 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f7f634-b9da-410d-8156-5c9014dddfd4 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:13.514000 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f7f634-b9da-410d-8156-5c9014dddfd4 None admin] PATH_INFO: `/v3/roles` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:13.514754 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f7f634-b9da-410d-8156-5c9014dddfd4 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:13.517259 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f7f634-b9da-410d-8156-5c9014dddfd4 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:13.540928 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 25/45] 10.222.0.57 () {60 vars in 1131 bytes} [Thu May 19 14:23:13 2022] POST /identity/v3/roles => generated 223 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 14:23:13.542531 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:13.584967 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653f526d-065e-4604-b66f-53e694a7425e None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:13.626541 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1154b30-ab70-4b83-920c-00c9889491ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:13.627238 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1154b30-ab70-4b83-920c-00c9889491ae None admin] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:13.627527 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1154b30-ab70-4b83-920c-00c9889491ae None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:13.627818 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1154b30-ab70-4b83-920c-00c9889491ae None admin] PATH_INFO: `/v3/projects` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:13.628548 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1154b30-ab70-4b83-920c-00c9889491ae None admin] RBAC: Authorizing `identity:create_project()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:13.630084 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1154b30-ab70-4b83-920c-00c9889491ae None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:13.669876 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 21/46] 10.222.0.57 () {60 vars in 1141 bytes} [Thu May 19 14:23:13 2022] POST /identity/v3/projects => generated 312 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 14:23:15.323997 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bb9d55-2e04-424d-9812-369aaa783dea None None] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:15.324568 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bb9d55-2e04-424d-9812-369aaa783dea None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:15.324568 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bb9d55-2e04-424d-9812-369aaa783dea None None] PATH_INFO: `/` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:15.325943 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 26/47] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:23:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:23:15.332104 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15622e68-ed80-486e-b08b-4d1b9350189f None None] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:15.332528 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15622e68-ed80-486e-b08b-4d1b9350189f None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:15.332667 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15622e68-ed80-486e-b08b-4d1b9350189f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:15.349808 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.auth.core [None req-15622e68-ed80-486e-b08b-4d1b9350189f None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143586) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:23:15.351264 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.fernet_utils [None req-15622e68-ed80-486e-b08b-4d1b9350189f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143586) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:23:15.416148 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 22/48] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:23:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:23:15.439467 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bb9d55-2e04-424d-9812-369aaa783dea None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:15.536340 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:15.544025 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bb9d55-2e04-424d-9812-369aaa783dea None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:15.578893 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec800db-a9d3-421b-be90-ecf8071b2d23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:15.579607 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec800db-a9d3-421b-be90-ecf8071b2d23 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:15.579891 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec800db-a9d3-421b-be90-ecf8071b2d23 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:15.580167 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec800db-a9d3-421b-be90-ecf8071b2d23 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:15.590050 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec800db-a9d3-421b-be90-ecf8071b2d23 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:23:15.598843 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-8ec800db-a9d3-421b-be90-ecf8071b2d23 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:23:15.598843 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:23:15.598843 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:23:15.598843 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:23:15.598843 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:23:15.598843 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:23:15.598843 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:23:15.598843 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:23:15.598843 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:23:15.598843 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:23:15.598843 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:23:15.598843 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:23:15.598843 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:23:15.598843 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:23:15.598843 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:23:15.598843 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:23:15.599893 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:23:15.599893 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:23:15.599893 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:23:15.599893 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:23:15.599893 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:23:15.599893 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:23:15.599893 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:23:15.599893 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:23:15.599893 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:23:15.599893 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:23:15.599893 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:23:15.599893 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:23:15.599893 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:23:15.599893 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:23:15.599893 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:23:15.599893 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:23:15.601118 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:23:15.601118 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:23:15.601118 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:23:15.601118 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:23:15.601118 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:23:15.601118 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:23:15.601118 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:23:15.601118 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:23:15.601118 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:23:15.601118 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:23:15.601118 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:23:15.601118 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:23:15.601118 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 27/49] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:23:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:23:15.606138 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15622e68-ed80-486e-b08b-4d1b9350189f None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:15.665935 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:15.710617 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15622e68-ed80-486e-b08b-4d1b9350189f None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:15.747209 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73213d6d-58b0-4d40-abbb-9f33ccbd8317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:15.747895 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73213d6d-58b0-4d40-abbb-9f33ccbd8317 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:15.748179 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73213d6d-58b0-4d40-abbb-9f33ccbd8317 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:15.748456 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73213d6d-58b0-4d40-abbb-9f33ccbd8317 None admin] PATH_INFO: `/v3/roles` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:15.749223 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73213d6d-58b0-4d40-abbb-9f33ccbd8317 None admin] RBAC: Adding query filter params (name=admin) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:23:15.750191 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73213d6d-58b0-4d40-abbb-9f33ccbd8317 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:15.751627 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73213d6d-58b0-4d40-abbb-9f33ccbd8317 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:15.763663 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 23/50] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:23:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:15.769928 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec800db-a9d3-421b-be90-ecf8071b2d23 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:15.869640 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec800db-a9d3-421b-be90-ecf8071b2d23 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:15.904125 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0365003b-530c-4d14-a5b8-e9e53038d86f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:15.905005 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0365003b-530c-4d14-a5b8-e9e53038d86f None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:15.905295 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0365003b-530c-4d14-a5b8-e9e53038d86f None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:15.905620 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0365003b-530c-4d14-a5b8-e9e53038d86f None admin] PATH_INFO: `/v3/users/4f2f0290e1b04ce48a90ce03cb93d060` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:15.906879 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0365003b-530c-4d14-a5b8-e9e53038d86f None admin] RBAC: Authorizing `identity:get_user(user_id=4f2f0290e1b04ce48a90ce03cb93d060)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:15.908343 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0365003b-530c-4d14-a5b8-e9e53038d86f None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:15.910034 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 28/51] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:23:15 2022] GET /identity/v3/users/4f2f0290e1b04ce48a90ce03cb93d060 => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:15.913892 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4492b26d-0163-4823-965d-5b693092efa4 None None] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:15.914213 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4492b26d-0163-4823-965d-5b693092efa4 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:15.914481 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4492b26d-0163-4823-965d-5b693092efa4 None None] PATH_INFO: `/` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:15.915217 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0365003b-530c-4d14-a5b8-e9e53038d86f None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:15.915995 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 24/52] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:23:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:23:15.920663 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d453d74-bad5-492d-a336-abdb649d79d1 None None] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:15.920953 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d453d74-bad5-492d-a336-abdb649d79d1 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:15.921279 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d453d74-bad5-492d-a336-abdb649d79d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:15.939983 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.auth.core [None req-1d453d74-bad5-492d-a336-abdb649d79d1 None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143586) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:23:15.941169 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.fernet_utils [None req-1d453d74-bad5-492d-a336-abdb649d79d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143586) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:23:15.942544 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:16.013394 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 25/53] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:23:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:23:16.017688 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0365003b-530c-4d14-a5b8-e9e53038d86f None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:16.036907 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d453d74-bad5-492d-a336-abdb649d79d1 None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:16.051353 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897a2ffe-cba0-4b3a-8ff7-abd2fa6030ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:16.052155 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897a2ffe-cba0-4b3a-8ff7-abd2fa6030ed None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:16.052443 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897a2ffe-cba0-4b3a-8ff7-abd2fa6030ed None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:16.052726 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897a2ffe-cba0-4b3a-8ff7-abd2fa6030ed None admin] PATH_INFO: `/v3/domains/default` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:16.053943 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897a2ffe-cba0-4b3a-8ff7-abd2fa6030ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:16.055414 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897a2ffe-cba0-4b3a-8ff7-abd2fa6030ed None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:16.057058 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 29/54] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:23:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:23:16.063545 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897a2ffe-cba0-4b3a-8ff7-abd2fa6030ed None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:16.085728 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:16.137281 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d453d74-bad5-492d-a336-abdb649d79d1 None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:16.156882 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:16.165937 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897a2ffe-cba0-4b3a-8ff7-abd2fa6030ed None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:16.175033 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5480483e-784b-458c-a0b5-9fa6ed70e5fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:16.175843 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5480483e-784b-458c-a0b5-9fa6ed70e5fa None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:16.176133 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5480483e-784b-458c-a0b5-9fa6ed70e5fa None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:16.176415 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5480483e-784b-458c-a0b5-9fa6ed70e5fa None admin] PATH_INFO: `/v3/domains/default` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:16.177599 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5480483e-784b-458c-a0b5-9fa6ed70e5fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:16.179102 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5480483e-784b-458c-a0b5-9fa6ed70e5fa None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:16.180913 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 26/55] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:23:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:23:16.186201 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5480483e-784b-458c-a0b5-9fa6ed70e5fa None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:16.200603 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c24408-b103-466f-b475-9d8dc806f7be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:16.201254 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c24408-b103-466f-b475-9d8dc806f7be None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:16.201539 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c24408-b103-466f-b475-9d8dc806f7be None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:16.201813 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c24408-b103-466f-b475-9d8dc806f7be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:16.203274 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c24408-b103-466f-b475-9d8dc806f7be None admin] RBAC: Adding query filter params (role.id=a3b793754e7b40b0bb21afe74eec3c18, scope.domain.id=default, user.id=4f2f0290e1b04ce48a90ce03cb93d060) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:23:16.203627 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c24408-b103-466f-b475-9d8dc806f7be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:16.205095 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c24408-b103-466f-b475-9d8dc806f7be None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:16.211055 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 30/56] 10.222.0.57 () {56 vars in 1322 bytes} [Thu May 19 14:23:16 2022] GET /identity/v3/role_assignments?user.id=4f2f0290e1b04ce48a90ce03cb93d060&scope.domain.id=default&role.id=a3b793754e7b40b0bb21afe74eec3c18 => generated 530 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:23:16.289659 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5480483e-784b-458c-a0b5-9fa6ed70e5fa None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:16.308038 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:16.331769 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7307a51-2628-4e19-9126-8386e8b74937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:16.332459 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7307a51-2628-4e19-9126-8386e8b74937 None admin] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:16.333036 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7307a51-2628-4e19-9126-8386e8b74937 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:16.333036 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7307a51-2628-4e19-9126-8386e8b74937 None admin] PATH_INFO: `/v3/projects` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:16.333742 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7307a51-2628-4e19-9126-8386e8b74937 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:16.335253 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7307a51-2628-4e19-9126-8386e8b74937 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:16.362787 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 27/57] 10.222.0.57 () {60 vars in 1140 bytes} [Thu May 19 14:23:16 2022] POST /identity/v3/projects => generated 298 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 14:23:18.614910 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f16029-e033-447e-8198-d0c075e23896 None None] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:18.615764 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f16029-e033-447e-8198-d0c075e23896 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:18.615961 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f16029-e033-447e-8198-d0c075e23896 None None] PATH_INFO: `/` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:18.617367 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 31/58] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:23:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:23:18.622626 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eb3eaa-3332-4fb6-b9a6-555e21e44be6 None None] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:18.623022 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eb3eaa-3332-4fb6-b9a6-555e21e44be6 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:18.623208 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eb3eaa-3332-4fb6-b9a6-555e21e44be6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:18.640926 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.auth.core [None req-d3eb3eaa-3332-4fb6-b9a6-555e21e44be6 None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143586) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:23:18.642166 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.fernet_utils [None req-d3eb3eaa-3332-4fb6-b9a6-555e21e44be6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143586) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:23:18.710810 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 28/59] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:23:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:23:18.735972 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f16029-e033-447e-8198-d0c075e23896 None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:18.836869 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f16029-e033-447e-8198-d0c075e23896 None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:18.869258 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50726fb-669b-46fc-acea-9370770d3e3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:18.870016 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50726fb-669b-46fc-acea-9370770d3e3f None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:18.870308 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50726fb-669b-46fc-acea-9370770d3e3f None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:18.870593 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50726fb-669b-46fc-acea-9370770d3e3f None admin] PATH_INFO: `/v3/domains/default` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:18.871782 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50726fb-669b-46fc-acea-9370770d3e3f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:18.873256 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50726fb-669b-46fc-acea-9370770d3e3f None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:18.875512 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 32/60] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:23:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:18.880947 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eb3eaa-3332-4fb6-b9a6-555e21e44be6 None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:18.989981 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:18.998032 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eb3eaa-3332-4fb6-b9a6-555e21e44be6 None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:19.032809 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39def30-8ed4-4011-a17b-713e17667ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:19.033476 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39def30-8ed4-4011-a17b-713e17667ebd None admin] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:19.033760 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39def30-8ed4-4011-a17b-713e17667ebd None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:19.034058 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39def30-8ed4-4011-a17b-713e17667ebd None admin] PATH_INFO: `/v3/users` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:19.034759 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39def30-8ed4-4011-a17b-713e17667ebd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:19.036196 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39def30-8ed4-4011-a17b-713e17667ebd None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:19.061277 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 29/61] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 19 14:23:18 2022] POST /identity/v3/users => generated 275 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:23:22.439094 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d81ba8-3f06-4d4b-9ae2-947843fc39ff None None] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:22.439659 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d81ba8-3f06-4d4b-9ae2-947843fc39ff None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:22.439659 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d81ba8-3f06-4d4b-9ae2-947843fc39ff None None] PATH_INFO: `/` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:22.441046 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 33/62] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:23:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:23:22.451829 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774b9080-2813-433e-9c76-358c93a45c61 None None] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:22.453300 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774b9080-2813-433e-9c76-358c93a45c61 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:22.453811 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774b9080-2813-433e-9c76-358c93a45c61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:22.478859 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.auth.core [None req-774b9080-2813-433e-9c76-358c93a45c61 None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143586) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:23:22.480464 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.fernet_utils [None req-774b9080-2813-433e-9c76-358c93a45c61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143586) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:23:22.567865 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c7126c-858f-4e67-bf0a-d07236145443 None None] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:22.568539 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c7126c-858f-4e67-bf0a-d07236145443 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:22.569045 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c7126c-858f-4e67-bf0a-d07236145443 None None] PATH_INFO: `/` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:22.570820 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 34/63] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:23:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:23:22.576231 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 30/64] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:23:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:23:22.576603 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40449fe-d790-41b1-85ab-3e6bb2cbfe93 None None] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:22.577107 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40449fe-d790-41b1-85ab-3e6bb2cbfe93 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:22.577696 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40449fe-d790-41b1-85ab-3e6bb2cbfe93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:22.596106 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.auth.core [None req-a40449fe-d790-41b1-85ab-3e6bb2cbfe93 None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143587) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:23:22.597664 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-a40449fe-d790-41b1-85ab-3e6bb2cbfe93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:23:22.623300 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774b9080-2813-433e-9c76-358c93a45c61 None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:22.626732 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:22.689035 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 35/65] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:23:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:23:22.718546 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40449fe-d790-41b1-85ab-3e6bb2cbfe93 None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:22.803686 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:22.821137 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774b9080-2813-433e-9c76-358c93a45c61 None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:22.885044 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499df1be-5bdb-4d2e-b5fc-18bd814ebd55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:22.885308 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40449fe-d790-41b1-85ab-3e6bb2cbfe93 None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:22.886377 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499df1be-5bdb-4d2e-b5fc-18bd814ebd55 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:22.886842 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499df1be-5bdb-4d2e-b5fc-18bd814ebd55 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:22.887342 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499df1be-5bdb-4d2e-b5fc-18bd814ebd55 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:22.908339 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499df1be-5bdb-4d2e-b5fc-18bd814ebd55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:23:22.918422 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-499df1be-5bdb-4d2e-b5fc-18bd814ebd55 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:23:22.918422 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:23:22.918422 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:23:22.918422 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:23:22.918422 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:23:22.918422 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:23:22.918422 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:23:22.918422 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:23:22.918422 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:23:22.918422 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:23:22.918422 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:23:22.918422 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:23:22.918422 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:23:22.918422 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:23:22.918422 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:23:22.918422 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:23:22.919739 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:23:22.919739 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:23:22.919739 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:23:22.919739 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:23:22.919739 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:23:22.919739 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:23:22.919739 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:23:22.919739 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:23:22.919739 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:23:22.919739 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:23:22.919739 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:23:22.919739 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:23:22.919739 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:23:22.919739 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:23:22.919739 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:23:22.919739 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:23:22.920860 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:23:22.920860 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:23:22.920860 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:23:22.920860 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:23:22.920860 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:23:22.920860 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:23:22.920860 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:23:22.920860 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:23:22.920860 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:23:22.920860 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:23:22.920860 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:23:22.920860 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:23:22.923296 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 31/66] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 19 14:23:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:23:22.924865 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b29036b-a198-4db3-ab41-daae002e7d55 None None] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:22.925337 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b29036b-a198-4db3-ab41-daae002e7d55 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:22.925772 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b29036b-a198-4db3-ab41-daae002e7d55 None None] PATH_INFO: `/` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:22.927256 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 32/67] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:23:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:23:22.931400 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7831694a-d15c-4783-8a5c-a99978f4a144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:22.931400 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7831694a-d15c-4783-8a5c-a99978f4a144 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:22.931400 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7831694a-d15c-4783-8a5c-a99978f4a144 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:22.931400 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7831694a-d15c-4783-8a5c-a99978f4a144 None admin] PATH_INFO: `/v3/roles/member` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:22.932867 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b29036b-a198-4db3-ab41-daae002e7d55 None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:22.942433 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7831694a-d15c-4783-8a5c-a99978f4a144 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:23:22.950619 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-7831694a-d15c-4783-8a5c-a99978f4a144 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:23:22.950619 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:23:22.950619 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:23:22.950619 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:23:22.950619 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:23:22.950619 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:23:22.950619 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:23:22.950619 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:23:22.950619 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:23:22.950619 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:23:22.950619 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:23:22.950619 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:23:22.950619 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:23:22.950619 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:23:22.950619 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:23:22.950619 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:23:22.951823 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:23:22.951823 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:23:22.951823 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:23:22.951823 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:23:22.951823 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:23:22.951823 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:23:22.951823 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:23:22.951823 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:23:22.951823 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:23:22.951823 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:23:22.951823 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:23:22.951823 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:23:22.951823 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:23:22.951823 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:23:22.951823 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:23:22.951823 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:23:22.953506 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:23:22.953506 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:23:22.953506 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:23:22.953506 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:23:22.953506 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:23:22.953506 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:23:22.953506 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:23:22.953506 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:23:22.953506 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:23:22.953506 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:23:22.953506 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:23:22.953506 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:23:22.953506 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 36/68] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:23:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:23:22.955063 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03343330-14f2-4d23-bfbf-db8f4bc0b19e None None] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:22.955063 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03343330-14f2-4d23-bfbf-db8f4bc0b19e None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:22.955392 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03343330-14f2-4d23-bfbf-db8f4bc0b19e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:22.974803 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.auth.core [None req-03343330-14f2-4d23-bfbf-db8f4bc0b19e None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143587) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:23:22.976196 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-03343330-14f2-4d23-bfbf-db8f4bc0b19e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:23:23.018243 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:23.053866 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 37/69] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:23:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:23:23.053866 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03343330-14f2-4d23-bfbf-db8f4bc0b19e None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:23.060791 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b29036b-a198-4db3-ab41-daae002e7d55 None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:23.110361 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bff0434-32d9-41ff-9c74-0cad8a9fcd27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:23.111036 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bff0434-32d9-41ff-9c74-0cad8a9fcd27 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:23.111326 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bff0434-32d9-41ff-9c74-0cad8a9fcd27 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:23.111608 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bff0434-32d9-41ff-9c74-0cad8a9fcd27 None admin] PATH_INFO: `/v3/roles` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:23.112382 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bff0434-32d9-41ff-9c74-0cad8a9fcd27 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:23:23.112701 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bff0434-32d9-41ff-9c74-0cad8a9fcd27 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:23.114141 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bff0434-32d9-41ff-9c74-0cad8a9fcd27 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:23.122286 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 33/70] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:23:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:23.123528 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae5611b-5081-4ae0-aa7c-e9d70ce2bc82 None None] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:23.123826 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae5611b-5081-4ae0-aa7c-e9d70ce2bc82 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:23.124363 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae5611b-5081-4ae0-aa7c-e9d70ce2bc82 None None] PATH_INFO: `/` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:23.126327 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 34/71] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:23:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:23:23.130781 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae5611b-5081-4ae0-aa7c-e9d70ce2bc82 None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:23.139779 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:23.157052 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03343330-14f2-4d23-bfbf-db8f4bc0b19e None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:23.191077 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a65a8d8-2c95-4dae-bc15-f3237dd622cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:23.191750 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a65a8d8-2c95-4dae-bc15-f3237dd622cf None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:23.192033 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a65a8d8-2c95-4dae-bc15-f3237dd622cf None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:23.192309 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a65a8d8-2c95-4dae-bc15-f3237dd622cf None admin] PATH_INFO: `/v3/roles` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:23.193085 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a65a8d8-2c95-4dae-bc15-f3237dd622cf None admin] RBAC: Adding query filter params (name=member) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:23:23.193393 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a65a8d8-2c95-4dae-bc15-f3237dd622cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:23.194826 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a65a8d8-2c95-4dae-bc15-f3237dd622cf None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:23.203025 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 38/72] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:23:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:23.204237 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5bbc2e-17e3-4436-8548-0e36597bc2d6 None None] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:23.204528 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5bbc2e-17e3-4436-8548-0e36597bc2d6 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:23.204809 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5bbc2e-17e3-4436-8548-0e36597bc2d6 None None] PATH_INFO: `/` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:23.206413 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 39/73] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:23:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:23:23.206809 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5bbc2e-17e3-4436-8548-0e36597bc2d6 None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:23.215128 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:23.244858 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae5611b-5081-4ae0-aa7c-e9d70ce2bc82 None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:23.278356 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab4be0a-bc2f-4385-acd2-d456e03d570c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:23.279167 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab4be0a-bc2f-4385-acd2-d456e03d570c None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:23.279696 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab4be0a-bc2f-4385-acd2-d456e03d570c None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:23.279815 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab4be0a-bc2f-4385-acd2-d456e03d570c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:23.290591 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab4be0a-bc2f-4385-acd2-d456e03d570c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:23:23.298988 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-4ab4be0a-bc2f-4385-acd2-d456e03d570c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:23:23.298988 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:23:23.298988 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:23:23.298988 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:23:23.298988 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:23:23.298988 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:23:23.298988 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:23:23.298988 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:23:23.298988 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:23:23.298988 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:23:23.298988 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:23:23.298988 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:23:23.298988 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:23:23.298988 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:23:23.298988 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:23:23.298988 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:23:23.300190 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:23:23.300190 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:23:23.300190 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:23:23.300190 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:23:23.300190 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:23:23.300190 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:23:23.300190 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:23:23.300190 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:23:23.300190 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:23:23.300190 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:23:23.300190 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:23:23.300190 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:23:23.300190 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:23:23.300190 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:23:23.300190 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:23:23.300190 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:23:23.306071 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:23:23.306071 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:23:23.306071 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:23:23.306071 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:23:23.306071 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:23:23.306071 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:23:23.306071 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:23:23.306071 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:23:23.306071 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:23:23.306071 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:23:23.306071 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:23:23.306071 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:23:23.306071 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 35/74] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:23:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:23:23.306071 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb03d31b-4b29-41ce-9e93-130f5e3ab692 None None] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:23.306971 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb03d31b-4b29-41ce-9e93-130f5e3ab692 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:23.306971 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb03d31b-4b29-41ce-9e93-130f5e3ab692 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:23.318424 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.auth.core [None req-eb03d31b-4b29-41ce-9e93-130f5e3ab692 None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143586) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:23:23.319633 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.fernet_utils [None req-eb03d31b-4b29-41ce-9e93-130f5e3ab692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143586) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:23:23.326252 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5bbc2e-17e3-4436-8548-0e36597bc2d6 None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:23.362017 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b538979-71d5-4519-b179-396f87560b5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:23.363381 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b538979-71d5-4519-b179-396f87560b5d None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:23.363831 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b538979-71d5-4519-b179-396f87560b5d None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:23.364301 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b538979-71d5-4519-b179-396f87560b5d None admin] PATH_INFO: `/v3/users/21249d0b4cb3470e83a9f256cbe52f18` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:23.383618 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b538979-71d5-4519-b179-396f87560b5d None admin] RBAC: Authorizing `identity:get_user(user_id=21249d0b4cb3470e83a9f256cbe52f18)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:23.385193 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b538979-71d5-4519-b179-396f87560b5d None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:23.386361 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:23.386930 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 40/75] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:23:23 2022] GET /identity/v3/users/21249d0b4cb3470e83a9f256cbe52f18 => generated 275 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:23.390744 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b538979-71d5-4519-b179-396f87560b5d None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:23.396262 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 36/76] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:23:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:23:23.397392 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95ea29d-9854-4ed1-9653-7deaded2e5a0 None None] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:23.397686 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95ea29d-9854-4ed1-9653-7deaded2e5a0 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:23.398004 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95ea29d-9854-4ed1-9653-7deaded2e5a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:23.416564 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.auth.core [None req-a95ea29d-9854-4ed1-9653-7deaded2e5a0 None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143586) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:23:23.417760 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.fernet_utils [None req-a95ea29d-9854-4ed1-9653-7deaded2e5a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143586) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:23:23.456634 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:23.486683 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 37/77] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:23:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:23:23.487537 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95ea29d-9854-4ed1-9653-7deaded2e5a0 None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:23.496602 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b538979-71d5-4519-b179-396f87560b5d None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:23.533130 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1aaa37-6c11-4f47-ac42-bb71b55f4701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:23.534046 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1aaa37-6c11-4f47-ac42-bb71b55f4701 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:23.534335 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1aaa37-6c11-4f47-ac42-bb71b55f4701 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:23.534611 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1aaa37-6c11-4f47-ac42-bb71b55f4701 None admin] PATH_INFO: `/v3/users/21249d0b4cb3470e83a9f256cbe52f18` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:23.535837 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1aaa37-6c11-4f47-ac42-bb71b55f4701 None admin] RBAC: Authorizing `identity:get_user(user_id=21249d0b4cb3470e83a9f256cbe52f18)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:23.537317 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1aaa37-6c11-4f47-ac42-bb71b55f4701 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:23.539000 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 41/78] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:23:23 2022] GET /identity/v3/users/21249d0b4cb3470e83a9f256cbe52f18 => generated 275 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:23.539774 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1aaa37-6c11-4f47-ac42-bb71b55f4701 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:23.591588 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95ea29d-9854-4ed1-9653-7deaded2e5a0 None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:23.617862 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:23.627804 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc36950-3a07-4dd4-94fd-430ab785768b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:23.628474 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc36950-3a07-4dd4-94fd-430ab785768b None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:23.628763 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc36950-3a07-4dd4-94fd-430ab785768b None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:23.629046 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc36950-3a07-4dd4-94fd-430ab785768b None admin] PATH_INFO: `/v3/roles` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:23.629807 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc36950-3a07-4dd4-94fd-430ab785768b None admin] RBAC: Adding query filter params (name=admin) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:23:23.630144 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc36950-3a07-4dd4-94fd-430ab785768b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:23.631537 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc36950-3a07-4dd4-94fd-430ab785768b None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:23.639365 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1aaa37-6c11-4f47-ac42-bb71b55f4701 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:23.639898 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 38/79] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:23:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:23.640571 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc36950-3a07-4dd4-94fd-430ab785768b None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:23.674115 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:23.679815 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45376e19-b6e6-4354-a708-2c86a4ba30f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:23.680469 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45376e19-b6e6-4354-a708-2c86a4ba30f5 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:23.680757 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45376e19-b6e6-4354-a708-2c86a4ba30f5 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:23.681038 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45376e19-b6e6-4354-a708-2c86a4ba30f5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:23.744269 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc36950-3a07-4dd4-94fd-430ab785768b None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:23.750116 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45376e19-b6e6-4354-a708-2c86a4ba30f5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:23.751564 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45376e19-b6e6-4354-a708-2c86a4ba30f5 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:23.777637 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0737e31-7e96-4d3f-bbe8-cb7d94c32f3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:23.778461 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0737e31-7e96-4d3f-bbe8-cb7d94c32f3a None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:23.778746 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0737e31-7e96-4d3f-bbe8-cb7d94c32f3a None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:23.779028 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0737e31-7e96-4d3f-bbe8-cb7d94c32f3a None admin] PATH_INFO: `/v3/roles/member` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:23.789349 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0737e31-7e96-4d3f-bbe8-cb7d94c32f3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:23:23.797625 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-d0737e31-7e96-4d3f-bbe8-cb7d94c32f3a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:23:23.797625 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:23:23.797625 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:23:23.797625 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:23:23.797625 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:23:23.797625 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:23:23.797625 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:23:23.797625 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:23:23.797625 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:23:23.797625 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:23:23.797625 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:23:23.797625 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:23:23.797625 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:23:23.797625 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:23:23.797625 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:23:23.797625 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:23:23.798854 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:23:23.798854 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:23:23.798854 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:23:23.798854 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:23:23.798854 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:23:23.798854 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:23:23.798854 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:23:23.798854 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:23:23.798854 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:23:23.798854 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:23:23.798854 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:23:23.798854 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:23:23.798854 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:23:23.798854 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:23:23.798854 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:23:23.798854 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:23:23.799771 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:23:23.799771 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:23:23.799771 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:23:23.799771 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:23:23.799771 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:23:23.799771 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:23:23.799771 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:23:23.799771 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:23:23.799771 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:23:23.799771 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:23:23.799771 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:23:23.799771 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:23:23.799771 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 39/80] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:23:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:23:23.799771 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0737e31-7e96-4d3f-bbe8-cb7d94c32f3a None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:23.848390 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:23.852417 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 42/81] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:23:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:23:23.853277 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45376e19-b6e6-4354-a708-2c86a4ba30f5 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:23.885308 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:23.900123 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0737e31-7e96-4d3f-bbe8-cb7d94c32f3a None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:23.939440 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7debe5ff-5207-4059-93a8-5672381b3fb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:23.940247 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7debe5ff-5207-4059-93a8-5672381b3fb9 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:23.940542 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7debe5ff-5207-4059-93a8-5672381b3fb9 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:23.940828 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7debe5ff-5207-4059-93a8-5672381b3fb9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:23.942056 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7debe5ff-5207-4059-93a8-5672381b3fb9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:23.943530 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7debe5ff-5207-4059-93a8-5672381b3fb9 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:23.945174 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 40/82] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:23:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:23.945976 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7debe5ff-5207-4059-93a8-5672381b3fb9 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:23.957428 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45376e19-b6e6-4354-a708-2c86a4ba30f5 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:23.992311 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d41d24-15f2-4565-8d96-301cbc6b4824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:23.993028 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d41d24-15f2-4565-8d96-301cbc6b4824 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:23.993245 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d41d24-15f2-4565-8d96-301cbc6b4824 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:23.993525 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d41d24-15f2-4565-8d96-301cbc6b4824 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:24.046818 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7debe5ff-5207-4059-93a8-5672381b3fb9 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:24.060060 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d41d24-15f2-4565-8d96-301cbc6b4824 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:24.061608 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d41d24-15f2-4565-8d96-301cbc6b4824 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:24.063353 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:24.082284 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71a2c26-60fd-4945-913c-8ad10ea56902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:24.083165 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71a2c26-60fd-4945-913c-8ad10ea56902 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:24.083450 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71a2c26-60fd-4945-913c-8ad10ea56902 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:24.083727 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71a2c26-60fd-4945-913c-8ad10ea56902 None admin] PATH_INFO: `/v3/users/4f2f0290e1b04ce48a90ce03cb93d060` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:24.085154 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71a2c26-60fd-4945-913c-8ad10ea56902 None admin] RBAC: Authorizing `identity:get_user(user_id=4f2f0290e1b04ce48a90ce03cb93d060)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:24.086624 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71a2c26-60fd-4945-913c-8ad10ea56902 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:24.088250 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 41/83] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:23:23 2022] GET /identity/v3/users/4f2f0290e1b04ce48a90ce03cb93d060 => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:24.090179 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71a2c26-60fd-4945-913c-8ad10ea56902 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:24.093685 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:24.163432 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 43/84] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:23:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:23:24.164279 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d41d24-15f2-4565-8d96-301cbc6b4824 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:24.193993 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71a2c26-60fd-4945-913c-8ad10ea56902 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:24.227567 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb4b303-fb6a-403f-a7d6-ec2415a312d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:24.228244 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb4b303-fb6a-403f-a7d6-ec2415a312d7 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:24.228539 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb4b303-fb6a-403f-a7d6-ec2415a312d7 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:24.228814 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb4b303-fb6a-403f-a7d6-ec2415a312d7 None admin] PATH_INFO: `/v3/roles` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:24.229570 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb4b303-fb6a-403f-a7d6-ec2415a312d7 None admin] RBAC: Adding query filter params (name=member) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:23:24.229940 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb4b303-fb6a-403f-a7d6-ec2415a312d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:24.231388 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb4b303-fb6a-403f-a7d6-ec2415a312d7 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:24.240583 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 42/85] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:23:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:24.241585 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb4b303-fb6a-403f-a7d6-ec2415a312d7 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:24.266599 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d41d24-15f2-4565-8d96-301cbc6b4824 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:24.289609 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:24.297338 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:24.303193 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424dc4f2-1b9e-45f5-992d-a3ea4bdde4eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:24.304055 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424dc4f2-1b9e-45f5-992d-a3ea4bdde4eb None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:24.304347 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424dc4f2-1b9e-45f5-992d-a3ea4bdde4eb None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:24.304628 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424dc4f2-1b9e-45f5-992d-a3ea4bdde4eb None admin] PATH_INFO: `/v3/projects/e7497d51ee8f4f0487b09541d4ad94d2` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:24.305836 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424dc4f2-1b9e-45f5-992d-a3ea4bdde4eb None admin] RBAC: Authorizing `identity:get_project(project_id=e7497d51ee8f4f0487b09541d4ad94d2)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:24.307328 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424dc4f2-1b9e-45f5-992d-a3ea4bdde4eb None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:24.309082 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 44/86] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:23:24 2022] GET /identity/v3/projects/e7497d51ee8f4f0487b09541d4ad94d2 => generated 298 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:24.310777 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424dc4f2-1b9e-45f5-992d-a3ea4bdde4eb None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:24.345972 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb4b303-fb6a-403f-a7d6-ec2415a312d7 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:24.379991 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f5760b-ce24-4520-817d-bcebf504cc22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:24.380659 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f5760b-ce24-4520-817d-bcebf504cc22 None admin] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:24.380942 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f5760b-ce24-4520-817d-bcebf504cc22 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:24.381236 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f5760b-ce24-4520-817d-bcebf504cc22 None admin] PATH_INFO: `/v3/users` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:24.381958 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f5760b-ce24-4520-817d-bcebf504cc22 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:24.383422 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f5760b-ce24-4520-817d-bcebf504cc22 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:24.403682 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 43/87] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 19 14:23:24 2022] POST /identity/v3/users => generated 289 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:23:24.404496 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f5760b-ce24-4520-817d-bcebf504cc22 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:24.423145 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424dc4f2-1b9e-45f5-992d-a3ea4bdde4eb None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:24.463182 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7415a0-a2ef-4c93-97b8-28fc13d53975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:24.463966 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7415a0-a2ef-4c93-97b8-28fc13d53975 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:24.464330 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7415a0-a2ef-4c93-97b8-28fc13d53975 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:24.464680 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7415a0-a2ef-4c93-97b8-28fc13d53975 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:24.526362 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f5760b-ce24-4520-817d-bcebf504cc22 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:24.528009 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:24.542665 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:24.545769 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7415a0-a2ef-4c93-97b8-28fc13d53975 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:24.548811 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7415a0-a2ef-4c93-97b8-28fc13d53975 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:24.569142 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63997e89-b37b-4cf7-97f7-344c53e85ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:24.570071 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63997e89-b37b-4cf7-97f7-344c53e85ce4 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:24.570365 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63997e89-b37b-4cf7-97f7-344c53e85ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:24.570645 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63997e89-b37b-4cf7-97f7-344c53e85ce4 None admin] PATH_INFO: `/v3/projects/e7497d51ee8f4f0487b09541d4ad94d2` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:24.571914 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63997e89-b37b-4cf7-97f7-344c53e85ce4 None admin] RBAC: Authorizing `identity:get_project(project_id=e7497d51ee8f4f0487b09541d4ad94d2)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:24.573406 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63997e89-b37b-4cf7-97f7-344c53e85ce4 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:24.575207 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 44/88] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:23:24 2022] GET /identity/v3/projects/e7497d51ee8f4f0487b09541d4ad94d2 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:24.578761 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63997e89-b37b-4cf7-97f7-344c53e85ce4 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:24.673870 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 45/89] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:23:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:23:24.675006 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7415a0-a2ef-4c93-97b8-28fc13d53975 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:24.724456 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63997e89-b37b-4cf7-97f7-344c53e85ce4 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:24.769998 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a703b755-e6fe-4d59-a303-aa0490679c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:24.770892 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a703b755-e6fe-4d59-a303-aa0490679c13 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:24.771179 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a703b755-e6fe-4d59-a303-aa0490679c13 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:24.771459 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a703b755-e6fe-4d59-a303-aa0490679c13 None admin] PATH_INFO: `/v3/users/21249d0b4cb3470e83a9f256cbe52f18` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:24.772693 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a703b755-e6fe-4d59-a303-aa0490679c13 None admin] RBAC: Authorizing `identity:get_user(user_id=21249d0b4cb3470e83a9f256cbe52f18)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:24.774195 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a703b755-e6fe-4d59-a303-aa0490679c13 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:24.775852 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 45/90] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:23:24 2022] GET /identity/v3/users/21249d0b4cb3470e83a9f256cbe52f18 => generated 275 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:24.776713 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a703b755-e6fe-4d59-a303-aa0490679c13 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:24.800876 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:24.804801 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7415a0-a2ef-4c93-97b8-28fc13d53975 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:24.834277 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:24.850191 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c218ca-a9cb-4af4-a652-9e827e6c898d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:24.851119 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c218ca-a9cb-4af4-a652-9e827e6c898d None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:24.851408 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c218ca-a9cb-4af4-a652-9e827e6c898d None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:24.851694 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c218ca-a9cb-4af4-a652-9e827e6c898d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:24.852715 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c218ca-a9cb-4af4-a652-9e827e6c898d None admin] RBAC: Adding query filter params (role.id=b954b2c7ce764aa0afac78e075a011a6, scope.project.id=e7497d51ee8f4f0487b09541d4ad94d2, user.id=21249d0b4cb3470e83a9f256cbe52f18) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:23:24.853536 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c218ca-a9cb-4af4-a652-9e827e6c898d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:24.855332 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c218ca-a9cb-4af4-a652-9e827e6c898d None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:24.863487 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 46/91] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:23:24 2022] GET /identity/v3/role_assignments?user.id=21249d0b4cb3470e83a9f256cbe52f18&scope.project.id=e7497d51ee8f4f0487b09541d4ad94d2&role.id=b954b2c7ce764aa0afac78e075a011a6 => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:24.864340 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c218ca-a9cb-4af4-a652-9e827e6c898d None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:24.892960 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a703b755-e6fe-4d59-a303-aa0490679c13 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:24.931856 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ca8335-5ede-4285-926f-1e51f220d464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:24.932527 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ca8335-5ede-4285-926f-1e51f220d464 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:24.932816 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ca8335-5ede-4285-926f-1e51f220d464 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:24.933102 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ca8335-5ede-4285-926f-1e51f220d464 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:24.934084 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ca8335-5ede-4285-926f-1e51f220d464 None admin] RBAC: Adding query filter params (role.id=3d5075293a4f45449eaef6af8fa53004, scope.project.id=e7497d51ee8f4f0487b09541d4ad94d2, user.id=21249d0b4cb3470e83a9f256cbe52f18) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:23:24.934405 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ca8335-5ede-4285-926f-1e51f220d464 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:24.936040 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ca8335-5ede-4285-926f-1e51f220d464 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:24.946112 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 46/92] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:23:24 2022] GET /identity/v3/role_assignments?user.id=21249d0b4cb3470e83a9f256cbe52f18&scope.project.id=e7497d51ee8f4f0487b09541d4ad94d2&role.id=3d5075293a4f45449eaef6af8fa53004 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:24.946875 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ca8335-5ede-4285-926f-1e51f220d464 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:25.001609 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c218ca-a9cb-4af4-a652-9e827e6c898d None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:25.044975 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1535f10-c3ce-47b6-8cbe-6b2199f270e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:25.046524 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1535f10-c3ce-47b6-8cbe-6b2199f270e6 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:25.047309 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1535f10-c3ce-47b6-8cbe-6b2199f270e6 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:25.047595 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1535f10-c3ce-47b6-8cbe-6b2199f270e6 None admin] PATH_INFO: `/v3/projects/e7497d51ee8f4f0487b09541d4ad94d2` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:25.049113 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1535f10-c3ce-47b6-8cbe-6b2199f270e6 None admin] RBAC: Authorizing `identity:get_project(project_id=e7497d51ee8f4f0487b09541d4ad94d2)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:25.050669 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1535f10-c3ce-47b6-8cbe-6b2199f270e6 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:25.053264 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 47/93] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:23:24 2022] GET /identity/v3/projects/e7497d51ee8f4f0487b09541d4ad94d2 => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:25.059895 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1535f10-c3ce-47b6-8cbe-6b2199f270e6 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:25.112237 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:25.195812 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ca8335-5ede-4285-926f-1e51f220d464 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:25.197731 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:25.204918 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1535f10-c3ce-47b6-8cbe-6b2199f270e6 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:25.247468 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c25771-f334-4052-a67f-688a7c2b2f28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:25.248158 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c25771-f334-4052-a67f-688a7c2b2f28 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:25.248453 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c25771-f334-4052-a67f-688a7c2b2f28 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:25.249970 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c25771-f334-4052-a67f-688a7c2b2f28 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:25.253885 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c25771-f334-4052-a67f-688a7c2b2f28 None admin] RBAC: Adding query filter params (role.id=a3b793754e7b40b0bb21afe74eec3c18, scope.project.id=e7497d51ee8f4f0487b09541d4ad94d2, user.id=4f2f0290e1b04ce48a90ce03cb93d060) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:23:25.253885 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c25771-f334-4052-a67f-688a7c2b2f28 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:25.253885 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c25771-f334-4052-a67f-688a7c2b2f28 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:25.259708 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 48/94] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:23:25 2022] GET /identity/v3/role_assignments?user.id=4f2f0290e1b04ce48a90ce03cb93d060&scope.project.id=e7497d51ee8f4f0487b09541d4ad94d2&role.id=a3b793754e7b40b0bb21afe74eec3c18 => generated 259 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:23:25.299491 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff7f1c3-2cd7-43b5-8aa7-57f557ccdd8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:25.300175 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff7f1c3-2cd7-43b5-8aa7-57f557ccdd8b None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:25.300461 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff7f1c3-2cd7-43b5-8aa7-57f557ccdd8b None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:25.300740 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff7f1c3-2cd7-43b5-8aa7-57f557ccdd8b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:25.386365 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff7f1c3-2cd7-43b5-8aa7-57f557ccdd8b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:25.387889 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff7f1c3-2cd7-43b5-8aa7-57f557ccdd8b None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:25.497573 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 47/95] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:23:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 551 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:23:25.503954 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c25771-f334-4052-a67f-688a7c2b2f28 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:25.609903 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:25.615078 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c25771-f334-4052-a67f-688a7c2b2f28 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:25.658021 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868988c8-a59b-421d-8c9f-5cda264a5744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:25.658925 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868988c8-a59b-421d-8c9f-5cda264a5744 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:25.659216 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868988c8-a59b-421d-8c9f-5cda264a5744 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:25.659499 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868988c8-a59b-421d-8c9f-5cda264a5744 None admin] PATH_INFO: `/v3/projects/5c7498adefdf4ff59ace12ff1685bd25` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:25.663244 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868988c8-a59b-421d-8c9f-5cda264a5744 None admin] RBAC: Authorizing `identity:get_project(project_id=5c7498adefdf4ff59ace12ff1685bd25)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:25.664755 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868988c8-a59b-421d-8c9f-5cda264a5744 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:25.666787 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 49/96] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:23:25 2022] GET /identity/v3/projects/5c7498adefdf4ff59ace12ff1685bd25 => generated 312 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:25.672803 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff7f1c3-2cd7-43b5-8aa7-57f557ccdd8b None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:25.690220 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:25.787730 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff7f1c3-2cd7-43b5-8aa7-57f557ccdd8b None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:25.826076 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc3f0af-bdee-4e9d-8418-2276548f2d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:25.826779 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc3f0af-bdee-4e9d-8418-2276548f2d05 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:25.827566 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc3f0af-bdee-4e9d-8418-2276548f2d05 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:25.828054 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc3f0af-bdee-4e9d-8418-2276548f2d05 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:25.828946 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc3f0af-bdee-4e9d-8418-2276548f2d05 None admin] RBAC: Adding query filter params (role.id=3d5075293a4f45449eaef6af8fa53004, scope.project.id=5c7498adefdf4ff59ace12ff1685bd25, user.id=21249d0b4cb3470e83a9f256cbe52f18) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:23:25.829267 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc3f0af-bdee-4e9d-8418-2276548f2d05 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:25.830772 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc3f0af-bdee-4e9d-8418-2276548f2d05 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:25.836983 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 48/97] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:23:25 2022] GET /identity/v3/role_assignments?user.id=21249d0b4cb3470e83a9f256cbe52f18&scope.project.id=5c7498adefdf4ff59ace12ff1685bd25&role.id=3d5075293a4f45449eaef6af8fa53004 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:27.678516 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300b6407-c30a-4cab-bf2c-adabff23fe09 None None] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:27.679536 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300b6407-c30a-4cab-bf2c-adabff23fe09 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:27.680047 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300b6407-c30a-4cab-bf2c-adabff23fe09 None None] PATH_INFO: `/` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:27.681674 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 50/98] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:23:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:23:27.688251 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be965931-1f44-4c2a-b143-38504b0da391 None None] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:27.689243 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be965931-1f44-4c2a-b143-38504b0da391 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:27.689762 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be965931-1f44-4c2a-b143-38504b0da391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:27.692900 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c02e846-f63a-49f9-88d0-51eacff31c7c None None] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:27.693192 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c02e846-f63a-49f9-88d0-51eacff31c7c None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:27.693473 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c02e846-f63a-49f9-88d0-51eacff31c7c None None] PATH_INFO: `/` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:27.695077 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 51/99] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:23:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:23:27.710425 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc672d1-0f24-4ec8-b6d9-f4dddfffd0d7 None None] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:27.710773 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc672d1-0f24-4ec8-b6d9-f4dddfffd0d7 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:27.711064 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc672d1-0f24-4ec8-b6d9-f4dddfffd0d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:27.726188 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.auth.core [None req-be965931-1f44-4c2a-b143-38504b0da391 None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143586) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:23:27.727413 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.fernet_utils [None req-be965931-1f44-4c2a-b143-38504b0da391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143586) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:23:27.740068 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.auth.core [None req-bdc672d1-0f24-4ec8-b6d9-f4dddfffd0d7 None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143587) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:23:27.741308 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-bdc672d1-0f24-4ec8-b6d9-f4dddfffd0d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:23:27.854251 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:27.883371 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 49/100] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:23:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:23:27.920572 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 52/101] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:23:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:23:27.923760 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be965931-1f44-4c2a-b143-38504b0da391 None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:27.966509 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc672d1-0f24-4ec8-b6d9-f4dddfffd0d7 None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:28.049346 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be965931-1f44-4c2a-b143-38504b0da391 None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:28.085616 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7038918a-5595-4d16-8629-9ea3bc0fdf10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:28.086838 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7038918a-5595-4d16-8629-9ea3bc0fdf10 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:28.087354 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7038918a-5595-4d16-8629-9ea3bc0fdf10 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:28.087858 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7038918a-5595-4d16-8629-9ea3bc0fdf10 None admin] PATH_INFO: `/v3/domains/default` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:28.089288 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7038918a-5595-4d16-8629-9ea3bc0fdf10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:28.091023 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7038918a-5595-4d16-8629-9ea3bc0fdf10 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:28.093019 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 50/102] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:23:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:28.098687 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7038918a-5595-4d16-8629-9ea3bc0fdf10 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:28.134294 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:28.141962 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc672d1-0f24-4ec8-b6d9-f4dddfffd0d7 None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:28.185035 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a667fc58-e81a-4cfa-a782-fdf925f23623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:28.186344 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a667fc58-e81a-4cfa-a782-fdf925f23623 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:28.187324 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a667fc58-e81a-4cfa-a782-fdf925f23623 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:28.187936 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a667fc58-e81a-4cfa-a782-fdf925f23623 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:28.218521 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a667fc58-e81a-4cfa-a782-fdf925f23623 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:23:28.226754 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-a667fc58-e81a-4cfa-a782-fdf925f23623 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:23:28.226754 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:23:28.226754 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:23:28.226754 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:23:28.226754 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:23:28.226754 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:23:28.226754 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:23:28.226754 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:23:28.226754 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:23:28.226754 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:23:28.226754 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:23:28.226754 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:23:28.226754 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:23:28.226754 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:23:28.226754 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:23:28.226754 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:23:28.228013 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:23:28.228013 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:23:28.228013 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:23:28.228013 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:23:28.228013 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:23:28.228013 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:23:28.228013 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:23:28.228013 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:23:28.228013 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:23:28.228013 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:23:28.228013 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:23:28.228013 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:23:28.228013 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:23:28.228013 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:23:28.228013 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:23:28.228013 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:23:28.228991 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:23:28.228991 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:23:28.228991 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:23:28.228991 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:23:28.228991 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:23:28.228991 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:23:28.228991 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:23:28.228991 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:23:28.228991 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:23:28.228991 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:23:28.228991 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:23:28.228991 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:23:28.228991 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 53/103] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:23:27 2022] GET /identity/v3/roles/reader => generated 84 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:23:28.229788 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d9d0b7-60a1-4446-b7a3-4bbb33d0fbfe None None] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:28.230242 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d9d0b7-60a1-4446-b7a3-4bbb33d0fbfe None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:28.230242 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d9d0b7-60a1-4446-b7a3-4bbb33d0fbfe None None] PATH_INFO: `/` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:28.231494 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 54/104] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:23:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:23:28.234333 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:28.236469 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d9d0b7-60a1-4446-b7a3-4bbb33d0fbfe None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:28.259053 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7038918a-5595-4d16-8629-9ea3bc0fdf10 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:28.298116 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da272f05-e3d0-421d-9f9e-9def4fccaa87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:28.298785 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da272f05-e3d0-421d-9f9e-9def4fccaa87 None admin] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:28.299071 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da272f05-e3d0-421d-9f9e-9def4fccaa87 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:28.299353 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da272f05-e3d0-421d-9f9e-9def4fccaa87 None admin] PATH_INFO: `/v3/projects` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:28.300068 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da272f05-e3d0-421d-9f9e-9def4fccaa87 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:28.301734 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da272f05-e3d0-421d-9f9e-9def4fccaa87 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:28.338739 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 51/105] 10.222.0.57 () {60 vars in 1141 bytes} [Thu May 19 14:23:28 2022] POST /identity/v3/projects => generated 302 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:23:28.340460 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe4e9e9-b819-4e41-a381-fb0209fbfb34 None None] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:28.340810 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe4e9e9-b819-4e41-a381-fb0209fbfb34 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:28.341101 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe4e9e9-b819-4e41-a381-fb0209fbfb34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:28.362528 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.auth.core [None req-7fe4e9e9-b819-4e41-a381-fb0209fbfb34 None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143586) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:23:28.364391 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.fernet_utils [None req-7fe4e9e9-b819-4e41-a381-fb0209fbfb34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143586) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:23:28.459882 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 52/106] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:23:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:23:28.504793 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe4e9e9-b819-4e41-a381-fb0209fbfb34 None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:28.526657 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d9d0b7-60a1-4446-b7a3-4bbb33d0fbfe None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:28.585211 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4b1ad2-0666-4484-9431-039e36c6f47a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:28.586373 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4b1ad2-0666-4484-9431-039e36c6f47a None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:28.586894 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4b1ad2-0666-4484-9431-039e36c6f47a None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:28.587392 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4b1ad2-0666-4484-9431-039e36c6f47a None admin] PATH_INFO: `/v3/roles` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:28.588398 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4b1ad2-0666-4484-9431-039e36c6f47a None admin] RBAC: Adding query filter params (name=reader) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:23:28.588938 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4b1ad2-0666-4484-9431-039e36c6f47a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:28.590639 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4b1ad2-0666-4484-9431-039e36c6f47a None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:28.601386 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 55/107] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:23:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 365 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:28.605440 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:28.615556 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4b1ad2-0666-4484-9431-039e36c6f47a None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:28.650576 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:28.725114 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe4e9e9-b819-4e41-a381-fb0209fbfb34 None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:28.785674 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4b1ad2-0666-4484-9431-039e36c6f47a None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:28.810602 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7b18db-10e8-44a6-a26f-14cf1acf8c19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:28.811653 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7b18db-10e8-44a6-a26f-14cf1acf8c19 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:28.811938 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7b18db-10e8-44a6-a26f-14cf1acf8c19 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:28.812221 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7b18db-10e8-44a6-a26f-14cf1acf8c19 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:28.832151 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7b18db-10e8-44a6-a26f-14cf1acf8c19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:23:28.843831 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-2b7b18db-10e8-44a6-a26f-14cf1acf8c19 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:23:28.843831 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:23:28.843831 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:23:28.843831 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:23:28.843831 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:23:28.843831 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:23:28.843831 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:23:28.843831 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:23:28.843831 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:23:28.843831 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:23:28.843831 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:23:28.843831 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:23:28.843831 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:23:28.843831 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:23:28.843831 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:23:28.843831 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:23:28.845074 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:23:28.845074 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:23:28.845074 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:23:28.845074 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:23:28.845074 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:23:28.845074 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:23:28.845074 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:23:28.845074 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:23:28.845074 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:23:28.845074 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:23:28.845074 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:23:28.845074 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:23:28.845074 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:23:28.845074 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:23:28.845074 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:23:28.845074 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:23:28.846166 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:23:28.846166 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:23:28.846166 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:23:28.846166 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:23:28.846166 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:23:28.846166 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:23:28.846166 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:23:28.846166 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:23:28.846166 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:23:28.846166 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:23:28.846166 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:23:28.846166 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:23:28.848758 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 53/108] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 19 14:23:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:23:28.851636 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7b18db-10e8-44a6-a26f-14cf1acf8c19 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:28.866617 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9aa203-0c1e-457e-836f-590297761278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:28.867811 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9aa203-0c1e-457e-836f-590297761278 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:28.868277 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9aa203-0c1e-457e-836f-590297761278 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:28.868717 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9aa203-0c1e-457e-836f-590297761278 None admin] PATH_INFO: `/v3/users/cd0294bae78b43ea80cea320f446e487` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:28.887880 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9aa203-0c1e-457e-836f-590297761278 None admin] RBAC: Authorizing `identity:get_user(user_id=cd0294bae78b43ea80cea320f446e487)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:28.890183 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9aa203-0c1e-457e-836f-590297761278 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:28.892154 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 56/109] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:23:28 2022] GET /identity/v3/users/cd0294bae78b43ea80cea320f446e487 => generated 289 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:28.898619 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9aa203-0c1e-457e-836f-590297761278 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:28.978761 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:28.991369 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7b18db-10e8-44a6-a26f-14cf1acf8c19 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:28.999730 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:29.005316 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9aa203-0c1e-457e-836f-590297761278 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:29.039847 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e531e5da-4672-46c4-9ca8-79c4a5f1da5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:29.040576 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e531e5da-4672-46c4-9ca8-79c4a5f1da5d None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:29.041048 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e531e5da-4672-46c4-9ca8-79c4a5f1da5d None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:29.041338 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e531e5da-4672-46c4-9ca8-79c4a5f1da5d None admin] PATH_INFO: `/v3/roles` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:29.042558 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e531e5da-4672-46c4-9ca8-79c4a5f1da5d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:23:29.043030 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e531e5da-4672-46c4-9ca8-79c4a5f1da5d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:29.044658 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e531e5da-4672-46c4-9ca8-79c4a5f1da5d None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:29.047409 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9f45db-6455-4d26-9313-b7c73f91da9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:29.048271 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9f45db-6455-4d26-9313-b7c73f91da9e None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:29.048877 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9f45db-6455-4d26-9313-b7c73f91da9e None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:29.049336 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9f45db-6455-4d26-9313-b7c73f91da9e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:29.058325 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 54/110] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:23:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:29.058325 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43c7fac-3174-464d-9c73-e088abc0534c None None] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:29.058325 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43c7fac-3174-464d-9c73-e088abc0534c None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:29.058325 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43c7fac-3174-464d-9c73-e088abc0534c None None] PATH_INFO: `/` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:29.058325 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 55/111] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:23:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:23:29.060559 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42bafe5-4b3c-44bc-a299-f9be7f051d8e None None] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:29.060559 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42bafe5-4b3c-44bc-a299-f9be7f051d8e None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:29.060559 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42bafe5-4b3c-44bc-a299-f9be7f051d8e None None] PATH_INFO: `/` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:29.061297 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 56/112] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:23:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:23:29.067222 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd494566-582d-48ea-926e-628d857cb843 None None] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:29.067510 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd494566-582d-48ea-926e-628d857cb843 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:29.067784 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd494566-582d-48ea-926e-628d857cb843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:29.092261 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.auth.core [None req-dd494566-582d-48ea-926e-628d857cb843 None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143586) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:23:29.094965 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.fernet_utils [None req-dd494566-582d-48ea-926e-628d857cb843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143586) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:23:29.143482 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9f45db-6455-4d26-9313-b7c73f91da9e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:29.145166 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9f45db-6455-4d26-9313-b7c73f91da9e None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:29.191618 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 57/113] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:23:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:23:29.193417 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd494566-582d-48ea-926e-628d857cb843 None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:29.237254 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:29.267167 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 57/114] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:23:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 14:23:29.268110 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f749e04e-8f4b-43df-893b-bae2972b32c4 None None] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:29.268410 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f749e04e-8f4b-43df-893b-bae2972b32c4 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:29.268698 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f749e04e-8f4b-43df-893b-bae2972b32c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:29.285563 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.auth.core [None req-f749e04e-8f4b-43df-893b-bae2972b32c4 None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143587) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:23:29.286848 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-f749e04e-8f4b-43df-893b-bae2972b32c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:23:29.322881 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:29.331105 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd494566-582d-48ea-926e-628d857cb843 None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:29.372274 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09306b54-197d-4bfc-96ef-02e1fcf4585a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:29.373177 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09306b54-197d-4bfc-96ef-02e1fcf4585a None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:29.373469 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09306b54-197d-4bfc-96ef-02e1fcf4585a None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:29.373776 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09306b54-197d-4bfc-96ef-02e1fcf4585a None admin] PATH_INFO: `/v3/users/21249d0b4cb3470e83a9f256cbe52f18` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:29.375810 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09306b54-197d-4bfc-96ef-02e1fcf4585a None admin] RBAC: Authorizing `identity:get_user(user_id=21249d0b4cb3470e83a9f256cbe52f18)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:29.377131 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09306b54-197d-4bfc-96ef-02e1fcf4585a None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:29.378830 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 58/115] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:23:29 2022] GET /identity/v3/users/21249d0b4cb3470e83a9f256cbe52f18 => generated 275 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:29.379627 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09306b54-197d-4bfc-96ef-02e1fcf4585a None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:29.394800 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 58/116] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:23:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:23:29.395504 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f749e04e-8f4b-43df-893b-bae2972b32c4 None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:29.500324 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09306b54-197d-4bfc-96ef-02e1fcf4585a None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:29.506161 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f749e04e-8f4b-43df-893b-bae2972b32c4 None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:29.509027 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:29.536439 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6e2eb4-bd10-4543-be00-ddb679cffc41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:29.537247 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6e2eb4-bd10-4543-be00-ddb679cffc41 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:29.537540 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6e2eb4-bd10-4543-be00-ddb679cffc41 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:29.537806 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6e2eb4-bd10-4543-be00-ddb679cffc41 None admin] PATH_INFO: `/v3/roles/member` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:29.549788 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6e2eb4-bd10-4543-be00-ddb679cffc41 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:23:29.553367 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbf41d5-ea18-4814-8709-bb109d3e91e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:29.554283 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbf41d5-ea18-4814-8709-bb109d3e91e9 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:29.554569 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbf41d5-ea18-4814-8709-bb109d3e91e9 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:29.554847 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbf41d5-ea18-4814-8709-bb109d3e91e9 None admin] PATH_INFO: `/v3/projects/e7497d51ee8f4f0487b09541d4ad94d2` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:29.556042 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbf41d5-ea18-4814-8709-bb109d3e91e9 None admin] RBAC: Authorizing `identity:get_project(project_id=e7497d51ee8f4f0487b09541d4ad94d2)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:29.557502 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbf41d5-ea18-4814-8709-bb109d3e91e9 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:29.557615 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-cf6e2eb4-bd10-4543-be00-ddb679cffc41 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:23:29.557615 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:23:29.557615 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:23:29.557615 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:23:29.557615 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:23:29.557615 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:23:29.557615 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:23:29.557615 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:23:29.557615 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:23:29.557615 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:23:29.557615 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:23:29.557615 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:23:29.557615 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:23:29.557615 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:23:29.557615 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:23:29.557615 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:23:29.558603 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:23:29.558603 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:23:29.558603 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:23:29.558603 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:23:29.558603 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:23:29.558603 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:23:29.558603 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:23:29.558603 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:23:29.558603 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:23:29.558603 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:23:29.558603 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:23:29.558603 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:23:29.558603 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:23:29.558603 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:23:29.558603 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:23:29.558603 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:23:29.559490 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:23:29.559490 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:23:29.559490 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:23:29.559490 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:23:29.559490 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:23:29.559490 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:23:29.559490 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:23:29.559490 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:23:29.559490 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:23:29.559490 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:23:29.559490 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:23:29.559490 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:23:29.559490 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 59/117] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:23:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:23:29.560255 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02489fb-f33e-4404-895e-dd72d44c2a26 None None] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:29.560340 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02489fb-f33e-4404-895e-dd72d44c2a26 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:29.566094 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02489fb-f33e-4404-895e-dd72d44c2a26 None None] PATH_INFO: `/` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:29.566094 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 60/118] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:23:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:23:29.566094 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02489fb-f33e-4404-895e-dd72d44c2a26 None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:29.566778 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 59/119] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:23:29 2022] GET /identity/v3/projects/e7497d51ee8f4f0487b09541d4ad94d2 => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:29.566778 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbf41d5-ea18-4814-8709-bb109d3e91e9 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:29.598075 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:29.673718 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbf41d5-ea18-4814-8709-bb109d3e91e9 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:29.693686 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02489fb-f33e-4404-895e-dd72d44c2a26 None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:29.711306 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dcdfd8-fa8f-4bad-b46e-4fc868cffaa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:29.712110 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dcdfd8-fa8f-4bad-b46e-4fc868cffaa1 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:29.712394 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dcdfd8-fa8f-4bad-b46e-4fc868cffaa1 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:29.712668 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dcdfd8-fa8f-4bad-b46e-4fc868cffaa1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:29.723998 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dcdfd8-fa8f-4bad-b46e-4fc868cffaa1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:23:29.730122 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a21ccaf-1490-41e0-8ebf-c14ad330f7fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:29.731041 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a21ccaf-1490-41e0-8ebf-c14ad330f7fe None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:29.731335 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a21ccaf-1490-41e0-8ebf-c14ad330f7fe None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:29.731614 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a21ccaf-1490-41e0-8ebf-c14ad330f7fe None admin] PATH_INFO: `/v3/projects/e7497d51ee8f4f0487b09541d4ad94d2` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:29.732516 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-d7dcdfd8-fa8f-4bad-b46e-4fc868cffaa1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:23:29.732516 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:23:29.732516 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:23:29.732516 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:23:29.732516 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:23:29.732516 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:23:29.732516 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:23:29.732516 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:23:29.732516 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:23:29.732516 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:23:29.732516 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:23:29.732516 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:23:29.732516 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:23:29.732516 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:23:29.732516 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:23:29.732516 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:23:29.734249 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:23:29.734249 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:23:29.734249 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:23:29.734249 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:23:29.734249 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:23:29.734249 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:23:29.734249 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:23:29.734249 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:23:29.734249 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:23:29.734249 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:23:29.734249 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:23:29.734249 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:23:29.734249 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:23:29.734249 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:23:29.734249 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:23:29.734249 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:23:29.740193 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:23:29.740193 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:23:29.740193 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:23:29.740193 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:23:29.740193 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:23:29.740193 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:23:29.740193 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:23:29.740193 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:23:29.740193 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:23:29.740193 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:23:29.740193 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:23:29.740193 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:23:29.740883 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a21ccaf-1490-41e0-8ebf-c14ad330f7fe None admin] RBAC: Authorizing `identity:get_project(project_id=e7497d51ee8f4f0487b09541d4ad94d2)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:29.740963 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 60/120] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:23:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:23:29.741049 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a21ccaf-1490-41e0-8ebf-c14ad330f7fe None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:29.741133 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dcdfd8-fa8f-4bad-b46e-4fc868cffaa1 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:29.741214 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 61/121] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:23:29 2022] GET /identity/v3/projects/e7497d51ee8f4f0487b09541d4ad94d2 => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:29.742344 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a21ccaf-1490-41e0-8ebf-c14ad330f7fe None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:29.774285 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:29.844842 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:29.852679 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dcdfd8-fa8f-4bad-b46e-4fc868cffaa1 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:29.853181 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a21ccaf-1490-41e0-8ebf-c14ad330f7fe None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:29.889576 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec641ac7-4023-4ef1-a0dd-4cf29973c49c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:29.890275 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec641ac7-4023-4ef1-a0dd-4cf29973c49c None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:29.890562 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec641ac7-4023-4ef1-a0dd-4cf29973c49c None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:29.890864 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec641ac7-4023-4ef1-a0dd-4cf29973c49c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:29.891675 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec641ac7-4023-4ef1-a0dd-4cf29973c49c None admin] RBAC: Adding query filter params (role.id=abff822b2f9d4abd96ff7414f6de1670, scope.project.id=e7497d51ee8f4f0487b09541d4ad94d2, user.id=cd0294bae78b43ea80cea320f446e487) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:23:29.892040 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec641ac7-4023-4ef1-a0dd-4cf29973c49c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:29.893467 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec641ac7-4023-4ef1-a0dd-4cf29973c49c None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:29.894879 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c704a77-9eee-4f48-807b-da6a0c256f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:29.895557 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c704a77-9eee-4f48-807b-da6a0c256f0b None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:29.895840 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c704a77-9eee-4f48-807b-da6a0c256f0b None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:29.896117 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c704a77-9eee-4f48-807b-da6a0c256f0b None admin] PATH_INFO: `/v3/roles` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:29.896865 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c704a77-9eee-4f48-807b-da6a0c256f0b None admin] RBAC: Adding query filter params (name=member) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:23:29.897175 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c704a77-9eee-4f48-807b-da6a0c256f0b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:29.898596 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c704a77-9eee-4f48-807b-da6a0c256f0b None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:29.899423 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 61/122] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:23:29 2022] GET /identity/v3/role_assignments?user.id=cd0294bae78b43ea80cea320f446e487&scope.project.id=e7497d51ee8f4f0487b09541d4ad94d2&role.id=abff822b2f9d4abd96ff7414f6de1670 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:29.913725 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 62/123] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:23:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:29.913725 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c704a77-9eee-4f48-807b-da6a0c256f0b None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:29.914062 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ad8ec4-72fc-4350-bd49-0fa382e7cf5e None None] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:29.914062 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ad8ec4-72fc-4350-bd49-0fa382e7cf5e None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:29.914062 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ad8ec4-72fc-4350-bd49-0fa382e7cf5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:29.936869 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.auth.core [None req-76ad8ec4-72fc-4350-bd49-0fa382e7cf5e None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143587) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:23:29.938881 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-76ad8ec4-72fc-4350-bd49-0fa382e7cf5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:23:30.037070 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 62/124] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:23:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:23:30.038286 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ad8ec4-72fc-4350-bd49-0fa382e7cf5e None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:30.049801 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c704a77-9eee-4f48-807b-da6a0c256f0b None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:30.082699 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:30.115527 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2af7312-6c68-456c-8661-3c6fd6f6b614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:30.116323 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2af7312-6c68-456c-8661-3c6fd6f6b614 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:30.116642 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2af7312-6c68-456c-8661-3c6fd6f6b614 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:30.116890 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2af7312-6c68-456c-8661-3c6fd6f6b614 None admin] PATH_INFO: `/v3/roles` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:30.117648 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2af7312-6c68-456c-8661-3c6fd6f6b614 None admin] RBAC: Adding query filter params (name=member) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:23:30.117987 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2af7312-6c68-456c-8661-3c6fd6f6b614 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:30.119634 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2af7312-6c68-456c-8661-3c6fd6f6b614 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:30.130801 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 63/125] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:23:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:30.131642 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2af7312-6c68-456c-8661-3c6fd6f6b614 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:30.160065 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:30.170419 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ad8ec4-72fc-4350-bd49-0fa382e7cf5e None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:30.220305 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc330c4-f202-42c7-847a-ac367938333e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:30.221264 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc330c4-f202-42c7-847a-ac367938333e None admin] REQUEST_METHOD: `PUT` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:30.221629 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc330c4-f202-42c7-847a-ac367938333e None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:30.221914 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc330c4-f202-42c7-847a-ac367938333e None admin] PATH_INFO: `/v3/projects/e7497d51ee8f4f0487b09541d4ad94d2/users/21249d0b4cb3470e83a9f256cbe52f18/roles/b954b2c7ce764aa0afac78e075a011a6` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:30.224855 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc330c4-f202-42c7-847a-ac367938333e None admin] RBAC: Authorizing `identity:create_grant(project_id=e7497d51ee8f4f0487b09541d4ad94d2, user_id=21249d0b4cb3470e83a9f256cbe52f18, role_id=b954b2c7ce764aa0afac78e075a011a6)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:30.226015 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc330c4-f202-42c7-847a-ac367938333e None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:30.242669 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 63/126] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 14:23:30 2022] PUT /identity/v3/projects/e7497d51ee8f4f0487b09541d4ad94d2/users/21249d0b4cb3470e83a9f256cbe52f18/roles/b954b2c7ce764aa0afac78e075a011a6 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:23:30.243856 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc330c4-f202-42c7-847a-ac367938333e None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:30.273330 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2af7312-6c68-456c-8661-3c6fd6f6b614 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:30.343346 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e11c01-c61c-4daa-9f12-b8a876629f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:30.344533 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e11c01-c61c-4daa-9f12-b8a876629f53 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:30.345004 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e11c01-c61c-4daa-9f12-b8a876629f53 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:30.345460 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e11c01-c61c-4daa-9f12-b8a876629f53 None admin] PATH_INFO: `/v3/users/21249d0b4cb3470e83a9f256cbe52f18` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:30.346972 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e11c01-c61c-4daa-9f12-b8a876629f53 None admin] RBAC: Authorizing `identity:get_user(user_id=21249d0b4cb3470e83a9f256cbe52f18)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:30.348616 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e11c01-c61c-4daa-9f12-b8a876629f53 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:30.350709 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 64/127] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:23:30 2022] GET /identity/v3/users/21249d0b4cb3470e83a9f256cbe52f18 => generated 275 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:30.351679 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:30.355077 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e11c01-c61c-4daa-9f12-b8a876629f53 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:30.370393 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc330c4-f202-42c7-847a-ac367938333e None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:30.430478 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf9c596-46fc-4120-9acd-2c98aa79ac2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:30.431651 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf9c596-46fc-4120-9acd-2c98aa79ac2c None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:30.432150 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf9c596-46fc-4120-9acd-2c98aa79ac2c None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:30.432637 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf9c596-46fc-4120-9acd-2c98aa79ac2c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:30.450290 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf9c596-46fc-4120-9acd-2c98aa79ac2c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:23:30.459899 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-0bf9c596-46fc-4120-9acd-2c98aa79ac2c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:23:30.459899 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:23:30.459899 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:23:30.459899 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:23:30.459899 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:23:30.459899 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:23:30.459899 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:23:30.459899 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:23:30.459899 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:23:30.459899 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:23:30.459899 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:23:30.459899 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:23:30.459899 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:23:30.459899 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:23:30.459899 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:23:30.459899 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:23:30.461002 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:23:30.461002 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:23:30.461002 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:23:30.461002 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:23:30.461002 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:23:30.461002 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:23:30.461002 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:23:30.461002 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:23:30.461002 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:23:30.461002 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:23:30.461002 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:23:30.461002 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:23:30.461002 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:23:30.461002 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:23:30.461002 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:23:30.461002 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:23:30.461925 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:23:30.461925 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:23:30.461925 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:23:30.461925 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:23:30.461925 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:23:30.461925 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:23:30.461925 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:23:30.461925 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:23:30.461925 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:23:30.461925 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:23:30.461925 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:23:30.461925 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:23:30.465023 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 64/128] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:23:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:23:30.470284 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf9c596-46fc-4120-9acd-2c98aa79ac2c None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:30.507211 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:30.556005 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e11c01-c61c-4daa-9f12-b8a876629f53 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:30.594284 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf9c596-46fc-4120-9acd-2c98aa79ac2c None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:30.599875 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13062b7c-e08d-4751-98e6-946bb49d4207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:30.601083 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13062b7c-e08d-4751-98e6-946bb49d4207 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:30.601378 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13062b7c-e08d-4751-98e6-946bb49d4207 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:30.601660 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13062b7c-e08d-4751-98e6-946bb49d4207 None admin] PATH_INFO: `/v3/users/21249d0b4cb3470e83a9f256cbe52f18` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:30.602981 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13062b7c-e08d-4751-98e6-946bb49d4207 None admin] RBAC: Authorizing `identity:get_user(user_id=21249d0b4cb3470e83a9f256cbe52f18)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:30.604483 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13062b7c-e08d-4751-98e6-946bb49d4207 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:30.606425 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 65/129] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:23:30 2022] GET /identity/v3/users/21249d0b4cb3470e83a9f256cbe52f18 => generated 275 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:30.607118 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13062b7c-e08d-4751-98e6-946bb49d4207 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:30.640196 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9935b7-9a86-43c6-bc58-4ca7612dd723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:30.641736 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9935b7-9a86-43c6-bc58-4ca7612dd723 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:30.642436 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9935b7-9a86-43c6-bc58-4ca7612dd723 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:30.642921 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9935b7-9a86-43c6-bc58-4ca7612dd723 None admin] PATH_INFO: `/v3/projects/e7497d51ee8f4f0487b09541d4ad94d2` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:30.644410 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9935b7-9a86-43c6-bc58-4ca7612dd723 None admin] RBAC: Authorizing `identity:get_project(project_id=e7497d51ee8f4f0487b09541d4ad94d2)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:30.646210 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9935b7-9a86-43c6-bc58-4ca7612dd723 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:30.648189 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 65/130] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:23:30 2022] GET /identity/v3/projects/e7497d51ee8f4f0487b09541d4ad94d2 => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:30.649307 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9935b7-9a86-43c6-bc58-4ca7612dd723 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:30.677814 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:30.719498 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13062b7c-e08d-4751-98e6-946bb49d4207 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:30.757317 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef80c9a8-c834-4126-8fd7-849e3ebd01bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:30.758900 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef80c9a8-c834-4126-8fd7-849e3ebd01bf None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:30.759249 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef80c9a8-c834-4126-8fd7-849e3ebd01bf None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:30.759543 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef80c9a8-c834-4126-8fd7-849e3ebd01bf None admin] PATH_INFO: `/v3/roles` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:30.760313 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef80c9a8-c834-4126-8fd7-849e3ebd01bf None admin] RBAC: Adding query filter params (name=admin) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:23:30.760630 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef80c9a8-c834-4126-8fd7-849e3ebd01bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:30.762047 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef80c9a8-c834-4126-8fd7-849e3ebd01bf None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:30.776356 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 66/131] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:23:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:30.776356 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef80c9a8-c834-4126-8fd7-849e3ebd01bf None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:30.790748 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9935b7-9a86-43c6-bc58-4ca7612dd723 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:30.805244 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:30.890621 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef80c9a8-c834-4126-8fd7-849e3ebd01bf None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:30.923155 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64542606-6b3f-4d8e-84e8-c41d9ff42fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:30.924158 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64542606-6b3f-4d8e-84e8-c41d9ff42fcd None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:30.924444 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64542606-6b3f-4d8e-84e8-c41d9ff42fcd None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:30.924728 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64542606-6b3f-4d8e-84e8-c41d9ff42fcd None admin] PATH_INFO: `/v3/projects/5c7498adefdf4ff59ace12ff1685bd25` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:30.926176 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64542606-6b3f-4d8e-84e8-c41d9ff42fcd None admin] RBAC: Authorizing `identity:get_project(project_id=5c7498adefdf4ff59ace12ff1685bd25)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:30.927352 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0f1f0d-882a-48f0-a124-21abad38403a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:30.927653 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64542606-6b3f-4d8e-84e8-c41d9ff42fcd None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:30.928570 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0f1f0d-882a-48f0-a124-21abad38403a None admin] REQUEST_METHOD: `PUT` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:30.928999 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0f1f0d-882a-48f0-a124-21abad38403a None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:30.929415 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 66/132] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:23:30 2022] GET /identity/v3/projects/5c7498adefdf4ff59ace12ff1685bd25 => generated 312 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:30.929666 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0f1f0d-882a-48f0-a124-21abad38403a None admin] PATH_INFO: `/v3/projects/e7497d51ee8f4f0487b09541d4ad94d2/users/21249d0b4cb3470e83a9f256cbe52f18/roles/3d5075293a4f45449eaef6af8fa53004` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:30.931894 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0f1f0d-882a-48f0-a124-21abad38403a None admin] RBAC: Authorizing `identity:create_grant(project_id=e7497d51ee8f4f0487b09541d4ad94d2, user_id=21249d0b4cb3470e83a9f256cbe52f18, role_id=3d5075293a4f45449eaef6af8fa53004)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:30.933398 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0f1f0d-882a-48f0-a124-21abad38403a None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:30.938377 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64542606-6b3f-4d8e-84e8-c41d9ff42fcd None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:30.955307 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 67/133] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 14:23:30 2022] PUT /identity/v3/projects/e7497d51ee8f4f0487b09541d4ad94d2/users/21249d0b4cb3470e83a9f256cbe52f18/roles/3d5075293a4f45449eaef6af8fa53004 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:23:30.956248 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0f1f0d-882a-48f0-a124-21abad38403a None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:31.136990 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:31.141144 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:31.150433 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0f1f0d-882a-48f0-a124-21abad38403a None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:31.185339 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64542606-6b3f-4d8e-84e8-c41d9ff42fcd None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:31.194973 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7c96e4-84d9-4ba0-9c46-1a8742c67418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:31.196325 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7c96e4-84d9-4ba0-9c46-1a8742c67418 None admin] REQUEST_METHOD: `PUT` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:31.196856 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7c96e4-84d9-4ba0-9c46-1a8742c67418 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:31.197358 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7c96e4-84d9-4ba0-9c46-1a8742c67418 None admin] PATH_INFO: `/v3/projects/5c7498adefdf4ff59ace12ff1685bd25/users/21249d0b4cb3470e83a9f256cbe52f18/roles/3d5075293a4f45449eaef6af8fa53004` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:31.200664 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7c96e4-84d9-4ba0-9c46-1a8742c67418 None admin] RBAC: Authorizing `identity:create_grant(project_id=5c7498adefdf4ff59ace12ff1685bd25, user_id=21249d0b4cb3470e83a9f256cbe52f18, role_id=3d5075293a4f45449eaef6af8fa53004)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:31.202905 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7c96e4-84d9-4ba0-9c46-1a8742c67418 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:31.221393 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 68/134] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 14:23:30 2022] PUT /identity/v3/projects/5c7498adefdf4ff59ace12ff1685bd25/users/21249d0b4cb3470e83a9f256cbe52f18/roles/3d5075293a4f45449eaef6af8fa53004 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:23:31.279454 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2a49e7-a63c-41e9-adce-5ae9e06a0ac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:31.280439 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2a49e7-a63c-41e9-adce-5ae9e06a0ac3 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:31.280729 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2a49e7-a63c-41e9-adce-5ae9e06a0ac3 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:31.281013 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2a49e7-a63c-41e9-adce-5ae9e06a0ac3 None admin] PATH_INFO: `/v3/users/4f2f0290e1b04ce48a90ce03cb93d060` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:31.282470 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2a49e7-a63c-41e9-adce-5ae9e06a0ac3 None admin] RBAC: Authorizing `identity:get_user(user_id=4f2f0290e1b04ce48a90ce03cb93d060)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:31.284018 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2a49e7-a63c-41e9-adce-5ae9e06a0ac3 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:31.287215 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 67/135] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:23:30 2022] GET /identity/v3/users/4f2f0290e1b04ce48a90ce03cb93d060 => generated 247 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:31.291509 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7c96e4-84d9-4ba0-9c46-1a8742c67418 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:31.351166 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2447dbe9-af80-4cf2-a8e7-d65b0c9c1d63 None None] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:31.351531 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2447dbe9-af80-4cf2-a8e7-d65b0c9c1d63 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:31.351782 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2447dbe9-af80-4cf2-a8e7-d65b0c9c1d63 None None] PATH_INFO: `/` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:31.353878 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 68/136] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:23:31 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:23:31.363405 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e0bff1-0ce4-4b5c-9f15-66a6c295083a None None] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:31.363745 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e0bff1-0ce4-4b5c-9f15-66a6c295083a None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:31.363978 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e0bff1-0ce4-4b5c-9f15-66a6c295083a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:31.398570 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.auth.core [None req-96e0bff1-0ce4-4b5c-9f15-66a6c295083a None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143587) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:23:31.399729 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-96e0bff1-0ce4-4b5c-9f15-66a6c295083a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:23:31.444963 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7c96e4-84d9-4ba0-9c46-1a8742c67418 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:31.488378 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5bb64b-37af-44d8-972c-20907d802052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:31.489665 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5bb64b-37af-44d8-972c-20907d802052 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:31.490286 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5bb64b-37af-44d8-972c-20907d802052 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:31.490782 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5bb64b-37af-44d8-972c-20907d802052 None admin] PATH_INFO: `/v3/projects/e7497d51ee8f4f0487b09541d4ad94d2` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:31.492592 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5bb64b-37af-44d8-972c-20907d802052 None admin] RBAC: Authorizing `identity:get_project(project_id=e7497d51ee8f4f0487b09541d4ad94d2)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:31.494416 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5bb64b-37af-44d8-972c-20907d802052 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:31.496935 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 69/137] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:23:31 2022] GET /identity/v3/projects/e7497d51ee8f4f0487b09541d4ad94d2 => generated 298 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:31.508669 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5bb64b-37af-44d8-972c-20907d802052 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:31.514177 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:31.595294 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 69/138] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:23:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:23:31.630295 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e0bff1-0ce4-4b5c-9f15-66a6c295083a None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:31.633621 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5bb64b-37af-44d8-972c-20907d802052 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:31.684299 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44f5f13-83c8-48a5-9c3a-c31c6c76a347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:31.685397 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44f5f13-83c8-48a5-9c3a-c31c6c76a347 None admin] REQUEST_METHOD: `PUT` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:31.685758 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44f5f13-83c8-48a5-9c3a-c31c6c76a347 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:31.686729 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44f5f13-83c8-48a5-9c3a-c31c6c76a347 None admin] PATH_INFO: `/v3/projects/e7497d51ee8f4f0487b09541d4ad94d2/users/4f2f0290e1b04ce48a90ce03cb93d060/roles/a3b793754e7b40b0bb21afe74eec3c18` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:31.692183 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44f5f13-83c8-48a5-9c3a-c31c6c76a347 None admin] RBAC: Authorizing `identity:create_grant(project_id=e7497d51ee8f4f0487b09541d4ad94d2, user_id=4f2f0290e1b04ce48a90ce03cb93d060, role_id=a3b793754e7b40b0bb21afe74eec3c18)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:31.693782 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44f5f13-83c8-48a5-9c3a-c31c6c76a347 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:31.727223 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 70/139] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 14:23:31 2022] PUT /identity/v3/projects/e7497d51ee8f4f0487b09541d4ad94d2/users/4f2f0290e1b04ce48a90ce03cb93d060/roles/a3b793754e7b40b0bb21afe74eec3c18 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 14:23:31.774044 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:31.803412 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e0bff1-0ce4-4b5c-9f15-66a6c295083a None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:31.843109 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b863923e-29a7-4d3f-acf1-839b3155fcc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:31.843940 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b863923e-29a7-4d3f-acf1-839b3155fcc8 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:31.844222 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b863923e-29a7-4d3f-acf1-839b3155fcc8 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:31.844501 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b863923e-29a7-4d3f-acf1-839b3155fcc8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:31.845748 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b863923e-29a7-4d3f-acf1-839b3155fcc8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:31.847332 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b863923e-29a7-4d3f-acf1-839b3155fcc8 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:31.849115 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 70/140] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:23:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:31.855282 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44f5f13-83c8-48a5-9c3a-c31c6c76a347 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:31.950101 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:32.007584 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44f5f13-83c8-48a5-9c3a-c31c6c76a347 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:32.059543 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88120818-37ff-4f5d-8ce5-426ae66f1106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:32.060235 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88120818-37ff-4f5d-8ce5-426ae66f1106 None admin] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:32.060522 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88120818-37ff-4f5d-8ce5-426ae66f1106 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:32.060802 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88120818-37ff-4f5d-8ce5-426ae66f1106 None admin] PATH_INFO: `/v3/users` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:32.062219 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88120818-37ff-4f5d-8ce5-426ae66f1106 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:32.063806 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88120818-37ff-4f5d-8ce5-426ae66f1106 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:32.099166 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 71/141] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 19 14:23:31 2022] POST /identity/v3/users => generated 283 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:23:32.787909 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c83db2-6796-478c-b6b5-58c824cab0c2 None None] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:32.788422 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c83db2-6796-478c-b6b5-58c824cab0c2 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:32.788524 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c83db2-6796-478c-b6b5-58c824cab0c2 None None] PATH_INFO: `/` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:32.789878 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 71/142] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:23:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:23:32.795616 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdd6373-aa8f-4569-b6f5-7a47a99ef7a5 None None] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:32.795929 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdd6373-aa8f-4569-b6f5-7a47a99ef7a5 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:32.796306 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdd6373-aa8f-4569-b6f5-7a47a99ef7a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:32.817979 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.auth.core [None req-bbdd6373-aa8f-4569-b6f5-7a47a99ef7a5 None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143586) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:23:32.819245 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.fernet_utils [None req-bbdd6373-aa8f-4569-b6f5-7a47a99ef7a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143586) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:23:32.916210 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 72/143] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:23:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:23:32.950748 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c83db2-6796-478c-b6b5-58c824cab0c2 None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:33.135140 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c83db2-6796-478c-b6b5-58c824cab0c2 None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:33.188543 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed85eb2-0422-4ff8-893b-7842d4d1f10a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:33.189290 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed85eb2-0422-4ff8-893b-7842d4d1f10a None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:33.189579 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed85eb2-0422-4ff8-893b-7842d4d1f10a None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:33.189904 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed85eb2-0422-4ff8-893b-7842d4d1f10a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:33.200715 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed85eb2-0422-4ff8-893b-7842d4d1f10a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:23:33.210263 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-0ed85eb2-0422-4ff8-893b-7842d4d1f10a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:23:33.210263 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:23:33.210263 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:23:33.210263 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:23:33.210263 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:23:33.210263 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:23:33.210263 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:23:33.210263 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:23:33.210263 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:23:33.210263 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:23:33.210263 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:23:33.210263 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:23:33.210263 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:23:33.210263 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:23:33.210263 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:23:33.210263 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:23:33.211310 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:23:33.211310 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:23:33.211310 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:23:33.211310 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:23:33.211310 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:23:33.211310 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:23:33.211310 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:23:33.211310 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:23:33.211310 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:23:33.211310 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:23:33.211310 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:23:33.211310 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:23:33.211310 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:23:33.211310 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:23:33.211310 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:23:33.211310 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:23:33.212214 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:23:33.212214 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:23:33.212214 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:23:33.212214 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:23:33.212214 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:23:33.212214 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:23:33.212214 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:23:33.212214 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:23:33.212214 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:23:33.212214 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:23:33.212214 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:23:33.212214 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:23:33.212214 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 72/144] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:23:32 2022] GET /identity/v3/roles/reader => generated 84 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:23:33.218776 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdd6373-aa8f-4569-b6f5-7a47a99ef7a5 None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:33.292013 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:33.355771 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdd6373-aa8f-4569-b6f5-7a47a99ef7a5 None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:33.398329 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c26b57-fb60-4e68-8477-6fb6ac14d503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:33.399015 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c26b57-fb60-4e68-8477-6fb6ac14d503 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:33.399297 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c26b57-fb60-4e68-8477-6fb6ac14d503 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:33.399576 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c26b57-fb60-4e68-8477-6fb6ac14d503 None admin] PATH_INFO: `/v3/roles` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:33.400337 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c26b57-fb60-4e68-8477-6fb6ac14d503 None admin] RBAC: Adding query filter params (name=reader) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:23:33.400647 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c26b57-fb60-4e68-8477-6fb6ac14d503 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:33.402054 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c26b57-fb60-4e68-8477-6fb6ac14d503 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:33.412510 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 73/145] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:23:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:33.418538 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed85eb2-0422-4ff8-893b-7842d4d1f10a None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:33.436202 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:33.542337 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed85eb2-0422-4ff8-893b-7842d4d1f10a None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:33.586553 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d4d988-94cf-464a-b70e-ac27051c34fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:33.587762 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d4d988-94cf-464a-b70e-ac27051c34fd None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:33.588236 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d4d988-94cf-464a-b70e-ac27051c34fd None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:33.588702 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d4d988-94cf-464a-b70e-ac27051c34fd None admin] PATH_INFO: `/v3/users/cd0294bae78b43ea80cea320f446e487` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:33.590186 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d4d988-94cf-464a-b70e-ac27051c34fd None admin] RBAC: Authorizing `identity:get_user(user_id=cd0294bae78b43ea80cea320f446e487)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:33.591941 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d4d988-94cf-464a-b70e-ac27051c34fd None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:33.594933 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 73/146] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:23:33 2022] GET /identity/v3/users/cd0294bae78b43ea80cea320f446e487 => generated 289 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:33.601052 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c26b57-fb60-4e68-8477-6fb6ac14d503 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:33.614304 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c4c35f-b656-44b7-b116-dd9df2a3867b None None] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:33.614887 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c4c35f-b656-44b7-b116-dd9df2a3867b None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:33.615388 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c4c35f-b656-44b7-b116-dd9df2a3867b None None] PATH_INFO: `/` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:33.622448 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 74/147] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:23:33 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:23:33.630768 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e834ca1-d58c-4cd9-ae42-f0eb58d703fd None None] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:33.631375 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e834ca1-d58c-4cd9-ae42-f0eb58d703fd None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:33.631855 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e834ca1-d58c-4cd9-ae42-f0eb58d703fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:33.658064 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.auth.core [None req-3e834ca1-d58c-4cd9-ae42-f0eb58d703fd None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143587) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:23:33.659760 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-3e834ca1-d58c-4cd9-ae42-f0eb58d703fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:23:33.781067 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c26b57-fb60-4e68-8477-6fb6ac14d503 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:33.788642 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 75/148] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:23:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:23:33.801120 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:33.825726 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e834ca1-d58c-4cd9-ae42-f0eb58d703fd None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:33.829103 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b2bf8c-aa82-41a7-a0c9-d47972011e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:33.834550 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b2bf8c-aa82-41a7-a0c9-d47972011e2d None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:33.834829 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b2bf8c-aa82-41a7-a0c9-d47972011e2d None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:33.835115 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b2bf8c-aa82-41a7-a0c9-d47972011e2d None admin] PATH_INFO: `/v3/projects/e7497d51ee8f4f0487b09541d4ad94d2` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:33.835247 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:33.838573 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b2bf8c-aa82-41a7-a0c9-d47972011e2d None admin] RBAC: Authorizing `identity:get_project(project_id=e7497d51ee8f4f0487b09541d4ad94d2)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:33.840053 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b2bf8c-aa82-41a7-a0c9-d47972011e2d None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:33.844772 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 74/149] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:23:33 2022] GET /identity/v3/projects/e7497d51ee8f4f0487b09541d4ad94d2 => generated 298 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:33.859579 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b2bf8c-aa82-41a7-a0c9-d47972011e2d None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:34.019610 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e834ca1-d58c-4cd9-ae42-f0eb58d703fd None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:34.070243 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b2bf8c-aa82-41a7-a0c9-d47972011e2d None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:34.095300 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74594267-d27b-47b0-8087-7580840a18ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:34.096277 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74594267-d27b-47b0-8087-7580840a18ef None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:34.096582 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74594267-d27b-47b0-8087-7580840a18ef None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:34.096871 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74594267-d27b-47b0-8087-7580840a18ef None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:34.116934 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74594267-d27b-47b0-8087-7580840a18ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:23:34.129928 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-74594267-d27b-47b0-8087-7580840a18ef None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:23:34.129928 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:23:34.129928 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:23:34.129928 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:23:34.129928 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:23:34.129928 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:23:34.129928 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:23:34.129928 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:23:34.129928 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:23:34.129928 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:23:34.129928 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:23:34.129928 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:23:34.129928 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:23:34.129928 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:23:34.129928 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:23:34.129928 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:23:34.130988 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:23:34.130988 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:23:34.130988 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:23:34.130988 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:23:34.130988 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:23:34.130988 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:23:34.130988 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:23:34.130988 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:23:34.130988 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:23:34.130988 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:23:34.130988 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:23:34.130988 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:23:34.130988 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:23:34.130988 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:23:34.130988 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:23:34.130988 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:23:34.131884 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:23:34.131884 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:23:34.131884 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:23:34.131884 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:23:34.131884 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:23:34.131884 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:23:34.131884 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:23:34.131884 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:23:34.131884 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:23:34.131884 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:23:34.131884 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:23:34.131884 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:23:34.132564 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0f49b2-1ccd-4f10-84d7-dd1d5b56210f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:34.132564 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0f49b2-1ccd-4f10-84d7-dd1d5b56210f None admin] REQUEST_METHOD: `PUT` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:34.132731 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0f49b2-1ccd-4f10-84d7-dd1d5b56210f None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:34.133337 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0f49b2-1ccd-4f10-84d7-dd1d5b56210f None admin] PATH_INFO: `/v3/projects/e7497d51ee8f4f0487b09541d4ad94d2/users/cd0294bae78b43ea80cea320f446e487/roles/abff822b2f9d4abd96ff7414f6de1670` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:34.134953 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0f49b2-1ccd-4f10-84d7-dd1d5b56210f None admin] RBAC: Authorizing `identity:create_grant(project_id=e7497d51ee8f4f0487b09541d4ad94d2, user_id=cd0294bae78b43ea80cea320f446e487, role_id=abff822b2f9d4abd96ff7414f6de1670)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:34.135411 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 76/150] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 19 14:23:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:23:34.137087 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0f49b2-1ccd-4f10-84d7-dd1d5b56210f None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:34.141195 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74594267-d27b-47b0-8087-7580840a18ef None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:34.165506 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 75/151] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 14:23:33 2022] PUT /identity/v3/projects/e7497d51ee8f4f0487b09541d4ad94d2/users/cd0294bae78b43ea80cea320f446e487/roles/abff822b2f9d4abd96ff7414f6de1670 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:23:34.263811 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:34.306146 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74594267-d27b-47b0-8087-7580840a18ef None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:34.353195 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa485ae-daff-4589-a5a7-89c9b0ed1e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:34.354281 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa485ae-daff-4589-a5a7-89c9b0ed1e24 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:34.354774 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa485ae-daff-4589-a5a7-89c9b0ed1e24 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:34.355276 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa485ae-daff-4589-a5a7-89c9b0ed1e24 None admin] PATH_INFO: `/v3/roles` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:34.356253 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa485ae-daff-4589-a5a7-89c9b0ed1e24 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:23:34.356785 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa485ae-daff-4589-a5a7-89c9b0ed1e24 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:34.358971 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa485ae-daff-4589-a5a7-89c9b0ed1e24 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:34.374210 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 77/152] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:23:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:34.380074 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0f49b2-1ccd-4f10-84d7-dd1d5b56210f None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:34.459236 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:34.541139 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0f49b2-1ccd-4f10-84d7-dd1d5b56210f None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:34.582924 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b869d1-c53f-4836-9ec4-4520e8eb750a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:34.584203 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b869d1-c53f-4836-9ec4-4520e8eb750a None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:34.584724 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b869d1-c53f-4836-9ec4-4520e8eb750a None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:34.585218 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b869d1-c53f-4836-9ec4-4520e8eb750a None admin] PATH_INFO: `/v3/users/21249d0b4cb3470e83a9f256cbe52f18` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:34.586716 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b869d1-c53f-4836-9ec4-4520e8eb750a None admin] RBAC: Authorizing `identity:get_user(user_id=21249d0b4cb3470e83a9f256cbe52f18)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:34.588394 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b869d1-c53f-4836-9ec4-4520e8eb750a None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:34.590309 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 76/153] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:23:34 2022] GET /identity/v3/users/21249d0b4cb3470e83a9f256cbe52f18 => generated 275 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:34.596702 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa485ae-daff-4589-a5a7-89c9b0ed1e24 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:34.613405 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a7f393-161f-4f0e-b190-3d3b5df7991a None None] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:34.614069 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a7f393-161f-4f0e-b190-3d3b5df7991a None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:34.614589 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a7f393-161f-4f0e-b190-3d3b5df7991a None None] PATH_INFO: `/` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:34.616212 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 77/154] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:23:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:23:34.631733 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6844df-1d0b-41bc-9fa2-abcc9c85ad28 None None] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:34.632388 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6844df-1d0b-41bc-9fa2-abcc9c85ad28 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:34.632877 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6844df-1d0b-41bc-9fa2-abcc9c85ad28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:34.665234 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.auth.core [None req-5d6844df-1d0b-41bc-9fa2-abcc9c85ad28 None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143586) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:23:34.666837 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.fernet_utils [None req-5d6844df-1d0b-41bc-9fa2-abcc9c85ad28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143586) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:23:34.744881 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 78/155] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:23:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:23:34.776414 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa485ae-daff-4589-a5a7-89c9b0ed1e24 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:34.805724 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6844df-1d0b-41bc-9fa2-abcc9c85ad28 None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:34.810977 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:34.825800 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0ae9f7-6a59-4674-85f4-34aca2b63924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:34.826510 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0ae9f7-6a59-4674-85f4-34aca2b63924 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:34.827325 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0ae9f7-6a59-4674-85f4-34aca2b63924 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:34.827611 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0ae9f7-6a59-4674-85f4-34aca2b63924 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:34.866320 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:34.926925 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6844df-1d0b-41bc-9fa2-abcc9c85ad28 None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:34.964640 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b25c0c3-ca9b-413e-93cf-288b89071427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:34.965919 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b25c0c3-ca9b-413e-93cf-288b89071427 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:34.966444 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b25c0c3-ca9b-413e-93cf-288b89071427 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:34.966930 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b25c0c3-ca9b-413e-93cf-288b89071427 None admin] PATH_INFO: `/v3/roles/member` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:34.979404 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b25c0c3-ca9b-413e-93cf-288b89071427 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:23:34.987836 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-1b25c0c3-ca9b-413e-93cf-288b89071427 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:23:34.987836 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:23:34.987836 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:23:34.987836 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:23:34.987836 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:23:34.987836 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:23:34.987836 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:23:34.987836 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:23:34.987836 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:23:34.987836 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:23:34.987836 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:23:34.987836 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:23:34.987836 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:23:34.987836 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:23:34.987836 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:23:34.987836 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:23:34.988881 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:23:34.988881 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:23:34.988881 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:23:34.988881 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:23:34.988881 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:23:34.988881 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:23:34.988881 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:23:34.988881 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:23:34.988881 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:23:34.988881 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:23:34.988881 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:23:34.988881 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:23:34.988881 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:23:34.988881 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:23:34.988881 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:23:34.988881 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:23:34.989780 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:23:34.989780 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:23:34.989780 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:23:34.989780 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:23:34.989780 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:23:34.989780 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:23:34.989780 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:23:34.989780 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:23:34.989780 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:23:34.989780 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:23:34.989780 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:23:34.989780 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:23:34.992433 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 79/156] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:23:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:23:34.997867 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0ae9f7-6a59-4674-85f4-34aca2b63924 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:35.002585 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0ae9f7-6a59-4674-85f4-34aca2b63924 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:35.003398 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b25c0c3-ca9b-413e-93cf-288b89071427 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:35.138489 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 78/157] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:23:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 542 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:23:35.144569 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0ae9f7-6a59-4674-85f4-34aca2b63924 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:35.194228 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:35.253266 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0ae9f7-6a59-4674-85f4-34aca2b63924 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:35.290538 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126a9542-2346-428c-a4d8-41bee5186970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:35.291429 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126a9542-2346-428c-a4d8-41bee5186970 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:35.291715 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126a9542-2346-428c-a4d8-41bee5186970 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:35.292324 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126a9542-2346-428c-a4d8-41bee5186970 None admin] PATH_INFO: `/v3/projects/e7497d51ee8f4f0487b09541d4ad94d2` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:35.293329 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126a9542-2346-428c-a4d8-41bee5186970 None admin] RBAC: Authorizing `identity:get_project(project_id=e7497d51ee8f4f0487b09541d4ad94d2)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:35.294826 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126a9542-2346-428c-a4d8-41bee5186970 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:35.301996 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 79/158] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:23:35 2022] GET /identity/v3/projects/e7497d51ee8f4f0487b09541d4ad94d2 => generated 298 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:35.303252 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126a9542-2346-428c-a4d8-41bee5186970 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:35.386223 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:35.391917 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b25c0c3-ca9b-413e-93cf-288b89071427 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:35.406386 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:35.414541 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126a9542-2346-428c-a4d8-41bee5186970 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:35.467585 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8114605f-a4e7-4da4-aeff-f4fab66f5ef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:35.468127 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8114605f-a4e7-4da4-aeff-f4fab66f5ef7 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:35.468417 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8114605f-a4e7-4da4-aeff-f4fab66f5ef7 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:35.468697 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8114605f-a4e7-4da4-aeff-f4fab66f5ef7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:35.469570 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8114605f-a4e7-4da4-aeff-f4fab66f5ef7 None admin] RBAC: Adding query filter params (role.id=b954b2c7ce764aa0afac78e075a011a6, scope.project.id=e7497d51ee8f4f0487b09541d4ad94d2, user.id=21249d0b4cb3470e83a9f256cbe52f18) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:23:35.469913 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8114605f-a4e7-4da4-aeff-f4fab66f5ef7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:35.471475 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8114605f-a4e7-4da4-aeff-f4fab66f5ef7 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:35.478578 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 80/159] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:23:35 2022] GET /identity/v3/role_assignments?user.id=21249d0b4cb3470e83a9f256cbe52f18&scope.project.id=e7497d51ee8f4f0487b09541d4ad94d2&role.id=b954b2c7ce764aa0afac78e075a011a6 => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:23:35.509539 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d6655a-d27a-4a13-9dc6-8099805bbe46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:35.510270 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d6655a-d27a-4a13-9dc6-8099805bbe46 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:35.510558 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d6655a-d27a-4a13-9dc6-8099805bbe46 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:35.510842 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d6655a-d27a-4a13-9dc6-8099805bbe46 None admin] PATH_INFO: `/v3/roles` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:35.511617 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d6655a-d27a-4a13-9dc6-8099805bbe46 None admin] RBAC: Adding query filter params (name=member) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:23:35.511933 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d6655a-d27a-4a13-9dc6-8099805bbe46 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:35.513331 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d6655a-d27a-4a13-9dc6-8099805bbe46 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:35.527159 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 80/160] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:23:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 525 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:23:35.533298 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8114605f-a4e7-4da4-aeff-f4fab66f5ef7 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:35.669897 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8114605f-a4e7-4da4-aeff-f4fab66f5ef7 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:35.704814 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b69db6-2faa-447c-8ddd-360f62d8183f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:35.705715 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b69db6-2faa-447c-8ddd-360f62d8183f None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:35.706030 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b69db6-2faa-447c-8ddd-360f62d8183f None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:35.706307 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b69db6-2faa-447c-8ddd-360f62d8183f None admin] PATH_INFO: `/v3/users/21249d0b4cb3470e83a9f256cbe52f18` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:35.707555 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b69db6-2faa-447c-8ddd-360f62d8183f None admin] RBAC: Authorizing `identity:get_user(user_id=21249d0b4cb3470e83a9f256cbe52f18)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:35.709029 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b69db6-2faa-447c-8ddd-360f62d8183f None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:35.711166 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 81/161] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:23:35 2022] GET /identity/v3/users/21249d0b4cb3470e83a9f256cbe52f18 => generated 275 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:35.719044 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d6655a-d27a-4a13-9dc6-8099805bbe46 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:35.787057 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bd266a-d6b4-4189-b108-32f48a54426a None None] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:35.787463 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bd266a-d6b4-4189-b108-32f48a54426a None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:35.787753 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bd266a-d6b4-4189-b108-32f48a54426a None None] PATH_INFO: `/` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:35.789149 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 82/162] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:23:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:23:35.807809 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e107ef7c-17aa-4ab2-9f64-c2e44666b9e1 None None] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:35.808159 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e107ef7c-17aa-4ab2-9f64-c2e44666b9e1 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:35.808375 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e107ef7c-17aa-4ab2-9f64-c2e44666b9e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:35.835271 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.auth.core [None req-e107ef7c-17aa-4ab2-9f64-c2e44666b9e1 None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143587) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:23:35.836517 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-e107ef7c-17aa-4ab2-9f64-c2e44666b9e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:23:35.875849 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:35.916163 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d6655a-d27a-4a13-9dc6-8099805bbe46 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:35.946110 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 83/163] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:23:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:23:35.979741 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6458a2cd-4df8-4a47-a68b-bea6bde3f9ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:35.980406 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6458a2cd-4df8-4a47-a68b-bea6bde3f9ae None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:35.980685 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6458a2cd-4df8-4a47-a68b-bea6bde3f9ae None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:35.980955 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6458a2cd-4df8-4a47-a68b-bea6bde3f9ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:35.999559 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e107ef7c-17aa-4ab2-9f64-c2e44666b9e1 None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:36.015000 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:36.085872 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6458a2cd-4df8-4a47-a68b-bea6bde3f9ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:36.085872 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6458a2cd-4df8-4a47-a68b-bea6bde3f9ae None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:36.199725 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e107ef7c-17aa-4ab2-9f64-c2e44666b9e1 None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:36.211627 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 81/164] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:23:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 493 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:23:36.219888 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6458a2cd-4df8-4a47-a68b-bea6bde3f9ae None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:36.250190 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6610cc4f-e086-40d2-8c75-3ce38090ac9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:36.251349 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6610cc4f-e086-40d2-8c75-3ce38090ac9a None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:36.251650 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6610cc4f-e086-40d2-8c75-3ce38090ac9a None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:36.251916 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6610cc4f-e086-40d2-8c75-3ce38090ac9a None admin] PATH_INFO: `/v3/roles/member` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:36.281373 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6610cc4f-e086-40d2-8c75-3ce38090ac9a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:23:36.289546 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-6610cc4f-e086-40d2-8c75-3ce38090ac9a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:23:36.289546 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:23:36.289546 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:23:36.289546 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:23:36.289546 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:23:36.289546 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:23:36.289546 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:23:36.289546 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:23:36.289546 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:23:36.289546 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:23:36.289546 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:23:36.289546 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:23:36.289546 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:23:36.289546 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:23:36.289546 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:23:36.289546 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:23:36.290601 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:23:36.290601 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:23:36.290601 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:23:36.290601 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:23:36.290601 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:23:36.290601 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:23:36.290601 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:23:36.290601 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:23:36.290601 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:23:36.290601 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:23:36.290601 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:23:36.290601 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:23:36.290601 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:23:36.290601 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:23:36.290601 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:23:36.290601 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:23:36.291535 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:23:36.291535 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:23:36.291535 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:23:36.291535 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:23:36.291535 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:23:36.291535 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:23:36.291535 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:23:36.291535 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:23:36.291535 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:23:36.291535 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:23:36.291535 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:23:36.291535 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:23:36.291535 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 84/165] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:23:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:23:36.297522 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6610cc4f-e086-40d2-8c75-3ce38090ac9a None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:36.341265 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:36.351507 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:36.415497 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6458a2cd-4df8-4a47-a68b-bea6bde3f9ae None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:36.503022 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaeef3f-dcc7-4f15-9ff9-a081b141687f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:36.504258 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaeef3f-dcc7-4f15-9ff9-a081b141687f None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:36.504756 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaeef3f-dcc7-4f15-9ff9-a081b141687f None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:36.505266 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaeef3f-dcc7-4f15-9ff9-a081b141687f None admin] PATH_INFO: `/v3/projects/e7497d51ee8f4f0487b09541d4ad94d2` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:36.510747 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eaeef3f-dcc7-4f15-9ff9-a081b141687f None admin] RBAC: Authorizing `identity:get_project(project_id=e7497d51ee8f4f0487b09541d4ad94d2)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:36.512490 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eaeef3f-dcc7-4f15-9ff9-a081b141687f None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:36.515380 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 82/166] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:23:36 2022] GET /identity/v3/projects/e7497d51ee8f4f0487b09541d4ad94d2 => generated 298 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:36.516902 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142ac08c-8302-4cfa-8b67-af705d6f535b None None] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:36.517429 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142ac08c-8302-4cfa-8b67-af705d6f535b None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:36.522054 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142ac08c-8302-4cfa-8b67-af705d6f535b None None] PATH_INFO: `/` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:36.524016 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 83/167] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:23:36 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:23:36.525985 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142ac08c-8302-4cfa-8b67-af705d6f535b None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:36.537792 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6610cc4f-e086-40d2-8c75-3ce38090ac9a None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:36.599498 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8d76a6-cfbf-4fe5-a40f-5845cb91dcb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:36.600428 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8d76a6-cfbf-4fe5-a40f-5845cb91dcb3 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:36.600724 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8d76a6-cfbf-4fe5-a40f-5845cb91dcb3 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:36.601006 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8d76a6-cfbf-4fe5-a40f-5845cb91dcb3 None admin] PATH_INFO: `/v3/roles` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:36.601900 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8d76a6-cfbf-4fe5-a40f-5845cb91dcb3 None admin] RBAC: Adding query filter params (name=member) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:23:36.602220 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8d76a6-cfbf-4fe5-a40f-5845cb91dcb3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:36.603647 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8d76a6-cfbf-4fe5-a40f-5845cb91dcb3 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:36.619820 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 85/168] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:23:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:36.621537 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481aff02-98fc-490d-8efb-cdd66d47e8e4 None None] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:36.621537 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481aff02-98fc-490d-8efb-cdd66d47e8e4 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:36.621908 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481aff02-98fc-490d-8efb-cdd66d47e8e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:36.650848 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.auth.core [None req-481aff02-98fc-490d-8efb-cdd66d47e8e4 None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143587) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:23:36.652053 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-481aff02-98fc-490d-8efb-cdd66d47e8e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:23:36.763561 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142ac08c-8302-4cfa-8b67-af705d6f535b None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:36.821756 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:36.832107 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 86/169] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:23:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:23:36.833061 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481aff02-98fc-490d-8efb-cdd66d47e8e4 None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:36.837811 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a870bee-fe54-48fa-aae2-18fb890b6256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:36.838603 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a870bee-fe54-48fa-aae2-18fb890b6256 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:36.838892 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a870bee-fe54-48fa-aae2-18fb890b6256 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:36.839173 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a870bee-fe54-48fa-aae2-18fb890b6256 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:36.840077 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a870bee-fe54-48fa-aae2-18fb890b6256 None admin] RBAC: Adding query filter params (role.id=3d5075293a4f45449eaef6af8fa53004, scope.project.id=e7497d51ee8f4f0487b09541d4ad94d2, user.id=21249d0b4cb3470e83a9f256cbe52f18) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:23:36.840395 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a870bee-fe54-48fa-aae2-18fb890b6256 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:36.841948 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a870bee-fe54-48fa-aae2-18fb890b6256 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:36.854235 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:36.855743 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 84/170] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:23:36 2022] GET /identity/v3/role_assignments?user.id=21249d0b4cb3470e83a9f256cbe52f18&scope.project.id=e7497d51ee8f4f0487b09541d4ad94d2&role.id=3d5075293a4f45449eaef6af8fa53004 => generated 608 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:36.874795 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a870bee-fe54-48fa-aae2-18fb890b6256 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:37.104254 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a870bee-fe54-48fa-aae2-18fb890b6256 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:37.109637 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481aff02-98fc-490d-8efb-cdd66d47e8e4 None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:37.172740 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a631ae-d9aa-4d84-b5c5-8f3d3b42be53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:37.173633 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a631ae-d9aa-4d84-b5c5-8f3d3b42be53 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:37.173940 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a631ae-d9aa-4d84-b5c5-8f3d3b42be53 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:37.174219 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a631ae-d9aa-4d84-b5c5-8f3d3b42be53 None admin] PATH_INFO: `/v3/users/21249d0b4cb3470e83a9f256cbe52f18` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:37.176272 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e951de1a-489b-46d2-82fd-8a7d9640e4bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:37.177074 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e951de1a-489b-46d2-82fd-8a7d9640e4bd None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:37.177354 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e951de1a-489b-46d2-82fd-8a7d9640e4bd None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:37.177630 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e951de1a-489b-46d2-82fd-8a7d9640e4bd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:37.178794 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a631ae-d9aa-4d84-b5c5-8f3d3b42be53 None admin] RBAC: Authorizing `identity:get_user(user_id=21249d0b4cb3470e83a9f256cbe52f18)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:37.180306 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a631ae-d9aa-4d84-b5c5-8f3d3b42be53 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:37.182096 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 87/171] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:23:36 2022] GET /identity/v3/users/21249d0b4cb3470e83a9f256cbe52f18 => generated 275 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:37.189922 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a631ae-d9aa-4d84-b5c5-8f3d3b42be53 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:37.203931 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e951de1a-489b-46d2-82fd-8a7d9640e4bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:23:37.211869 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-e951de1a-489b-46d2-82fd-8a7d9640e4bd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:23:37.211869 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:23:37.211869 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:23:37.211869 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:23:37.211869 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:23:37.211869 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:23:37.211869 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:23:37.211869 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:23:37.211869 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:23:37.211869 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:23:37.211869 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:23:37.211869 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:23:37.211869 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:23:37.211869 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:23:37.211869 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:23:37.211869 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:23:37.212957 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:23:37.212957 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:23:37.212957 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:23:37.212957 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:23:37.212957 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:23:37.212957 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:23:37.212957 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:23:37.212957 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:23:37.212957 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:23:37.212957 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:23:37.212957 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:23:37.212957 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:23:37.212957 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:23:37.212957 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:23:37.212957 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:23:37.212957 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:23:37.213908 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:23:37.213908 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:23:37.213908 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:23:37.213908 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:23:37.213908 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:23:37.213908 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:23:37.213908 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:23:37.213908 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:23:37.213908 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:23:37.213908 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:23:37.213908 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:23:37.213908 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:23:37.213908 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 85/172] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:23:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:23:37.220475 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e951de1a-489b-46d2-82fd-8a7d9640e4bd None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:37.292480 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:37.360629 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:37.384780 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e951de1a-489b-46d2-82fd-8a7d9640e4bd None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:37.402656 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a631ae-d9aa-4d84-b5c5-8f3d3b42be53 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:37.463092 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16372204-eb0e-4eac-bc4c-962c66d974a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:37.464465 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16372204-eb0e-4eac-bc4c-962c66d974a1 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:37.464939 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16372204-eb0e-4eac-bc4c-962c66d974a1 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:37.465414 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16372204-eb0e-4eac-bc4c-962c66d974a1 None admin] PATH_INFO: `/v3/roles` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:37.466411 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16372204-eb0e-4eac-bc4c-962c66d974a1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:23:37.466929 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16372204-eb0e-4eac-bc4c-962c66d974a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:37.469409 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16372204-eb0e-4eac-bc4c-962c66d974a1 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:37.473865 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cae52b-6542-42be-8772-24d9dffd9478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:37.473865 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cae52b-6542-42be-8772-24d9dffd9478 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:37.473865 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cae52b-6542-42be-8772-24d9dffd9478 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:37.474174 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cae52b-6542-42be-8772-24d9dffd9478 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:37.484550 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 86/173] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:23:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:37.485950 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8f67b6-c1d2-42ac-9c64-137bf4c520dd None None] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:37.486248 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8f67b6-c1d2-42ac-9c64-137bf4c520dd None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:37.486531 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8f67b6-c1d2-42ac-9c64-137bf4c520dd None None] PATH_INFO: `/` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:37.496642 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 87/174] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:23:37 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:23:37.497499 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8f67b6-c1d2-42ac-9c64-137bf4c520dd None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:37.622327 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8f67b6-c1d2-42ac-9c64-137bf4c520dd None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:37.646207 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:37.667476 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a6ae92-1279-4ccc-842f-ac798067f7bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:37.668373 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a6ae92-1279-4ccc-842f-ac798067f7bf None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:37.668660 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a6ae92-1279-4ccc-842f-ac798067f7bf None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:37.668945 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a6ae92-1279-4ccc-842f-ac798067f7bf None admin] PATH_INFO: `/v3/users/4f2f0290e1b04ce48a90ce03cb93d060` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:37.670220 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a6ae92-1279-4ccc-842f-ac798067f7bf None admin] RBAC: Authorizing `identity:get_user(user_id=4f2f0290e1b04ce48a90ce03cb93d060)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:37.672466 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a6ae92-1279-4ccc-842f-ac798067f7bf None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:37.674333 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 88/175] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:23:37 2022] GET /identity/v3/users/4f2f0290e1b04ce48a90ce03cb93d060 => generated 247 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:37.675859 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897a6e22-9e8c-41e0-abe0-7d3c547c75f6 None None] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:37.676151 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897a6e22-9e8c-41e0-abe0-7d3c547c75f6 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:37.676434 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897a6e22-9e8c-41e0-abe0-7d3c547c75f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:37.678119 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cae52b-6542-42be-8772-24d9dffd9478 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:37.679618 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cae52b-6542-42be-8772-24d9dffd9478 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:37.695344 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.auth.core [None req-897a6e22-9e8c-41e0-abe0-7d3c547c75f6 None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143586) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:23:37.696559 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.fernet_utils [None req-897a6e22-9e8c-41e0-abe0-7d3c547c75f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143586) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:23:37.773347 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 89/176] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:23:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:23:37.774308 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897a6e22-9e8c-41e0-abe0-7d3c547c75f6 None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:37.831299 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:37.877157 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 88/177] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:23:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 688 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:23:37.878284 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cae52b-6542-42be-8772-24d9dffd9478 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:37.888652 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897a6e22-9e8c-41e0-abe0-7d3c547c75f6 None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:37.906192 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:37.927097 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ed0b8e-0a08-4eb3-9fe4-797d9b56ff25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:37.928000 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ed0b8e-0a08-4eb3-9fe4-797d9b56ff25 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:37.928663 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ed0b8e-0a08-4eb3-9fe4-797d9b56ff25 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:37.929134 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ed0b8e-0a08-4eb3-9fe4-797d9b56ff25 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:38.004890 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cae52b-6542-42be-8772-24d9dffd9478 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:38.022939 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ed0b8e-0a08-4eb3-9fe4-797d9b56ff25 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:38.025362 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ed0b8e-0a08-4eb3-9fe4-797d9b56ff25 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:38.047766 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d675640-6cf9-4bf3-aa4d-84e7434381f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:38.048574 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d675640-6cf9-4bf3-aa4d-84e7434381f4 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:38.048859 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d675640-6cf9-4bf3-aa4d-84e7434381f4 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:38.049137 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d675640-6cf9-4bf3-aa4d-84e7434381f4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:38.060144 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d675640-6cf9-4bf3-aa4d-84e7434381f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:23:38.068430 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-7d675640-6cf9-4bf3-aa4d-84e7434381f4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:23:38.068430 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:23:38.068430 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:23:38.068430 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:23:38.068430 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:23:38.068430 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:23:38.068430 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:23:38.068430 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:23:38.068430 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:23:38.068430 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:23:38.068430 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:23:38.068430 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:23:38.068430 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:23:38.068430 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:23:38.068430 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:23:38.068430 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:23:38.069442 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:23:38.069442 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:23:38.069442 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:23:38.069442 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:23:38.069442 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:23:38.069442 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:23:38.069442 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:23:38.069442 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:23:38.069442 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:23:38.069442 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:23:38.069442 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:23:38.069442 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:23:38.069442 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:23:38.069442 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:23:38.069442 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:23:38.069442 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:23:38.070387 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:23:38.070387 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:23:38.070387 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:23:38.070387 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:23:38.070387 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:23:38.070387 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:23:38.070387 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:23:38.070387 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:23:38.070387 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:23:38.070387 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:23:38.070387 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:23:38.070387 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:23:38.070387 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 89/178] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:23:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:23:38.071191 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d675640-6cf9-4bf3-aa4d-84e7434381f4 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:38.120639 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:38.143644 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 90/179] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:23:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:23:38.143644 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea23313-9605-460f-8192-3c0cc418a82b None None] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:38.143644 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea23313-9605-460f-8192-3c0cc418a82b None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:38.143644 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea23313-9605-460f-8192-3c0cc418a82b None None] PATH_INFO: `/` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:38.143644 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 91/180] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:23:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:23:38.147675 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89924dec-76a7-4a6b-a53c-31dd355d1dde None None] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:38.147964 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89924dec-76a7-4a6b-a53c-31dd355d1dde None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:38.148246 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89924dec-76a7-4a6b-a53c-31dd355d1dde None None] PATH_INFO: `/` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:38.149541 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 92/181] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:23:38 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:23:38.150430 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89924dec-76a7-4a6b-a53c-31dd355d1dde None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:38.166489 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:38.202190 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d675640-6cf9-4bf3-aa4d-84e7434381f4 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:38.242577 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9306902a-0063-40e4-89c7-b5e6a17ffed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:38.243456 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9306902a-0063-40e4-89c7-b5e6a17ffed8 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:38.243747 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9306902a-0063-40e4-89c7-b5e6a17ffed8 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:38.244086 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9306902a-0063-40e4-89c7-b5e6a17ffed8 None admin] PATH_INFO: `/v3/projects/5c7498adefdf4ff59ace12ff1685bd25` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:38.245723 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9306902a-0063-40e4-89c7-b5e6a17ffed8 None admin] RBAC: Authorizing `identity:get_project(project_id=5c7498adefdf4ff59ace12ff1685bd25)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:38.247682 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9306902a-0063-40e4-89c7-b5e6a17ffed8 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:38.254928 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 90/182] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:23:38 2022] GET /identity/v3/projects/5c7498adefdf4ff59ace12ff1685bd25 => generated 312 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:38.254928 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9306902a-0063-40e4-89c7-b5e6a17ffed8 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:38.257507 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89924dec-76a7-4a6b-a53c-31dd355d1dde None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:38.293574 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b50b88-da7c-40c9-8e1c-3dd2fd21a12b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:38.294272 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b50b88-da7c-40c9-8e1c-3dd2fd21a12b None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:38.294755 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b50b88-da7c-40c9-8e1c-3dd2fd21a12b None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:38.295203 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b50b88-da7c-40c9-8e1c-3dd2fd21a12b None admin] PATH_INFO: `/v3/roles` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:38.296924 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b50b88-da7c-40c9-8e1c-3dd2fd21a12b None admin] RBAC: Adding query filter params (name=admin) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:23:38.297393 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b50b88-da7c-40c9-8e1c-3dd2fd21a12b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:38.298982 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b50b88-da7c-40c9-8e1c-3dd2fd21a12b None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:38.308197 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 93/183] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:23:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:38.309609 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aae374-9ae7-46dd-a9f6-dc41f4060d04 None None] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:38.309942 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aae374-9ae7-46dd-a9f6-dc41f4060d04 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:38.310228 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aae374-9ae7-46dd-a9f6-dc41f4060d04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:38.328383 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.auth.core [None req-d1aae374-9ae7-46dd-a9f6-dc41f4060d04 None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143586) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:23:38.329576 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.fernet_utils [None req-d1aae374-9ae7-46dd-a9f6-dc41f4060d04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143586) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:23:38.369508 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9306902a-0063-40e4-89c7-b5e6a17ffed8 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:38.377131 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:38.398645 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 94/184] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:23:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:23:38.403338 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30ca5c3-5bf7-488b-93ae-936ed7be47b8 None None] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:38.403666 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30ca5c3-5bf7-488b-93ae-936ed7be47b8 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:38.403907 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30ca5c3-5bf7-488b-93ae-936ed7be47b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:38.411992 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180dcf29-1e82-4ea7-9732-c6475ca0d3ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:38.412687 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:38.412895 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180dcf29-1e82-4ea7-9732-c6475ca0d3ce None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:38.413182 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180dcf29-1e82-4ea7-9732-c6475ca0d3ce None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:38.413529 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180dcf29-1e82-4ea7-9732-c6475ca0d3ce None admin] PATH_INFO: `/v3/projects/e7497d51ee8f4f0487b09541d4ad94d2` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:38.415372 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180dcf29-1e82-4ea7-9732-c6475ca0d3ce None admin] RBAC: Authorizing `identity:get_project(project_id=e7497d51ee8f4f0487b09541d4ad94d2)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:38.417609 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180dcf29-1e82-4ea7-9732-c6475ca0d3ce None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:38.425291 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 91/185] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:23:38 2022] GET /identity/v3/projects/e7497d51ee8f4f0487b09541d4ad94d2 => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:38.425291 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180dcf29-1e82-4ea7-9732-c6475ca0d3ce None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:38.434027 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.auth.core [None req-d30ca5c3-5bf7-488b-93ae-936ed7be47b8 None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143586) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:23:38.435243 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.fernet_utils [None req-d30ca5c3-5bf7-488b-93ae-936ed7be47b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143586) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:23:38.508506 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 95/186] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:23:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:23:38.509366 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30ca5c3-5bf7-488b-93ae-936ed7be47b8 None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:38.544499 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180dcf29-1e82-4ea7-9732-c6475ca0d3ce None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:38.582749 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f39b437-0923-4407-a3ab-92139b80c1cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:38.583785 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f39b437-0923-4407-a3ab-92139b80c1cc None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:38.584294 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f39b437-0923-4407-a3ab-92139b80c1cc None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:38.584820 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f39b437-0923-4407-a3ab-92139b80c1cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:38.585991 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f39b437-0923-4407-a3ab-92139b80c1cc None admin] RBAC: Adding query filter params (role.id=3d5075293a4f45449eaef6af8fa53004, scope.project.id=5c7498adefdf4ff59ace12ff1685bd25, user.id=21249d0b4cb3470e83a9f256cbe52f18) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:23:38.586517 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f39b437-0923-4407-a3ab-92139b80c1cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:38.588270 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f39b437-0923-4407-a3ab-92139b80c1cc None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:38.595840 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 92/187] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:23:38 2022] GET /identity/v3/role_assignments?user.id=21249d0b4cb3470e83a9f256cbe52f18&scope.project.id=5c7498adefdf4ff59ace12ff1685bd25&role.id=3d5075293a4f45449eaef6af8fa53004 => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:38.597006 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f39b437-0923-4407-a3ab-92139b80c1cc None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:38.631630 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30ca5c3-5bf7-488b-93ae-936ed7be47b8 None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:38.656337 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:38.677070 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cb92a1-0017-49a3-961c-0bf2ac7990ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:38.678012 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cb92a1-0017-49a3-961c-0bf2ac7990ea None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:38.678296 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cb92a1-0017-49a3-961c-0bf2ac7990ea None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:38.678572 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cb92a1-0017-49a3-961c-0bf2ac7990ea None admin] PATH_INFO: `/v3/users/89cde8a97d784539962d5992b9d7727f` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:38.681005 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:38.698801 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cb92a1-0017-49a3-961c-0bf2ac7990ea None admin] RBAC: Authorizing `identity:get_user(user_id=89cde8a97d784539962d5992b9d7727f)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:38.700344 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cb92a1-0017-49a3-961c-0bf2ac7990ea None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:38.707734 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 96/188] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:23:38 2022] GET /identity/v3/users/89cde8a97d784539962d5992b9d7727f => generated 283 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:38.707734 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cb92a1-0017-49a3-961c-0bf2ac7990ea None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:38.739327 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f39b437-0923-4407-a3ab-92139b80c1cc None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:38.783277 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328cdffb-7ce4-405f-9d6d-e6f830021099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:38.784422 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328cdffb-7ce4-405f-9d6d-e6f830021099 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:38.784422 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328cdffb-7ce4-405f-9d6d-e6f830021099 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:38.784625 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328cdffb-7ce4-405f-9d6d-e6f830021099 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:38.798937 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328cdffb-7ce4-405f-9d6d-e6f830021099 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:23:38.807573 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-328cdffb-7ce4-405f-9d6d-e6f830021099 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:23:38.807573 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:23:38.807573 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:23:38.807573 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:23:38.807573 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:23:38.807573 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:23:38.807573 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:23:38.807573 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:23:38.807573 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:23:38.807573 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:23:38.807573 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:23:38.807573 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:23:38.807573 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:23:38.807573 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:23:38.807573 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:23:38.807573 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:23:38.808606 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:23:38.808606 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:23:38.808606 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:23:38.808606 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:23:38.808606 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:23:38.808606 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:23:38.808606 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:23:38.808606 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:23:38.808606 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:23:38.808606 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:23:38.808606 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:23:38.808606 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:23:38.808606 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:23:38.808606 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:23:38.808606 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:23:38.808606 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:23:38.809961 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:23:38.809961 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:23:38.809961 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:23:38.809961 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:23:38.809961 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:23:38.809961 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:23:38.809961 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:23:38.809961 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:23:38.809961 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:23:38.809961 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:23:38.809961 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:23:38.809961 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:23:38.809961 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 93/189] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 19 14:23:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:23:38.809961 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328cdffb-7ce4-405f-9d6d-e6f830021099 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:38.830788 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cb92a1-0017-49a3-961c-0bf2ac7990ea None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:38.871724 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b433241b-4d24-46bc-8ba5-c0dcddc57334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:38.872404 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b433241b-4d24-46bc-8ba5-c0dcddc57334 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:38.872693 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b433241b-4d24-46bc-8ba5-c0dcddc57334 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:38.872974 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b433241b-4d24-46bc-8ba5-c0dcddc57334 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:38.874048 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b433241b-4d24-46bc-8ba5-c0dcddc57334 None admin] RBAC: Adding query filter params (role.id=a3b793754e7b40b0bb21afe74eec3c18, scope.project.id=e7497d51ee8f4f0487b09541d4ad94d2, user.id=4f2f0290e1b04ce48a90ce03cb93d060) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:23:38.874372 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b433241b-4d24-46bc-8ba5-c0dcddc57334 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:38.875820 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b433241b-4d24-46bc-8ba5-c0dcddc57334 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:38.881659 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 97/190] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:23:38 2022] GET /identity/v3/role_assignments?user.id=4f2f0290e1b04ce48a90ce03cb93d060&scope.project.id=e7497d51ee8f4f0487b09541d4ad94d2&role.id=a3b793754e7b40b0bb21afe74eec3c18 => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:38.882929 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f1de0a-0d74-4f94-bdfc-4fbd16a2f9fd None None] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:38.883223 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f1de0a-0d74-4f94-bdfc-4fbd16a2f9fd None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:38.883496 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f1de0a-0d74-4f94-bdfc-4fbd16a2f9fd None None] PATH_INFO: `/` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:38.884750 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 98/191] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:23:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:23:38.885992 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f1de0a-0d74-4f94-bdfc-4fbd16a2f9fd None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:38.943158 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:38.947730 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328cdffb-7ce4-405f-9d6d-e6f830021099 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:38.958725 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:38.989300 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79106860-af9e-451c-9981-21838cd10018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:38.990252 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79106860-af9e-451c-9981-21838cd10018 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:38.990543 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79106860-af9e-451c-9981-21838cd10018 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:38.990820 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79106860-af9e-451c-9981-21838cd10018 None admin] PATH_INFO: `/v3/domains/default` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:38.992045 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79106860-af9e-451c-9981-21838cd10018 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:38.993552 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79106860-af9e-451c-9981-21838cd10018 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:38.995512 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 94/192] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:23:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:38.996107 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79106860-af9e-451c-9981-21838cd10018 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:39.019477 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f1de0a-0d74-4f94-bdfc-4fbd16a2f9fd None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:39.067250 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de15e40-f600-4852-a4d0-62c1ded4ae43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:39.068440 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de15e40-f600-4852-a4d0-62c1ded4ae43 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:39.068440 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de15e40-f600-4852-a4d0-62c1ded4ae43 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:39.068440 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de15e40-f600-4852-a4d0-62c1ded4ae43 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:39.118182 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79106860-af9e-451c-9981-21838cd10018 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:39.144449 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de15e40-f600-4852-a4d0-62c1ded4ae43 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:39.146022 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de15e40-f600-4852-a4d0-62c1ded4ae43 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:39.159336 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe4c355-8286-4dd3-a8fd-9c1f70b47d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:39.160040 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe4c355-8286-4dd3-a8fd-9c1f70b47d51 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:39.160325 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe4c355-8286-4dd3-a8fd-9c1f70b47d51 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:39.160608 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe4c355-8286-4dd3-a8fd-9c1f70b47d51 None admin] PATH_INFO: `/v3/roles` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:39.161368 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe4c355-8286-4dd3-a8fd-9c1f70b47d51 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:23:39.161685 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe4c355-8286-4dd3-a8fd-9c1f70b47d51 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:39.163144 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe4c355-8286-4dd3-a8fd-9c1f70b47d51 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:39.171846 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 95/193] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:23:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:39.177554 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3022244-df06-4f25-9a54-7271ee2b96e9 None None] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:39.177875 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3022244-df06-4f25-9a54-7271ee2b96e9 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:39.178157 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3022244-df06-4f25-9a54-7271ee2b96e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:39.194425 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.auth.core [None req-c3022244-df06-4f25-9a54-7271ee2b96e9 None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143587) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:23:39.195550 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-c3022244-df06-4f25-9a54-7271ee2b96e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:23:39.201778 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:39.218631 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:39.248005 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 99/194] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:23:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:23:39.248859 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de15e40-f600-4852-a4d0-62c1ded4ae43 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:39.263371 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 96/195] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:23:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:23:39.264434 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3022244-df06-4f25-9a54-7271ee2b96e9 None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:39.352239 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de15e40-f600-4852-a4d0-62c1ded4ae43 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:39.368575 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3022244-df06-4f25-9a54-7271ee2b96e9 None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:39.386064 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab30b3ab-cd7f-45d3-b0cc-432ce5f52684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:39.386740 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab30b3ab-cd7f-45d3-b0cc-432ce5f52684 None admin] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:39.387025 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab30b3ab-cd7f-45d3-b0cc-432ce5f52684 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:39.387306 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab30b3ab-cd7f-45d3-b0cc-432ce5f52684 None admin] PATH_INFO: `/v3/users` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:39.388027 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab30b3ab-cd7f-45d3-b0cc-432ce5f52684 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:39.390198 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab30b3ab-cd7f-45d3-b0cc-432ce5f52684 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:39.405290 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5eadf34-f5fa-4a29-957f-e9fa41d83234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:39.406214 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eadf34-f5fa-4a29-957f-e9fa41d83234 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:39.406498 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eadf34-f5fa-4a29-957f-e9fa41d83234 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:39.406773 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eadf34-f5fa-4a29-957f-e9fa41d83234 None admin] PATH_INFO: `/v3/users/89cde8a97d784539962d5992b9d7727f` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:39.408061 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5eadf34-f5fa-4a29-957f-e9fa41d83234 None admin] RBAC: Authorizing `identity:get_user(user_id=89cde8a97d784539962d5992b9d7727f)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:39.409507 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5eadf34-f5fa-4a29-957f-e9fa41d83234 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:39.410838 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 100/196] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 19 14:23:39 2022] POST /identity/v3/users => generated 297 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:23:39.411334 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 97/197] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:23:39 2022] GET /identity/v3/users/89cde8a97d784539962d5992b9d7727f => generated 283 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:39.411938 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5eadf34-f5fa-4a29-957f-e9fa41d83234 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:39.415153 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab30b3ab-cd7f-45d3-b0cc-432ce5f52684 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:39.442707 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:39.447821 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:39.527128 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5eadf34-f5fa-4a29-957f-e9fa41d83234 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:39.558803 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab30b3ab-cd7f-45d3-b0cc-432ce5f52684 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:39.567015 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecdc6a4-cac8-4790-b513-8b9462f6e66c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:39.567853 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecdc6a4-cac8-4790-b513-8b9462f6e66c None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:39.568136 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecdc6a4-cac8-4790-b513-8b9462f6e66c None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:39.568412 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecdc6a4-cac8-4790-b513-8b9462f6e66c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:39.582467 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecdc6a4-cac8-4790-b513-8b9462f6e66c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:23:39.590310 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-9ecdc6a4-cac8-4790-b513-8b9462f6e66c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:23:39.590310 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:23:39.590310 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:23:39.590310 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:23:39.590310 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:23:39.590310 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:23:39.590310 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:23:39.590310 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:23:39.590310 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:23:39.590310 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:23:39.590310 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:23:39.590310 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:23:39.590310 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:23:39.590310 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:23:39.590310 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:23:39.590310 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:23:39.591343 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:23:39.591343 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:23:39.591343 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:23:39.591343 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:23:39.591343 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:23:39.591343 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:23:39.591343 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:23:39.591343 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:23:39.591343 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:23:39.591343 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:23:39.591343 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:23:39.591343 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:23:39.591343 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:23:39.591343 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:23:39.591343 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:23:39.591343 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:23:39.599771 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:23:39.599771 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:23:39.599771 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:23:39.599771 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:23:39.599771 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:23:39.599771 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:23:39.599771 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:23:39.599771 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:23:39.599771 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:23:39.599771 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:23:39.599771 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:23:39.599771 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:23:39.599771 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 98/198] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:23:39 2022] GET /identity/v3/roles/reader => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:23:39.599771 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecdc6a4-cac8-4790-b513-8b9462f6e66c None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:39.606494 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b39dc15-32d0-4bcb-84c2-3c7095482405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:39.607381 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b39dc15-32d0-4bcb-84c2-3c7095482405 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:39.608138 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b39dc15-32d0-4bcb-84c2-3c7095482405 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:39.608138 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b39dc15-32d0-4bcb-84c2-3c7095482405 None admin] PATH_INFO: `/v3/projects/24fafd1589784882b754b5f8543954bb` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:39.609170 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b39dc15-32d0-4bcb-84c2-3c7095482405 None admin] RBAC: Authorizing `identity:get_project(project_id=24fafd1589784882b754b5f8543954bb)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:39.610717 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b39dc15-32d0-4bcb-84c2-3c7095482405 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:39.612607 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 101/199] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:23:39 2022] GET /identity/v3/projects/24fafd1589784882b754b5f8543954bb => generated 302 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:39.613203 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b39dc15-32d0-4bcb-84c2-3c7095482405 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:39.709060 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:39.719351 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:39.730669 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecdc6a4-cac8-4790-b513-8b9462f6e66c None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:39.746974 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b39dc15-32d0-4bcb-84c2-3c7095482405 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:39.775988 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61748f2-6a9f-4c6a-adeb-62d87004a190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:39.776640 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61748f2-6a9f-4c6a-adeb-62d87004a190 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:39.776922 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61748f2-6a9f-4c6a-adeb-62d87004a190 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:39.777200 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61748f2-6a9f-4c6a-adeb-62d87004a190 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:39.788872 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bf6253-f47e-48b7-b164-553310c5e0b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:39.789947 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bf6253-f47e-48b7-b164-553310c5e0b4 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:39.790236 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bf6253-f47e-48b7-b164-553310c5e0b4 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:39.790518 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bf6253-f47e-48b7-b164-553310c5e0b4 None admin] PATH_INFO: `/v3/roles` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:39.791601 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bf6253-f47e-48b7-b164-553310c5e0b4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:23:39.791913 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bf6253-f47e-48b7-b164-553310c5e0b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:39.793290 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bf6253-f47e-48b7-b164-553310c5e0b4 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:39.805956 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 102/200] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:23:39 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:39.806871 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bf6253-f47e-48b7-b164-553310c5e0b4 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:39.857378 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61748f2-6a9f-4c6a-adeb-62d87004a190 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:39.859733 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61748f2-6a9f-4c6a-adeb-62d87004a190 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:39.945995 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bf6253-f47e-48b7-b164-553310c5e0b4 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:39.959479 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:39.979107 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 99/201] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:23:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:23:39.980420 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61748f2-6a9f-4c6a-adeb-62d87004a190 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:39.994622 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd700ccc-0813-49ee-978b-f7b6556b093c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:39.995295 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd700ccc-0813-49ee-978b-f7b6556b093c None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:39.995582 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd700ccc-0813-49ee-978b-f7b6556b093c None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:39.995866 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd700ccc-0813-49ee-978b-f7b6556b093c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:39.996991 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd700ccc-0813-49ee-978b-f7b6556b093c None admin] RBAC: Adding query filter params (role.id=a3b793754e7b40b0bb21afe74eec3c18, scope.project.id=24fafd1589784882b754b5f8543954bb, user.id=89cde8a97d784539962d5992b9d7727f) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:23:39.997317 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd700ccc-0813-49ee-978b-f7b6556b093c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:39.999806 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd700ccc-0813-49ee-978b-f7b6556b093c None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:40.007647 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 103/202] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:23:39 2022] GET /identity/v3/role_assignments?user.id=89cde8a97d784539962d5992b9d7727f&scope.project.id=24fafd1589784882b754b5f8543954bb&role.id=a3b793754e7b40b0bb21afe74eec3c18 => generated 259 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:40.008504 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd700ccc-0813-49ee-978b-f7b6556b093c None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:40.033115 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:40.134308 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61748f2-6a9f-4c6a-adeb-62d87004a190 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:40.177589 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a0a4d6-12d3-4080-8700-ce57a13693de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:40.178536 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a0a4d6-12d3-4080-8700-ce57a13693de None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:40.179055 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a0a4d6-12d3-4080-8700-ce57a13693de None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:40.179476 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a0a4d6-12d3-4080-8700-ce57a13693de None admin] PATH_INFO: `/v3/users/cd0294bae78b43ea80cea320f446e487` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:40.180983 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a0a4d6-12d3-4080-8700-ce57a13693de None admin] RBAC: Authorizing `identity:get_user(user_id=cd0294bae78b43ea80cea320f446e487)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:40.182513 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a0a4d6-12d3-4080-8700-ce57a13693de None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:40.184425 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 100/203] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:23:39 2022] GET /identity/v3/users/cd0294bae78b43ea80cea320f446e487 => generated 289 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:40.190587 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a0a4d6-12d3-4080-8700-ce57a13693de None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:40.228856 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd700ccc-0813-49ee-978b-f7b6556b093c None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:40.269585 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e31c259-3579-433c-affe-5125ba05eb97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:40.270945 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e31c259-3579-433c-affe-5125ba05eb97 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:40.271460 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e31c259-3579-433c-affe-5125ba05eb97 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:40.271979 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e31c259-3579-433c-affe-5125ba05eb97 None admin] PATH_INFO: `/v3/projects/24fafd1589784882b754b5f8543954bb` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:40.273433 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e31c259-3579-433c-affe-5125ba05eb97 None admin] RBAC: Authorizing `identity:get_project(project_id=24fafd1589784882b754b5f8543954bb)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:40.275124 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e31c259-3579-433c-affe-5125ba05eb97 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:40.277413 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 104/204] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:23:40 2022] GET /identity/v3/projects/24fafd1589784882b754b5f8543954bb => generated 302 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:40.282915 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:40.285089 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e31c259-3579-433c-affe-5125ba05eb97 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:40.325306 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a0a4d6-12d3-4080-8700-ce57a13693de None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:40.373995 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302a52b3-c31d-4bc8-ae88-73a0861ad16f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:40.374825 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302a52b3-c31d-4bc8-ae88-73a0861ad16f None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:40.375124 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302a52b3-c31d-4bc8-ae88-73a0861ad16f None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:40.375400 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302a52b3-c31d-4bc8-ae88-73a0861ad16f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:40.426892 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:40.474023 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302a52b3-c31d-4bc8-ae88-73a0861ad16f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:40.475774 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302a52b3-c31d-4bc8-ae88-73a0861ad16f None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:40.483640 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e31c259-3579-433c-affe-5125ba05eb97 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:40.552380 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a322a5ca-00b5-4208-9570-8d057fc62841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:40.553044 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a322a5ca-00b5-4208-9570-8d057fc62841 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:40.553327 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a322a5ca-00b5-4208-9570-8d057fc62841 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:40.553601 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a322a5ca-00b5-4208-9570-8d057fc62841 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:40.554471 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a322a5ca-00b5-4208-9570-8d057fc62841 None admin] RBAC: Adding query filter params (role.id=b954b2c7ce764aa0afac78e075a011a6, scope.project.id=24fafd1589784882b754b5f8543954bb, user.id=89cde8a97d784539962d5992b9d7727f) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:23:40.554781 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a322a5ca-00b5-4208-9570-8d057fc62841 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:40.556225 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a322a5ca-00b5-4208-9570-8d057fc62841 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:40.567922 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 105/205] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:23:40 2022] GET /identity/v3/role_assignments?user.id=89cde8a97d784539962d5992b9d7727f&scope.project.id=24fafd1589784882b754b5f8543954bb&role.id=b954b2c7ce764aa0afac78e075a011a6 => generated 259 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:40.580925 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:40.649018 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 101/206] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:23:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 14:23:40.656313 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a322a5ca-00b5-4208-9570-8d057fc62841 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:40.803983 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a322a5ca-00b5-4208-9570-8d057fc62841 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:40.839494 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bd1946-b15e-408f-9cf4-eee8c7eef03d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:40.840355 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bd1946-b15e-408f-9cf4-eee8c7eef03d None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:40.840634 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bd1946-b15e-408f-9cf4-eee8c7eef03d None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:40.840905 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bd1946-b15e-408f-9cf4-eee8c7eef03d None admin] PATH_INFO: `/v3/projects/e7497d51ee8f4f0487b09541d4ad94d2` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:40.842849 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bd1946-b15e-408f-9cf4-eee8c7eef03d None admin] RBAC: Authorizing `identity:get_project(project_id=e7497d51ee8f4f0487b09541d4ad94d2)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:40.844445 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bd1946-b15e-408f-9cf4-eee8c7eef03d None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:40.846377 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 106/207] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:23:40 2022] GET /identity/v3/projects/e7497d51ee8f4f0487b09541d4ad94d2 => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:40.852461 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302a52b3-c31d-4bc8-ae88-73a0861ad16f None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:40.982109 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302a52b3-c31d-4bc8-ae88-73a0861ad16f None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:41.023699 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc26174-ef9e-4d71-804f-fbdc1f91b409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:41.024354 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc26174-ef9e-4d71-804f-fbdc1f91b409 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:41.024635 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc26174-ef9e-4d71-804f-fbdc1f91b409 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:41.024907 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc26174-ef9e-4d71-804f-fbdc1f91b409 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:41.025741 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc26174-ef9e-4d71-804f-fbdc1f91b409 None admin] RBAC: Adding query filter params (role.id=abff822b2f9d4abd96ff7414f6de1670, scope.project.id=e7497d51ee8f4f0487b09541d4ad94d2, user.id=cd0294bae78b43ea80cea320f446e487) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:23:41.026082 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc26174-ef9e-4d71-804f-fbdc1f91b409 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:41.027696 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc26174-ef9e-4d71-804f-fbdc1f91b409 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:41.038126 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 102/208] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:23:40 2022] GET /identity/v3/role_assignments?user.id=cd0294bae78b43ea80cea320f446e487&scope.project.id=e7497d51ee8f4f0487b09541d4ad94d2&role.id=abff822b2f9d4abd96ff7414f6de1670 => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:42.029028 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206ebfa3-46c7-48de-aee6-7eab64eb993d None None] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:42.030314 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206ebfa3-46c7-48de-aee6-7eab64eb993d None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:42.030871 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206ebfa3-46c7-48de-aee6-7eab64eb993d None None] PATH_INFO: `/` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:42.032784 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 107/209] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:23:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:23:42.039088 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558876a9-c121-4ebe-beb6-160ac33feae9 None None] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:42.040313 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558876a9-c121-4ebe-beb6-160ac33feae9 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:42.040830 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558876a9-c121-4ebe-beb6-160ac33feae9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:42.059448 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.auth.core [None req-558876a9-c121-4ebe-beb6-160ac33feae9 None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143587) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:23:42.061570 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-558876a9-c121-4ebe-beb6-160ac33feae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:23:42.076114 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:42.108645 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dff342-3b7a-43d4-840a-f433c5fd354f None None] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:42.109008 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dff342-3b7a-43d4-840a-f433c5fd354f None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:42.109209 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dff342-3b7a-43d4-840a-f433c5fd354f None None] PATH_INFO: `/` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:42.111750 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 108/210] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:23:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:23:42.118490 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9b8408-b3d6-4f5a-a4f2-ad9303253cb5 None None] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:42.119059 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9b8408-b3d6-4f5a-a4f2-ad9303253cb5 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:42.119549 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9b8408-b3d6-4f5a-a4f2-ad9303253cb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:42.133913 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:42.141635 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.auth.core [None req-6b9b8408-b3d6-4f5a-a4f2-ad9303253cb5 None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143586) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:23:42.142926 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 103/211] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:23:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:23:42.143428 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.fernet_utils [None req-6b9b8408-b3d6-4f5a-a4f2-ad9303253cb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143586) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:23:42.168739 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558876a9-c121-4ebe-beb6-160ac33feae9 None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:42.233914 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 109/212] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:23:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:23:42.260351 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9b8408-b3d6-4f5a-a4f2-ad9303253cb5 None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:42.293594 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558876a9-c121-4ebe-beb6-160ac33feae9 None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:42.330856 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c97db5d-7307-4e65-b5b8-ffe19597fc7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:42.331702 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c97db5d-7307-4e65-b5b8-ffe19597fc7c None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:42.331989 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c97db5d-7307-4e65-b5b8-ffe19597fc7c None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:42.332255 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c97db5d-7307-4e65-b5b8-ffe19597fc7c None admin] PATH_INFO: `/v3/roles/member` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:42.344318 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c97db5d-7307-4e65-b5b8-ffe19597fc7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:23:42.352347 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-9c97db5d-7307-4e65-b5b8-ffe19597fc7c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:23:42.352347 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:23:42.352347 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:23:42.352347 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:23:42.352347 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:23:42.352347 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:23:42.352347 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:23:42.352347 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:23:42.352347 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:23:42.352347 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:23:42.352347 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:23:42.352347 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:23:42.352347 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:23:42.352347 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:23:42.352347 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:23:42.352347 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:23:42.353387 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:23:42.353387 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:23:42.353387 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:23:42.353387 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:23:42.353387 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:23:42.353387 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:23:42.353387 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:23:42.353387 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:23:42.353387 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:23:42.353387 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:23:42.353387 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:23:42.353387 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:23:42.353387 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:23:42.353387 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:23:42.353387 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:23:42.353387 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:23:42.359736 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:23:42.359736 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:23:42.359736 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:23:42.359736 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:23:42.359736 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:23:42.359736 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:23:42.359736 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:23:42.359736 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:23:42.359736 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:23:42.359736 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:23:42.359736 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:23:42.359736 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:23:42.359736 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 104/213] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:23:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:23:42.359736 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c97db5d-7307-4e65-b5b8-ffe19597fc7c None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:42.371259 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:42.381107 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9b8408-b3d6-4f5a-a4f2-ad9303253cb5 None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:42.419853 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd0b1cc-b2f4-41b4-b9b4-dc8122399d68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:42.421073 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd0b1cc-b2f4-41b4-b9b4-dc8122399d68 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:42.421688 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd0b1cc-b2f4-41b4-b9b4-dc8122399d68 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:42.422230 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd0b1cc-b2f4-41b4-b9b4-dc8122399d68 None admin] PATH_INFO: `/v3/domains/default` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:42.423759 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd0b1cc-b2f4-41b4-b9b4-dc8122399d68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:42.425653 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd0b1cc-b2f4-41b4-b9b4-dc8122399d68 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:42.432249 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 110/214] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:23:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:42.433561 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd0b1cc-b2f4-41b4-b9b4-dc8122399d68 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:42.438125 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:42.465874 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c97db5d-7307-4e65-b5b8-ffe19597fc7c None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:42.502374 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef8cedc-cec0-4ba8-8c61-1723429eb137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:42.502998 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef8cedc-cec0-4ba8-8c61-1723429eb137 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:42.503343 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef8cedc-cec0-4ba8-8c61-1723429eb137 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:42.503623 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef8cedc-cec0-4ba8-8c61-1723429eb137 None admin] PATH_INFO: `/v3/roles` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:42.504420 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef8cedc-cec0-4ba8-8c61-1723429eb137 None admin] RBAC: Adding query filter params (name=member) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:23:42.504804 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef8cedc-cec0-4ba8-8c61-1723429eb137 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:42.506170 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef8cedc-cec0-4ba8-8c61-1723429eb137 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:42.519615 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 105/215] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:23:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:23:42.520654 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef8cedc-cec0-4ba8-8c61-1723429eb137 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:42.567461 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd0b1cc-b2f4-41b4-b9b4-dc8122399d68 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:42.612988 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd166b53-b82c-41d5-9b8c-64fc6f2bf0ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:42.614076 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd166b53-b82c-41d5-9b8c-64fc6f2bf0ab None admin] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:42.614643 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:42.615127 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd166b53-b82c-41d5-9b8c-64fc6f2bf0ab None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:42.616245 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd166b53-b82c-41d5-9b8c-64fc6f2bf0ab None admin] PATH_INFO: `/v3/users` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:42.617209 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd166b53-b82c-41d5-9b8c-64fc6f2bf0ab None admin] RBAC: Authorizing `identity:create_user()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:42.618901 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd166b53-b82c-41d5-9b8c-64fc6f2bf0ab None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:42.644027 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 111/216] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 19 14:23:42 2022] POST /identity/v3/users => generated 297 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:23:42.650789 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef8cedc-cec0-4ba8-8c61-1723429eb137 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:42.692060 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16878630-c452-425a-8f53-811c7661848d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:42.693132 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16878630-c452-425a-8f53-811c7661848d None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:42.693598 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16878630-c452-425a-8f53-811c7661848d None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:42.694011 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16878630-c452-425a-8f53-811c7661848d None admin] PATH_INFO: `/v3/users/c441f5a4e9a94ad1872cdc44fab83b7c` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:42.710534 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16878630-c452-425a-8f53-811c7661848d None admin] RBAC: Authorizing `identity:get_user(user_id=c441f5a4e9a94ad1872cdc44fab83b7c)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:42.712254 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16878630-c452-425a-8f53-811c7661848d None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:42.714598 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 106/217] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:23:42 2022] GET /identity/v3/users/c441f5a4e9a94ad1872cdc44fab83b7c => generated 297 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:42.720863 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd166b53-b82c-41d5-9b8c-64fc6f2bf0ab None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:42.750365 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a9beec-78d8-49f7-8474-957abbb18378 None None] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:42.750798 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a9beec-78d8-49f7-8474-957abbb18378 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:42.750965 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a9beec-78d8-49f7-8474-957abbb18378 None None] PATH_INFO: `/` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:42.752298 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 107/218] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:23:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:23:42.758227 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcb506e-f88d-4e3d-8f65-2a803941849f None None] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:42.758654 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcb506e-f88d-4e3d-8f65-2a803941849f None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:42.759034 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcb506e-f88d-4e3d-8f65-2a803941849f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:42.777829 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.auth.core [None req-2dcb506e-f88d-4e3d-8f65-2a803941849f None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143587) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:23:42.779305 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-2dcb506e-f88d-4e3d-8f65-2a803941849f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:23:42.796753 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:42.851012 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd166b53-b82c-41d5-9b8c-64fc6f2bf0ab None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:42.882368 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 108/219] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:23:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:23:42.898399 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92284a02-97c9-4411-a559-c574c9626452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:42.898983 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcb506e-f88d-4e3d-8f65-2a803941849f None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:42.899102 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92284a02-97c9-4411-a559-c574c9626452 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:42.899315 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92284a02-97c9-4411-a559-c574c9626452 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:42.899594 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92284a02-97c9-4411-a559-c574c9626452 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:42.982184 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:42.997687 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92284a02-97c9-4411-a559-c574c9626452 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:43.000349 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92284a02-97c9-4411-a559-c574c9626452 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:43.039345 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcb506e-f88d-4e3d-8f65-2a803941849f None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:43.083318 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:43.090723 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d22341-60cb-44ba-ad3c-5f892c1f19de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:43.091562 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d22341-60cb-44ba-ad3c-5f892c1f19de None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:43.091848 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d22341-60cb-44ba-ad3c-5f892c1f19de None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:43.092291 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d22341-60cb-44ba-ad3c-5f892c1f19de None admin] PATH_INFO: `/v3/roles/admin` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:43.106210 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d22341-60cb-44ba-ad3c-5f892c1f19de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:23:43.114922 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-15d22341-60cb-44ba-ad3c-5f892c1f19de None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:23:43.114922 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:23:43.114922 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:23:43.114922 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:23:43.114922 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:23:43.114922 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:23:43.114922 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:23:43.114922 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:23:43.114922 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:23:43.114922 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:23:43.114922 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:23:43.114922 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:23:43.114922 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:23:43.114922 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:23:43.114922 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:23:43.114922 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:23:43.116297 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:23:43.116297 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:23:43.116297 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:23:43.116297 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:23:43.116297 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:23:43.116297 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:23:43.116297 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:23:43.116297 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:23:43.116297 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:23:43.116297 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:23:43.116297 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:23:43.116297 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:23:43.116297 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:23:43.116297 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:23:43.116297 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:23:43.116297 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:23:43.117445 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:23:43.117445 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:23:43.117445 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:23:43.117445 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:23:43.117445 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:23:43.117445 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:23:43.117445 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:23:43.117445 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:23:43.117445 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:23:43.117445 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:23:43.117445 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:23:43.117445 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:23:43.117445 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 109/220] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:23:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:23:43.122410 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d22341-60cb-44ba-ad3c-5f892c1f19de None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:43.146369 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 112/221] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:23:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:23:43.155221 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92284a02-97c9-4411-a559-c574c9626452 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:43.247415 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d22341-60cb-44ba-ad3c-5f892c1f19de None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:43.276951 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:43.308194 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92284a02-97c9-4411-a559-c574c9626452 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:43.311806 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6959cb90-465e-4354-bf60-c7bb62d537f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:43.312963 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6959cb90-465e-4354-bf60-c7bb62d537f5 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:43.313471 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6959cb90-465e-4354-bf60-c7bb62d537f5 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:43.314016 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6959cb90-465e-4354-bf60-c7bb62d537f5 None admin] PATH_INFO: `/v3/roles` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:43.314998 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6959cb90-465e-4354-bf60-c7bb62d537f5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:23:43.315564 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6959cb90-465e-4354-bf60-c7bb62d537f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:43.317278 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6959cb90-465e-4354-bf60-c7bb62d537f5 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:43.329479 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 110/222] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:23:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:43.331973 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2d1201-cace-4cbc-a48d-da17c0979681 None None] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:43.333188 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2d1201-cace-4cbc-a48d-da17c0979681 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:43.333777 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2d1201-cace-4cbc-a48d-da17c0979681 None None] PATH_INFO: `/` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:43.335861 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 111/223] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:23:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:23:43.337119 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2d1201-cace-4cbc-a48d-da17c0979681 None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:43.361222 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26beb8f6-9d12-4ab1-988f-72b6ac3a3f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:43.362055 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26beb8f6-9d12-4ab1-988f-72b6ac3a3f27 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:43.362344 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26beb8f6-9d12-4ab1-988f-72b6ac3a3f27 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:43.362625 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26beb8f6-9d12-4ab1-988f-72b6ac3a3f27 None admin] PATH_INFO: `/v3/projects/24fafd1589784882b754b5f8543954bb` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:43.368516 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26beb8f6-9d12-4ab1-988f-72b6ac3a3f27 None admin] RBAC: Authorizing `identity:get_project(project_id=24fafd1589784882b754b5f8543954bb)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:43.370149 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26beb8f6-9d12-4ab1-988f-72b6ac3a3f27 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:43.371901 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 113/224] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:23:43 2022] GET /identity/v3/projects/24fafd1589784882b754b5f8543954bb => generated 302 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:43.376614 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb16efc6-328c-4366-9adf-103d18af1a3e None None] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:43.376912 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb16efc6-328c-4366-9adf-103d18af1a3e None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:43.377196 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb16efc6-328c-4366-9adf-103d18af1a3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:43.415116 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.auth.core [None req-bb16efc6-328c-4366-9adf-103d18af1a3e None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143586) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:23:43.416347 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.fernet_utils [None req-bb16efc6-328c-4366-9adf-103d18af1a3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143586) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:23:43.457417 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:43.490440 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2d1201-cace-4cbc-a48d-da17c0979681 None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:43.528848 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105be0e2-1f33-4d71-91e2-3317d80e893f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:43.530328 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105be0e2-1f33-4d71-91e2-3317d80e893f None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:43.530864 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105be0e2-1f33-4d71-91e2-3317d80e893f None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:43.531411 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105be0e2-1f33-4d71-91e2-3317d80e893f None admin] PATH_INFO: `/v3/users/89cde8a97d784539962d5992b9d7727f` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:43.532999 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105be0e2-1f33-4d71-91e2-3317d80e893f None admin] RBAC: Authorizing `identity:get_user(user_id=89cde8a97d784539962d5992b9d7727f)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:43.534834 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105be0e2-1f33-4d71-91e2-3317d80e893f None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:43.537482 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 112/225] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:23:43 2022] GET /identity/v3/users/89cde8a97d784539962d5992b9d7727f => generated 283 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:43.538641 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105be0e2-1f33-4d71-91e2-3317d80e893f None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:43.556843 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 114/226] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:23:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:23:43.557973 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb16efc6-328c-4366-9adf-103d18af1a3e None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:43.600242 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:43.678754 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105be0e2-1f33-4d71-91e2-3317d80e893f None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:43.743981 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492438a7-e978-4fd7-9fde-da233727bbab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:43.748627 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492438a7-e978-4fd7-9fde-da233727bbab None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:43.748627 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492438a7-e978-4fd7-9fde-da233727bbab None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:43.748627 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492438a7-e978-4fd7-9fde-da233727bbab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:43.748627 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492438a7-e978-4fd7-9fde-da233727bbab None admin] RBAC: Adding query filter params (role.id=3d5075293a4f45449eaef6af8fa53004, scope.project.id=24fafd1589784882b754b5f8543954bb, user.id=c441f5a4e9a94ad1872cdc44fab83b7c) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:23:43.748627 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492438a7-e978-4fd7-9fde-da233727bbab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:43.752400 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492438a7-e978-4fd7-9fde-da233727bbab None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:43.764455 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 113/227] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:23:43 2022] GET /identity/v3/role_assignments?user.id=c441f5a4e9a94ad1872cdc44fab83b7c&scope.project.id=24fafd1589784882b754b5f8543954bb&role.id=3d5075293a4f45449eaef6af8fa53004 => generated 259 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:43.765645 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492438a7-e978-4fd7-9fde-da233727bbab None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:43.781880 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb16efc6-328c-4366-9adf-103d18af1a3e None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:43.859617 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eae72b0-37fe-4cd9-b692-9ec4a132329f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:43.860852 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eae72b0-37fe-4cd9-b692-9ec4a132329f None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:43.861331 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eae72b0-37fe-4cd9-b692-9ec4a132329f None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:43.861791 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eae72b0-37fe-4cd9-b692-9ec4a132329f None admin] PATH_INFO: `/v3/projects/24fafd1589784882b754b5f8543954bb` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:43.866848 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eae72b0-37fe-4cd9-b692-9ec4a132329f None admin] RBAC: Authorizing `identity:get_project(project_id=24fafd1589784882b754b5f8543954bb)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:43.868659 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eae72b0-37fe-4cd9-b692-9ec4a132329f None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:43.870691 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 115/228] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:23:43 2022] GET /identity/v3/projects/24fafd1589784882b754b5f8543954bb => generated 302 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:43.876726 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eae72b0-37fe-4cd9-b692-9ec4a132329f None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:43.933369 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492438a7-e978-4fd7-9fde-da233727bbab None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:43.944022 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:43.952795 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:43.982286 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee4e66e-8683-4de8-a9bc-3180b9fdd9e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:43.983829 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee4e66e-8683-4de8-a9bc-3180b9fdd9e3 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:43.984121 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee4e66e-8683-4de8-a9bc-3180b9fdd9e3 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:43.984399 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee4e66e-8683-4de8-a9bc-3180b9fdd9e3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:43.998199 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee4e66e-8683-4de8-a9bc-3180b9fdd9e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:23:44.008278 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-dee4e66e-8683-4de8-a9bc-3180b9fdd9e3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:23:44.008278 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:23:44.008278 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:23:44.008278 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:23:44.008278 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:23:44.008278 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:23:44.008278 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:23:44.008278 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:23:44.008278 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:23:44.008278 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:23:44.008278 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:23:44.008278 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:23:44.008278 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:23:44.008278 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:23:44.008278 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:23:44.008278 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:23:44.009322 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:23:44.009322 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:23:44.009322 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:23:44.009322 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:23:44.009322 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:23:44.009322 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:23:44.009322 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:23:44.009322 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:23:44.009322 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:23:44.009322 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:23:44.009322 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:23:44.009322 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:23:44.009322 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:23:44.009322 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:23:44.009322 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:23:44.009322 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:23:44.010253 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:23:44.010253 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:23:44.010253 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:23:44.010253 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:23:44.010253 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:23:44.010253 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:23:44.010253 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:23:44.010253 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:23:44.010253 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:23:44.010253 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:23:44.010253 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:23:44.010253 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:23:44.010253 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 114/229] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 19 14:23:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:23:44.017505 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee4e66e-8683-4de8-a9bc-3180b9fdd9e3 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:44.105040 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eae72b0-37fe-4cd9-b692-9ec4a132329f None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:44.134487 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee4e66e-8683-4de8-a9bc-3180b9fdd9e3 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:44.174975 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59222014-c9ce-408f-a35f-aea27cc72f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:44.175810 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59222014-c9ce-408f-a35f-aea27cc72f9e None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:44.176193 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59222014-c9ce-408f-a35f-aea27cc72f9e None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:44.176550 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59222014-c9ce-408f-a35f-aea27cc72f9e None admin] PATH_INFO: `/v3/roles` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:44.177390 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59222014-c9ce-408f-a35f-aea27cc72f9e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:23:44.177775 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59222014-c9ce-408f-a35f-aea27cc72f9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:44.179505 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59222014-c9ce-408f-a35f-aea27cc72f9e None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:44.186747 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a7b6df-7396-4025-8532-87bff639f930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:44.187499 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a7b6df-7396-4025-8532-87bff639f930 None admin] REQUEST_METHOD: `PUT` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:44.187784 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a7b6df-7396-4025-8532-87bff639f930 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:44.188062 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a7b6df-7396-4025-8532-87bff639f930 None admin] PATH_INFO: `/v3/projects/24fafd1589784882b754b5f8543954bb/users/89cde8a97d784539962d5992b9d7727f/roles/a3b793754e7b40b0bb21afe74eec3c18` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:44.189464 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 115/230] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:23:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:44.195064 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a7b6df-7396-4025-8532-87bff639f930 None admin] RBAC: Authorizing `identity:create_grant(project_id=24fafd1589784882b754b5f8543954bb, user_id=89cde8a97d784539962d5992b9d7727f, role_id=a3b793754e7b40b0bb21afe74eec3c18)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:44.196049 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59222014-c9ce-408f-a35f-aea27cc72f9e None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:44.196529 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a7b6df-7396-4025-8532-87bff639f930 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:44.223363 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 116/231] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 14:23:43 2022] PUT /identity/v3/projects/24fafd1589784882b754b5f8543954bb/users/89cde8a97d784539962d5992b9d7727f/roles/a3b793754e7b40b0bb21afe74eec3c18 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 14:23:44.260280 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:44.348859 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59222014-c9ce-408f-a35f-aea27cc72f9e None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:44.392853 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f474e160-d9fc-402e-99fb-bf3b369edd6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:44.393812 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f474e160-d9fc-402e-99fb-bf3b369edd6b None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:44.394124 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f474e160-d9fc-402e-99fb-bf3b369edd6b None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:44.394401 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f474e160-d9fc-402e-99fb-bf3b369edd6b None admin] PATH_INFO: `/v3/users/89cde8a97d784539962d5992b9d7727f` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:44.395752 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f474e160-d9fc-402e-99fb-bf3b369edd6b None admin] RBAC: Authorizing `identity:get_user(user_id=89cde8a97d784539962d5992b9d7727f)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:44.397250 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f474e160-d9fc-402e-99fb-bf3b369edd6b None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:44.399301 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 116/232] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:23:44 2022] GET /identity/v3/users/89cde8a97d784539962d5992b9d7727f => generated 283 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:44.405653 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a7b6df-7396-4025-8532-87bff639f930 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:44.519706 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:44.535442 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a7b6df-7396-4025-8532-87bff639f930 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:44.574980 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fed5a0-696a-439b-a544-522da2456c35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:44.575851 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fed5a0-696a-439b-a544-522da2456c35 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:44.576133 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fed5a0-696a-439b-a544-522da2456c35 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:44.576439 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fed5a0-696a-439b-a544-522da2456c35 None admin] PATH_INFO: `/v3/projects/24fafd1589784882b754b5f8543954bb` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:44.577760 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fed5a0-696a-439b-a544-522da2456c35 None admin] RBAC: Authorizing `identity:get_project(project_id=24fafd1589784882b754b5f8543954bb)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:44.579248 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fed5a0-696a-439b-a544-522da2456c35 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:44.581263 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 117/233] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:23:44 2022] GET /identity/v3/projects/24fafd1589784882b754b5f8543954bb => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:44.586992 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f474e160-d9fc-402e-99fb-bf3b369edd6b None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:44.722047 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f474e160-d9fc-402e-99fb-bf3b369edd6b None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:44.723670 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:44.758936 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa0d17d-d0a5-4902-9b01-446701616d49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:44.759838 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa0d17d-d0a5-4902-9b01-446701616d49 None admin] REQUEST_METHOD: `PUT` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:44.760129 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa0d17d-d0a5-4902-9b01-446701616d49 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:44.760411 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa0d17d-d0a5-4902-9b01-446701616d49 None admin] PATH_INFO: `/v3/projects/24fafd1589784882b754b5f8543954bb/users/89cde8a97d784539962d5992b9d7727f/roles/b954b2c7ce764aa0afac78e075a011a6` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:44.762740 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa0d17d-d0a5-4902-9b01-446701616d49 None admin] RBAC: Authorizing `identity:create_grant(project_id=24fafd1589784882b754b5f8543954bb, user_id=89cde8a97d784539962d5992b9d7727f, role_id=b954b2c7ce764aa0afac78e075a011a6)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:44.764404 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa0d17d-d0a5-4902-9b01-446701616d49 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:44.788274 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 117/234] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 14:23:44 2022] PUT /identity/v3/projects/24fafd1589784882b754b5f8543954bb/users/89cde8a97d784539962d5992b9d7727f/roles/b954b2c7ce764aa0afac78e075a011a6 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:23:45.634361 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22742695-3e6f-4e44-ae5f-91d456d4335d None None] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:45.634882 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22742695-3e6f-4e44-ae5f-91d456d4335d None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:45.635005 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22742695-3e6f-4e44-ae5f-91d456d4335d None None] PATH_INFO: `/` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:45.635287 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75400352-385d-41eb-b722-8f1b585d3f63 None None] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:45.635657 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75400352-385d-41eb-b722-8f1b585d3f63 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:45.635823 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75400352-385d-41eb-b722-8f1b585d3f63 None None] PATH_INFO: `/` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:45.636299 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 118/235] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:23:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:23:45.637684 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 118/236] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:23:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:23:45.642904 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4b44ec-503a-4b7f-81cf-fcaec0ecab00 None None] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:45.643194 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4b44ec-503a-4b7f-81cf-fcaec0ecab00 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:45.643470 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4b44ec-503a-4b7f-81cf-fcaec0ecab00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:45.654984 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c07af9-084e-4e57-a89b-d0fba1d80269 None None] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:45.655292 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c07af9-084e-4e57-a89b-d0fba1d80269 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:45.655546 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c07af9-084e-4e57-a89b-d0fba1d80269 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:45.681615 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.auth.core [None req-a8c07af9-084e-4e57-a89b-d0fba1d80269 None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143587) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:23:45.682870 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-a8c07af9-084e-4e57-a89b-d0fba1d80269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:23:45.683962 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.auth.core [None req-6c4b44ec-503a-4b7f-81cf-fcaec0ecab00 None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143586) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:23:45.685179 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.fernet_utils [None req-6c4b44ec-503a-4b7f-81cf-fcaec0ecab00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143586) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:23:45.775353 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 119/237] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:23:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:23:45.803870 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c07af9-084e-4e57-a89b-d0fba1d80269 None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:45.813306 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 119/238] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:23:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:23:45.851697 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4b44ec-503a-4b7f-81cf-fcaec0ecab00 None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:45.942537 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:45.961426 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4b44ec-503a-4b7f-81cf-fcaec0ecab00 None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:45.997724 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9417f720-5af7-4219-b146-13afcff4eddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:45.999092 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9417f720-5af7-4219-b146-13afcff4eddb None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:45.999377 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9417f720-5af7-4219-b146-13afcff4eddb None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:45.999654 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9417f720-5af7-4219-b146-13afcff4eddb None admin] PATH_INFO: `/v3/domains/default` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:46.000242 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:46.000870 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9417f720-5af7-4219-b146-13afcff4eddb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:46.002369 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9417f720-5af7-4219-b146-13afcff4eddb None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:46.004022 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 120/239] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:23:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:23:46.009535 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9417f720-5af7-4219-b146-13afcff4eddb None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:46.023803 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c07af9-084e-4e57-a89b-d0fba1d80269 None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:46.089623 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c517354-ab57-452c-a32a-cc6bdfa96271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:46.090458 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c517354-ab57-452c-a32a-cc6bdfa96271 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:46.090740 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c517354-ab57-452c-a32a-cc6bdfa96271 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:46.091018 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c517354-ab57-452c-a32a-cc6bdfa96271 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:46.126917 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c517354-ab57-452c-a32a-cc6bdfa96271 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:23:46.134734 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-3c517354-ab57-452c-a32a-cc6bdfa96271 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:23:46.134734 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:23:46.134734 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:23:46.134734 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:23:46.134734 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:23:46.134734 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:23:46.134734 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:23:46.134734 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:23:46.134734 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:23:46.134734 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:23:46.134734 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:23:46.134734 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:23:46.134734 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:23:46.134734 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:23:46.134734 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:23:46.134734 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:23:46.135759 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:23:46.135759 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:23:46.135759 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:23:46.135759 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:23:46.135759 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:23:46.135759 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:23:46.135759 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:23:46.135759 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:23:46.135759 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:23:46.135759 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:23:46.135759 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:23:46.135759 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:23:46.135759 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:23:46.135759 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:23:46.135759 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:23:46.135759 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:23:46.136719 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:23:46.136719 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:23:46.136719 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:23:46.136719 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:23:46.136719 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:23:46.136719 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:23:46.136719 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:23:46.136719 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:23:46.136719 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:23:46.136719 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:23:46.136719 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:23:46.136719 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:23:46.142309 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 120/240] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:23:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:23:46.149129 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c517354-ab57-452c-a32a-cc6bdfa96271 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:46.153127 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9417f720-5af7-4219-b146-13afcff4eddb None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:46.191197 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716a5f5d-c762-485c-82fe-fc12ec43671f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:46.191926 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716a5f5d-c762-485c-82fe-fc12ec43671f None admin] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:46.192211 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716a5f5d-c762-485c-82fe-fc12ec43671f None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:46.192492 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716a5f5d-c762-485c-82fe-fc12ec43671f None admin] PATH_INFO: `/v3/users` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:46.193272 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716a5f5d-c762-485c-82fe-fc12ec43671f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:46.194695 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716a5f5d-c762-485c-82fe-fc12ec43671f None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:46.223037 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 121/241] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 19 14:23:46 2022] POST /identity/v3/users => generated 293 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 14:23:46.334260 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c517354-ab57-452c-a32a-cc6bdfa96271 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:46.372710 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73439de5-60a4-491a-a535-108436946a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:46.373560 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73439de5-60a4-491a-a535-108436946a06 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:46.374099 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73439de5-60a4-491a-a535-108436946a06 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:46.374424 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73439de5-60a4-491a-a535-108436946a06 None admin] PATH_INFO: `/v3/roles` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:46.375204 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73439de5-60a4-491a-a535-108436946a06 None admin] RBAC: Adding query filter params (name=reader) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:23:46.375516 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73439de5-60a4-491a-a535-108436946a06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:46.376948 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73439de5-60a4-491a-a535-108436946a06 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:46.386786 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 121/242] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:23:46 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:46.393398 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716a5f5d-c762-485c-82fe-fc12ec43671f None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:46.407239 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:46.513383 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716a5f5d-c762-485c-82fe-fc12ec43671f None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:46.550415 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea211e9-f81b-4e9c-ad49-597324d1ee2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:46.551303 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea211e9-f81b-4e9c-ad49-597324d1ee2a None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:46.551654 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea211e9-f81b-4e9c-ad49-597324d1ee2a None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:46.551932 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea211e9-f81b-4e9c-ad49-597324d1ee2a None admin] PATH_INFO: `/v3/users/e8b14cd4ac074742a8c629d0a42902b8` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:46.566650 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea211e9-f81b-4e9c-ad49-597324d1ee2a None admin] RBAC: Authorizing `identity:get_user(user_id=e8b14cd4ac074742a8c629d0a42902b8)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:46.568185 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea211e9-f81b-4e9c-ad49-597324d1ee2a None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:46.570255 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 122/243] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:23:46 2022] GET /identity/v3/users/e8b14cd4ac074742a8c629d0a42902b8 => generated 297 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:46.575938 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73439de5-60a4-491a-a535-108436946a06 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:46.593615 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:46.705724 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73439de5-60a4-491a-a535-108436946a06 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:46.744869 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106ff266-6e2a-4391-afad-611232a9902f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:46.745808 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106ff266-6e2a-4391-afad-611232a9902f None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:46.746275 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106ff266-6e2a-4391-afad-611232a9902f None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:46.746709 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106ff266-6e2a-4391-afad-611232a9902f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:46.809985 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbb68af-53c8-4ab5-8c34-bdc58b54d5ae None None] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:46.810553 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbb68af-53c8-4ab5-8c34-bdc58b54d5ae None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:46.810766 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbb68af-53c8-4ab5-8c34-bdc58b54d5ae None None] PATH_INFO: `/` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:46.812564 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 123/244] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:23:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:23:46.817968 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a6023c-f74c-4166-b44f-a6f30dac820a None None] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:46.818457 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a6023c-f74c-4166-b44f-a6f30dac820a None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:46.818898 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a6023c-f74c-4166-b44f-a6f30dac820a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:46.825117 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106ff266-6e2a-4391-afad-611232a9902f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:46.827500 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106ff266-6e2a-4391-afad-611232a9902f None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:46.838221 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.auth.core [None req-43a6023c-f74c-4166-b44f-a6f30dac820a None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143586) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:23:46.841503 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.fernet_utils [None req-43a6023c-f74c-4166-b44f-a6f30dac820a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143586) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:23:46.847406 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:46.916485 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:46.927684 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 124/245] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:23:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:23:46.938841 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 122/246] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:23:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:23:46.944490 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a6023c-f74c-4166-b44f-a6f30dac820a None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:46.954456 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106ff266-6e2a-4391-afad-611232a9902f None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:47.073329 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a6023c-f74c-4166-b44f-a6f30dac820a None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:47.089239 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106ff266-6e2a-4391-afad-611232a9902f None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:47.114216 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:47.121640 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba020222-39fe-4e17-a5c0-eff8e27cd1af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:47.123600 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba020222-39fe-4e17-a5c0-eff8e27cd1af None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:47.124781 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba020222-39fe-4e17-a5c0-eff8e27cd1af None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:47.124781 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba020222-39fe-4e17-a5c0-eff8e27cd1af None admin] PATH_INFO: `/v3/projects/24fafd1589784882b754b5f8543954bb` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:47.126283 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba020222-39fe-4e17-a5c0-eff8e27cd1af None admin] RBAC: Authorizing `identity:get_project(project_id=24fafd1589784882b754b5f8543954bb)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:47.128309 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba020222-39fe-4e17-a5c0-eff8e27cd1af None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:47.131205 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 125/247] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:23:46 2022] GET /identity/v3/projects/24fafd1589784882b754b5f8543954bb => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:47.133415 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5d7121-6863-43dc-87e9-a552a5fc7e28 None None] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:47.133710 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5d7121-6863-43dc-87e9-a552a5fc7e28 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:47.134025 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5d7121-6863-43dc-87e9-a552a5fc7e28 None None] PATH_INFO: `/` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:47.135454 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 126/248] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:23:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:23:47.138498 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc49514-e3a7-4c2d-8d19-1f15fd4b7ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:47.139862 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc49514-e3a7-4c2d-8d19-1f15fd4b7ffc None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:47.140147 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc49514-e3a7-4c2d-8d19-1f15fd4b7ffc None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:47.140496 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5d7121-6863-43dc-87e9-a552a5fc7e28 None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:47.140631 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc49514-e3a7-4c2d-8d19-1f15fd4b7ffc None admin] PATH_INFO: `/v3/roles/member` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:47.153517 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc49514-e3a7-4c2d-8d19-1f15fd4b7ffc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:23:47.162179 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-7bc49514-e3a7-4c2d-8d19-1f15fd4b7ffc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:23:47.162179 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:23:47.162179 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:23:47.162179 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:23:47.162179 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:23:47.162179 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:23:47.162179 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:23:47.162179 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:23:47.162179 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:23:47.162179 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:23:47.162179 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:23:47.162179 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:23:47.162179 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:23:47.162179 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:23:47.162179 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:23:47.162179 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:23:47.164446 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:23:47.164446 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:23:47.164446 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:23:47.164446 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:23:47.164446 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:23:47.164446 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:23:47.164446 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:23:47.164446 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:23:47.164446 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:23:47.164446 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:23:47.164446 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:23:47.164446 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:23:47.164446 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:23:47.164446 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:23:47.164446 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:23:47.164446 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:23:47.166535 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:23:47.166535 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:23:47.166535 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:23:47.166535 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:23:47.166535 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:23:47.166535 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:23:47.166535 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:23:47.166535 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:23:47.166535 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:23:47.166535 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:23:47.166535 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:23:47.166535 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:23:47.166535 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 123/249] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:23:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:23:47.166535 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07d2704-feda-41af-ac8e-179254ab2e40 None None] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:47.168699 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07d2704-feda-41af-ac8e-179254ab2e40 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:47.168699 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07d2704-feda-41af-ac8e-179254ab2e40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:47.192265 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.auth.core [None req-d07d2704-feda-41af-ac8e-179254ab2e40 None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143587) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:23:47.194213 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-d07d2704-feda-41af-ac8e-179254ab2e40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:23:47.227105 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:47.290301 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 124/250] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:23:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:23:47.291141 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07d2704-feda-41af-ac8e-179254ab2e40 None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:47.301478 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5d7121-6863-43dc-87e9-a552a5fc7e28 None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:47.373641 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a9c411-cf5a-42fd-b4a5-71c845230813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:47.375525 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a9c411-cf5a-42fd-b4a5-71c845230813 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:47.375809 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a9c411-cf5a-42fd-b4a5-71c845230813 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:47.376085 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a9c411-cf5a-42fd-b4a5-71c845230813 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:47.376936 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a9c411-cf5a-42fd-b4a5-71c845230813 None admin] RBAC: Adding query filter params (role.id=abff822b2f9d4abd96ff7414f6de1670, scope.project.id=24fafd1589784882b754b5f8543954bb, user.id=e8b14cd4ac074742a8c629d0a42902b8) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:23:47.377257 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a9c411-cf5a-42fd-b4a5-71c845230813 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:47.378754 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a9c411-cf5a-42fd-b4a5-71c845230813 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:47.395070 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 127/251] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:23:47 2022] GET /identity/v3/role_assignments?user.id=e8b14cd4ac074742a8c629d0a42902b8&scope.project.id=24fafd1589784882b754b5f8543954bb&role.id=abff822b2f9d4abd96ff7414f6de1670 => generated 259 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:47.395932 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a9c411-cf5a-42fd-b4a5-71c845230813 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:47.446413 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07d2704-feda-41af-ac8e-179254ab2e40 None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:47.450947 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:47.557240 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4922a227-8511-4b64-a1e1-eb45b93fc1cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:47.558156 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4922a227-8511-4b64-a1e1-eb45b93fc1cf None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:47.558351 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4922a227-8511-4b64-a1e1-eb45b93fc1cf None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:47.558624 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4922a227-8511-4b64-a1e1-eb45b93fc1cf None admin] PATH_INFO: `/v3/roles` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:47.559402 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4922a227-8511-4b64-a1e1-eb45b93fc1cf None admin] RBAC: Adding query filter params (name=member) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:23:47.559724 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4922a227-8511-4b64-a1e1-eb45b93fc1cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:47.561333 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4922a227-8511-4b64-a1e1-eb45b93fc1cf None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:47.566244 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a9c411-cf5a-42fd-b4a5-71c845230813 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:47.580904 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 125/252] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:23:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:47.586683 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4922a227-8511-4b64-a1e1-eb45b93fc1cf None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:47.601669 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:47.611134 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462d06b5-8899-4aae-922b-43261350d494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:47.612415 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462d06b5-8899-4aae-922b-43261350d494 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:47.612940 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462d06b5-8899-4aae-922b-43261350d494 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:47.613471 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462d06b5-8899-4aae-922b-43261350d494 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:47.633515 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462d06b5-8899-4aae-922b-43261350d494 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:23:47.648892 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-462d06b5-8899-4aae-922b-43261350d494 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:23:47.648892 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:23:47.648892 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:23:47.648892 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:23:47.648892 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:23:47.648892 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:23:47.648892 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:23:47.648892 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:23:47.648892 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:23:47.648892 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:23:47.648892 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:23:47.648892 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:23:47.648892 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:23:47.648892 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:23:47.648892 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:23:47.648892 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:23:47.650112 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:23:47.650112 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:23:47.650112 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:23:47.650112 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:23:47.650112 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:23:47.650112 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:23:47.650112 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:23:47.650112 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:23:47.650112 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:23:47.650112 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:23:47.650112 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:23:47.650112 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:23:47.650112 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:23:47.650112 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:23:47.650112 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:23:47.650112 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:23:47.651022 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:23:47.651022 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:23:47.651022 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:23:47.651022 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:23:47.651022 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:23:47.651022 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:23:47.651022 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:23:47.651022 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:23:47.651022 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:23:47.651022 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:23:47.651022 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:23:47.651022 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:23:47.651022 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 128/253] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:23:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:23:47.663011 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462d06b5-8899-4aae-922b-43261350d494 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:47.803476 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4922a227-8511-4b64-a1e1-eb45b93fc1cf None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:47.845296 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462d06b5-8899-4aae-922b-43261350d494 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:47.854732 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffedf30-0c92-439a-93d1-da27c7aaab66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:47.855617 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffedf30-0c92-439a-93d1-da27c7aaab66 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:47.855933 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffedf30-0c92-439a-93d1-da27c7aaab66 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:47.856638 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffedf30-0c92-439a-93d1-da27c7aaab66 None admin] PATH_INFO: `/v3/users/c441f5a4e9a94ad1872cdc44fab83b7c` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:47.858943 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffedf30-0c92-439a-93d1-da27c7aaab66 None admin] RBAC: Authorizing `identity:get_user(user_id=c441f5a4e9a94ad1872cdc44fab83b7c)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:47.860701 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffedf30-0c92-439a-93d1-da27c7aaab66 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:47.863716 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 126/254] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:23:47 2022] GET /identity/v3/users/c441f5a4e9a94ad1872cdc44fab83b7c => generated 297 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:47.865296 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62fbebd-5a6c-4d18-acd9-9cb1afb2f511 None None] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:47.865583 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62fbebd-5a6c-4d18-acd9-9cb1afb2f511 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:47.866274 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62fbebd-5a6c-4d18-acd9-9cb1afb2f511 None None] PATH_INFO: `/` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:47.867861 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 127/255] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:23:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:23:47.868575 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62fbebd-5a6c-4d18-acd9-9cb1afb2f511 None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:47.900583 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af39753-24aa-4755-9e19-c2c0aee2b948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:47.901188 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af39753-24aa-4755-9e19-c2c0aee2b948 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:47.901471 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af39753-24aa-4755-9e19-c2c0aee2b948 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:47.901749 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af39753-24aa-4755-9e19-c2c0aee2b948 None admin] PATH_INFO: `/v3/roles` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:47.902545 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af39753-24aa-4755-9e19-c2c0aee2b948 None admin] RBAC: Adding query filter params (name=admin) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:23:47.902857 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af39753-24aa-4755-9e19-c2c0aee2b948 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:47.904860 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af39753-24aa-4755-9e19-c2c0aee2b948 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:47.924702 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 129/256] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:23:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:47.930197 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0fe7b4-5672-46a2-baaf-aa2f16ec5406 None None] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:47.930495 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0fe7b4-5672-46a2-baaf-aa2f16ec5406 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:47.930777 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0fe7b4-5672-46a2-baaf-aa2f16ec5406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:47.944258 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:47.967626 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.auth.core [None req-0f0fe7b4-5672-46a2-baaf-aa2f16ec5406 None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143586) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:23:47.969072 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.fernet_utils [None req-0f0fe7b4-5672-46a2-baaf-aa2f16ec5406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143586) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:23:48.043955 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62fbebd-5a6c-4d18-acd9-9cb1afb2f511 None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:48.068767 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:48.110263 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c3b4af-9439-4cc9-92e2-16c20dcf6f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:48.111133 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c3b4af-9439-4cc9-92e2-16c20dcf6f7b None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:48.111411 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c3b4af-9439-4cc9-92e2-16c20dcf6f7b None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:48.111686 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c3b4af-9439-4cc9-92e2-16c20dcf6f7b None admin] PATH_INFO: `/v3/projects/24fafd1589784882b754b5f8543954bb` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:48.112283 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 130/257] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:23:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:23:48.113155 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0fe7b4-5672-46a2-baaf-aa2f16ec5406 None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:48.114220 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c3b4af-9439-4cc9-92e2-16c20dcf6f7b None admin] RBAC: Authorizing `identity:get_project(project_id=24fafd1589784882b754b5f8543954bb)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:48.117721 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c3b4af-9439-4cc9-92e2-16c20dcf6f7b None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:48.120589 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 128/258] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:23:47 2022] GET /identity/v3/projects/24fafd1589784882b754b5f8543954bb => generated 302 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:48.125016 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c3b4af-9439-4cc9-92e2-16c20dcf6f7b None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:48.268155 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c3b4af-9439-4cc9-92e2-16c20dcf6f7b None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:48.269692 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:48.304663 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae3cca9-3e0d-49a8-a95e-b1687a0e5704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:48.305556 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae3cca9-3e0d-49a8-a95e-b1687a0e5704 None admin] REQUEST_METHOD: `PUT` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:48.305832 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae3cca9-3e0d-49a8-a95e-b1687a0e5704 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:48.306135 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae3cca9-3e0d-49a8-a95e-b1687a0e5704 None admin] PATH_INFO: `/v3/projects/24fafd1589784882b754b5f8543954bb/users/c441f5a4e9a94ad1872cdc44fab83b7c/roles/3d5075293a4f45449eaef6af8fa53004` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:48.308235 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae3cca9-3e0d-49a8-a95e-b1687a0e5704 None admin] RBAC: Authorizing `identity:create_grant(project_id=24fafd1589784882b754b5f8543954bb, user_id=c441f5a4e9a94ad1872cdc44fab83b7c, role_id=3d5075293a4f45449eaef6af8fa53004)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:48.309816 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae3cca9-3e0d-49a8-a95e-b1687a0e5704 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:48.327927 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 129/259] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 14:23:48 2022] PUT /identity/v3/projects/24fafd1589784882b754b5f8543954bb/users/c441f5a4e9a94ad1872cdc44fab83b7c/roles/3d5075293a4f45449eaef6af8fa53004 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:23:48.329087 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae3cca9-3e0d-49a8-a95e-b1687a0e5704 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:48.348079 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0fe7b4-5672-46a2-baaf-aa2f16ec5406 None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:48.436232 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099ae2b3-54f0-4b67-9b93-9ebaca401707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:48.437573 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099ae2b3-54f0-4b67-9b93-9ebaca401707 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:48.438134 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099ae2b3-54f0-4b67-9b93-9ebaca401707 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:48.438650 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099ae2b3-54f0-4b67-9b93-9ebaca401707 None admin] PATH_INFO: `/v3/users/89cde8a97d784539962d5992b9d7727f` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:48.441823 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099ae2b3-54f0-4b67-9b93-9ebaca401707 None admin] RBAC: Authorizing `identity:get_user(user_id=89cde8a97d784539962d5992b9d7727f)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:48.443694 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099ae2b3-54f0-4b67-9b93-9ebaca401707 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:48.445649 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 131/260] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:23:48 2022] GET /identity/v3/users/89cde8a97d784539962d5992b9d7727f => generated 283 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:48.450184 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae3cca9-3e0d-49a8-a95e-b1687a0e5704 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:48.463087 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099ae2b3-54f0-4b67-9b93-9ebaca401707 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:48.490358 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08de9c76-ecba-49a1-af75-5398e5e6ba4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:48.491187 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08de9c76-ecba-49a1-af75-5398e5e6ba4d None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:48.491472 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08de9c76-ecba-49a1-af75-5398e5e6ba4d None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:48.491747 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08de9c76-ecba-49a1-af75-5398e5e6ba4d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:48.504755 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08de9c76-ecba-49a1-af75-5398e5e6ba4d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:23:48.513273 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-08de9c76-ecba-49a1-af75-5398e5e6ba4d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:23:48.513273 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:23:48.513273 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:23:48.513273 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:23:48.513273 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:23:48.513273 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:23:48.513273 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:23:48.513273 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:23:48.513273 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:23:48.513273 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:23:48.513273 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:23:48.513273 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:23:48.513273 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:23:48.513273 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:23:48.513273 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:23:48.513273 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:23:48.515047 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:23:48.515047 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:23:48.515047 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:23:48.515047 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:23:48.515047 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:23:48.515047 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:23:48.515047 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:23:48.515047 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:23:48.515047 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:23:48.515047 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:23:48.515047 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:23:48.515047 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:23:48.515047 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:23:48.515047 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:23:48.515047 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:23:48.515047 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:23:48.516842 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:23:48.516842 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:23:48.516842 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:23:48.516842 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:23:48.516842 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:23:48.516842 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:23:48.516842 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:23:48.516842 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:23:48.516842 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:23:48.516842 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:23:48.516842 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:23:48.516842 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:23:48.516842 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 130/261] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 19 14:23:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:23:48.523552 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08de9c76-ecba-49a1-af75-5398e5e6ba4d None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:48.565996 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:48.579497 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:48.648027 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08de9c76-ecba-49a1-af75-5398e5e6ba4d None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:48.681551 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099ae2b3-54f0-4b67-9b93-9ebaca401707 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:48.691426 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32db5e8-8000-45b7-804b-82787dfa5c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:48.692107 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32db5e8-8000-45b7-804b-82787dfa5c6d None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:48.692389 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32db5e8-8000-45b7-804b-82787dfa5c6d None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:48.692665 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32db5e8-8000-45b7-804b-82787dfa5c6d None admin] PATH_INFO: `/v3/roles` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:48.693425 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32db5e8-8000-45b7-804b-82787dfa5c6d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:23:48.693736 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32db5e8-8000-45b7-804b-82787dfa5c6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:48.695178 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32db5e8-8000-45b7-804b-82787dfa5c6d None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:48.718122 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 131/262] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:23:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:48.724677 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32db5e8-8000-45b7-804b-82787dfa5c6d None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:48.761564 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73f0896-3428-4f49-8f66-92c873d1b91a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:48.762777 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73f0896-3428-4f49-8f66-92c873d1b91a None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:48.763355 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73f0896-3428-4f49-8f66-92c873d1b91a None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:48.764174 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73f0896-3428-4f49-8f66-92c873d1b91a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:48.882717 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73f0896-3428-4f49-8f66-92c873d1b91a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:48.884202 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73f0896-3428-4f49-8f66-92c873d1b91a None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:48.921622 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:48.945766 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32db5e8-8000-45b7-804b-82787dfa5c6d None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:48.995898 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:49.019316 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f212db31-f01e-4527-9407-c4d7877396bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:49.020208 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f212db31-f01e-4527-9407-c4d7877396bf None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:49.020494 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f212db31-f01e-4527-9407-c4d7877396bf None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:49.020802 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f212db31-f01e-4527-9407-c4d7877396bf None admin] PATH_INFO: `/v3/users/89cde8a97d784539962d5992b9d7727f` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:49.023076 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f212db31-f01e-4527-9407-c4d7877396bf None admin] RBAC: Authorizing `identity:get_user(user_id=89cde8a97d784539962d5992b9d7727f)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:49.024546 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f212db31-f01e-4527-9407-c4d7877396bf None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:49.034467 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 132/263] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:23:48 2022] GET /identity/v3/users/89cde8a97d784539962d5992b9d7727f => generated 283 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:23:49.040755 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f212db31-f01e-4527-9407-c4d7877396bf None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:49.153071 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 132/264] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:23:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 690 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:23:49.159190 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73f0896-3428-4f49-8f66-92c873d1b91a None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:49.207826 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f212db31-f01e-4527-9407-c4d7877396bf None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:49.250569 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6878ea4b-b75d-4f40-990c-582d350c130e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:49.251231 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6878ea4b-b75d-4f40-990c-582d350c130e None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:49.251519 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6878ea4b-b75d-4f40-990c-582d350c130e None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:49.251803 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6878ea4b-b75d-4f40-990c-582d350c130e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:49.276611 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:49.390773 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6878ea4b-b75d-4f40-990c-582d350c130e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:49.392279 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6878ea4b-b75d-4f40-990c-582d350c130e None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:49.412345 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73f0896-3428-4f49-8f66-92c873d1b91a None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:49.514928 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50dce45-b3aa-4265-a31f-69ede6beff9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:49.515813 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50dce45-b3aa-4265-a31f-69ede6beff9b None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:49.516099 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50dce45-b3aa-4265-a31f-69ede6beff9b None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:49.516383 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50dce45-b3aa-4265-a31f-69ede6beff9b None admin] PATH_INFO: `/v3/projects/24fafd1589784882b754b5f8543954bb` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:49.517682 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50dce45-b3aa-4265-a31f-69ede6beff9b None admin] RBAC: Authorizing `identity:get_project(project_id=24fafd1589784882b754b5f8543954bb)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:49.519176 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50dce45-b3aa-4265-a31f-69ede6beff9b None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:49.521234 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 133/265] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:23:49 2022] GET /identity/v3/projects/24fafd1589784882b754b5f8543954bb => generated 302 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:23:49.527955 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50dce45-b3aa-4265-a31f-69ede6beff9b None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:49.554297 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 133/266] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:23:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 514 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 14:23:49.559629 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6878ea4b-b75d-4f40-990c-582d350c130e None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:49.566050 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:49.663800 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:49.728193 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50dce45-b3aa-4265-a31f-69ede6beff9b None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:49.772489 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6878ea4b-b75d-4f40-990c-582d350c130e None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:49.806981 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406fec1a-f55d-4033-8183-883e13acdf6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:49.807652 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406fec1a-f55d-4033-8183-883e13acdf6f None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:49.807940 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406fec1a-f55d-4033-8183-883e13acdf6f None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:49.808215 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406fec1a-f55d-4033-8183-883e13acdf6f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:49.809054 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406fec1a-f55d-4033-8183-883e13acdf6f None admin] RBAC: Adding query filter params (role.id=a3b793754e7b40b0bb21afe74eec3c18, scope.project.id=24fafd1589784882b754b5f8543954bb, user.id=89cde8a97d784539962d5992b9d7727f) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:23:49.809369 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406fec1a-f55d-4033-8183-883e13acdf6f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:49.810879 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406fec1a-f55d-4033-8183-883e13acdf6f None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:49.824346 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 134/267] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:23:49 2022] GET /identity/v3/role_assignments?user.id=89cde8a97d784539962d5992b9d7727f&scope.project.id=24fafd1589784882b754b5f8543954bb&role.id=a3b793754e7b40b0bb21afe74eec3c18 => generated 608 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:49.834881 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30deafc0-371c-4978-8fd7-6002b5d22d6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:49.835772 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30deafc0-371c-4978-8fd7-6002b5d22d6f None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:49.836056 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30deafc0-371c-4978-8fd7-6002b5d22d6f None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:49.836332 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30deafc0-371c-4978-8fd7-6002b5d22d6f None admin] PATH_INFO: `/v3/projects/24fafd1589784882b754b5f8543954bb` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:49.837660 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30deafc0-371c-4978-8fd7-6002b5d22d6f None admin] RBAC: Authorizing `identity:get_project(project_id=24fafd1589784882b754b5f8543954bb)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:49.839214 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30deafc0-371c-4978-8fd7-6002b5d22d6f None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:49.841038 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 134/268] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:23:49 2022] GET /identity/v3/projects/24fafd1589784882b754b5f8543954bb => generated 302 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:49.847913 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406fec1a-f55d-4033-8183-883e13acdf6f None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:49.957509 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:49.985874 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406fec1a-f55d-4033-8183-883e13acdf6f None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:50.026125 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0307a8-feb3-440c-bdf3-be564bdf8e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:50.026803 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0307a8-feb3-440c-bdf3-be564bdf8e41 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:50.027090 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0307a8-feb3-440c-bdf3-be564bdf8e41 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:50.027368 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0307a8-feb3-440c-bdf3-be564bdf8e41 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:50.028264 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0307a8-feb3-440c-bdf3-be564bdf8e41 None admin] RBAC: Adding query filter params (role.id=b954b2c7ce764aa0afac78e075a011a6, scope.project.id=24fafd1589784882b754b5f8543954bb, user.id=89cde8a97d784539962d5992b9d7727f) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:23:50.028587 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0307a8-feb3-440c-bdf3-be564bdf8e41 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:50.030644 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0307a8-feb3-440c-bdf3-be564bdf8e41 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:50.038270 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 135/269] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:23:49 2022] GET /identity/v3/role_assignments?user.id=89cde8a97d784539962d5992b9d7727f&scope.project.id=24fafd1589784882b754b5f8543954bb&role.id=b954b2c7ce764aa0afac78e075a011a6 => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:50.188450 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ca0cea-b6ae-4db0-8315-2824de8ce2c6 None None] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:50.189127 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ca0cea-b6ae-4db0-8315-2824de8ce2c6 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:50.189127 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ca0cea-b6ae-4db0-8315-2824de8ce2c6 None None] PATH_INFO: `/` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:50.190819 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 135/270] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:23:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:23:50.196664 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853ad19a-207e-4f9c-aa56-a4d35a345665 None None] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:50.197296 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853ad19a-207e-4f9c-aa56-a4d35a345665 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:50.197789 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853ad19a-207e-4f9c-aa56-a4d35a345665 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:50.222884 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02880883-2fb6-4532-95cb-d2a4cad2d872 None None] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:50.223067 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02880883-2fb6-4532-95cb-d2a4cad2d872 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:50.223327 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02880883-2fb6-4532-95cb-d2a4cad2d872 None None] PATH_INFO: `/` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:50.225317 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 136/271] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:23:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:23:50.226357 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.auth.core [None req-853ad19a-207e-4f9c-aa56-a4d35a345665 None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143586) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:23:50.227836 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.fernet_utils [None req-853ad19a-207e-4f9c-aa56-a4d35a345665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143586) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:23:50.230394 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e707b62b-b532-4d63-b44f-36a8a64de483 None None] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:50.230682 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e707b62b-b532-4d63-b44f-36a8a64de483 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:50.231030 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e707b62b-b532-4d63-b44f-36a8a64de483 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:50.259936 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.auth.core [None req-e707b62b-b532-4d63-b44f-36a8a64de483 None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143587) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:23:50.261156 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-e707b62b-b532-4d63-b44f-36a8a64de483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:23:50.355851 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 137/272] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:23:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:23:50.384073 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e707b62b-b532-4d63-b44f-36a8a64de483 None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:50.385810 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 136/273] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:23:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:23:50.426267 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853ad19a-207e-4f9c-aa56-a4d35a345665 None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:50.450283 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:50.506880 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e707b62b-b532-4d63-b44f-36a8a64de483 None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:50.547574 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:50.555211 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13c3e9b-082b-46b1-a416-2af8ecec4753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:50.556416 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13c3e9b-082b-46b1-a416-2af8ecec4753 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:50.557257 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13c3e9b-082b-46b1-a416-2af8ecec4753 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:50.557694 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13c3e9b-082b-46b1-a416-2af8ecec4753 None admin] PATH_INFO: `/v3/domains/default` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:50.559364 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13c3e9b-082b-46b1-a416-2af8ecec4753 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:50.562371 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13c3e9b-082b-46b1-a416-2af8ecec4753 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:50.565008 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 138/274] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:23:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:23:50.569762 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13c3e9b-082b-46b1-a416-2af8ecec4753 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:50.583004 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853ad19a-207e-4f9c-aa56-a4d35a345665 None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:50.629748 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843e18e0-14d5-43d5-b659-bdfd73d008f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:50.630599 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843e18e0-14d5-43d5-b659-bdfd73d008f2 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:50.630884 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843e18e0-14d5-43d5-b659-bdfd73d008f2 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:50.631165 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843e18e0-14d5-43d5-b659-bdfd73d008f2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:50.651133 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843e18e0-14d5-43d5-b659-bdfd73d008f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:23:50.660324 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-843e18e0-14d5-43d5-b659-bdfd73d008f2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:23:50.660324 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:23:50.660324 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:23:50.660324 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:23:50.660324 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:23:50.660324 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:23:50.660324 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:23:50.660324 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:23:50.660324 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:23:50.660324 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:23:50.660324 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:23:50.660324 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:23:50.660324 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:23:50.660324 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:23:50.660324 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:23:50.660324 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:23:50.661447 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:23:50.661447 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:23:50.661447 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:23:50.661447 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:23:50.661447 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:23:50.661447 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:23:50.661447 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:23:50.661447 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:23:50.661447 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:23:50.661447 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:23:50.661447 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:23:50.661447 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:23:50.661447 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:23:50.661447 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:23:50.661447 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:23:50.661447 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:23:50.667201 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:23:50.667201 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:23:50.667201 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:23:50.667201 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:23:50.667201 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:23:50.667201 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:23:50.667201 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:23:50.667201 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:23:50.667201 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:23:50.667201 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:23:50.667201 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:23:50.667201 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:23:50.667201 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 137/275] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:23:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:23:50.674142 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843e18e0-14d5-43d5-b659-bdfd73d008f2 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:50.721895 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13c3e9b-082b-46b1-a416-2af8ecec4753 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:50.743134 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:50.766575 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46056c81-3f60-41e6-abbd-032039bc69b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:50.767691 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46056c81-3f60-41e6-abbd-032039bc69b4 None admin] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:50.768227 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46056c81-3f60-41e6-abbd-032039bc69b4 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:50.768725 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46056c81-3f60-41e6-abbd-032039bc69b4 None admin] PATH_INFO: `/v3/users` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:50.769706 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46056c81-3f60-41e6-abbd-032039bc69b4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:50.771529 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46056c81-3f60-41e6-abbd-032039bc69b4 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:50.796131 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843e18e0-14d5-43d5-b659-bdfd73d008f2 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:50.813881 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 139/276] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 19 14:23:50 2022] POST /identity/v3/users => generated 293 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:23:50.842433 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c16f136-e526-4f12-a24f-1219e4e851d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:50.843244 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c16f136-e526-4f12-a24f-1219e4e851d8 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:50.843531 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c16f136-e526-4f12-a24f-1219e4e851d8 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:50.843811 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c16f136-e526-4f12-a24f-1219e4e851d8 None admin] PATH_INFO: `/v3/roles` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:50.844697 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c16f136-e526-4f12-a24f-1219e4e851d8 None admin] RBAC: Adding query filter params (name=member) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:23:50.845016 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c16f136-e526-4f12-a24f-1219e4e851d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:50.846586 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c16f136-e526-4f12-a24f-1219e4e851d8 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:50.848346 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:50.852223 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d12624-a584-4d48-8c22-8f86aa1e3ce6 None None] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:50.854111 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d12624-a584-4d48-8c22-8f86aa1e3ce6 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:50.854111 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d12624-a584-4d48-8c22-8f86aa1e3ce6 None None] PATH_INFO: `/` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:50.854676 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 140/277] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:23:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:23:50.861731 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb7e0c0-ff6c-419a-9361-6519d1d0727e None None] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:50.862209 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb7e0c0-ff6c-419a-9361-6519d1d0727e None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:50.862588 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb7e0c0-ff6c-419a-9361-6519d1d0727e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:50.865653 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 138/278] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:23:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:23:50.872178 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c16f136-e526-4f12-a24f-1219e4e851d8 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:50.882179 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.auth.core [None req-9bb7e0c0-ff6c-419a-9361-6519d1d0727e None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143587) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:23:50.883518 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-9bb7e0c0-ff6c-419a-9361-6519d1d0727e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:23:50.992840 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 141/279] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:23:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:23:50.994800 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c16f136-e526-4f12-a24f-1219e4e851d8 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:51.016901 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb7e0c0-ff6c-419a-9361-6519d1d0727e None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:51.044084 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044a740d-ca01-474e-bdbc-efc9379c2438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:51.045550 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044a740d-ca01-474e-bdbc-efc9379c2438 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:51.047204 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044a740d-ca01-474e-bdbc-efc9379c2438 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:51.047488 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044a740d-ca01-474e-bdbc-efc9379c2438 None admin] PATH_INFO: `/v3/users/4ef6ced909414456a7e6e8a61083bd4f` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:51.065279 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044a740d-ca01-474e-bdbc-efc9379c2438 None admin] RBAC: Authorizing `identity:get_user(user_id=4ef6ced909414456a7e6e8a61083bd4f)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:51.066804 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044a740d-ca01-474e-bdbc-efc9379c2438 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:51.068721 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 139/280] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:23:50 2022] GET /identity/v3/users/4ef6ced909414456a7e6e8a61083bd4f => generated 293 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:23:51.074211 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044a740d-ca01-474e-bdbc-efc9379c2438 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:51.141818 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:51.156191 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb7e0c0-ff6c-419a-9361-6519d1d0727e None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:51.162363 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:51.202820 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044a740d-ca01-474e-bdbc-efc9379c2438 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:51.203212 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec0bbae-b12a-466c-b040-55ecadae703e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:51.204528 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec0bbae-b12a-466c-b040-55ecadae703e None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:51.204972 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec0bbae-b12a-466c-b040-55ecadae703e None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:51.205404 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec0bbae-b12a-466c-b040-55ecadae703e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:51.219931 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec0bbae-b12a-466c-b040-55ecadae703e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:23:51.229726 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-3ec0bbae-b12a-466c-b040-55ecadae703e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:23:51.229726 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:23:51.229726 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:23:51.229726 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:23:51.229726 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:23:51.229726 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:23:51.229726 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:23:51.229726 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:23:51.229726 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:23:51.229726 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:23:51.229726 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:23:51.229726 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:23:51.229726 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:23:51.229726 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:23:51.229726 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:23:51.229726 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:23:51.230819 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:23:51.230819 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:23:51.230819 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:23:51.230819 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:23:51.230819 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:23:51.230819 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:23:51.230819 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:23:51.230819 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:23:51.230819 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:23:51.230819 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:23:51.230819 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:23:51.230819 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:23:51.230819 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:23:51.230819 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:23:51.230819 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:23:51.230819 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:23:51.231804 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:23:51.231804 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:23:51.231804 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:23:51.231804 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:23:51.231804 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:23:51.231804 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:23:51.231804 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:23:51.231804 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:23:51.231804 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:23:51.231804 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:23:51.231804 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:23:51.231804 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:23:51.231804 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 142/281] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:23:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:23:51.240094 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec0bbae-b12a-466c-b040-55ecadae703e None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:51.249358 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d79ffce-2662-4bb3-b980-1a3b44f3e037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:51.250151 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d79ffce-2662-4bb3-b980-1a3b44f3e037 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:51.250875 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d79ffce-2662-4bb3-b980-1a3b44f3e037 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:51.250875 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d79ffce-2662-4bb3-b980-1a3b44f3e037 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:51.251544 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d79ffce-2662-4bb3-b980-1a3b44f3e037 None admin] RBAC: Adding query filter params (role.id=3d5075293a4f45449eaef6af8fa53004, user.id=4ef6ced909414456a7e6e8a61083bd4f, scope.system=all) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:23:51.251855 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d79ffce-2662-4bb3-b980-1a3b44f3e037 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:51.253287 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d79ffce-2662-4bb3-b980-1a3b44f3e037 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:51.260711 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 140/282] 10.222.0.57 () {56 vars in 1308 bytes} [Thu May 19 14:23:51 2022] GET /identity/v3/role_assignments?user.id=4ef6ced909414456a7e6e8a61083bd4f&scope.system=all&role.id=3d5075293a4f45449eaef6af8fa53004 => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:23:51.262954 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16547a45-ac08-49e0-b0fb-dde559ef424c None None] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:51.263555 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16547a45-ac08-49e0-b0fb-dde559ef424c None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:51.264134 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16547a45-ac08-49e0-b0fb-dde559ef424c None None] PATH_INFO: `/` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:51.267310 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 141/283] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:23:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:23:51.273696 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad07a85-d7d4-4dc3-8d35-7e949d744081 None None] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:51.274329 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad07a85-d7d4-4dc3-8d35-7e949d744081 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:51.274817 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad07a85-d7d4-4dc3-8d35-7e949d744081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:51.294506 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.auth.core [None req-3ad07a85-d7d4-4dc3-8d35-7e949d744081 None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143586) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:23:51.296736 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.fernet_utils [None req-3ad07a85-d7d4-4dc3-8d35-7e949d744081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143586) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:23:51.360989 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec0bbae-b12a-466c-b040-55ecadae703e None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:51.377668 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 142/284] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:23:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:23:51.399883 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a217746-c0c8-4a96-9285-35eee802003c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:51.400579 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a217746-c0c8-4a96-9285-35eee802003c None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:51.400866 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a217746-c0c8-4a96-9285-35eee802003c None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:51.401198 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a217746-c0c8-4a96-9285-35eee802003c None admin] PATH_INFO: `/v3/roles` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:51.402007 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a217746-c0c8-4a96-9285-35eee802003c None admin] RBAC: Adding query filter params (name=reader) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:23:51.402326 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a217746-c0c8-4a96-9285-35eee802003c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:51.403884 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a217746-c0c8-4a96-9285-35eee802003c None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:51.415048 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 143/285] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:23:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:51.421091 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad07a85-d7d4-4dc3-8d35-7e949d744081 None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:51.421422 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a217746-c0c8-4a96-9285-35eee802003c None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:51.454598 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:51.517986 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:51.542236 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a217746-c0c8-4a96-9285-35eee802003c None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:51.579797 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d789bdd-b5f6-426c-b8d5-a79477348af2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:51.580595 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d789bdd-b5f6-426c-b8d5-a79477348af2 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:51.580880 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d789bdd-b5f6-426c-b8d5-a79477348af2 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:51.581156 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d789bdd-b5f6-426c-b8d5-a79477348af2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:51.584050 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad07a85-d7d4-4dc3-8d35-7e949d744081 None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:51.594439 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d789bdd-b5f6-426c-b8d5-a79477348af2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:23:51.603042 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-6d789bdd-b5f6-426c-b8d5-a79477348af2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:23:51.603042 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:23:51.603042 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:23:51.603042 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:23:51.603042 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:23:51.603042 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:23:51.603042 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:23:51.603042 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:23:51.603042 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:23:51.603042 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:23:51.603042 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:23:51.603042 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:23:51.603042 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:23:51.603042 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:23:51.603042 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:23:51.603042 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:23:51.604086 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:23:51.604086 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:23:51.604086 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:23:51.604086 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:23:51.604086 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:23:51.604086 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:23:51.604086 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:23:51.604086 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:23:51.604086 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:23:51.604086 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:23:51.604086 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:23:51.604086 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:23:51.604086 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:23:51.604086 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:23:51.604086 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:23:51.604086 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:23:51.605015 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:23:51.605015 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:23:51.605015 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:23:51.605015 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:23:51.605015 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:23:51.605015 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:23:51.605015 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:23:51.605015 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:23:51.605015 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:23:51.605015 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:23:51.605015 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:23:51.605015 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:23:51.605015 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 144/286] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:23:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:23:51.610640 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d789bdd-b5f6-426c-b8d5-a79477348af2 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:51.642659 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe06a41c-d47c-4f84-9866-5c77a624a8a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:51.643536 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe06a41c-d47c-4f84-9866-5c77a624a8a3 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:51.643817 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe06a41c-d47c-4f84-9866-5c77a624a8a3 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:51.644116 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe06a41c-d47c-4f84-9866-5c77a624a8a3 None admin] PATH_INFO: `/v3/users/e8b14cd4ac074742a8c629d0a42902b8` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:51.645546 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe06a41c-d47c-4f84-9866-5c77a624a8a3 None admin] RBAC: Authorizing `identity:get_user(user_id=e8b14cd4ac074742a8c629d0a42902b8)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:51.647027 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe06a41c-d47c-4f84-9866-5c77a624a8a3 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:51.648840 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 143/287] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:23:51 2022] GET /identity/v3/users/e8b14cd4ac074742a8c629d0a42902b8 => generated 297 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:51.656883 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe06a41c-d47c-4f84-9866-5c77a624a8a3 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:51.794288 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:51.805171 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d789bdd-b5f6-426c-b8d5-a79477348af2 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:51.808453 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe06a41c-d47c-4f84-9866-5c77a624a8a3 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:51.834616 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:51.863464 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f529f37-c1d2-45a7-afe1-e13323cdba08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:51.864337 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f529f37-c1d2-45a7-afe1-e13323cdba08 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:51.864622 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f529f37-c1d2-45a7-afe1-e13323cdba08 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:51.864907 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f529f37-c1d2-45a7-afe1-e13323cdba08 None admin] PATH_INFO: `/v3/projects/24fafd1589784882b754b5f8543954bb` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:51.866089 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ff6abb-bd37-4f13-831a-5286f4c59796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:51.866756 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ff6abb-bd37-4f13-831a-5286f4c59796 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:51.867033 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ff6abb-bd37-4f13-831a-5286f4c59796 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:51.867307 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ff6abb-bd37-4f13-831a-5286f4c59796 None admin] PATH_INFO: `/v3/roles` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:51.868077 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ff6abb-bd37-4f13-831a-5286f4c59796 None admin] RBAC: Adding query filter params (name=member) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:23:51.868394 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ff6abb-bd37-4f13-831a-5286f4c59796 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:51.869818 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ff6abb-bd37-4f13-831a-5286f4c59796 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:51.872053 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f529f37-c1d2-45a7-afe1-e13323cdba08 None admin] RBAC: Authorizing `identity:get_project(project_id=24fafd1589784882b754b5f8543954bb)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:51.874251 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f529f37-c1d2-45a7-afe1-e13323cdba08 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:51.880372 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 144/288] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:23:51 2022] GET /identity/v3/projects/24fafd1589784882b754b5f8543954bb => generated 302 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:51.881760 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f529f37-c1d2-45a7-afe1-e13323cdba08 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:51.885536 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 145/289] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:23:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:23:51.896212 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ff6abb-bd37-4f13-831a-5286f4c59796 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:52.024628 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ff6abb-bd37-4f13-831a-5286f4c59796 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:52.039870 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f529f37-c1d2-45a7-afe1-e13323cdba08 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:52.064933 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a36cee-133d-4592-bf35-6bcfa0faa0df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:52.065738 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a36cee-133d-4592-bf35-6bcfa0faa0df None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:52.066065 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a36cee-133d-4592-bf35-6bcfa0faa0df None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:52.066342 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a36cee-133d-4592-bf35-6bcfa0faa0df None admin] PATH_INFO: `/v3/users/c441f5a4e9a94ad1872cdc44fab83b7c` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:52.067570 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a36cee-133d-4592-bf35-6bcfa0faa0df None admin] RBAC: Authorizing `identity:get_user(user_id=c441f5a4e9a94ad1872cdc44fab83b7c)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:52.069037 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a36cee-133d-4592-bf35-6bcfa0faa0df None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:52.073958 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 146/290] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:23:51 2022] GET /identity/v3/users/c441f5a4e9a94ad1872cdc44fab83b7c => generated 297 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:23:52.076779 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a36cee-133d-4592-bf35-6bcfa0faa0df None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:52.097619 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:52.140449 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc84d81-eb78-4532-93a8-4082dbe63497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:52.141343 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc84d81-eb78-4532-93a8-4082dbe63497 None admin] REQUEST_METHOD: `PUT` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:52.141622 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc84d81-eb78-4532-93a8-4082dbe63497 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:52.141917 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc84d81-eb78-4532-93a8-4082dbe63497 None admin] PATH_INFO: `/v3/projects/24fafd1589784882b754b5f8543954bb/users/e8b14cd4ac074742a8c629d0a42902b8/roles/abff822b2f9d4abd96ff7414f6de1670` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:52.148466 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc84d81-eb78-4532-93a8-4082dbe63497 None admin] RBAC: Authorizing `identity:create_grant(project_id=24fafd1589784882b754b5f8543954bb, user_id=e8b14cd4ac074742a8c629d0a42902b8, role_id=abff822b2f9d4abd96ff7414f6de1670)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:52.149940 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc84d81-eb78-4532-93a8-4082dbe63497 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:52.192387 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a36cee-133d-4592-bf35-6bcfa0faa0df None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:52.196372 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 145/291] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 14:23:51 2022] PUT /identity/v3/projects/24fafd1589784882b754b5f8543954bb/users/e8b14cd4ac074742a8c629d0a42902b8/roles/abff822b2f9d4abd96ff7414f6de1670 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:23:52.235951 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566f161a-518a-4b02-ac20-f117dcce532c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:52.236731 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566f161a-518a-4b02-ac20-f117dcce532c None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:52.237022 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566f161a-518a-4b02-ac20-f117dcce532c None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:52.237304 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566f161a-518a-4b02-ac20-f117dcce532c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:52.340612 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566f161a-518a-4b02-ac20-f117dcce532c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:52.342208 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566f161a-518a-4b02-ac20-f117dcce532c None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:52.367062 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:52.484529 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 147/292] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:23:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:23:52.490348 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc84d81-eb78-4532-93a8-4082dbe63497 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:52.560923 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:52.629932 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc84d81-eb78-4532-93a8-4082dbe63497 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:52.673790 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c2fc24-8b4a-4e2a-87e9-3568f77bdf8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:52.674694 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c2fc24-8b4a-4e2a-87e9-3568f77bdf8d None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:52.674978 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c2fc24-8b4a-4e2a-87e9-3568f77bdf8d None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:52.675278 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c2fc24-8b4a-4e2a-87e9-3568f77bdf8d None admin] PATH_INFO: `/v3/projects/24fafd1589784882b754b5f8543954bb` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:52.676501 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c2fc24-8b4a-4e2a-87e9-3568f77bdf8d None admin] RBAC: Authorizing `identity:get_project(project_id=24fafd1589784882b754b5f8543954bb)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:52.677980 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c2fc24-8b4a-4e2a-87e9-3568f77bdf8d None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:52.680017 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 146/293] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:23:52 2022] GET /identity/v3/projects/24fafd1589784882b754b5f8543954bb => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:52.686539 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566f161a-518a-4b02-ac20-f117dcce532c None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:52.816657 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566f161a-518a-4b02-ac20-f117dcce532c None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:52.836632 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:52.863954 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c55cf40-abe1-4b2a-bfa8-2d9b8af9c30a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:52.864680 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c55cf40-abe1-4b2a-bfa8-2d9b8af9c30a None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:52.864969 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c55cf40-abe1-4b2a-bfa8-2d9b8af9c30a None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:52.865250 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c55cf40-abe1-4b2a-bfa8-2d9b8af9c30a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:52.866123 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c55cf40-abe1-4b2a-bfa8-2d9b8af9c30a None admin] RBAC: Adding query filter params (role.id=3d5075293a4f45449eaef6af8fa53004, scope.project.id=24fafd1589784882b754b5f8543954bb, user.id=c441f5a4e9a94ad1872cdc44fab83b7c) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:23:52.866442 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c55cf40-abe1-4b2a-bfa8-2d9b8af9c30a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:52.867898 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c55cf40-abe1-4b2a-bfa8-2d9b8af9c30a None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:52.876471 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 148/294] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:23:52 2022] GET /identity/v3/role_assignments?user.id=c441f5a4e9a94ad1872cdc44fab83b7c&scope.project.id=24fafd1589784882b754b5f8543954bb&role.id=3d5075293a4f45449eaef6af8fa53004 => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:53.603374 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a881a9-4dee-44cc-9e84-7d0d306c08c0 None None] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:53.604449 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a881a9-4dee-44cc-9e84-7d0d306c08c0 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:53.604990 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a881a9-4dee-44cc-9e84-7d0d306c08c0 None None] PATH_INFO: `/` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:53.607042 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 147/295] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:23:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:23:53.614030 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c9081b-3fb4-4658-9f7c-3ab66d25b240 None None] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:53.615055 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c9081b-3fb4-4658-9f7c-3ab66d25b240 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:53.615559 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c9081b-3fb4-4658-9f7c-3ab66d25b240 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:53.635388 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.auth.core [None req-d4c9081b-3fb4-4658-9f7c-3ab66d25b240 None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143587) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:23:53.636958 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-d4c9081b-3fb4-4658-9f7c-3ab66d25b240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:23:53.726217 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 149/296] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:23:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:23:53.750775 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a881a9-4dee-44cc-9e84-7d0d306c08c0 None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:53.798840 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cccb5fa-116f-46a6-b411-a5003bc9347e None None] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:53.799484 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cccb5fa-116f-46a6-b411-a5003bc9347e None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:53.799976 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cccb5fa-116f-46a6-b411-a5003bc9347e None None] PATH_INFO: `/` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:53.802954 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 150/297] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:23:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:23:53.807269 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3671cb8-6011-47af-bc1b-8c213c4a1e33 None None] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:53.807912 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3671cb8-6011-47af-bc1b-8c213c4a1e33 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:53.808405 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3671cb8-6011-47af-bc1b-8c213c4a1e33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:53.828992 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.auth.core [None req-d3671cb8-6011-47af-bc1b-8c213c4a1e33 None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143587) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:23:53.830836 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-d3671cb8-6011-47af-bc1b-8c213c4a1e33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:23:53.875329 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a881a9-4dee-44cc-9e84-7d0d306c08c0 None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:53.892832 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:53.921391 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab1a365-ce17-4d5a-8a32-dfc3ffe9ac06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:53.922644 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab1a365-ce17-4d5a-8a32-dfc3ffe9ac06 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:53.923133 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab1a365-ce17-4d5a-8a32-dfc3ffe9ac06 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:53.923626 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab1a365-ce17-4d5a-8a32-dfc3ffe9ac06 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:53.925933 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 151/298] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:23:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:23:53.935122 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab1a365-ce17-4d5a-8a32-dfc3ffe9ac06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:23:53.943536 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-bab1a365-ce17-4d5a-8a32-dfc3ffe9ac06 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:23:53.943536 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:23:53.943536 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:23:53.943536 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:23:53.943536 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:23:53.943536 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:23:53.943536 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:23:53.943536 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:23:53.943536 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:23:53.943536 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:23:53.943536 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:23:53.943536 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:23:53.943536 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:23:53.943536 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:23:53.943536 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:23:53.943536 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:23:53.944560 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:23:53.944560 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:23:53.944560 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:23:53.944560 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:23:53.944560 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:23:53.944560 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:23:53.944560 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:23:53.944560 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:23:53.944560 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:23:53.944560 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:23:53.944560 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:23:53.944560 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:23:53.944560 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:23:53.944560 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:23:53.944560 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:23:53.944560 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:23:53.945460 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:23:53.945460 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:23:53.945460 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:23:53.945460 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:23:53.945460 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:23:53.945460 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:23:53.945460 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:23:53.945460 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:23:53.945460 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:23:53.945460 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:23:53.945460 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:23:53.945460 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:23:53.946660 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3671cb8-6011-47af-bc1b-8c213c4a1e33 None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:53.948418 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 148/299] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:23:53 2022] GET /identity/v3/roles/reader => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:23:53.954010 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab1a365-ce17-4d5a-8a32-dfc3ffe9ac06 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:53.996259 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:54.063614 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3671cb8-6011-47af-bc1b-8c213c4a1e33 None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:54.068564 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab1a365-ce17-4d5a-8a32-dfc3ffe9ac06 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:54.102127 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609a3dda-1866-4e61-a7d2-8864bd946916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:54.102919 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609a3dda-1866-4e61-a7d2-8864bd946916 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:54.103269 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609a3dda-1866-4e61-a7d2-8864bd946916 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:54.103487 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609a3dda-1866-4e61-a7d2-8864bd946916 None admin] PATH_INFO: `/v3/domains/default` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:54.104656 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609a3dda-1866-4e61-a7d2-8864bd946916 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:54.106147 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609a3dda-1866-4e61-a7d2-8864bd946916 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:54.106498 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118410fe-1d73-4b1a-bf2c-871e69de6d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:54.107806 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118410fe-1d73-4b1a-bf2c-871e69de6d2a None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:54.108154 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118410fe-1d73-4b1a-bf2c-871e69de6d2a None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:54.108437 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118410fe-1d73-4b1a-bf2c-871e69de6d2a None admin] PATH_INFO: `/v3/roles` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:54.108897 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 152/300] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:23:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:23:54.110023 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9170d8-4c4a-4481-bf0d-f92dd1686611 None None] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:54.110314 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9170d8-4c4a-4481-bf0d-f92dd1686611 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:54.110598 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9170d8-4c4a-4481-bf0d-f92dd1686611 None None] PATH_INFO: `/` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:54.111245 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118410fe-1d73-4b1a-bf2c-871e69de6d2a None admin] RBAC: Adding query filter params (name=reader) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:23:54.111768 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118410fe-1d73-4b1a-bf2c-871e69de6d2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:54.113392 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118410fe-1d73-4b1a-bf2c-871e69de6d2a None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:54.116017 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 153/301] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:23:54 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:23:54.116700 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9170d8-4c4a-4481-bf0d-f92dd1686611 None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:54.123863 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 149/302] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:23:53 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:23:54.125051 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18966ff-d5e4-4e11-b9c8-ce9345d24d82 None None] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:54.125396 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18966ff-d5e4-4e11-b9c8-ce9345d24d82 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:54.125622 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18966ff-d5e4-4e11-b9c8-ce9345d24d82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:54.143256 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.auth.core [None req-c18966ff-d5e4-4e11-b9c8-ce9345d24d82 None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143586) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:23:54.144543 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.fernet_utils [None req-c18966ff-d5e4-4e11-b9c8-ce9345d24d82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143586) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:23:54.173824 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:54.216989 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 150/303] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:23:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:23:54.218440 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18966ff-d5e4-4e11-b9c8-ce9345d24d82 None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:54.234473 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9170d8-4c4a-4481-bf0d-f92dd1686611 None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:54.245604 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:54.276101 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbef6d1-033f-4820-9959-9f26d8f1f9d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:54.276872 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbef6d1-033f-4820-9959-9f26d8f1f9d7 None admin] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:54.277151 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbef6d1-033f-4820-9959-9f26d8f1f9d7 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:54.277706 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbef6d1-033f-4820-9959-9f26d8f1f9d7 None admin] PATH_INFO: `/v3/groups` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:54.278251 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbef6d1-033f-4820-9959-9f26d8f1f9d7 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:54.280397 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbef6d1-033f-4820-9959-9f26d8f1f9d7 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:54.302019 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 154/304] 10.222.0.57 () {60 vars in 1134 bytes} [Thu May 19 14:23:54 2022] POST /identity/v3/groups => generated 229 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:23:54.303397 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbef6d1-033f-4820-9959-9f26d8f1f9d7 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:54.339787 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18966ff-d5e4-4e11-b9c8-ce9345d24d82 None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:54.392682 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b887de-0d81-4268-a65a-c3b0c72ccc39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:54.394807 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b887de-0d81-4268-a65a-c3b0c72ccc39 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:54.395323 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b887de-0d81-4268-a65a-c3b0c72ccc39 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:54.396109 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b887de-0d81-4268-a65a-c3b0c72ccc39 None admin] PATH_INFO: `/v3/users/b47eff9c105c4e2fa106c63dcd162914` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:54.417346 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b887de-0d81-4268-a65a-c3b0c72ccc39 None admin] RBAC: Authorizing `identity:get_user(user_id=b47eff9c105c4e2fa106c63dcd162914)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:54.419296 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b887de-0d81-4268-a65a-c3b0c72ccc39 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:54.423108 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 151/305] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:23:54 2022] GET /identity/v3/users/b47eff9c105c4e2fa106c63dcd162914 => generated 293 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:54.429418 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b887de-0d81-4268-a65a-c3b0c72ccc39 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:54.434050 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbef6d1-033f-4820-9959-9f26d8f1f9d7 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:54.473996 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4d8750-b962-42e7-8d6b-d858cb905be5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:54.475116 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4d8750-b962-42e7-8d6b-d858cb905be5 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:54.475403 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4d8750-b962-42e7-8d6b-d858cb905be5 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:54.475686 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4d8750-b962-42e7-8d6b-d858cb905be5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:54.482851 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:54.489717 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4d8750-b962-42e7-8d6b-d858cb905be5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:23:54.500019 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-4a4d8750-b962-42e7-8d6b-d858cb905be5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:23:54.500019 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:23:54.500019 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:23:54.500019 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:23:54.500019 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:23:54.500019 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:23:54.500019 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:23:54.500019 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:23:54.500019 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:23:54.500019 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:23:54.500019 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:23:54.500019 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:23:54.500019 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:23:54.500019 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:23:54.500019 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:23:54.500019 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:23:54.501078 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:23:54.501078 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:23:54.501078 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:23:54.501078 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:23:54.501078 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:23:54.501078 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:23:54.501078 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:23:54.501078 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:23:54.501078 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:23:54.501078 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:23:54.501078 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:23:54.501078 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:23:54.501078 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:23:54.501078 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:23:54.501078 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:23:54.501078 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:23:54.502052 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:23:54.502052 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:23:54.502052 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:23:54.502052 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:23:54.502052 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:23:54.502052 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:23:54.502052 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:23:54.502052 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:23:54.502052 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:23:54.502052 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:23:54.502052 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:23:54.502052 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:23:54.502052 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 155/306] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:23:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:23:54.508741 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4d8750-b962-42e7-8d6b-d858cb905be5 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:54.545949 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:54.579096 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b887de-0d81-4268-a65a-c3b0c72ccc39 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:54.619507 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c90d268-141d-4dd1-abc7-b62469b2d7b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:54.620155 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c90d268-141d-4dd1-abc7-b62469b2d7b9 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:54.620241 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c90d268-141d-4dd1-abc7-b62469b2d7b9 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:54.620526 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c90d268-141d-4dd1-abc7-b62469b2d7b9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:54.621325 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c90d268-141d-4dd1-abc7-b62469b2d7b9 None admin] RBAC: Adding query filter params (role.id=abff822b2f9d4abd96ff7414f6de1670, user.id=b47eff9c105c4e2fa106c63dcd162914, scope.system=all) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:23:54.621708 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c90d268-141d-4dd1-abc7-b62469b2d7b9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:54.623808 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c90d268-141d-4dd1-abc7-b62469b2d7b9 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:54.630153 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 152/307] 10.222.0.57 () {56 vars in 1308 bytes} [Thu May 19 14:23:54 2022] GET /identity/v3/role_assignments?user.id=b47eff9c105c4e2fa106c63dcd162914&scope.system=all&role.id=abff822b2f9d4abd96ff7414f6de1670 => generated 226 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:54.647569 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4d8750-b962-42e7-8d6b-d858cb905be5 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:54.696821 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074f5bdb-67a3-49b5-895b-7828647973a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:54.697827 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074f5bdb-67a3-49b5-895b-7828647973a3 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:54.698310 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074f5bdb-67a3-49b5-895b-7828647973a3 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:54.698770 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074f5bdb-67a3-49b5-895b-7828647973a3 None admin] PATH_INFO: `/v3/roles` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:54.699725 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074f5bdb-67a3-49b5-895b-7828647973a3 None admin] RBAC: Adding query filter params (name=member) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:23:54.700290 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074f5bdb-67a3-49b5-895b-7828647973a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:54.702170 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074f5bdb-67a3-49b5-895b-7828647973a3 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:54.712365 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 156/308] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:23:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:23:54.719060 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c90d268-141d-4dd1-abc7-b62469b2d7b9 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:54.816674 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:54.833770 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c90d268-141d-4dd1-abc7-b62469b2d7b9 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:54.873684 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416e4725-96ca-4f88-9e15-89588c9e5690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:54.874625 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416e4725-96ca-4f88-9e15-89588c9e5690 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:54.874912 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416e4725-96ca-4f88-9e15-89588c9e5690 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:54.875193 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416e4725-96ca-4f88-9e15-89588c9e5690 None admin] PATH_INFO: `/v3/users/4ef6ced909414456a7e6e8a61083bd4f` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:54.876440 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416e4725-96ca-4f88-9e15-89588c9e5690 None admin] RBAC: Authorizing `identity:get_user(user_id=4ef6ced909414456a7e6e8a61083bd4f)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:54.877915 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416e4725-96ca-4f88-9e15-89588c9e5690 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:54.879948 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 153/309] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:23:54 2022] GET /identity/v3/users/4ef6ced909414456a7e6e8a61083bd4f => generated 293 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:54.885704 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074f5bdb-67a3-49b5-895b-7828647973a3 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:54.984576 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:54.995663 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074f5bdb-67a3-49b5-895b-7828647973a3 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:55.032974 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934d9561-375e-4512-82aa-6a69f3e7f1c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:55.033892 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934d9561-375e-4512-82aa-6a69f3e7f1c3 None admin] REQUEST_METHOD: `PUT` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:55.034187 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934d9561-375e-4512-82aa-6a69f3e7f1c3 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:55.034468 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934d9561-375e-4512-82aa-6a69f3e7f1c3 None admin] PATH_INFO: `/v3/system/users/4ef6ced909414456a7e6e8a61083bd4f/roles/3d5075293a4f45449eaef6af8fa53004` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:55.036071 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934d9561-375e-4512-82aa-6a69f3e7f1c3 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=4ef6ced909414456a7e6e8a61083bd4f, role_id=3d5075293a4f45449eaef6af8fa53004)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:55.037573 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934d9561-375e-4512-82aa-6a69f3e7f1c3 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:55.049907 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 157/310] 10.222.0.57 () {58 vars in 1334 bytes} [Thu May 19 14:23:54 2022] PUT /identity/v3/system/users/4ef6ced909414456a7e6e8a61083bd4f/roles/3d5075293a4f45449eaef6af8fa53004 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:23:55.234823 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d424c45-e960-4de7-9303-11e5ca3342b2 None None] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:55.235022 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d424c45-e960-4de7-9303-11e5ca3342b2 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:55.235312 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d424c45-e960-4de7-9303-11e5ca3342b2 None None] PATH_INFO: `/` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:55.237162 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 154/311] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:23:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:23:55.242897 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3d2c8f-0529-4c16-a8aa-cc192e641d17 None None] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:55.243195 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3d2c8f-0529-4c16-a8aa-cc192e641d17 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:55.243487 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3d2c8f-0529-4c16-a8aa-cc192e641d17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:55.261677 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.auth.core [None req-3e3d2c8f-0529-4c16-a8aa-cc192e641d17 None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143587) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:23:55.262945 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-3e3d2c8f-0529-4c16-a8aa-cc192e641d17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:23:55.334916 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 158/312] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:23:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:23:55.360510 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d424c45-e960-4de7-9303-11e5ca3342b2 None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:55.475755 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d424c45-e960-4de7-9303-11e5ca3342b2 None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:55.514737 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed34648d-0651-4375-abc2-113001292c6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:55.515492 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed34648d-0651-4375-abc2-113001292c6c None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:55.515776 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed34648d-0651-4375-abc2-113001292c6c None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:55.516058 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed34648d-0651-4375-abc2-113001292c6c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:55.527647 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed34648d-0651-4375-abc2-113001292c6c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:23:55.536414 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-ed34648d-0651-4375-abc2-113001292c6c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:23:55.536414 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:23:55.536414 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:23:55.536414 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:23:55.536414 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:23:55.536414 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:23:55.536414 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:23:55.536414 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:23:55.536414 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:23:55.536414 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:23:55.536414 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:23:55.536414 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:23:55.536414 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:23:55.536414 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:23:55.536414 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:23:55.536414 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:23:55.537476 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:23:55.537476 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:23:55.537476 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:23:55.537476 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:23:55.537476 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:23:55.537476 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:23:55.537476 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:23:55.537476 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:23:55.537476 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:23:55.537476 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:23:55.537476 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:23:55.537476 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:23:55.537476 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:23:55.537476 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:23:55.537476 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:23:55.537476 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:23:55.538464 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:23:55.538464 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:23:55.538464 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:23:55.538464 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:23:55.538464 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:23:55.538464 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:23:55.538464 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:23:55.538464 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:23:55.538464 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:23:55.538464 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:23:55.538464 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:23:55.538464 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:23:55.538464 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 155/313] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:23:55 2022] GET /identity/v3/roles/reader => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:23:55.546347 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3d2c8f-0529-4c16-a8aa-cc192e641d17 None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:55.638749 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:55.657738 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3d2c8f-0529-4c16-a8aa-cc192e641d17 None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:55.699214 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3544ec1f-330b-4413-9eb1-520adbb63707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:55.699895 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3544ec1f-330b-4413-9eb1-520adbb63707 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:55.700179 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3544ec1f-330b-4413-9eb1-520adbb63707 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:55.700460 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3544ec1f-330b-4413-9eb1-520adbb63707 None admin] PATH_INFO: `/v3/roles` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:55.701263 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3544ec1f-330b-4413-9eb1-520adbb63707 None admin] RBAC: Adding query filter params (name=reader) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:23:55.701575 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3544ec1f-330b-4413-9eb1-520adbb63707 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:55.703097 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3544ec1f-330b-4413-9eb1-520adbb63707 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:55.712561 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 159/314] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:23:55 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:23:55.718798 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed34648d-0651-4375-abc2-113001292c6c None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:55.738284 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:55.846097 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed34648d-0651-4375-abc2-113001292c6c None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:55.885293 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f570097c-939e-4adf-aac1-1174c8de89c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:55.886630 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f570097c-939e-4adf-aac1-1174c8de89c9 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:55.887194 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f570097c-939e-4adf-aac1-1174c8de89c9 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:55.887681 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f570097c-939e-4adf-aac1-1174c8de89c9 None admin] PATH_INFO: `/v3/users/e8b14cd4ac074742a8c629d0a42902b8` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:55.889813 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f570097c-939e-4adf-aac1-1174c8de89c9 None admin] RBAC: Authorizing `identity:get_user(user_id=e8b14cd4ac074742a8c629d0a42902b8)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:55.891545 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f570097c-939e-4adf-aac1-1174c8de89c9 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:55.893714 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 156/315] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:23:55 2022] GET /identity/v3/users/e8b14cd4ac074742a8c629d0a42902b8 => generated 297 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:55.905889 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3544ec1f-330b-4413-9eb1-520adbb63707 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:56.007688 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3544ec1f-330b-4413-9eb1-520adbb63707 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:56.044366 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4b25d5-4658-4f5a-b0a3-19136d38405c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:56.045069 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4b25d5-4658-4f5a-b0a3-19136d38405c None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:56.045357 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4b25d5-4658-4f5a-b0a3-19136d38405c None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:56.045640 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4b25d5-4658-4f5a-b0a3-19136d38405c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:56.063100 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:56.121075 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4b25d5-4658-4f5a-b0a3-19136d38405c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:56.122513 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4b25d5-4658-4f5a-b0a3-19136d38405c None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:56.234203 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 160/316] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:23:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:23:56.236809 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f570097c-939e-4adf-aac1-1174c8de89c9 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:56.329415 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:56.349579 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f570097c-939e-4adf-aac1-1174c8de89c9 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:56.385886 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afcbc2b-b7b1-41a9-b8df-84e705f86f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:56.387174 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afcbc2b-b7b1-41a9-b8df-84e705f86f69 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:56.387688 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afcbc2b-b7b1-41a9-b8df-84e705f86f69 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:56.388172 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afcbc2b-b7b1-41a9-b8df-84e705f86f69 None admin] PATH_INFO: `/v3/projects/24fafd1589784882b754b5f8543954bb` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:56.389651 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afcbc2b-b7b1-41a9-b8df-84e705f86f69 None admin] RBAC: Authorizing `identity:get_project(project_id=24fafd1589784882b754b5f8543954bb)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:56.391368 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afcbc2b-b7b1-41a9-b8df-84e705f86f69 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:56.393634 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 157/317] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:23:56 2022] GET /identity/v3/projects/24fafd1589784882b754b5f8543954bb => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:56.405882 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4b25d5-4658-4f5a-b0a3-19136d38405c None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:56.449260 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:56.519391 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4b25d5-4658-4f5a-b0a3-19136d38405c None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:56.556667 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce4bf53-ba7d-4d39-a1e4-5f352a8bfeca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:56.557383 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce4bf53-ba7d-4d39-a1e4-5f352a8bfeca None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:56.557673 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce4bf53-ba7d-4d39-a1e4-5f352a8bfeca None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:56.558027 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce4bf53-ba7d-4d39-a1e4-5f352a8bfeca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:56.558878 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce4bf53-ba7d-4d39-a1e4-5f352a8bfeca None admin] RBAC: Adding query filter params (role.id=abff822b2f9d4abd96ff7414f6de1670, scope.project.id=24fafd1589784882b754b5f8543954bb, user.id=e8b14cd4ac074742a8c629d0a42902b8) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:23:56.559192 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce4bf53-ba7d-4d39-a1e4-5f352a8bfeca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:56.560848 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce4bf53-ba7d-4d39-a1e4-5f352a8bfeca None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:56.567419 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 161/318] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:23:56 2022] GET /identity/v3/role_assignments?user.id=e8b14cd4ac074742a8c629d0a42902b8&scope.project.id=24fafd1589784882b754b5f8543954bb&role.id=abff822b2f9d4abd96ff7414f6de1670 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:56.685001 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0d4eeb-2aaa-416b-aeff-5cedb3171429 None None] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:56.685981 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0d4eeb-2aaa-416b-aeff-5cedb3171429 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:56.686332 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0d4eeb-2aaa-416b-aeff-5cedb3171429 None None] PATH_INFO: `/` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:56.688089 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 158/319] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:23:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:23:56.693971 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0d46c2-1373-4e2a-9fb4-2e7561d930d8 None None] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:56.694503 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0d46c2-1373-4e2a-9fb4-2e7561d930d8 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:56.694503 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0d46c2-1373-4e2a-9fb4-2e7561d930d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:56.712375 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.auth.core [None req-aa0d46c2-1373-4e2a-9fb4-2e7561d930d8 None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143587) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:23:56.713596 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-aa0d46c2-1373-4e2a-9fb4-2e7561d930d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:23:56.789937 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 162/320] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:23:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:23:56.813824 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0d4eeb-2aaa-416b-aeff-5cedb3171429 None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:56.929617 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0d4eeb-2aaa-416b-aeff-5cedb3171429 None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:56.964411 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a143b95e-f456-475e-bf7c-97a70477395c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:56.965212 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a143b95e-f456-475e-bf7c-97a70477395c None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:56.965496 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a143b95e-f456-475e-bf7c-97a70477395c None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:56.965774 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a143b95e-f456-475e-bf7c-97a70477395c None admin] PATH_INFO: `/v3/domains/default` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:56.966977 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a143b95e-f456-475e-bf7c-97a70477395c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:56.968423 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a143b95e-f456-475e-bf7c-97a70477395c None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:56.970258 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 159/321] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:23:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:56.976102 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0d46c2-1373-4e2a-9fb4-2e7561d930d8 None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:57.018597 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:57.061297 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bbd630-455f-441f-8a16-ce7e0c5df989 None None] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:57.061684 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bbd630-455f-441f-8a16-ce7e0c5df989 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:57.061921 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bbd630-455f-441f-8a16-ce7e0c5df989 None None] PATH_INFO: `/` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:57.063545 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 160/322] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:23:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:23:57.068551 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc66d0f6-eba2-4151-893c-1912788a51b2 None None] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:57.068849 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc66d0f6-eba2-4151-893c-1912788a51b2 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:57.069130 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc66d0f6-eba2-4151-893c-1912788a51b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:57.079702 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0d46c2-1373-4e2a-9fb4-2e7561d930d8 None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:57.086697 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.auth.core [None req-cc66d0f6-eba2-4151-893c-1912788a51b2 None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143586) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:23:57.087896 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.fernet_utils [None req-cc66d0f6-eba2-4151-893c-1912788a51b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143586) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:23:57.102705 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:57.117893 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60e38fe-1c21-4bc4-a89f-0bd4d5270862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:57.118566 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60e38fe-1c21-4bc4-a89f-0bd4d5270862 None admin] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:57.118848 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60e38fe-1c21-4bc4-a89f-0bd4d5270862 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:57.119129 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60e38fe-1c21-4bc4-a89f-0bd4d5270862 None admin] PATH_INFO: `/v3/groups` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:57.119815 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60e38fe-1c21-4bc4-a89f-0bd4d5270862 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:57.121299 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60e38fe-1c21-4bc4-a89f-0bd4d5270862 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:57.136605 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 163/323] 10.222.0.57 () {60 vars in 1134 bytes} [Thu May 19 14:23:56 2022] POST /identity/v3/groups => generated 226 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:23:57.168690 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 161/324] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:23:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:23:57.191809 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60e38fe-1c21-4bc4-a89f-0bd4d5270862 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:57.309441 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:57.314766 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60e38fe-1c21-4bc4-a89f-0bd4d5270862 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:57.358309 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62315e42-5778-4852-95a8-a0dfc806d9fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:57.359110 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62315e42-5778-4852-95a8-a0dfc806d9fa None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:57.359395 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62315e42-5778-4852-95a8-a0dfc806d9fa None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:57.359675 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62315e42-5778-4852-95a8-a0dfc806d9fa None admin] PATH_INFO: `/v3/roles/reader` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:57.370415 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62315e42-5778-4852-95a8-a0dfc806d9fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:23:57.378239 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-62315e42-5778-4852-95a8-a0dfc806d9fa None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:23:57.378239 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:23:57.378239 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:23:57.378239 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:23:57.378239 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:23:57.378239 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:23:57.378239 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:23:57.378239 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:23:57.378239 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:23:57.378239 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:23:57.378239 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:23:57.378239 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:23:57.378239 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:23:57.378239 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:23:57.378239 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:23:57.378239 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:23:57.379356 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:23:57.379356 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:23:57.379356 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:23:57.379356 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:23:57.379356 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:23:57.379356 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:23:57.379356 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:23:57.379356 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:23:57.379356 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:23:57.379356 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:23:57.379356 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:23:57.379356 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:23:57.379356 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:23:57.379356 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:23:57.379356 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:23:57.379356 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:23:57.380374 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:23:57.380374 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:23:57.380374 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:23:57.380374 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:23:57.380374 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:23:57.380374 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:23:57.380374 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:23:57.380374 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:23:57.380374 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:23:57.380374 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:23:57.380374 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:23:57.380374 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:23:57.380374 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 164/325] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:23:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:23:57.385426 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc66d0f6-eba2-4151-893c-1912788a51b2 None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:57.487118 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c64518-86bb-4575-9495-5364a24f03d3 None None] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:57.487651 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c64518-86bb-4575-9495-5364a24f03d3 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:57.487943 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c64518-86bb-4575-9495-5364a24f03d3 None None] PATH_INFO: `/` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:57.489554 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 165/326] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:23:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:23:57.496142 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e24f89f-6674-49f0-b57c-27f366ea8155 None None] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:57.496436 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e24f89f-6674-49f0-b57c-27f366ea8155 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:57.496719 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e24f89f-6674-49f0-b57c-27f366ea8155 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:57.504475 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc66d0f6-eba2-4151-893c-1912788a51b2 None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:57.522929 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.auth.core [None req-2e24f89f-6674-49f0-b57c-27f366ea8155 None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143587) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:23:57.524167 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-2e24f89f-6674-49f0-b57c-27f366ea8155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:23:57.542465 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57888a47-1d9f-456e-afd1-39834c54dc74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:57.543134 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57888a47-1d9f-456e-afd1-39834c54dc74 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:57.543412 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57888a47-1d9f-456e-afd1-39834c54dc74 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:57.543690 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57888a47-1d9f-456e-afd1-39834c54dc74 None admin] PATH_INFO: `/v3/roles` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:57.544536 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57888a47-1d9f-456e-afd1-39834c54dc74 None admin] RBAC: Adding query filter params (name=reader) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:23:57.544955 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57888a47-1d9f-456e-afd1-39834c54dc74 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:57.546511 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57888a47-1d9f-456e-afd1-39834c54dc74 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:57.557894 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 162/327] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:23:57 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:57.561778 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57888a47-1d9f-456e-afd1-39834c54dc74 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:57.579502 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:57.624668 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 166/328] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:23:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:23:57.649691 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e24f89f-6674-49f0-b57c-27f366ea8155 None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:57.706851 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57888a47-1d9f-456e-afd1-39834c54dc74 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:57.763950 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e26df3-5ebe-47ab-8e04-afef9a907eb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:57.764766 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e26df3-5ebe-47ab-8e04-afef9a907eb2 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:57.765050 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e26df3-5ebe-47ab-8e04-afef9a907eb2 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:57.765325 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e26df3-5ebe-47ab-8e04-afef9a907eb2 None admin] PATH_INFO: `/v3/users/b47eff9c105c4e2fa106c63dcd162914` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:57.767886 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e26df3-5ebe-47ab-8e04-afef9a907eb2 None admin] RBAC: Authorizing `identity:get_user(user_id=b47eff9c105c4e2fa106c63dcd162914)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:57.769549 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e26df3-5ebe-47ab-8e04-afef9a907eb2 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:57.772136 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 163/329] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:23:57 2022] GET /identity/v3/users/b47eff9c105c4e2fa106c63dcd162914 => generated 293 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:23:57.779254 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e26df3-5ebe-47ab-8e04-afef9a907eb2 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:57.792816 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:57.809795 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e24f89f-6674-49f0-b57c-27f366ea8155 None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:57.849544 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf3a888-333b-491d-a136-37f98fc2108a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:57.850801 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf3a888-333b-491d-a136-37f98fc2108a None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:57.851183 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf3a888-333b-491d-a136-37f98fc2108a None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:57.851561 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf3a888-333b-491d-a136-37f98fc2108a None admin] PATH_INFO: `/v3/roles/member` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:57.864419 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf3a888-333b-491d-a136-37f98fc2108a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:23:57.874498 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-4bf3a888-333b-491d-a136-37f98fc2108a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:23:57.874498 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:23:57.874498 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:23:57.874498 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:23:57.874498 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:23:57.874498 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:23:57.874498 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:23:57.874498 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:23:57.874498 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:23:57.874498 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:23:57.874498 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:23:57.874498 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:23:57.874498 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:23:57.874498 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:23:57.874498 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:23:57.874498 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:23:57.875568 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:23:57.875568 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:23:57.875568 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:23:57.875568 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:23:57.875568 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:23:57.875568 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:23:57.875568 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:23:57.875568 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:23:57.875568 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:23:57.875568 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:23:57.875568 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:23:57.875568 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:23:57.875568 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:23:57.875568 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:23:57.875568 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:23:57.875568 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:23:57.876535 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:23:57.876535 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:23:57.876535 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:23:57.876535 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:23:57.876535 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:23:57.876535 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:23:57.876535 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:23:57.876535 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:23:57.876535 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:23:57.876535 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:23:57.876535 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:23:57.876535 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:23:57.876535 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 167/330] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:23:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:23:57.881973 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf3a888-333b-491d-a136-37f98fc2108a None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:57.891943 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:57.924648 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e26df3-5ebe-47ab-8e04-afef9a907eb2 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:57.979431 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daccd452-6115-472a-9145-2c3175a25d92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:57.981464 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daccd452-6115-472a-9145-2c3175a25d92 None admin] REQUEST_METHOD: `PUT` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:57.981992 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daccd452-6115-472a-9145-2c3175a25d92 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:57.982457 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daccd452-6115-472a-9145-2c3175a25d92 None admin] PATH_INFO: `/v3/system/users/b47eff9c105c4e2fa106c63dcd162914/roles/abff822b2f9d4abd96ff7414f6de1670` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:57.985018 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daccd452-6115-472a-9145-2c3175a25d92 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=b47eff9c105c4e2fa106c63dcd162914, role_id=abff822b2f9d4abd96ff7414f6de1670)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:57.992520 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daccd452-6115-472a-9145-2c3175a25d92 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:58.009965 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 164/331] 10.222.0.57 () {58 vars in 1334 bytes} [Thu May 19 14:23:57 2022] PUT /identity/v3/system/users/b47eff9c105c4e2fa106c63dcd162914/roles/abff822b2f9d4abd96ff7414f6de1670 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:23:58.033448 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf3a888-333b-491d-a136-37f98fc2108a None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:58.082466 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268bc116-ec72-4a18-9684-a98f5526601b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:58.083145 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268bc116-ec72-4a18-9684-a98f5526601b None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:58.083428 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268bc116-ec72-4a18-9684-a98f5526601b None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:58.083697 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268bc116-ec72-4a18-9684-a98f5526601b None admin] PATH_INFO: `/v3/roles` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:58.084450 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268bc116-ec72-4a18-9684-a98f5526601b None admin] RBAC: Adding query filter params (name=member) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:23:58.084760 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268bc116-ec72-4a18-9684-a98f5526601b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:58.086211 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268bc116-ec72-4a18-9684-a98f5526601b None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:58.097544 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 168/332] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:23:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:23:58.103522 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daccd452-6115-472a-9145-2c3175a25d92 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:58.242101 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daccd452-6115-472a-9145-2c3175a25d92 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:58.280726 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a7f7dc-ebe1-4fa9-996e-e68188377fb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:58.282072 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a7f7dc-ebe1-4fa9-996e-e68188377fb6 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:58.282597 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a7f7dc-ebe1-4fa9-996e-e68188377fb6 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:58.283097 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a7f7dc-ebe1-4fa9-996e-e68188377fb6 None admin] PATH_INFO: `/v3/users/4ef6ced909414456a7e6e8a61083bd4f` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:58.284616 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a7f7dc-ebe1-4fa9-996e-e68188377fb6 None admin] RBAC: Authorizing `identity:get_user(user_id=4ef6ced909414456a7e6e8a61083bd4f)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:58.286355 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a7f7dc-ebe1-4fa9-996e-e68188377fb6 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:58.288560 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 165/333] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:23:58 2022] GET /identity/v3/users/4ef6ced909414456a7e6e8a61083bd4f => generated 293 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:58.293999 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268bc116-ec72-4a18-9684-a98f5526601b None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:58.321495 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:58.433338 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268bc116-ec72-4a18-9684-a98f5526601b None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:58.474979 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dbc4b1-01ba-459b-bbf8-b70ed3c1ebfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:58.476038 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dbc4b1-01ba-459b-bbf8-b70ed3c1ebfa None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:58.476534 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dbc4b1-01ba-459b-bbf8-b70ed3c1ebfa None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:58.477024 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dbc4b1-01ba-459b-bbf8-b70ed3c1ebfa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:58.478121 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dbc4b1-01ba-459b-bbf8-b70ed3c1ebfa None admin] RBAC: Adding query filter params (role.id=3d5075293a4f45449eaef6af8fa53004, user.id=4ef6ced909414456a7e6e8a61083bd4f, scope.system=all) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:23:58.478677 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dbc4b1-01ba-459b-bbf8-b70ed3c1ebfa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:58.480399 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dbc4b1-01ba-459b-bbf8-b70ed3c1ebfa None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:58.498571 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 169/334] 10.222.0.57 () {56 vars in 1308 bytes} [Thu May 19 14:23:58 2022] GET /identity/v3/role_assignments?user.id=4ef6ced909414456a7e6e8a61083bd4f&scope.system=all&role.id=3d5075293a4f45449eaef6af8fa53004 => generated 510 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:59.909115 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07d4b86-fd35-4cdd-8792-bff5296cf7c6 None None] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:59.909636 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07d4b86-fd35-4cdd-8792-bff5296cf7c6 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:59.909636 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07d4b86-fd35-4cdd-8792-bff5296cf7c6 None None] PATH_INFO: `/` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:59.911281 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 166/335] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:23:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:23:59.926543 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b4fd78-c7df-44f1-82ac-a8c14d8b3425 None None] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:59.927034 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b4fd78-c7df-44f1-82ac-a8c14d8b3425 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:59.927152 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b4fd78-c7df-44f1-82ac-a8c14d8b3425 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:59.950163 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.auth.core [None req-c7b4fd78-c7df-44f1-82ac-a8c14d8b3425 None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143587) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:23:59.952246 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-c7b4fd78-c7df-44f1-82ac-a8c14d8b3425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:24:00.043018 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:00.048106 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 170/336] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:23:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:24:00.091573 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07d4b86-fd35-4cdd-8792-bff5296cf7c6 None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:00.110868 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:00.226340 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07d4b86-fd35-4cdd-8792-bff5296cf7c6 None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:00.269245 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6fd79a-2938-4351-b794-0dca7d4c4934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:00.270070 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6fd79a-2938-4351-b794-0dca7d4c4934 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:00.270353 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6fd79a-2938-4351-b794-0dca7d4c4934 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:00.270624 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6fd79a-2938-4351-b794-0dca7d4c4934 None admin] PATH_INFO: `/v3/roles/member` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:00.287010 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6fd79a-2938-4351-b794-0dca7d4c4934 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:24:00.295276 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-2d6fd79a-2938-4351-b794-0dca7d4c4934 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:24:00.295276 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:00.295276 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:00.295276 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:00.295276 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:00.295276 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:00.295276 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:00.295276 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:00.295276 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:00.295276 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:00.295276 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:00.295276 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:00.295276 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:24:00.295276 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:24:00.295276 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:24:00.295276 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:24:00.296287 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:24:00.296287 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:24:00.296287 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:24:00.296287 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:24:00.296287 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:00.296287 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:00.296287 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:00.296287 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:00.296287 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:00.296287 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:00.296287 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:00.296287 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:24:00.296287 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:00.296287 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:00.296287 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:00.296287 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:00.297593 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:00.297593 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:00.297593 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:00.297593 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:00.297593 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:24:00.297593 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:24:00.297593 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:24:00.297593 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:24:00.297593 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:24:00.297593 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:24:00.297593 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:24:00.297593 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:24:00.297593 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 167/337] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:24:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:00.304480 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b4fd78-c7df-44f1-82ac-a8c14d8b3425 None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:00.448314 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b4fd78-c7df-44f1-82ac-a8c14d8b3425 None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:00.493052 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1026575-7108-480a-80b6-be2f5d8bd548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:00.493655 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1026575-7108-480a-80b6-be2f5d8bd548 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:00.493975 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1026575-7108-480a-80b6-be2f5d8bd548 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:00.494251 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1026575-7108-480a-80b6-be2f5d8bd548 None admin] PATH_INFO: `/v3/roles` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:00.495014 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1026575-7108-480a-80b6-be2f5d8bd548 None admin] RBAC: Adding query filter params (name=member) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:00.495322 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1026575-7108-480a-80b6-be2f5d8bd548 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:00.496716 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1026575-7108-480a-80b6-be2f5d8bd548 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:00.506923 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 171/338] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:24:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:00.514198 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6fd79a-2938-4351-b794-0dca7d4c4934 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:00.588434 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:00.631083 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6fd79a-2938-4351-b794-0dca7d4c4934 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:00.687385 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be7a016-9497-4f37-939d-323a2df3d5d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:00.688045 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be7a016-9497-4f37-939d-323a2df3d5d5 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:00.688331 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be7a016-9497-4f37-939d-323a2df3d5d5 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:00.688608 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be7a016-9497-4f37-939d-323a2df3d5d5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:00.777651 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be7a016-9497-4f37-939d-323a2df3d5d5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:00.783197 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be7a016-9497-4f37-939d-323a2df3d5d5 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:00.789815 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34a46f8-dc8f-45ca-9655-c6b2b9a7b3fa None None] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:00.790126 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34a46f8-dc8f-45ca-9655-c6b2b9a7b3fa None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:00.790400 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34a46f8-dc8f-45ca-9655-c6b2b9a7b3fa None None] PATH_INFO: `/` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:00.791784 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 172/339] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:24:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:24:00.797805 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ed26f7-7ed0-44a7-8683-eab7be154d45 None None] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:00.798417 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ed26f7-7ed0-44a7-8683-eab7be154d45 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:00.798933 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ed26f7-7ed0-44a7-8683-eab7be154d45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:00.831894 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.auth.core [None req-07ed26f7-7ed0-44a7-8683-eab7be154d45 None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143587) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:24:00.833460 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-07ed26f7-7ed0-44a7-8683-eab7be154d45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:24:00.883858 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:00.925537 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 168/340] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:24:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:24:00.934180 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be7a016-9497-4f37-939d-323a2df3d5d5 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:01.003516 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:01.025508 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 173/341] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:24:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:24:01.069274 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ed26f7-7ed0-44a7-8683-eab7be154d45 None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:01.095179 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be7a016-9497-4f37-939d-323a2df3d5d5 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:01.157301 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8568b6c8-603d-4486-8650-48efc4909a36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:01.158228 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8568b6c8-603d-4486-8650-48efc4909a36 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:01.158512 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8568b6c8-603d-4486-8650-48efc4909a36 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:01.158791 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8568b6c8-603d-4486-8650-48efc4909a36 None admin] PATH_INFO: `/v3/projects/e7497d51ee8f4f0487b09541d4ad94d2` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:01.161581 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8568b6c8-603d-4486-8650-48efc4909a36 None admin] RBAC: Authorizing `identity:get_project(project_id=e7497d51ee8f4f0487b09541d4ad94d2)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:01.164068 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8568b6c8-603d-4486-8650-48efc4909a36 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:01.165957 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 169/342] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:24:00 2022] GET /identity/v3/projects/e7497d51ee8f4f0487b09541d4ad94d2 => generated 298 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:01.172000 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8568b6c8-603d-4486-8650-48efc4909a36 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:01.286218 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:01.302819 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ed26f7-7ed0-44a7-8683-eab7be154d45 None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:01.384562 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5052a896-b7b8-42dc-b29b-d5893fe1283b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:01.385439 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5052a896-b7b8-42dc-b29b-d5893fe1283b None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:01.385729 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5052a896-b7b8-42dc-b29b-d5893fe1283b None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:01.386030 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5052a896-b7b8-42dc-b29b-d5893fe1283b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:01.388210 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8568b6c8-603d-4486-8650-48efc4909a36 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:01.416352 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5052a896-b7b8-42dc-b29b-d5893fe1283b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:24:01.428919 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-5052a896-b7b8-42dc-b29b-d5893fe1283b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:24:01.428919 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:01.428919 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:01.428919 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:01.428919 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:01.428919 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:01.428919 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:01.428919 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:01.428919 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:01.428919 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:01.428919 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:01.428919 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:01.428919 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:24:01.428919 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:24:01.428919 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:24:01.428919 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:24:01.430033 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:24:01.430033 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:24:01.430033 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:24:01.430033 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:24:01.430033 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:01.430033 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:01.430033 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:01.430033 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:01.430033 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:01.430033 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:01.430033 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:01.430033 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:24:01.430033 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:01.430033 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:01.430033 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:01.430033 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:01.430971 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:01.430971 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:01.430971 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:01.430971 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:01.430971 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:24:01.430971 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:24:01.430971 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:24:01.430971 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:24:01.430971 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:24:01.430971 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:24:01.430971 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:24:01.430971 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:24:01.447164 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1ba80a-fb11-40fb-b3b7-68e3d0e92f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:01.447749 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 174/343] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 19 14:24:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 371 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:24:01.450098 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1ba80a-fb11-40fb-b3b7-68e3d0e92f97 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:01.450789 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f4ea90-686a-4afe-b905-c8d15d959628 None None] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:01.451604 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1ba80a-fb11-40fb-b3b7-68e3d0e92f97 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:01.452115 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f4ea90-686a-4afe-b905-c8d15d959628 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:01.452622 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1ba80a-fb11-40fb-b3b7-68e3d0e92f97 None admin] PATH_INFO: `/v3/groups/6f88b9d7a32742db86b87c6d666173ac` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:01.453113 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f4ea90-686a-4afe-b905-c8d15d959628 None None] PATH_INFO: `/` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:01.457151 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 175/344] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:24:01 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:24:01.461147 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acdd2ff-93a1-43c6-bb44-ea184c6f207b None None] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:01.461664 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acdd2ff-93a1-43c6-bb44-ea184c6f207b None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:01.462160 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acdd2ff-93a1-43c6-bb44-ea184c6f207b None None] PATH_INFO: `/` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:01.463731 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 176/345] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:24:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:24:01.464651 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acdd2ff-93a1-43c6-bb44-ea184c6f207b None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:01.473268 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1ba80a-fb11-40fb-b3b7-68e3d0e92f97 None admin] RBAC: Authorizing `identity:get_group(group_id=6f88b9d7a32742db86b87c6d666173ac)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:01.475599 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1ba80a-fb11-40fb-b3b7-68e3d0e92f97 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:01.478839 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 170/346] 10.222.0.57 () {56 vars in 1180 bytes} [Thu May 19 14:24:01 2022] GET /identity/v3/groups/6f88b9d7a32742db86b87c6d666173ac => generated 226 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:01.480310 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119af8e3-63d7-47e8-8429-8b2ac8bc03ad None None] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:01.480851 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119af8e3-63d7-47e8-8429-8b2ac8bc03ad None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:01.481404 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119af8e3-63d7-47e8-8429-8b2ac8bc03ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:01.505050 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.auth.core [None req-119af8e3-63d7-47e8-8429-8b2ac8bc03ad None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143586) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:24:01.508943 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.fernet_utils [None req-119af8e3-63d7-47e8-8429-8b2ac8bc03ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143586) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:24:01.537427 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:01.612122 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acdd2ff-93a1-43c6-bb44-ea184c6f207b None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:01.621541 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 171/347] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:24:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:24:01.623476 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef924ba1-0070-47ac-9878-80efebc195c1 None None] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:01.624031 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef924ba1-0070-47ac-9878-80efebc195c1 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:01.624566 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef924ba1-0070-47ac-9878-80efebc195c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:01.648374 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.auth.core [None req-ef924ba1-0070-47ac-9878-80efebc195c1 None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143586) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:24:01.650009 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.fernet_utils [None req-ef924ba1-0070-47ac-9878-80efebc195c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143586) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:24:01.685727 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177a8b16-5171-4aab-b2eb-c1f9d53c68a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:01.686784 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177a8b16-5171-4aab-b2eb-c1f9d53c68a2 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:01.687074 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177a8b16-5171-4aab-b2eb-c1f9d53c68a2 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:01.687409 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177a8b16-5171-4aab-b2eb-c1f9d53c68a2 None admin] PATH_INFO: `/v3/roles` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:01.688181 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177a8b16-5171-4aab-b2eb-c1f9d53c68a2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:01.688496 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177a8b16-5171-4aab-b2eb-c1f9d53c68a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:01.690047 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177a8b16-5171-4aab-b2eb-c1f9d53c68a2 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:01.704205 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:01.704425 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 177/348] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:24:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:01.704425 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177a8b16-5171-4aab-b2eb-c1f9d53c68a2 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:01.754746 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 172/349] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:24:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:24:01.756048 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef924ba1-0070-47ac-9878-80efebc195c1 None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:01.830661 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177a8b16-5171-4aab-b2eb-c1f9d53c68a2 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:01.849401 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:01.876026 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac90a38-e409-4e3f-a011-aa7fe872ad00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:01.876685 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac90a38-e409-4e3f-a011-aa7fe872ad00 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:01.876966 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac90a38-e409-4e3f-a011-aa7fe872ad00 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:01.877339 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac90a38-e409-4e3f-a011-aa7fe872ad00 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:01.878119 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac90a38-e409-4e3f-a011-aa7fe872ad00 None admin] RBAC: Adding query filter params (group.id=6f88b9d7a32742db86b87c6d666173ac, role.id=3d5075293a4f45449eaef6af8fa53004, scope.project.id=e7497d51ee8f4f0487b09541d4ad94d2) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:01.878431 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac90a38-e409-4e3f-a011-aa7fe872ad00 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:01.879884 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac90a38-e409-4e3f-a011-aa7fe872ad00 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:01.886354 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 178/350] 10.222.0.57 () {56 vars in 1376 bytes} [Thu May 19 14:24:01 2022] GET /identity/v3/role_assignments?group.id=6f88b9d7a32742db86b87c6d666173ac&scope.project.id=e7497d51ee8f4f0487b09541d4ad94d2&role.id=3d5075293a4f45449eaef6af8fa53004 => generated 260 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:01.887308 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac90a38-e409-4e3f-a011-aa7fe872ad00 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:01.896658 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef924ba1-0070-47ac-9878-80efebc195c1 None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:01.940590 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cbd20e-e9a4-4bbd-8a26-777f755d6d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:01.941373 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cbd20e-e9a4-4bbd-8a26-777f755d6d4f None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:01.941626 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cbd20e-e9a4-4bbd-8a26-777f755d6d4f None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:01.941942 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cbd20e-e9a4-4bbd-8a26-777f755d6d4f None admin] PATH_INFO: `/v3/roles/member` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:01.961959 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cbd20e-e9a4-4bbd-8a26-777f755d6d4f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:24:01.970771 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-b7cbd20e-e9a4-4bbd-8a26-777f755d6d4f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:24:01.970771 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:01.970771 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:01.970771 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:01.970771 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:01.970771 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:01.970771 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:01.970771 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:01.970771 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:01.970771 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:01.970771 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:01.970771 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:01.970771 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:24:01.970771 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:24:01.970771 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:24:01.970771 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:24:01.971876 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:24:01.971876 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:24:01.971876 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:24:01.971876 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:24:01.971876 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:01.971876 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:01.971876 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:01.971876 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:01.971876 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:01.971876 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:01.971876 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:01.971876 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:24:01.971876 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:01.971876 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:01.971876 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:01.971876 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:01.973243 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:01.973243 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:01.973243 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:01.973243 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:01.973243 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:24:01.973243 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:24:01.973243 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:24:01.973243 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:24:01.973243 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:24:01.973243 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:24:01.973243 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:24:01.973243 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:24:01.973243 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 173/351] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:24:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:01.973243 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cbd20e-e9a4-4bbd-8a26-777f755d6d4f None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:02.004384 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:02.038538 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac90a38-e409-4e3f-a011-aa7fe872ad00 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:02.086905 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98595be-79f1-4476-b356-44f8600982e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:02.087578 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98595be-79f1-4476-b356-44f8600982e2 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:02.088484 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98595be-79f1-4476-b356-44f8600982e2 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:02.088771 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98595be-79f1-4476-b356-44f8600982e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:02.102578 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cbd20e-e9a4-4bbd-8a26-777f755d6d4f None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:02.156023 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0fb3d1-8d94-41df-994f-88e37d96851e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:02.157231 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0fb3d1-8d94-41df-994f-88e37d96851e None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:02.157753 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0fb3d1-8d94-41df-994f-88e37d96851e None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:02.158300 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0fb3d1-8d94-41df-994f-88e37d96851e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:02.161421 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:02.174036 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0fb3d1-8d94-41df-994f-88e37d96851e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:24:02.186859 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-fe0fb3d1-8d94-41df-994f-88e37d96851e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:24:02.186859 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:02.186859 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:02.186859 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:02.186859 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:02.186859 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:02.186859 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:02.186859 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:02.186859 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:02.186859 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:02.186859 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:02.186859 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:02.186859 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:24:02.186859 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:24:02.186859 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:24:02.186859 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:24:02.188027 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:24:02.188027 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:24:02.188027 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:24:02.188027 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:24:02.188027 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:02.188027 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:02.188027 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:02.188027 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:02.188027 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:02.188027 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:02.188027 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:02.188027 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:24:02.188027 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:02.188027 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:02.188027 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:02.188027 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:02.188944 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:02.188944 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:02.188944 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:02.188944 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:02.188944 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:24:02.188944 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:24:02.188944 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:24:02.188944 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:24:02.188944 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:24:02.188944 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:24:02.188944 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:24:02.188944 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:24:02.191929 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 174/352] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 19 14:24:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:02.193052 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98595be-79f1-4476-b356-44f8600982e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:02.193902 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33181319-21b4-41d8-b5bf-c6c73c13b83c None None] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:02.194410 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33181319-21b4-41d8-b5bf-c6c73c13b83c None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:02.194612 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98595be-79f1-4476-b356-44f8600982e2 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:02.195090 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33181319-21b4-41d8-b5bf-c6c73c13b83c None None] PATH_INFO: `/` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:02.198332 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 175/353] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:24:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:24:02.199404 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33181319-21b4-41d8-b5bf-c6c73c13b83c None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:02.294280 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:02.317989 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33181319-21b4-41d8-b5bf-c6c73c13b83c None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:02.320205 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 179/354] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:24:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:24:02.320884 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98595be-79f1-4476-b356-44f8600982e2 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:02.370101 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feea7bf4-20bf-4b44-9746-390ddf2f8083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:02.370785 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feea7bf4-20bf-4b44-9746-390ddf2f8083 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:02.371068 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feea7bf4-20bf-4b44-9746-390ddf2f8083 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:02.371344 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feea7bf4-20bf-4b44-9746-390ddf2f8083 None admin] PATH_INFO: `/v3/roles` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:02.372109 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feea7bf4-20bf-4b44-9746-390ddf2f8083 None admin] RBAC: Adding query filter params (name=member) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:02.372459 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feea7bf4-20bf-4b44-9746-390ddf2f8083 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:02.374354 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feea7bf4-20bf-4b44-9746-390ddf2f8083 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:02.389313 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 176/355] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:24:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:02.389313 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a56ebc2-569f-442f-b83f-f9042d32754b None None] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:02.389313 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a56ebc2-569f-442f-b83f-f9042d32754b None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:02.389313 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a56ebc2-569f-442f-b83f-f9042d32754b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:02.408069 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.auth.core [None req-9a56ebc2-569f-442f-b83f-f9042d32754b None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143586) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:24:02.409273 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.fernet_utils [None req-9a56ebc2-569f-442f-b83f-f9042d32754b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143586) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:24:02.431456 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:02.461933 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98595be-79f1-4476-b356-44f8600982e2 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:02.504102 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 177/356] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:24:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:24:02.505183 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a56ebc2-569f-442f-b83f-f9042d32754b None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:02.525996 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dec17dd-ff75-441b-a109-4c1ee5c546c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:02.526679 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dec17dd-ff75-441b-a109-4c1ee5c546c1 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:02.526958 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dec17dd-ff75-441b-a109-4c1ee5c546c1 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:02.527235 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dec17dd-ff75-441b-a109-4c1ee5c546c1 None admin] PATH_INFO: `/v3/roles` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:02.527993 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dec17dd-ff75-441b-a109-4c1ee5c546c1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:02.528302 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dec17dd-ff75-441b-a109-4c1ee5c546c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:02.530387 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dec17dd-ff75-441b-a109-4c1ee5c546c1 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:02.546435 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 180/357] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:24:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:02.547380 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dec17dd-ff75-441b-a109-4c1ee5c546c1 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:02.624209 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:02.671132 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a56ebc2-569f-442f-b83f-f9042d32754b None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:02.712516 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dec17dd-ff75-441b-a109-4c1ee5c546c1 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:02.721804 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23a7407-0dde-412b-8995-343b25544a08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:02.723501 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23a7407-0dde-412b-8995-343b25544a08 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:02.724014 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23a7407-0dde-412b-8995-343b25544a08 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:02.725191 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23a7407-0dde-412b-8995-343b25544a08 None admin] PATH_INFO: `/v3/projects/e7497d51ee8f4f0487b09541d4ad94d2` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:02.727024 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23a7407-0dde-412b-8995-343b25544a08 None admin] RBAC: Authorizing `identity:get_project(project_id=e7497d51ee8f4f0487b09541d4ad94d2)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:02.729393 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23a7407-0dde-412b-8995-343b25544a08 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:02.733244 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 178/358] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:24:02 2022] GET /identity/v3/projects/e7497d51ee8f4f0487b09541d4ad94d2 => generated 298 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:02.734507 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23a7407-0dde-412b-8995-343b25544a08 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:02.758418 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4077f258-008b-4f82-8f36-af8b667476c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:02.759620 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4077f258-008b-4f82-8f36-af8b667476c2 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:02.759933 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4077f258-008b-4f82-8f36-af8b667476c2 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:02.760163 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4077f258-008b-4f82-8f36-af8b667476c2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:02.774488 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:02.853470 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4077f258-008b-4f82-8f36-af8b667476c2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:02.855002 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4077f258-008b-4f82-8f36-af8b667476c2 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:02.886700 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23a7407-0dde-412b-8995-343b25544a08 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:02.926056 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772d0e14-fd12-40c4-9448-1422450ebf88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:02.926949 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772d0e14-fd12-40c4-9448-1422450ebf88 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:02.927233 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772d0e14-fd12-40c4-9448-1422450ebf88 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:02.927503 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772d0e14-fd12-40c4-9448-1422450ebf88 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:02.938642 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772d0e14-fd12-40c4-9448-1422450ebf88 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:24:02.947250 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-772d0e14-fd12-40c4-9448-1422450ebf88 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:24:02.947250 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:02.947250 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:02.947250 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:02.947250 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:02.947250 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:02.947250 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:02.947250 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:02.947250 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:02.947250 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:02.947250 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:02.947250 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:02.947250 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:24:02.947250 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:24:02.947250 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:24:02.947250 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:24:02.948722 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:24:02.948722 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:24:02.948722 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:24:02.948722 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:24:02.948722 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:02.948722 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:02.948722 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:02.948722 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:02.948722 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:02.948722 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:02.948722 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:02.948722 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:24:02.948722 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:02.948722 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:02.948722 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:02.948722 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:02.949689 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:02.949689 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:02.949689 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:02.949689 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:02.949689 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:24:02.949689 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:24:02.949689 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:24:02.949689 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:24:02.949689 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:24:02.949689 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:24:02.949689 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:24:02.949689 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:24:02.949689 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 179/359] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:24:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:02.949689 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772d0e14-fd12-40c4-9448-1422450ebf88 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:02.952482 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:02.995929 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 181/360] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:24:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:24:02.996520 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4077f258-008b-4f82-8f36-af8b667476c2 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:03.055854 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:03.086906 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772d0e14-fd12-40c4-9448-1422450ebf88 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:03.135623 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4077f258-008b-4f82-8f36-af8b667476c2 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:03.137787 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474485db-8cc3-4cd1-9b3b-133206ba047e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:03.139778 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474485db-8cc3-4cd1-9b3b-133206ba047e None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:03.140067 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474485db-8cc3-4cd1-9b3b-133206ba047e None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:03.140346 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474485db-8cc3-4cd1-9b3b-133206ba047e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:03.188649 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4fa880-ab39-4d53-9dcf-b5c0cca72340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:03.190526 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4fa880-ab39-4d53-9dcf-b5c0cca72340 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:03.190807 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4fa880-ab39-4d53-9dcf-b5c0cca72340 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:03.191085 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4fa880-ab39-4d53-9dcf-b5c0cca72340 None admin] PATH_INFO: `/v3/groups/6f88b9d7a32742db86b87c6d666173ac` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:03.192568 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4fa880-ab39-4d53-9dcf-b5c0cca72340 None admin] RBAC: Authorizing `identity:get_group(group_id=6f88b9d7a32742db86b87c6d666173ac)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:03.195273 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4fa880-ab39-4d53-9dcf-b5c0cca72340 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:03.198238 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 182/361] 10.222.0.57 () {56 vars in 1180 bytes} [Thu May 19 14:24:02 2022] GET /identity/v3/groups/6f88b9d7a32742db86b87c6d666173ac => generated 226 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:03.198238 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80a56fe-da00-4f4d-9697-94ff83e1cc2b None None] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:03.198238 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80a56fe-da00-4f4d-9697-94ff83e1cc2b None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:03.198238 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80a56fe-da00-4f4d-9697-94ff83e1cc2b None None] PATH_INFO: `/` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:03.199379 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 183/362] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:24:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:24:03.200163 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80a56fe-da00-4f4d-9697-94ff83e1cc2b None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:03.242235 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474485db-8cc3-4cd1-9b3b-133206ba047e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:03.243873 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474485db-8cc3-4cd1-9b3b-133206ba047e None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:03.246304 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:03.355608 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80a56fe-da00-4f4d-9697-94ff83e1cc2b None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:03.368424 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:03.398208 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 180/363] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:24:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:24:03.399223 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474485db-8cc3-4cd1-9b3b-133206ba047e None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:03.403208 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3ba2c7-09a7-4a4b-99a3-efe668b2a52d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:03.404487 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3ba2c7-09a7-4a4b-99a3-efe668b2a52d None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:03.404768 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3ba2c7-09a7-4a4b-99a3-efe668b2a52d None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:03.405039 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3ba2c7-09a7-4a4b-99a3-efe668b2a52d None admin] PATH_INFO: `/v3/roles` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:03.405895 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3ba2c7-09a7-4a4b-99a3-efe668b2a52d None admin] RBAC: Adding query filter params (name=reader) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:03.406207 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3ba2c7-09a7-4a4b-99a3-efe668b2a52d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:03.407601 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3ba2c7-09a7-4a4b-99a3-efe668b2a52d None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:03.416650 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 184/364] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:24:03 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:03.417780 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3ba2c7-09a7-4a4b-99a3-efe668b2a52d None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:03.545301 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474485db-8cc3-4cd1-9b3b-133206ba047e None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:03.547087 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:03.553567 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3ba2c7-09a7-4a4b-99a3-efe668b2a52d None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:03.593877 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1355b0-32ae-4be6-b809-d2621b74e893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:03.594940 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1355b0-32ae-4be6-b809-d2621b74e893 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:03.595315 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1355b0-32ae-4be6-b809-d2621b74e893 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:03.595600 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1355b0-32ae-4be6-b809-d2621b74e893 None admin] PATH_INFO: `/v3/projects/24fafd1589784882b754b5f8543954bb` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:03.596805 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1355b0-32ae-4be6-b809-d2621b74e893 None admin] RBAC: Authorizing `identity:get_project(project_id=24fafd1589784882b754b5f8543954bb)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:03.596996 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e44cb1-0c4a-4917-b522-8809b3ffb642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:03.597650 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e44cb1-0c4a-4917-b522-8809b3ffb642 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:03.600783 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e44cb1-0c4a-4917-b522-8809b3ffb642 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:03.600783 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e44cb1-0c4a-4917-b522-8809b3ffb642 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:03.600997 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1355b0-32ae-4be6-b809-d2621b74e893 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:03.601091 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e44cb1-0c4a-4917-b522-8809b3ffb642 None admin] RBAC: Adding query filter params (group.id=6f88b9d7a32742db86b87c6d666173ac, role.id=b954b2c7ce764aa0afac78e075a011a6, scope.project.id=e7497d51ee8f4f0487b09541d4ad94d2) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:03.601091 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e44cb1-0c4a-4917-b522-8809b3ffb642 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:03.601251 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 181/365] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:24:03 2022] GET /identity/v3/projects/24fafd1589784882b754b5f8543954bb => generated 302 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:03.601475 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e44cb1-0c4a-4917-b522-8809b3ffb642 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:03.601576 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5619d2d2-b56a-4a7a-8a81-39612d9d194d None None] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:03.601653 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5619d2d2-b56a-4a7a-8a81-39612d9d194d None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:03.601964 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5619d2d2-b56a-4a7a-8a81-39612d9d194d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:03.608606 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 185/366] 10.222.0.57 () {56 vars in 1376 bytes} [Thu May 19 14:24:03 2022] GET /identity/v3/role_assignments?group.id=6f88b9d7a32742db86b87c6d666173ac&scope.project.id=e7497d51ee8f4f0487b09541d4ad94d2&role.id=b954b2c7ce764aa0afac78e075a011a6 => generated 260 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:03.609438 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e44cb1-0c4a-4917-b522-8809b3ffb642 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:03.625395 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.auth.core [None req-5619d2d2-b56a-4a7a-8a81-39612d9d194d None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143586) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:24:03.627186 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.fernet_utils [None req-5619d2d2-b56a-4a7a-8a81-39612d9d194d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143586) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:24:03.670301 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:03.754018 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 182/367] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:24:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:24:03.755029 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5619d2d2-b56a-4a7a-8a81-39612d9d194d None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:03.764724 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e44cb1-0c4a-4917-b522-8809b3ffb642 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:03.813962 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29a4982-2f02-4aed-8d83-16bac9ab1925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:03.815317 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29a4982-2f02-4aed-8d83-16bac9ab1925 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:03.815831 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29a4982-2f02-4aed-8d83-16bac9ab1925 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:03.816418 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29a4982-2f02-4aed-8d83-16bac9ab1925 None admin] PATH_INFO: `/v3/projects/24fafd1589784882b754b5f8543954bb` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:03.817980 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29a4982-2f02-4aed-8d83-16bac9ab1925 None admin] RBAC: Authorizing `identity:get_project(project_id=24fafd1589784882b754b5f8543954bb)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:03.819865 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29a4982-2f02-4aed-8d83-16bac9ab1925 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:03.821870 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 186/368] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:24:03 2022] GET /identity/v3/projects/24fafd1589784882b754b5f8543954bb => generated 302 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:03.822887 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29a4982-2f02-4aed-8d83-16bac9ab1925 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:03.895751 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:03.899839 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5619d2d2-b56a-4a7a-8a81-39612d9d194d None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:03.946491 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e950d432-7606-4f00-8a79-7690831ecd91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:03.947377 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e950d432-7606-4f00-8a79-7690831ecd91 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:03.947916 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e950d432-7606-4f00-8a79-7690831ecd91 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:03.947916 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e950d432-7606-4f00-8a79-7690831ecd91 None admin] PATH_INFO: `/v3/users/b47eff9c105c4e2fa106c63dcd162914` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:03.949154 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e950d432-7606-4f00-8a79-7690831ecd91 None admin] RBAC: Authorizing `identity:get_user(user_id=b47eff9c105c4e2fa106c63dcd162914)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:03.950673 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e950d432-7606-4f00-8a79-7690831ecd91 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:03.952911 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 183/369] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:24:03 2022] GET /identity/v3/users/b47eff9c105c4e2fa106c63dcd162914 => generated 293 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:03.953790 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e950d432-7606-4f00-8a79-7690831ecd91 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:03.970855 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29a4982-2f02-4aed-8d83-16bac9ab1925 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:03.972970 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:04.017810 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51814ce-1a6a-4bc4-bb16-b0c9b2d19d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:04.018811 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51814ce-1a6a-4bc4-bb16-b0c9b2d19d00 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:04.019096 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51814ce-1a6a-4bc4-bb16-b0c9b2d19d00 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:04.019373 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51814ce-1a6a-4bc4-bb16-b0c9b2d19d00 None admin] PATH_INFO: `/v3/groups/6f88b9d7a32742db86b87c6d666173ac` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:04.020747 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51814ce-1a6a-4bc4-bb16-b0c9b2d19d00 None admin] RBAC: Authorizing `identity:get_group(group_id=6f88b9d7a32742db86b87c6d666173ac)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:04.022234 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51814ce-1a6a-4bc4-bb16-b0c9b2d19d00 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:04.024112 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 187/370] 10.222.0.57 () {56 vars in 1180 bytes} [Thu May 19 14:24:03 2022] GET /identity/v3/groups/6f88b9d7a32742db86b87c6d666173ac => generated 226 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:04.024888 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51814ce-1a6a-4bc4-bb16-b0c9b2d19d00 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:04.084994 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e950d432-7606-4f00-8a79-7690831ecd91 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:04.126664 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad63ef2a-4f26-49b6-a468-6bc17f6c6975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:04.131713 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad63ef2a-4f26-49b6-a468-6bc17f6c6975 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:04.132071 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad63ef2a-4f26-49b6-a468-6bc17f6c6975 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:04.132348 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad63ef2a-4f26-49b6-a468-6bc17f6c6975 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:04.144790 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51814ce-1a6a-4bc4-bb16-b0c9b2d19d00 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:04.146293 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad63ef2a-4f26-49b6-a468-6bc17f6c6975 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:24:04.162900 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-ad63ef2a-4f26-49b6-a468-6bc17f6c6975 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:24:04.162900 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:04.162900 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:04.162900 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:04.162900 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:04.162900 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:04.162900 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:04.162900 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:04.162900 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:04.162900 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:04.162900 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:04.162900 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:04.162900 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:24:04.162900 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:24:04.162900 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:24:04.162900 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:24:04.166715 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:24:04.166715 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:24:04.166715 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:24:04.166715 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:24:04.166715 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:04.166715 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:04.166715 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:04.166715 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:04.166715 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:04.166715 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:04.166715 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:04.166715 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:24:04.166715 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:04.166715 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:04.166715 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:04.166715 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:04.168924 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:04.168924 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:04.168924 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:04.168924 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:04.168924 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:24:04.168924 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:24:04.168924 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:24:04.168924 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:24:04.168924 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:24:04.168924 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:24:04.168924 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:24:04.168924 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:24:04.168924 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 184/371] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:24:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:04.168924 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad63ef2a-4f26-49b6-a468-6bc17f6c6975 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:04.184240 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7d69f8-1b0b-4431-b32d-187b188e7f75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:04.186129 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7d69f8-1b0b-4431-b32d-187b188e7f75 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:04.186418 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7d69f8-1b0b-4431-b32d-187b188e7f75 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:04.186795 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7d69f8-1b0b-4431-b32d-187b188e7f75 None admin] PATH_INFO: `/v3/groups/6f88b9d7a32742db86b87c6d666173ac` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:04.189323 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7d69f8-1b0b-4431-b32d-187b188e7f75 None admin] RBAC: Authorizing `identity:get_group(group_id=6f88b9d7a32742db86b87c6d666173ac)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:04.193676 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7d69f8-1b0b-4431-b32d-187b188e7f75 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:04.196840 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 188/372] 10.222.0.57 () {56 vars in 1180 bytes} [Thu May 19 14:24:04 2022] GET /identity/v3/groups/6f88b9d7a32742db86b87c6d666173ac => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:04.197767 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7d69f8-1b0b-4431-b32d-187b188e7f75 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:04.203755 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:04.256751 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:04.300736 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad63ef2a-4f26-49b6-a468-6bc17f6c6975 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:04.334423 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7d69f8-1b0b-4431-b32d-187b188e7f75 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:04.350484 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee4f47c-ed2d-4498-8563-aed6afe89d24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:04.351181 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee4f47c-ed2d-4498-8563-aed6afe89d24 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:04.351535 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee4f47c-ed2d-4498-8563-aed6afe89d24 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:04.351709 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee4f47c-ed2d-4498-8563-aed6afe89d24 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:04.352626 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee4f47c-ed2d-4498-8563-aed6afe89d24 None admin] RBAC: Adding query filter params (role.id=abff822b2f9d4abd96ff7414f6de1670, user.id=b47eff9c105c4e2fa106c63dcd162914, scope.system=all) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:04.352943 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee4f47c-ed2d-4498-8563-aed6afe89d24 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:04.356433 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee4f47c-ed2d-4498-8563-aed6afe89d24 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:04.368151 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 185/373] 10.222.0.57 () {56 vars in 1308 bytes} [Thu May 19 14:24:04 2022] GET /identity/v3/role_assignments?user.id=b47eff9c105c4e2fa106c63dcd162914&scope.system=all&role.id=abff822b2f9d4abd96ff7414f6de1670 => generated 510 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:04.369014 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee4f47c-ed2d-4498-8563-aed6afe89d24 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:04.399840 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63b2866-14d5-4804-a48b-959fcd2a591e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:04.400902 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63b2866-14d5-4804-a48b-959fcd2a591e None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:04.401380 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63b2866-14d5-4804-a48b-959fcd2a591e None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:04.401878 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63b2866-14d5-4804-a48b-959fcd2a591e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:04.403071 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63b2866-14d5-4804-a48b-959fcd2a591e None admin] RBAC: Adding query filter params (group.id=6f88b9d7a32742db86b87c6d666173ac, role.id=3d5075293a4f45449eaef6af8fa53004, scope.project.id=24fafd1589784882b754b5f8543954bb) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:04.404358 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63b2866-14d5-4804-a48b-959fcd2a591e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:04.406473 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63b2866-14d5-4804-a48b-959fcd2a591e None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:04.423163 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 189/374] 10.222.0.57 () {56 vars in 1376 bytes} [Thu May 19 14:24:04 2022] GET /identity/v3/role_assignments?group.id=6f88b9d7a32742db86b87c6d666173ac&scope.project.id=24fafd1589784882b754b5f8543954bb&role.id=3d5075293a4f45449eaef6af8fa53004 => generated 260 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:04.424363 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63b2866-14d5-4804-a48b-959fcd2a591e None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:04.481400 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:04.503662 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee4f47c-ed2d-4498-8563-aed6afe89d24 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:04.546295 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de50b0f-a546-4443-a406-0c094128fa35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:04.547103 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de50b0f-a546-4443-a406-0c094128fa35 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:04.547418 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de50b0f-a546-4443-a406-0c094128fa35 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:04.547756 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de50b0f-a546-4443-a406-0c094128fa35 None admin] PATH_INFO: `/v3/roles` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:04.548626 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de50b0f-a546-4443-a406-0c094128fa35 None admin] RBAC: Adding query filter params (name=admin) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:04.549143 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de50b0f-a546-4443-a406-0c094128fa35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:04.551048 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de50b0f-a546-4443-a406-0c094128fa35 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:04.563692 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 186/375] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:24:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:04.570063 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de50b0f-a546-4443-a406-0c094128fa35 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:04.610661 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63b2866-14d5-4804-a48b-959fcd2a591e None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:04.612475 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:04.675173 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61a1361-59a0-4bc7-965b-c8c09bbf8aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:04.675946 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61a1361-59a0-4bc7-965b-c8c09bbf8aa3 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:04.676226 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61a1361-59a0-4bc7-965b-c8c09bbf8aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:04.676512 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61a1361-59a0-4bc7-965b-c8c09bbf8aa3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:04.677352 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61a1361-59a0-4bc7-965b-c8c09bbf8aa3 None admin] RBAC: Adding query filter params (group.id=6f88b9d7a32742db86b87c6d666173ac, role.id=b954b2c7ce764aa0afac78e075a011a6, scope.project.id=24fafd1589784882b754b5f8543954bb) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:04.677674 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61a1361-59a0-4bc7-965b-c8c09bbf8aa3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:04.679285 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61a1361-59a0-4bc7-965b-c8c09bbf8aa3 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:04.691586 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 190/376] 10.222.0.57 () {56 vars in 1376 bytes} [Thu May 19 14:24:04 2022] GET /identity/v3/role_assignments?group.id=6f88b9d7a32742db86b87c6d666173ac&scope.project.id=24fafd1589784882b754b5f8543954bb&role.id=b954b2c7ce764aa0afac78e075a011a6 => generated 260 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:04.692666 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a86652-7b61-4597-981d-d61926549b5b None None] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:04.692960 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a86652-7b61-4597-981d-d61926549b5b None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:04.693243 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a86652-7b61-4597-981d-d61926549b5b None None] PATH_INFO: `/` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:04.695097 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 191/377] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:24:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:24:04.705154 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3fbf01-212e-46ac-9f2c-31eea211697d None None] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:04.705540 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3fbf01-212e-46ac-9f2c-31eea211697d None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:04.705720 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3fbf01-212e-46ac-9f2c-31eea211697d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:04.737723 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de50b0f-a546-4443-a406-0c094128fa35 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:04.753333 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.auth.core [None req-cf3fbf01-212e-46ac-9f2c-31eea211697d None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143587) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:24:04.754673 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-cf3fbf01-212e-46ac-9f2c-31eea211697d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:24:04.830521 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d7afd6-ed44-4871-b2ad-f25d2fd2a7dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:04.831244 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d7afd6-ed44-4871-b2ad-f25d2fd2a7dd None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:04.831529 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d7afd6-ed44-4871-b2ad-f25d2fd2a7dd None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:04.831878 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d7afd6-ed44-4871-b2ad-f25d2fd2a7dd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:04.863108 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:04.931373 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 192/378] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:24:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:24:04.957340 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d7afd6-ed44-4871-b2ad-f25d2fd2a7dd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:04.959007 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d7afd6-ed44-4871-b2ad-f25d2fd2a7dd None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:04.968647 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3fbf01-212e-46ac-9f2c-31eea211697d None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:05.089832 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 187/379] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:24:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 520 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 14:24:05.095564 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d7afd6-ed44-4871-b2ad-f25d2fd2a7dd None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:05.135397 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:05.146081 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3fbf01-212e-46ac-9f2c-31eea211697d None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:05.190832 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:05.234186 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86eda4cf-623c-4fe0-9b7a-8dadc44a0d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:05.235447 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eda4cf-623c-4fe0-9b7a-8dadc44a0d3e None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:05.236067 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eda4cf-623c-4fe0-9b7a-8dadc44a0d3e None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:05.236573 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eda4cf-623c-4fe0-9b7a-8dadc44a0d3e None admin] PATH_INFO: `/v3/roles/member` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:05.260308 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d7afd6-ed44-4871-b2ad-f25d2fd2a7dd None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:05.270872 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86eda4cf-623c-4fe0-9b7a-8dadc44a0d3e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:24:05.285868 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-86eda4cf-623c-4fe0-9b7a-8dadc44a0d3e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:24:05.285868 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:05.285868 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:05.285868 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:05.285868 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:05.285868 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:05.285868 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:05.285868 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:05.285868 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:05.285868 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:05.285868 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:05.285868 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:05.285868 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:24:05.285868 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:24:05.285868 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:24:05.285868 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:24:05.286944 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:24:05.286944 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:24:05.286944 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:24:05.286944 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:24:05.286944 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:05.286944 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:05.286944 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:05.286944 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:05.286944 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:05.286944 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:05.286944 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:05.286944 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:24:05.286944 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:05.286944 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:05.286944 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:05.286944 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:05.287876 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:05.287876 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:05.287876 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:05.287876 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:05.287876 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:24:05.287876 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:24:05.287876 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:24:05.287876 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:24:05.287876 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:24:05.287876 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:24:05.287876 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:24:05.287876 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:24:05.290842 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 193/380] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:24:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:24:05.300493 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86eda4cf-623c-4fe0-9b7a-8dadc44a0d3e None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:05.309152 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd35dc8-fa74-40aa-9a71-ad9db49a82d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:05.310278 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd35dc8-fa74-40aa-9a71-ad9db49a82d1 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:05.310568 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd35dc8-fa74-40aa-9a71-ad9db49a82d1 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:05.310853 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd35dc8-fa74-40aa-9a71-ad9db49a82d1 None admin] PATH_INFO: `/v3/projects/9471b834f91741fbaf409539b1810552` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:05.312125 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd35dc8-fa74-40aa-9a71-ad9db49a82d1 None admin] RBAC: Authorizing `identity:get_project(project_id=9471b834f91741fbaf409539b1810552)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:05.313691 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd35dc8-fa74-40aa-9a71-ad9db49a82d1 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:05.316715 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 188/381] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:24:05 2022] GET /identity/v3/projects/9471b834f91741fbaf409539b1810552 => generated 344 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:24:05.325197 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd35dc8-fa74-40aa-9a71-ad9db49a82d1 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:05.460855 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86eda4cf-623c-4fe0-9b7a-8dadc44a0d3e None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:05.510040 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd35dc8-fa74-40aa-9a71-ad9db49a82d1 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:05.511904 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fc7489-1919-4da2-9a7a-9b15bf0f67b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:05.512566 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fc7489-1919-4da2-9a7a-9b15bf0f67b0 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:05.512897 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fc7489-1919-4da2-9a7a-9b15bf0f67b0 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:05.513380 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fc7489-1919-4da2-9a7a-9b15bf0f67b0 None admin] PATH_INFO: `/v3/roles` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:05.514605 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fc7489-1919-4da2-9a7a-9b15bf0f67b0 None admin] RBAC: Adding query filter params (name=member) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:05.514605 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fc7489-1919-4da2-9a7a-9b15bf0f67b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:05.515930 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fc7489-1919-4da2-9a7a-9b15bf0f67b0 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:05.531770 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:05.533656 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 194/382] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:24:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:05.539012 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fc7489-1919-4da2-9a7a-9b15bf0f67b0 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:05.558613 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038bf1e7-9cbf-4fcd-bb8d-39afed0e3776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:05.560039 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038bf1e7-9cbf-4fcd-bb8d-39afed0e3776 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:05.560558 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038bf1e7-9cbf-4fcd-bb8d-39afed0e3776 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:05.561129 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038bf1e7-9cbf-4fcd-bb8d-39afed0e3776 None admin] PATH_INFO: `/v3/groups/741b95495ffd44ce8e040cc5e23b155e` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:05.569821 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038bf1e7-9cbf-4fcd-bb8d-39afed0e3776 None admin] RBAC: Authorizing `identity:get_group(group_id=741b95495ffd44ce8e040cc5e23b155e)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:05.573746 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038bf1e7-9cbf-4fcd-bb8d-39afed0e3776 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:05.575978 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:05.584664 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 189/383] 10.222.0.57 () {56 vars in 1180 bytes} [Thu May 19 14:24:05 2022] GET /identity/v3/groups/741b95495ffd44ce8e040cc5e23b155e => generated 229 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:24:05.586331 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038bf1e7-9cbf-4fcd-bb8d-39afed0e3776 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:05.754407 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fc7489-1919-4da2-9a7a-9b15bf0f67b0 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:05.765869 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038bf1e7-9cbf-4fcd-bb8d-39afed0e3776 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:05.803926 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9107e7a0-0d41-4615-9259-56f102c15fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:05.804606 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9107e7a0-0d41-4615-9259-56f102c15fef None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:05.804888 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9107e7a0-0d41-4615-9259-56f102c15fef None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:05.805181 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9107e7a0-0d41-4615-9259-56f102c15fef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:05.806086 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9107e7a0-0d41-4615-9259-56f102c15fef None admin] RBAC: Adding query filter params (group.id=741b95495ffd44ce8e040cc5e23b155e, role.id=a3b793754e7b40b0bb21afe74eec3c18, scope.project.id=9471b834f91741fbaf409539b1810552) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:05.806399 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9107e7a0-0d41-4615-9259-56f102c15fef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:05.807858 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9107e7a0-0d41-4615-9259-56f102c15fef None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:05.811723 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7942f178-35f6-4ad8-aa31-999a545641fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:05.812928 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7942f178-35f6-4ad8-aa31-999a545641fa None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:05.813435 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7942f178-35f6-4ad8-aa31-999a545641fa None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:05.813960 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7942f178-35f6-4ad8-aa31-999a545641fa None admin] PATH_INFO: `/v3/groups/6f88b9d7a32742db86b87c6d666173ac` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:05.817752 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7942f178-35f6-4ad8-aa31-999a545641fa None admin] RBAC: Authorizing `identity:get_group(group_id=6f88b9d7a32742db86b87c6d666173ac)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:05.818020 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 190/384] 10.222.0.57 () {56 vars in 1376 bytes} [Thu May 19 14:24:05 2022] GET /identity/v3/role_assignments?group.id=741b95495ffd44ce8e040cc5e23b155e&scope.project.id=9471b834f91741fbaf409539b1810552&role.id=a3b793754e7b40b0bb21afe74eec3c18 => generated 260 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:24:05.819744 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7942f178-35f6-4ad8-aa31-999a545641fa None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:05.821808 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 195/385] 10.222.0.57 () {56 vars in 1180 bytes} [Thu May 19 14:24:05 2022] GET /identity/v3/groups/6f88b9d7a32742db86b87c6d666173ac => generated 226 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:24:05.828305 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9107e7a0-0d41-4615-9259-56f102c15fef None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:05.884116 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:05.946836 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9107e7a0-0d41-4615-9259-56f102c15fef None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:05.986298 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143a86de-4dd0-4bc5-bd21-1cdea177b60a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:05.987233 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143a86de-4dd0-4bc5-bd21-1cdea177b60a None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:05.987519 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143a86de-4dd0-4bc5-bd21-1cdea177b60a None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:05.987798 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143a86de-4dd0-4bc5-bd21-1cdea177b60a None admin] PATH_INFO: `/v3/projects/e7497d51ee8f4f0487b09541d4ad94d2` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:05.989090 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143a86de-4dd0-4bc5-bd21-1cdea177b60a None admin] RBAC: Authorizing `identity:get_project(project_id=e7497d51ee8f4f0487b09541d4ad94d2)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:05.990600 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143a86de-4dd0-4bc5-bd21-1cdea177b60a None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:05.992738 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 191/386] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:24:05 2022] GET /identity/v3/projects/e7497d51ee8f4f0487b09541d4ad94d2 => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:05.998733 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7942f178-35f6-4ad8-aa31-999a545641fa None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:06.110029 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:06.128346 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7942f178-35f6-4ad8-aa31-999a545641fa None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:06.164895 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e892999-7398-498e-8b2d-e243c344bc04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:06.165780 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e892999-7398-498e-8b2d-e243c344bc04 None admin] REQUEST_METHOD: `PUT` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:06.166087 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e892999-7398-498e-8b2d-e243c344bc04 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:06.166360 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e892999-7398-498e-8b2d-e243c344bc04 None admin] PATH_INFO: `/v3/projects/e7497d51ee8f4f0487b09541d4ad94d2/groups/6f88b9d7a32742db86b87c6d666173ac/roles/3d5075293a4f45449eaef6af8fa53004` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:06.168318 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e892999-7398-498e-8b2d-e243c344bc04 None admin] RBAC: Authorizing `identity:create_grant(project_id=e7497d51ee8f4f0487b09541d4ad94d2, group_id=6f88b9d7a32742db86b87c6d666173ac, role_id=3d5075293a4f45449eaef6af8fa53004)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:06.170288 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e892999-7398-498e-8b2d-e243c344bc04 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:06.190141 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 196/387] 10.222.0.57 () {58 vars in 1442 bytes} [Thu May 19 14:24:05 2022] PUT /identity/v3/projects/e7497d51ee8f4f0487b09541d4ad94d2/groups/6f88b9d7a32742db86b87c6d666173ac/roles/3d5075293a4f45449eaef6af8fa53004 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 14:24:06.720039 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42777c43-f941-4fd0-a352-b5da68f23f47 None None] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:06.720495 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42777c43-f941-4fd0-a352-b5da68f23f47 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:06.720884 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42777c43-f941-4fd0-a352-b5da68f23f47 None None] PATH_INFO: `/` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:06.722674 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 192/388] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:24:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:24:06.728370 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653a0218-3688-4c78-b3d5-0bc2468428ee None None] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:06.728778 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653a0218-3688-4c78-b3d5-0bc2468428ee None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:06.729170 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653a0218-3688-4c78-b3d5-0bc2468428ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:06.747263 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.auth.core [None req-653a0218-3688-4c78-b3d5-0bc2468428ee None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143587) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:24:06.748544 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-653a0218-3688-4c78-b3d5-0bc2468428ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:24:06.820973 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 197/389] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:24:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:24:06.844065 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42777c43-f941-4fd0-a352-b5da68f23f47 None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:06.955031 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42777c43-f941-4fd0-a352-b5da68f23f47 None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:06.970232 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:06.991968 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67ba7c5-c635-4e80-9124-f1d1fe4b2284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:06.992788 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67ba7c5-c635-4e80-9124-f1d1fe4b2284 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:06.993075 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67ba7c5-c635-4e80-9124-f1d1fe4b2284 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:06.993356 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67ba7c5-c635-4e80-9124-f1d1fe4b2284 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:07.005873 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67ba7c5-c635-4e80-9124-f1d1fe4b2284 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:24:07.014877 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-f67ba7c5-c635-4e80-9124-f1d1fe4b2284 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:24:07.014877 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:07.014877 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:07.014877 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:07.014877 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:07.014877 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:07.014877 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:07.014877 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:07.014877 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:07.014877 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:07.014877 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:07.014877 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:07.014877 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:24:07.014877 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:24:07.014877 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:24:07.014877 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:24:07.015913 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:24:07.015913 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:24:07.015913 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:24:07.015913 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:24:07.015913 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:07.015913 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:07.015913 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:07.015913 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:07.015913 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:07.015913 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:07.015913 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:07.015913 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:24:07.015913 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:07.015913 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:07.015913 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:07.015913 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:07.016823 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:07.016823 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:07.016823 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:07.016823 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:07.016823 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:24:07.016823 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:24:07.016823 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:24:07.016823 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:24:07.016823 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:24:07.016823 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:24:07.016823 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:24:07.016823 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:24:07.016823 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 193/390] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 19 14:24:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:07.022239 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653a0218-3688-4c78-b3d5-0bc2468428ee None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:07.102995 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:07.128857 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653a0218-3688-4c78-b3d5-0bc2468428ee None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:07.167893 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7024091c-5ad4-430a-bd07-0da8053df95d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:07.168718 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7024091c-5ad4-430a-bd07-0da8053df95d None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:07.168911 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7024091c-5ad4-430a-bd07-0da8053df95d None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:07.169706 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7024091c-5ad4-430a-bd07-0da8053df95d None admin] PATH_INFO: `/v3/roles` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:07.170679 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7024091c-5ad4-430a-bd07-0da8053df95d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:07.170980 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7024091c-5ad4-430a-bd07-0da8053df95d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:07.172380 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7024091c-5ad4-430a-bd07-0da8053df95d None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:07.181516 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 198/391] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:24:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:07.188123 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67ba7c5-c635-4e80-9124-f1d1fe4b2284 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:07.300472 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67ba7c5-c635-4e80-9124-f1d1fe4b2284 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:07.336162 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa0ffa9-3b86-454a-85d4-fc44a5729821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:07.337027 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa0ffa9-3b86-454a-85d4-fc44a5729821 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:07.337305 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa0ffa9-3b86-454a-85d4-fc44a5729821 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:07.337577 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa0ffa9-3b86-454a-85d4-fc44a5729821 None admin] PATH_INFO: `/v3/groups/6f88b9d7a32742db86b87c6d666173ac` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:07.338902 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa0ffa9-3b86-454a-85d4-fc44a5729821 None admin] RBAC: Authorizing `identity:get_group(group_id=6f88b9d7a32742db86b87c6d666173ac)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:07.340353 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa0ffa9-3b86-454a-85d4-fc44a5729821 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:07.343291 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 194/392] 10.222.0.57 () {56 vars in 1180 bytes} [Thu May 19 14:24:07 2022] GET /identity/v3/groups/6f88b9d7a32742db86b87c6d666173ac => generated 226 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:07.351172 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7024091c-5ad4-430a-bd07-0da8053df95d None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:07.463629 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7024091c-5ad4-430a-bd07-0da8053df95d None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:07.503104 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de808a4f-eb48-4d07-b95d-fa362def9061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:07.503980 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de808a4f-eb48-4d07-b95d-fa362def9061 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:07.504262 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de808a4f-eb48-4d07-b95d-fa362def9061 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:07.504536 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de808a4f-eb48-4d07-b95d-fa362def9061 None admin] PATH_INFO: `/v3/projects/e7497d51ee8f4f0487b09541d4ad94d2` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:07.505736 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de808a4f-eb48-4d07-b95d-fa362def9061 None admin] RBAC: Authorizing `identity:get_project(project_id=e7497d51ee8f4f0487b09541d4ad94d2)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:07.507215 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de808a4f-eb48-4d07-b95d-fa362def9061 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:07.508916 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 199/393] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:24:07 2022] GET /identity/v3/projects/e7497d51ee8f4f0487b09541d4ad94d2 => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:07.516316 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa0ffa9-3b86-454a-85d4-fc44a5729821 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:07.566298 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:07.590667 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aa7afa-d415-4185-a438-c51d6beef194 None None] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:07.591506 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aa7afa-d415-4185-a438-c51d6beef194 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:07.592256 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aa7afa-d415-4185-a438-c51d6beef194 None None] PATH_INFO: `/` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:07.593396 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 200/394] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:24:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:24:07.600428 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a25e49-aa85-4b5a-950c-a571057f88a0 None None] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:07.600828 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a25e49-aa85-4b5a-950c-a571057f88a0 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:07.601207 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a25e49-aa85-4b5a-950c-a571057f88a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:07.623043 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.auth.core [None req-84a25e49-aa85-4b5a-950c-a571057f88a0 None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143587) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:24:07.624240 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-84a25e49-aa85-4b5a-950c-a571057f88a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:24:07.646236 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:07.699379 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 201/395] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:24:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:24:07.716750 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa0ffa9-3b86-454a-85d4-fc44a5729821 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:07.735672 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a25e49-aa85-4b5a-950c-a571057f88a0 None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:07.819181 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1779f06c-4835-40e3-81a5-d08cacc03b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:07.820001 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1779f06c-4835-40e3-81a5-d08cacc03b9c None admin] REQUEST_METHOD: `PUT` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:07.820285 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1779f06c-4835-40e3-81a5-d08cacc03b9c None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:07.820564 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1779f06c-4835-40e3-81a5-d08cacc03b9c None admin] PATH_INFO: `/v3/projects/e7497d51ee8f4f0487b09541d4ad94d2/groups/6f88b9d7a32742db86b87c6d666173ac/roles/b954b2c7ce764aa0afac78e075a011a6` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:07.823031 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1779f06c-4835-40e3-81a5-d08cacc03b9c None admin] RBAC: Authorizing `identity:create_grant(project_id=e7497d51ee8f4f0487b09541d4ad94d2, group_id=6f88b9d7a32742db86b87c6d666173ac, role_id=b954b2c7ce764aa0afac78e075a011a6)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:07.824477 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1779f06c-4835-40e3-81a5-d08cacc03b9c None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:07.842672 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a25e49-aa85-4b5a-950c-a571057f88a0 None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:07.873059 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 195/396] 10.222.0.57 () {58 vars in 1442 bytes} [Thu May 19 14:24:07 2022] PUT /identity/v3/projects/e7497d51ee8f4f0487b09541d4ad94d2/groups/6f88b9d7a32742db86b87c6d666173ac/roles/b954b2c7ce764aa0afac78e075a011a6 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:24:07.881367 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71765e2-1554-4363-8f8f-872ef6611ecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:07.882337 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71765e2-1554-4363-8f8f-872ef6611ecd None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:07.883664 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71765e2-1554-4363-8f8f-872ef6611ecd None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:07.883965 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71765e2-1554-4363-8f8f-872ef6611ecd None admin] PATH_INFO: `/v3/roles/member` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:07.895897 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:07.902559 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71765e2-1554-4363-8f8f-872ef6611ecd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:24:07.911584 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-b71765e2-1554-4363-8f8f-872ef6611ecd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:24:07.911584 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:07.911584 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:07.911584 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:07.911584 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:07.911584 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:07.911584 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:07.911584 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:07.911584 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:07.911584 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:07.911584 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:07.911584 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:07.911584 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:24:07.911584 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:24:07.911584 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:24:07.911584 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:24:07.912987 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:24:07.912987 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:24:07.912987 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:24:07.912987 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:24:07.912987 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:07.912987 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:07.912987 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:07.912987 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:07.912987 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:07.912987 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:07.912987 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:07.912987 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:24:07.912987 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:07.912987 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:07.912987 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:07.912987 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:07.914092 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:07.914092 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:07.914092 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:07.914092 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:07.914092 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:24:07.914092 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:24:07.914092 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:24:07.914092 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:24:07.914092 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:24:07.914092 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:24:07.914092 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:24:07.914092 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:24:07.914092 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 202/397] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:24:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:24:07.919408 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1779f06c-4835-40e3-81a5-d08cacc03b9c None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:07.956432 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc0e7a8-c8a6-4ea0-87c0-b33e1e2fceb9 None None] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:07.956786 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc0e7a8-c8a6-4ea0-87c0-b33e1e2fceb9 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:07.957028 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc0e7a8-c8a6-4ea0-87c0-b33e1e2fceb9 None None] PATH_INFO: `/` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:07.958930 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 203/398] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:24:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:24:07.967867 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffef5ca-d621-4851-8e63-9738933b155f None None] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:07.968204 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffef5ca-d621-4851-8e63-9738933b155f None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:07.968433 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffef5ca-d621-4851-8e63-9738933b155f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:08.011093 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.auth.core [None req-5ffef5ca-d621-4851-8e63-9738933b155f None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143587) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:24:08.012319 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-5ffef5ca-d621-4851-8e63-9738933b155f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:24:08.039493 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:08.043685 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1779f06c-4835-40e3-81a5-d08cacc03b9c None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:08.087787 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72d39ca-f372-43e7-96f5-6ff7cb2d5e9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:08.088383 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72d39ca-f372-43e7-96f5-6ff7cb2d5e9c None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:08.088669 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72d39ca-f372-43e7-96f5-6ff7cb2d5e9c None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:08.088946 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72d39ca-f372-43e7-96f5-6ff7cb2d5e9c None admin] PATH_INFO: `/v3/roles` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:08.089709 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72d39ca-f372-43e7-96f5-6ff7cb2d5e9c None admin] RBAC: Adding query filter params (name=member) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:08.090051 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72d39ca-f372-43e7-96f5-6ff7cb2d5e9c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:08.091967 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72d39ca-f372-43e7-96f5-6ff7cb2d5e9c None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:08.102624 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 196/399] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:24:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:08.109302 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72d39ca-f372-43e7-96f5-6ff7cb2d5e9c None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:08.209158 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 204/400] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:24:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:24:08.227813 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72d39ca-f372-43e7-96f5-6ff7cb2d5e9c None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:08.233886 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffef5ca-d621-4851-8e63-9738933b155f None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:08.274481 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd442ea-8f69-448d-988a-d19b66d51811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:08.275348 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd442ea-8f69-448d-988a-d19b66d51811 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:08.275633 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd442ea-8f69-448d-988a-d19b66d51811 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:08.275912 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd442ea-8f69-448d-988a-d19b66d51811 None admin] PATH_INFO: `/v3/groups/6f88b9d7a32742db86b87c6d666173ac` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:08.277098 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd442ea-8f69-448d-988a-d19b66d51811 None admin] RBAC: Authorizing `identity:get_group(group_id=6f88b9d7a32742db86b87c6d666173ac)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:08.280356 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd442ea-8f69-448d-988a-d19b66d51811 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:08.283309 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 197/401] 10.222.0.57 () {56 vars in 1180 bytes} [Thu May 19 14:24:08 2022] GET /identity/v3/groups/6f88b9d7a32742db86b87c6d666173ac => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:24:08.288902 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd442ea-8f69-448d-988a-d19b66d51811 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:08.329773 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:08.375539 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffef5ca-d621-4851-8e63-9738933b155f None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:08.407284 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:08.417765 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc644f05-8006-442b-b6c8-dfdb063e2c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:08.418922 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc644f05-8006-442b-b6c8-dfdb063e2c8d None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:08.419765 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc644f05-8006-442b-b6c8-dfdb063e2c8d None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:08.420963 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc644f05-8006-442b-b6c8-dfdb063e2c8d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:08.434269 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc644f05-8006-442b-b6c8-dfdb063e2c8d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:24:08.444236 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-bc644f05-8006-442b-b6c8-dfdb063e2c8d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:24:08.444236 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:08.444236 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:08.444236 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:08.444236 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:08.444236 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:08.444236 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:08.444236 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:08.444236 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:08.444236 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:08.444236 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:08.444236 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:08.444236 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:24:08.444236 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:24:08.444236 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:24:08.444236 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:24:08.445287 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:24:08.445287 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:24:08.445287 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:24:08.445287 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:24:08.445287 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:08.445287 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:08.445287 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:08.445287 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:08.445287 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:08.445287 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:08.445287 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:08.445287 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:24:08.445287 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:08.445287 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:08.445287 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:08.445287 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:08.446247 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:08.446247 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:08.446247 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:08.446247 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:08.446247 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:24:08.446247 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:24:08.446247 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:24:08.446247 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:24:08.446247 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:24:08.446247 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:24:08.446247 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:24:08.446247 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:24:08.449437 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 205/402] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 19 14:24:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:24:08.456639 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc644f05-8006-442b-b6c8-dfdb063e2c8d None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:08.486901 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd442ea-8f69-448d-988a-d19b66d51811 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:08.572177 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35f9e7c-9554-434d-ab5c-7f94b8a9dd8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:08.573054 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35f9e7c-9554-434d-ab5c-7f94b8a9dd8d None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:08.573336 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35f9e7c-9554-434d-ab5c-7f94b8a9dd8d None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:08.573614 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35f9e7c-9554-434d-ab5c-7f94b8a9dd8d None admin] PATH_INFO: `/v3/projects/24fafd1589784882b754b5f8543954bb` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:08.578605 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35f9e7c-9554-434d-ab5c-7f94b8a9dd8d None admin] RBAC: Authorizing `identity:get_project(project_id=24fafd1589784882b754b5f8543954bb)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:08.580130 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35f9e7c-9554-434d-ab5c-7f94b8a9dd8d None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:08.582109 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc644f05-8006-442b-b6c8-dfdb063e2c8d None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:08.582236 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 198/403] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:24:08 2022] GET /identity/v3/projects/24fafd1589784882b754b5f8543954bb => generated 302 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:24:08.587902 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35f9e7c-9554-434d-ab5c-7f94b8a9dd8d None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:08.622637 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897577aa-6aa2-4f1f-bc06-08569218decc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:08.623760 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897577aa-6aa2-4f1f-bc06-08569218decc None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:08.624275 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897577aa-6aa2-4f1f-bc06-08569218decc None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:08.624760 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897577aa-6aa2-4f1f-bc06-08569218decc None admin] PATH_INFO: `/v3/roles` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:08.626277 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897577aa-6aa2-4f1f-bc06-08569218decc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:08.626925 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897577aa-6aa2-4f1f-bc06-08569218decc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:08.628678 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897577aa-6aa2-4f1f-bc06-08569218decc None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:08.639850 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 206/404] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:24:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:24:08.646257 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897577aa-6aa2-4f1f-bc06-08569218decc None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:08.710114 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:08.734958 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:08.738841 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35f9e7c-9554-434d-ab5c-7f94b8a9dd8d None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:08.775404 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f258a4-dcb9-43e6-ba3a-f6b7a67373e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:08.776330 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f258a4-dcb9-43e6-ba3a-f6b7a67373e3 None admin] REQUEST_METHOD: `PUT` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:08.776617 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f258a4-dcb9-43e6-ba3a-f6b7a67373e3 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:08.776890 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f258a4-dcb9-43e6-ba3a-f6b7a67373e3 None admin] PATH_INFO: `/v3/projects/24fafd1589784882b754b5f8543954bb/groups/6f88b9d7a32742db86b87c6d666173ac/roles/3d5075293a4f45449eaef6af8fa53004` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:08.777014 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897577aa-6aa2-4f1f-bc06-08569218decc None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:08.780012 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f258a4-dcb9-43e6-ba3a-f6b7a67373e3 None admin] RBAC: Authorizing `identity:create_grant(project_id=24fafd1589784882b754b5f8543954bb, group_id=6f88b9d7a32742db86b87c6d666173ac, role_id=3d5075293a4f45449eaef6af8fa53004)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:08.781520 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f258a4-dcb9-43e6-ba3a-f6b7a67373e3 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:08.800606 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 199/405] 10.222.0.57 () {58 vars in 1442 bytes} [Thu May 19 14:24:08 2022] PUT /identity/v3/projects/24fafd1589784882b754b5f8543954bb/groups/6f88b9d7a32742db86b87c6d666173ac/roles/3d5075293a4f45449eaef6af8fa53004 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 14:24:08.801942 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10987cf-55e2-4b70-b5b6-e773d055e35f None None] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:08.802232 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10987cf-55e2-4b70-b5b6-e773d055e35f None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:08.802514 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10987cf-55e2-4b70-b5b6-e773d055e35f None None] PATH_INFO: `/` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:08.803963 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 200/406] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:24:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:24:08.810565 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45273fed-b4b6-420f-bf0c-9c7657a23d6b None None] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:08.811501 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45273fed-b4b6-420f-bf0c-9c7657a23d6b None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:08.812043 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45273fed-b4b6-420f-bf0c-9c7657a23d6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:08.840927 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.auth.core [None req-45273fed-b4b6-420f-bf0c-9c7657a23d6b None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143586) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:24:08.844222 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.fernet_utils [None req-45273fed-b4b6-420f-bf0c-9c7657a23d6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143586) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:24:08.860208 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817ea737-1b0e-43f7-aa51-41866cf459ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:08.861482 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817ea737-1b0e-43f7-aa51-41866cf459ce None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:08.862035 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817ea737-1b0e-43f7-aa51-41866cf459ce None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:08.862571 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817ea737-1b0e-43f7-aa51-41866cf459ce None admin] PATH_INFO: `/v3/groups/6f88b9d7a32742db86b87c6d666173ac` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:08.866205 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817ea737-1b0e-43f7-aa51-41866cf459ce None admin] RBAC: Authorizing `identity:get_group(group_id=6f88b9d7a32742db86b87c6d666173ac)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:08.868125 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817ea737-1b0e-43f7-aa51-41866cf459ce None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:08.870244 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 207/407] 10.222.0.57 () {56 vars in 1180 bytes} [Thu May 19 14:24:08 2022] GET /identity/v3/groups/6f88b9d7a32742db86b87c6d666173ac => generated 226 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:24:08.877827 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817ea737-1b0e-43f7-aa51-41866cf459ce None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:08.997737 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 201/408] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:24:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:24:09.034920 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817ea737-1b0e-43f7-aa51-41866cf459ce None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:09.040856 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45273fed-b4b6-420f-bf0c-9c7657a23d6b None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:09.057264 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:09.078466 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d54c62-4dd9-4afd-8989-cf22086e370f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:09.078839 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d54c62-4dd9-4afd-8989-cf22086e370f None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:09.079018 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d54c62-4dd9-4afd-8989-cf22086e370f None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:09.079279 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d54c62-4dd9-4afd-8989-cf22086e370f None admin] PATH_INFO: `/v3/projects/24fafd1589784882b754b5f8543954bb` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:09.081000 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d54c62-4dd9-4afd-8989-cf22086e370f None admin] RBAC: Authorizing `identity:get_project(project_id=24fafd1589784882b754b5f8543954bb)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:09.082132 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d54c62-4dd9-4afd-8989-cf22086e370f None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:09.084241 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 208/409] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:24:08 2022] GET /identity/v3/projects/24fafd1589784882b754b5f8543954bb => generated 302 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:24:09.090430 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d54c62-4dd9-4afd-8989-cf22086e370f None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:09.149174 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:09.175872 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45273fed-b4b6-420f-bf0c-9c7657a23d6b None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:09.236285 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e521967e-241a-41e1-9f6d-7ec8957a5e8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:09.237482 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e521967e-241a-41e1-9f6d-7ec8957a5e8c None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:09.238028 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e521967e-241a-41e1-9f6d-7ec8957a5e8c None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:09.238522 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e521967e-241a-41e1-9f6d-7ec8957a5e8c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:09.243773 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d54c62-4dd9-4afd-8989-cf22086e370f None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:09.250401 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e521967e-241a-41e1-9f6d-7ec8957a5e8c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:24:09.258677 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-e521967e-241a-41e1-9f6d-7ec8957a5e8c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:24:09.258677 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:09.258677 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:09.258677 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:09.258677 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:09.258677 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:09.258677 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:09.258677 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:09.258677 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:09.258677 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:09.258677 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:09.258677 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:09.258677 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:24:09.258677 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:24:09.258677 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:24:09.258677 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:24:09.259685 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:24:09.259685 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:24:09.259685 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:24:09.259685 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:24:09.259685 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:09.259685 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:09.259685 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:09.259685 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:09.259685 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:09.259685 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:09.259685 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:09.259685 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:24:09.259685 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:09.259685 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:09.259685 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:09.259685 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:09.260695 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:09.260695 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:09.260695 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:09.260695 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:09.260695 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:24:09.260695 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:24:09.260695 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:24:09.260695 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:24:09.260695 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:24:09.260695 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:24:09.260695 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:24:09.260695 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:24:09.263494 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 202/410] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:24:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:24:09.269915 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7078d6-d9a7-4ff6-bb2f-8c445b0dea78 None None] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:09.270475 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7078d6-d9a7-4ff6-bb2f-8c445b0dea78 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:09.270998 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7078d6-d9a7-4ff6-bb2f-8c445b0dea78 None None] PATH_INFO: `/` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:09.272532 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 203/411] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:24:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:24:09.273749 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7078d6-d9a7-4ff6-bb2f-8c445b0dea78 None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:09.290333 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2856aeef-880e-4e6e-9401-eb1677fc755c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:09.291795 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2856aeef-880e-4e6e-9401-eb1677fc755c None admin] REQUEST_METHOD: `PUT` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:09.292626 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2856aeef-880e-4e6e-9401-eb1677fc755c None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:09.293081 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2856aeef-880e-4e6e-9401-eb1677fc755c None admin] PATH_INFO: `/v3/projects/24fafd1589784882b754b5f8543954bb/groups/6f88b9d7a32742db86b87c6d666173ac/roles/b954b2c7ce764aa0afac78e075a011a6` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:09.297572 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2856aeef-880e-4e6e-9401-eb1677fc755c None admin] RBAC: Authorizing `identity:create_grant(project_id=24fafd1589784882b754b5f8543954bb, group_id=6f88b9d7a32742db86b87c6d666173ac, role_id=b954b2c7ce764aa0afac78e075a011a6)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:09.299337 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2856aeef-880e-4e6e-9401-eb1677fc755c None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:09.317874 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 209/412] 10.222.0.57 () {58 vars in 1442 bytes} [Thu May 19 14:24:09 2022] PUT /identity/v3/projects/24fafd1589784882b754b5f8543954bb/groups/6f88b9d7a32742db86b87c6d666173ac/roles/b954b2c7ce764aa0afac78e075a011a6 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:24:09.320706 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22235bf6-7112-4875-a301-600b00e0fecf None None] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:09.321227 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22235bf6-7112-4875-a301-600b00e0fecf None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:09.321780 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22235bf6-7112-4875-a301-600b00e0fecf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:09.346369 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.auth.core [None req-22235bf6-7112-4875-a301-600b00e0fecf None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143587) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:24:09.348183 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-22235bf6-7112-4875-a301-600b00e0fecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:24:09.407595 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:09.410685 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7078d6-d9a7-4ff6-bb2f-8c445b0dea78 None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:09.453707 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d226001-6d7d-4ffd-9490-2cf9a2015397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:09.454802 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d226001-6d7d-4ffd-9490-2cf9a2015397 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:09.455005 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 210/413] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:24:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:24:09.455152 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d226001-6d7d-4ffd-9490-2cf9a2015397 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:09.455492 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d226001-6d7d-4ffd-9490-2cf9a2015397 None admin] PATH_INFO: `/v3/roles` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:09.465919 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d226001-6d7d-4ffd-9490-2cf9a2015397 None admin] RBAC: Adding query filter params (name=admin) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:09.465919 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d226001-6d7d-4ffd-9490-2cf9a2015397 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:09.465919 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d226001-6d7d-4ffd-9490-2cf9a2015397 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:09.484776 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 204/414] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:24:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:24:09.491070 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22235bf6-7112-4875-a301-600b00e0fecf None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:09.494859 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d226001-6d7d-4ffd-9490-2cf9a2015397 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:09.523982 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:09.635842 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22235bf6-7112-4875-a301-600b00e0fecf None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:09.690187 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0dfa8e-6029-4fd5-94e1-9fbaae5379f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:09.691536 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0dfa8e-6029-4fd5-94e1-9fbaae5379f6 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:09.692032 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0dfa8e-6029-4fd5-94e1-9fbaae5379f6 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:09.692520 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0dfa8e-6029-4fd5-94e1-9fbaae5379f6 None admin] PATH_INFO: `/v3/groups/741b95495ffd44ce8e040cc5e23b155e` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:09.694052 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0dfa8e-6029-4fd5-94e1-9fbaae5379f6 None admin] RBAC: Authorizing `identity:get_group(group_id=741b95495ffd44ce8e040cc5e23b155e)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:09.695759 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0dfa8e-6029-4fd5-94e1-9fbaae5379f6 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:09.697976 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 211/415] 10.222.0.57 () {56 vars in 1180 bytes} [Thu May 19 14:24:09 2022] GET /identity/v3/groups/741b95495ffd44ce8e040cc5e23b155e => generated 229 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:09.702785 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d226001-6d7d-4ffd-9490-2cf9a2015397 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:09.706890 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0dfa8e-6029-4fd5-94e1-9fbaae5379f6 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:09.743976 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe596d5-9480-49c4-8040-8469f0106edf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:09.745206 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe596d5-9480-49c4-8040-8469f0106edf None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:09.745718 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe596d5-9480-49c4-8040-8469f0106edf None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:09.746314 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe596d5-9480-49c4-8040-8469f0106edf None admin] PATH_INFO: `/v3/roles/member` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:09.758359 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe596d5-9480-49c4-8040-8469f0106edf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:24:09.767142 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:09.768237 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-ffe596d5-9480-49c4-8040-8469f0106edf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:24:09.768237 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:09.768237 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:09.768237 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:09.768237 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:09.768237 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:09.768237 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:09.768237 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:09.768237 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:09.768237 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:09.768237 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:09.768237 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:09.768237 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:24:09.768237 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:24:09.768237 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:24:09.768237 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:24:09.769257 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:24:09.769257 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:24:09.769257 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:24:09.769257 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:24:09.769257 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:09.769257 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:09.769257 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:09.769257 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:09.769257 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:09.769257 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:09.769257 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:09.769257 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:24:09.769257 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:09.769257 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:09.769257 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:09.769257 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:09.775502 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:09.775502 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:09.775502 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:09.775502 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:09.775502 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:24:09.775502 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:24:09.775502 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:24:09.775502 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:24:09.775502 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:24:09.775502 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:24:09.775502 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:24:09.775502 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:24:09.775502 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 205/416] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:24:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:09.780375 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe596d5-9480-49c4-8040-8469f0106edf None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:09.846604 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0dfa8e-6029-4fd5-94e1-9fbaae5379f6 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:09.857140 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:09.894485 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494c9a15-9744-487c-a622-1c0396dc51bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:09.895347 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494c9a15-9744-487c-a622-1c0396dc51bc None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:09.895634 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494c9a15-9744-487c-a622-1c0396dc51bc None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:09.895912 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494c9a15-9744-487c-a622-1c0396dc51bc None admin] PATH_INFO: `/v3/projects/9471b834f91741fbaf409539b1810552` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:09.899348 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494c9a15-9744-487c-a622-1c0396dc51bc None admin] RBAC: Authorizing `identity:get_project(project_id=9471b834f91741fbaf409539b1810552)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:09.900808 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494c9a15-9744-487c-a622-1c0396dc51bc None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:09.912582 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 212/417] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:24:09 2022] GET /identity/v3/projects/9471b834f91741fbaf409539b1810552 => generated 344 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:09.924758 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe596d5-9480-49c4-8040-8469f0106edf None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:09.927568 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494c9a15-9744-487c-a622-1c0396dc51bc None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:10.019409 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97d847f-82a0-4e58-8fe5-d7d898c74add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:10.020082 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97d847f-82a0-4e58-8fe5-d7d898c74add None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:10.020365 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97d847f-82a0-4e58-8fe5-d7d898c74add None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:10.020640 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97d847f-82a0-4e58-8fe5-d7d898c74add None admin] PATH_INFO: `/v3/roles` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:10.021390 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97d847f-82a0-4e58-8fe5-d7d898c74add None admin] RBAC: Adding query filter params (name=member) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:10.021694 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97d847f-82a0-4e58-8fe5-d7d898c74add None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:10.023081 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97d847f-82a0-4e58-8fe5-d7d898c74add None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:10.041255 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 206/418] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:24:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:24:10.047886 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97d847f-82a0-4e58-8fe5-d7d898c74add None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:10.061833 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494c9a15-9744-487c-a622-1c0396dc51bc None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:10.074520 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:10.122787 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d16000b-01a7-4ec7-ac03-ce0d05c10482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:10.123497 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d16000b-01a7-4ec7-ac03-ce0d05c10482 None admin] REQUEST_METHOD: `PUT` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:10.123779 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d16000b-01a7-4ec7-ac03-ce0d05c10482 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:10.124055 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d16000b-01a7-4ec7-ac03-ce0d05c10482 None admin] PATH_INFO: `/v3/projects/9471b834f91741fbaf409539b1810552/groups/741b95495ffd44ce8e040cc5e23b155e/roles/a3b793754e7b40b0bb21afe74eec3c18` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:10.131015 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d16000b-01a7-4ec7-ac03-ce0d05c10482 None admin] RBAC: Authorizing `identity:create_grant(project_id=9471b834f91741fbaf409539b1810552, group_id=741b95495ffd44ce8e040cc5e23b155e, role_id=a3b793754e7b40b0bb21afe74eec3c18)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:10.132497 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d16000b-01a7-4ec7-ac03-ce0d05c10482 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:10.151383 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 213/419] 10.222.0.57 () {58 vars in 1442 bytes} [Thu May 19 14:24:09 2022] PUT /identity/v3/projects/9471b834f91741fbaf409539b1810552/groups/741b95495ffd44ce8e040cc5e23b155e/roles/a3b793754e7b40b0bb21afe74eec3c18 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 14:24:10.224524 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97d847f-82a0-4e58-8fe5-d7d898c74add None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:10.257197 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:10.265960 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ee867e-cd5f-4082-8cd8-9e2e7237a877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:10.266930 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ee867e-cd5f-4082-8cd8-9e2e7237a877 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:10.266930 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ee867e-cd5f-4082-8cd8-9e2e7237a877 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:10.267173 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ee867e-cd5f-4082-8cd8-9e2e7237a877 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:10.370730 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ee867e-cd5f-4082-8cd8-9e2e7237a877 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:10.372275 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ee867e-cd5f-4082-8cd8-9e2e7237a877 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:10.498261 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 207/420] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:24:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 14:24:10.503845 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d16000b-01a7-4ec7-ac03-ce0d05c10482 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:10.673634 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d16000b-01a7-4ec7-ac03-ce0d05c10482 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:10.716714 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e09e57b-445d-4a64-b00f-bfd6417d15c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:10.718232 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e09e57b-445d-4a64-b00f-bfd6417d15c5 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:10.718779 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e09e57b-445d-4a64-b00f-bfd6417d15c5 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:10.719424 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e09e57b-445d-4a64-b00f-bfd6417d15c5 None admin] PATH_INFO: `/v3/projects/e7497d51ee8f4f0487b09541d4ad94d2` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:10.722734 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e09e57b-445d-4a64-b00f-bfd6417d15c5 None admin] RBAC: Authorizing `identity:get_project(project_id=e7497d51ee8f4f0487b09541d4ad94d2)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:10.724222 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e09e57b-445d-4a64-b00f-bfd6417d15c5 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:10.726659 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 214/421] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:24:10 2022] GET /identity/v3/projects/e7497d51ee8f4f0487b09541d4ad94d2 => generated 298 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:10.732651 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ee867e-cd5f-4082-8cd8-9e2e7237a877 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:10.804746 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:10.913601 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ee867e-cd5f-4082-8cd8-9e2e7237a877 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:10.921664 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fb50b1-aac5-4ad2-a08a-74c9e03666c8 None None] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:10.921991 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fb50b1-aac5-4ad2-a08a-74c9e03666c8 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:10.922304 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fb50b1-aac5-4ad2-a08a-74c9e03666c8 None None] PATH_INFO: `/` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:10.923989 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 215/422] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:24:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:24:10.929225 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d044e009-1606-4668-a605-2dde9996c5a7 None None] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:10.929520 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d044e009-1606-4668-a605-2dde9996c5a7 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:10.929801 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d044e009-1606-4668-a605-2dde9996c5a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:10.958993 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.auth.core [None req-d044e009-1606-4668-a605-2dde9996c5a7 None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143587) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:24:10.961455 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-d044e009-1606-4668-a605-2dde9996c5a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:24:10.984326 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4d6019-7fdb-479c-b6f6-817e61a096cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:10.985506 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4d6019-7fdb-479c-b6f6-817e61a096cc None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:10.985998 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4d6019-7fdb-479c-b6f6-817e61a096cc None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:10.986485 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4d6019-7fdb-479c-b6f6-817e61a096cc None admin] PATH_INFO: `/v3/groups/6f88b9d7a32742db86b87c6d666173ac` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:10.987889 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4d6019-7fdb-479c-b6f6-817e61a096cc None admin] RBAC: Authorizing `identity:get_group(group_id=6f88b9d7a32742db86b87c6d666173ac)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:10.990333 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4d6019-7fdb-479c-b6f6-817e61a096cc None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:10.995021 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 208/423] 10.222.0.57 () {56 vars in 1180 bytes} [Thu May 19 14:24:10 2022] GET /identity/v3/groups/6f88b9d7a32742db86b87c6d666173ac => generated 226 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:10.999943 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4d6019-7fdb-479c-b6f6-817e61a096cc None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:11.028099 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:11.096617 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 216/424] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:24:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:24:11.131572 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d044e009-1606-4668-a605-2dde9996c5a7 None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:11.191787 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:11.196234 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4d6019-7fdb-479c-b6f6-817e61a096cc None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:11.246507 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5539b2fb-7bb5-49b6-bd3b-96226a9bea7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:11.247450 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5539b2fb-7bb5-49b6-bd3b-96226a9bea7b None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:11.247825 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5539b2fb-7bb5-49b6-bd3b-96226a9bea7b None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:11.248067 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5539b2fb-7bb5-49b6-bd3b-96226a9bea7b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:11.248924 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5539b2fb-7bb5-49b6-bd3b-96226a9bea7b None admin] RBAC: Adding query filter params (group.id=6f88b9d7a32742db86b87c6d666173ac, role.id=3d5075293a4f45449eaef6af8fa53004, scope.project.id=e7497d51ee8f4f0487b09541d4ad94d2) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:11.249298 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5539b2fb-7bb5-49b6-bd3b-96226a9bea7b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:11.251223 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5539b2fb-7bb5-49b6-bd3b-96226a9bea7b None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:11.258245 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 209/425] 10.222.0.57 () {56 vars in 1376 bytes} [Thu May 19 14:24:10 2022] GET /identity/v3/role_assignments?group.id=6f88b9d7a32742db86b87c6d666173ac&scope.project.id=e7497d51ee8f4f0487b09541d4ad94d2&role.id=3d5075293a4f45449eaef6af8fa53004 => generated 611 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:24:11.305422 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d044e009-1606-4668-a605-2dde9996c5a7 None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:11.346228 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac469e1-be4f-459f-ab8b-2e6c5ed6aa7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:11.347115 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac469e1-be4f-459f-ab8b-2e6c5ed6aa7f None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:11.347334 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac469e1-be4f-459f-ab8b-2e6c5ed6aa7f None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:11.347614 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac469e1-be4f-459f-ab8b-2e6c5ed6aa7f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:11.361037 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac469e1-be4f-459f-ab8b-2e6c5ed6aa7f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:24:11.369685 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-cac469e1-be4f-459f-ab8b-2e6c5ed6aa7f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:24:11.369685 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:11.369685 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:11.369685 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:11.369685 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:11.369685 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:11.369685 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:11.369685 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:11.369685 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:11.369685 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:11.369685 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:11.369685 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:11.369685 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:24:11.369685 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:24:11.369685 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:24:11.369685 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:24:11.370822 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:24:11.370822 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:24:11.370822 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:24:11.370822 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:24:11.370822 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:11.370822 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:11.370822 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:11.370822 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:11.370822 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:11.370822 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:11.370822 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:11.370822 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:24:11.370822 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:11.370822 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:11.370822 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:11.370822 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:11.371762 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:11.371762 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:11.371762 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:11.371762 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:11.371762 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:24:11.371762 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:24:11.371762 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:24:11.371762 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:24:11.371762 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:24:11.371762 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:24:11.371762 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:24:11.371762 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:24:11.371762 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 217/426] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 19 14:24:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:24:11.377208 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5539b2fb-7bb5-49b6-bd3b-96226a9bea7b None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:11.527687 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5539b2fb-7bb5-49b6-bd3b-96226a9bea7b None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:11.571241 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5b948a-3f2d-4a2f-979a-f289d25a7a61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:11.571919 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5b948a-3f2d-4a2f-979a-f289d25a7a61 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:11.572198 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5b948a-3f2d-4a2f-979a-f289d25a7a61 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:11.572468 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5b948a-3f2d-4a2f-979a-f289d25a7a61 None admin] PATH_INFO: `/v3/roles` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:11.573225 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5b948a-3f2d-4a2f-979a-f289d25a7a61 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:11.573526 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5b948a-3f2d-4a2f-979a-f289d25a7a61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:11.574923 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5b948a-3f2d-4a2f-979a-f289d25a7a61 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:11.584742 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 210/427] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:24:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:11.590085 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac469e1-be4f-459f-ab8b-2e6c5ed6aa7f None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:11.642334 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:11.722949 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae38ed05-a6e4-4de0-9feb-f76ca4d1dcca None None] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:11.723249 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae38ed05-a6e4-4de0-9feb-f76ca4d1dcca None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:11.723508 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae38ed05-a6e4-4de0-9feb-f76ca4d1dcca None None] PATH_INFO: `/` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:11.725282 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 211/428] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:24:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:24:11.730585 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701e1463-82cd-42d2-8552-0ed0893f6b37 None None] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:11.731110 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701e1463-82cd-42d2-8552-0ed0893f6b37 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:11.732259 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701e1463-82cd-42d2-8552-0ed0893f6b37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:11.750191 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac469e1-be4f-459f-ab8b-2e6c5ed6aa7f None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:11.758140 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.auth.core [None req-701e1463-82cd-42d2-8552-0ed0893f6b37 None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143586) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:24:11.759851 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.fernet_utils [None req-701e1463-82cd-42d2-8552-0ed0893f6b37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143586) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:24:11.785784 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6244498-5642-4690-bff3-11c5068c4198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:11.786455 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6244498-5642-4690-bff3-11c5068c4198 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:11.786737 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6244498-5642-4690-bff3-11c5068c4198 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:11.787039 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6244498-5642-4690-bff3-11c5068c4198 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:11.790202 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:11.859440 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 212/429] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:24:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:24:11.883853 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701e1463-82cd-42d2-8552-0ed0893f6b37 None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:11.884611 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6244498-5642-4690-bff3-11c5068c4198 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:11.886666 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6244498-5642-4690-bff3-11c5068c4198 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:11.926457 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:12.007165 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701e1463-82cd-42d2-8552-0ed0893f6b37 None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:12.033874 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 218/430] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:24:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:24:12.036010 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6244498-5642-4690-bff3-11c5068c4198 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:12.057131 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca9558e-d5c0-469a-bd57-f4507ce58589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:12.058111 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca9558e-d5c0-469a-bd57-f4507ce58589 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:12.058509 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca9558e-d5c0-469a-bd57-f4507ce58589 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:12.058990 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca9558e-d5c0-469a-bd57-f4507ce58589 None admin] PATH_INFO: `/v3/roles/member` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:12.073175 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca9558e-d5c0-469a-bd57-f4507ce58589 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:24:12.083210 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-fca9558e-d5c0-469a-bd57-f4507ce58589 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:24:12.083210 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:12.083210 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:12.083210 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:12.083210 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:12.083210 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:12.083210 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:12.083210 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:12.083210 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:12.083210 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:12.083210 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:12.083210 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:12.083210 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:24:12.083210 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:24:12.083210 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:24:12.083210 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:24:12.084290 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:24:12.084290 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:24:12.084290 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:24:12.084290 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:24:12.084290 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:12.084290 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:12.084290 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:12.084290 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:12.084290 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:12.084290 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:12.084290 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:12.084290 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:24:12.084290 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:12.084290 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:12.084290 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:12.084290 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:12.085239 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:12.085239 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:12.085239 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:12.085239 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:12.085239 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:24:12.085239 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:24:12.085239 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:24:12.085239 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:24:12.085239 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:24:12.085239 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:24:12.085239 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:24:12.085239 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:24:12.085239 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 213/431] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:24:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:24:12.096051 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca9558e-d5c0-469a-bd57-f4507ce58589 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:12.130072 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:12.171040 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6244498-5642-4690-bff3-11c5068c4198 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:12.204371 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:12.214555 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3d197b-e627-4786-a741-5c5c743da28a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:12.215417 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3d197b-e627-4786-a741-5c5c743da28a None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:12.216229 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3d197b-e627-4786-a741-5c5c743da28a None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:12.216588 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3d197b-e627-4786-a741-5c5c743da28a None admin] PATH_INFO: `/v3/projects/e7497d51ee8f4f0487b09541d4ad94d2` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:12.218822 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3d197b-e627-4786-a741-5c5c743da28a None admin] RBAC: Authorizing `identity:get_project(project_id=e7497d51ee8f4f0487b09541d4ad94d2)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:12.219551 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3d197b-e627-4786-a741-5c5c743da28a None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:12.221620 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 219/432] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:24:12 2022] GET /identity/v3/projects/e7497d51ee8f4f0487b09541d4ad94d2 => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:12.223645 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b735fcc-8401-4c80-83bc-628223d14dc9 None None] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:12.223645 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b735fcc-8401-4c80-83bc-628223d14dc9 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:12.223645 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b735fcc-8401-4c80-83bc-628223d14dc9 None None] PATH_INFO: `/` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:12.224951 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 220/433] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:24:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:24:12.234804 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54956f3-ab76-4a57-a7ad-4c76b546afd9 None None] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:12.234804 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54956f3-ab76-4a57-a7ad-4c76b546afd9 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:12.235098 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54956f3-ab76-4a57-a7ad-4c76b546afd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:12.244675 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca9558e-d5c0-469a-bd57-f4507ce58589 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:12.258499 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.auth.core [None req-c54956f3-ab76-4a57-a7ad-4c76b546afd9 None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143587) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:24:12.259695 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-c54956f3-ab76-4a57-a7ad-4c76b546afd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:24:12.287549 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f74f0a6-7a29-4acc-841a-d5393507af47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:12.288230 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f74f0a6-7a29-4acc-841a-d5393507af47 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:12.288513 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f74f0a6-7a29-4acc-841a-d5393507af47 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:12.288794 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f74f0a6-7a29-4acc-841a-d5393507af47 None admin] PATH_INFO: `/v3/roles` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:12.289543 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f74f0a6-7a29-4acc-841a-d5393507af47 None admin] RBAC: Adding query filter params (name=member) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:12.289892 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f74f0a6-7a29-4acc-841a-d5393507af47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:12.291261 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f74f0a6-7a29-4acc-841a-d5393507af47 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:12.300799 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 214/434] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:24:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:24:12.301693 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f74f0a6-7a29-4acc-841a-d5393507af47 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:12.337153 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 221/435] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:24:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:24:12.338315 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54956f3-ab76-4a57-a7ad-4c76b546afd9 None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:12.398982 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:12.424106 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f74f0a6-7a29-4acc-841a-d5393507af47 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:12.451005 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54956f3-ab76-4a57-a7ad-4c76b546afd9 None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:12.462774 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28ca1a3-5c0c-471d-b9a0-6319f741f0f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:12.463666 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28ca1a3-5c0c-471d-b9a0-6319f741f0f9 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:12.463956 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28ca1a3-5c0c-471d-b9a0-6319f741f0f9 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:12.464234 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28ca1a3-5c0c-471d-b9a0-6319f741f0f9 None admin] PATH_INFO: `/v3/groups/6f88b9d7a32742db86b87c6d666173ac` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:12.465705 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28ca1a3-5c0c-471d-b9a0-6319f741f0f9 None admin] RBAC: Authorizing `identity:get_group(group_id=6f88b9d7a32742db86b87c6d666173ac)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:12.467749 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28ca1a3-5c0c-471d-b9a0-6319f741f0f9 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:12.469622 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 215/436] 10.222.0.57 () {56 vars in 1180 bytes} [Thu May 19 14:24:12 2022] GET /identity/v3/groups/6f88b9d7a32742db86b87c6d666173ac => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:12.474831 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28ca1a3-5c0c-471d-b9a0-6319f741f0f9 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:12.480925 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:12.490788 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53904f38-1ec7-44e5-8bf6-3860f18bea1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:12.491436 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53904f38-1ec7-44e5-8bf6-3860f18bea1d None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:12.491717 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53904f38-1ec7-44e5-8bf6-3860f18bea1d None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:12.491991 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53904f38-1ec7-44e5-8bf6-3860f18bea1d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:12.562742 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53904f38-1ec7-44e5-8bf6-3860f18bea1d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:12.564284 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53904f38-1ec7-44e5-8bf6-3860f18bea1d None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:12.584690 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28ca1a3-5c0c-471d-b9a0-6319f741f0f9 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:12.623233 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169f1fd1-a0e0-40c9-bac5-85ce7add1ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:12.624054 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169f1fd1-a0e0-40c9-bac5-85ce7add1ea5 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:12.624337 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169f1fd1-a0e0-40c9-bac5-85ce7add1ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:12.624611 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169f1fd1-a0e0-40c9-bac5-85ce7add1ea5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:12.635705 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169f1fd1-a0e0-40c9-bac5-85ce7add1ea5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:24:12.644418 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-169f1fd1-a0e0-40c9-bac5-85ce7add1ea5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:24:12.644418 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:12.644418 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:12.644418 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:12.644418 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:12.644418 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:12.644418 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:12.644418 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:12.644418 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:12.644418 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:12.644418 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:12.644418 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:12.644418 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:24:12.644418 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:24:12.644418 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:24:12.644418 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:24:12.645471 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:24:12.645471 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:24:12.645471 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:24:12.645471 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:24:12.645471 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:12.645471 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:12.645471 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:12.645471 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:12.645471 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:12.645471 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:12.645471 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:12.645471 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:24:12.645471 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:12.645471 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:12.645471 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:12.645471 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:12.646446 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:12.646446 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:12.646446 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:12.646446 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:12.646446 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:24:12.646446 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:24:12.646446 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:24:12.646446 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:24:12.646446 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:24:12.646446 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:24:12.646446 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:24:12.646446 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:24:12.646446 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 216/437] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 19 14:24:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:12.646446 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169f1fd1-a0e0-40c9-bac5-85ce7add1ea5 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:12.660575 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:12.688903 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 222/438] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:24:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:24:12.690019 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53904f38-1ec7-44e5-8bf6-3860f18bea1d None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:12.723401 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:12.773590 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169f1fd1-a0e0-40c9-bac5-85ce7add1ea5 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:12.806152 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53904f38-1ec7-44e5-8bf6-3860f18bea1d None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:12.811370 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7840253-cd5a-4790-8de4-615fe559a5d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:12.812268 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7840253-cd5a-4790-8de4-615fe559a5d6 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:12.812722 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7840253-cd5a-4790-8de4-615fe559a5d6 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:12.813154 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7840253-cd5a-4790-8de4-615fe559a5d6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:12.814283 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7840253-cd5a-4790-8de4-615fe559a5d6 None admin] RBAC: Adding query filter params (group.id=6f88b9d7a32742db86b87c6d666173ac, role.id=b954b2c7ce764aa0afac78e075a011a6, scope.project.id=e7497d51ee8f4f0487b09541d4ad94d2) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:12.814773 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7840253-cd5a-4790-8de4-615fe559a5d6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:12.816483 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7840253-cd5a-4790-8de4-615fe559a5d6 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:12.823433 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 217/439] 10.222.0.57 () {56 vars in 1376 bytes} [Thu May 19 14:24:12 2022] GET /identity/v3/role_assignments?group.id=6f88b9d7a32742db86b87c6d666173ac&scope.project.id=e7497d51ee8f4f0487b09541d4ad94d2&role.id=b954b2c7ce764aa0afac78e075a011a6 => generated 611 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:12.824549 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7840253-cd5a-4790-8de4-615fe559a5d6 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:12.858233 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fa6cf6-1289-49fd-840a-7d461b247671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:12.858969 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fa6cf6-1289-49fd-840a-7d461b247671 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:12.859253 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fa6cf6-1289-49fd-840a-7d461b247671 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:12.859528 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fa6cf6-1289-49fd-840a-7d461b247671 None admin] PATH_INFO: `/v3/roles` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:12.860295 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fa6cf6-1289-49fd-840a-7d461b247671 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:12.860602 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fa6cf6-1289-49fd-840a-7d461b247671 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:12.862106 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fa6cf6-1289-49fd-840a-7d461b247671 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:12.871708 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 223/440] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:24:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:12.877766 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a44c4bf-f521-4b7b-9389-73152c78319a None None] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:12.878087 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a44c4bf-f521-4b7b-9389-73152c78319a None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:12.878366 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a44c4bf-f521-4b7b-9389-73152c78319a None None] PATH_INFO: `/` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:12.879954 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 224/441] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:24:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:24:12.883826 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a44c4bf-f521-4b7b-9389-73152c78319a None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:12.926168 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:12.965122 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7840253-cd5a-4790-8de4-615fe559a5d6 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:12.986960 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:13.003205 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a44c4bf-f521-4b7b-9389-73152c78319a None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:13.005552 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8e4fcf-f653-4379-8097-a89e742287dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:13.006485 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8e4fcf-f653-4379-8097-a89e742287dd None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:13.006766 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8e4fcf-f653-4379-8097-a89e742287dd None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:13.007043 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8e4fcf-f653-4379-8097-a89e742287dd None admin] PATH_INFO: `/v3/projects/24fafd1589784882b754b5f8543954bb` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:13.008287 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8e4fcf-f653-4379-8097-a89e742287dd None admin] RBAC: Authorizing `identity:get_project(project_id=24fafd1589784882b754b5f8543954bb)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:13.009762 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8e4fcf-f653-4379-8097-a89e742287dd None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:13.011767 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 218/442] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:24:12 2022] GET /identity/v3/projects/24fafd1589784882b754b5f8543954bb => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:13.018702 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc171650-8eac-4df0-8290-2363b518c3d7 None None] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:13.018992 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc171650-8eac-4df0-8290-2363b518c3d7 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:13.019258 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc171650-8eac-4df0-8290-2363b518c3d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:13.036954 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.auth.core [None req-dc171650-8eac-4df0-8290-2363b518c3d7 None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143586) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:24:13.038192 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.fernet_utils [None req-dc171650-8eac-4df0-8290-2363b518c3d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143586) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:24:13.047335 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fffc7a-2e0b-44b7-8777-658e5c96ade5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:13.047987 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fffc7a-2e0b-44b7-8777-658e5c96ade5 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:13.048270 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fffc7a-2e0b-44b7-8777-658e5c96ade5 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:13.048546 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fffc7a-2e0b-44b7-8777-658e5c96ade5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:13.123507 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 219/443] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:24:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:24:13.124437 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc171650-8eac-4df0-8290-2363b518c3d7 None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:13.134873 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fffc7a-2e0b-44b7-8777-658e5c96ade5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:13.136342 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fffc7a-2e0b-44b7-8777-658e5c96ade5 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:13.205975 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:13.244799 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:13.250162 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc171650-8eac-4df0-8290-2363b518c3d7 None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:13.262224 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 225/444] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:24:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:24:13.263077 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fffc7a-2e0b-44b7-8777-658e5c96ade5 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:13.287635 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b0d174-863b-4c06-bb95-fd1fb6a1aee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:13.288497 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b0d174-863b-4c06-bb95-fd1fb6a1aee9 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:13.288789 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b0d174-863b-4c06-bb95-fd1fb6a1aee9 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:13.289067 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b0d174-863b-4c06-bb95-fd1fb6a1aee9 None admin] PATH_INFO: `/v3/groups/6f88b9d7a32742db86b87c6d666173ac` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:13.290277 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b0d174-863b-4c06-bb95-fd1fb6a1aee9 None admin] RBAC: Authorizing `identity:get_group(group_id=6f88b9d7a32742db86b87c6d666173ac)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:13.291716 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b0d174-863b-4c06-bb95-fd1fb6a1aee9 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:13.293339 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 220/445] 10.222.0.57 () {56 vars in 1180 bytes} [Thu May 19 14:24:13 2022] GET /identity/v3/groups/6f88b9d7a32742db86b87c6d666173ac => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:13.294653 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b0d174-863b-4c06-bb95-fd1fb6a1aee9 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:13.365333 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fffc7a-2e0b-44b7-8777-658e5c96ade5 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:13.398181 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b0d174-863b-4c06-bb95-fd1fb6a1aee9 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:13.399041 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a6920f-63ad-407d-9da4-56ca3ec36f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:13.399846 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a6920f-63ad-407d-9da4-56ca3ec36f3c None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:13.400132 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a6920f-63ad-407d-9da4-56ca3ec36f3c None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:13.400423 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a6920f-63ad-407d-9da4-56ca3ec36f3c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:13.411009 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a6920f-63ad-407d-9da4-56ca3ec36f3c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:24:13.418677 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-d2a6920f-63ad-407d-9da4-56ca3ec36f3c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:24:13.418677 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:13.418677 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:13.418677 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:13.418677 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:13.418677 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:13.418677 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:13.418677 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:13.418677 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:13.418677 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:13.418677 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:13.418677 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:13.418677 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:24:13.418677 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:24:13.418677 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:24:13.418677 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:24:13.420335 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:24:13.420335 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:24:13.420335 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:24:13.420335 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:24:13.420335 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:13.420335 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:13.420335 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:13.420335 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:13.420335 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:13.420335 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:13.420335 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:13.420335 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:24:13.420335 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:13.420335 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:13.420335 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:13.420335 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:13.421279 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:13.421279 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:13.421279 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:13.421279 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:13.421279 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:24:13.421279 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:24:13.421279 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:24:13.421279 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:24:13.421279 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:24:13.421279 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:24:13.421279 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:24:13.421279 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:24:13.421279 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 226/446] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:24:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:13.422174 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a6920f-63ad-407d-9da4-56ca3ec36f3c None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:13.438317 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b466bd44-550f-4e81-a1ff-479afc20e026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:13.439181 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b466bd44-550f-4e81-a1ff-479afc20e026 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:13.439463 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b466bd44-550f-4e81-a1ff-479afc20e026 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:13.439737 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b466bd44-550f-4e81-a1ff-479afc20e026 None admin] PATH_INFO: `/v3/projects/24fafd1589784882b754b5f8543954bb` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:13.440928 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b466bd44-550f-4e81-a1ff-479afc20e026 None admin] RBAC: Authorizing `identity:get_project(project_id=24fafd1589784882b754b5f8543954bb)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:13.442420 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b466bd44-550f-4e81-a1ff-479afc20e026 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:13.444110 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 221/447] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:24:13 2022] GET /identity/v3/projects/24fafd1589784882b754b5f8543954bb => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:13.444865 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b466bd44-550f-4e81-a1ff-479afc20e026 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:13.448924 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:13.474801 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:13.528856 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a6920f-63ad-407d-9da4-56ca3ec36f3c None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:13.545968 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b466bd44-550f-4e81-a1ff-479afc20e026 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:13.562572 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c141849-6611-467b-a909-0dcbf77eff4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:13.563242 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c141849-6611-467b-a909-0dcbf77eff4b None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:13.563522 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c141849-6611-467b-a909-0dcbf77eff4b None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:13.563794 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c141849-6611-467b-a909-0dcbf77eff4b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:13.564645 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c141849-6611-467b-a909-0dcbf77eff4b None admin] RBAC: Adding query filter params (group.id=6f88b9d7a32742db86b87c6d666173ac, role.id=3d5075293a4f45449eaef6af8fa53004, scope.project.id=24fafd1589784882b754b5f8543954bb) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:13.564957 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c141849-6611-467b-a909-0dcbf77eff4b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:13.566415 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c141849-6611-467b-a909-0dcbf77eff4b None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:13.572250 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 227/448] 10.222.0.57 () {56 vars in 1376 bytes} [Thu May 19 14:24:13 2022] GET /identity/v3/role_assignments?group.id=6f88b9d7a32742db86b87c6d666173ac&scope.project.id=24fafd1589784882b754b5f8543954bb&role.id=3d5075293a4f45449eaef6af8fa53004 => generated 611 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:13.573014 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c141849-6611-467b-a909-0dcbf77eff4b None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:13.595404 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244217ac-06d8-41d6-8f82-c1f6b411115e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:13.596218 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244217ac-06d8-41d6-8f82-c1f6b411115e None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:13.596575 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244217ac-06d8-41d6-8f82-c1f6b411115e None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:13.596926 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244217ac-06d8-41d6-8f82-c1f6b411115e None admin] PATH_INFO: `/v3/roles` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:13.598427 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244217ac-06d8-41d6-8f82-c1f6b411115e None admin] RBAC: Adding query filter params (name=admin) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:13.598815 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244217ac-06d8-41d6-8f82-c1f6b411115e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:13.600697 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244217ac-06d8-41d6-8f82-c1f6b411115e None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:13.615210 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 222/449] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:24:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:13.616462 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244217ac-06d8-41d6-8f82-c1f6b411115e None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:13.698272 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:13.699901 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c141849-6611-467b-a909-0dcbf77eff4b None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:13.715536 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:13.739781 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794dbc85-d71b-49f1-b09a-9bbc75e143c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:13.740654 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794dbc85-d71b-49f1-b09a-9bbc75e143c1 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:13.740939 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794dbc85-d71b-49f1-b09a-9bbc75e143c1 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:13.741212 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794dbc85-d71b-49f1-b09a-9bbc75e143c1 None admin] PATH_INFO: `/v3/groups/6f88b9d7a32742db86b87c6d666173ac` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:13.742496 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794dbc85-d71b-49f1-b09a-9bbc75e143c1 None admin] RBAC: Authorizing `identity:get_group(group_id=6f88b9d7a32742db86b87c6d666173ac)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:13.743272 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244217ac-06d8-41d6-8f82-c1f6b411115e None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:13.743975 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794dbc85-d71b-49f1-b09a-9bbc75e143c1 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:13.751107 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 228/450] 10.222.0.57 () {56 vars in 1180 bytes} [Thu May 19 14:24:13 2022] GET /identity/v3/groups/6f88b9d7a32742db86b87c6d666173ac => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:13.752144 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794dbc85-d71b-49f1-b09a-9bbc75e143c1 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:13.793014 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c77854-c07c-4457-b5a6-437fe9f2872b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:13.794602 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c77854-c07c-4457-b5a6-437fe9f2872b None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:13.794888 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c77854-c07c-4457-b5a6-437fe9f2872b None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:13.795216 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c77854-c07c-4457-b5a6-437fe9f2872b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:13.875991 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794dbc85-d71b-49f1-b09a-9bbc75e143c1 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:13.876743 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c77854-c07c-4457-b5a6-437fe9f2872b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:13.878263 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c77854-c07c-4457-b5a6-437fe9f2872b None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:13.921737 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e8dad3-f134-402f-a91d-db6acfb36a0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:13.922446 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e8dad3-f134-402f-a91d-db6acfb36a0d None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:13.922728 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e8dad3-f134-402f-a91d-db6acfb36a0d None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:13.923003 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e8dad3-f134-402f-a91d-db6acfb36a0d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:13.923831 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e8dad3-f134-402f-a91d-db6acfb36a0d None admin] RBAC: Adding query filter params (group.id=6f88b9d7a32742db86b87c6d666173ac, role.id=b954b2c7ce764aa0afac78e075a011a6, scope.project.id=24fafd1589784882b754b5f8543954bb) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:13.924142 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e8dad3-f134-402f-a91d-db6acfb36a0d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:13.925588 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e8dad3-f134-402f-a91d-db6acfb36a0d None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:13.931836 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 229/451] 10.222.0.57 () {56 vars in 1376 bytes} [Thu May 19 14:24:13 2022] GET /identity/v3/role_assignments?group.id=6f88b9d7a32742db86b87c6d666173ac&scope.project.id=24fafd1589784882b754b5f8543954bb&role.id=b954b2c7ce764aa0afac78e075a011a6 => generated 611 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:24:13.963316 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:14.005159 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 223/452] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:24:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 14:24:14.010836 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e8dad3-f134-402f-a91d-db6acfb36a0d None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:14.061569 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:14.120131 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e8dad3-f134-402f-a91d-db6acfb36a0d None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:14.156530 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f5ae00-3354-4c1e-8da8-df45c23d42e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:14.157384 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f5ae00-3354-4c1e-8da8-df45c23d42e3 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:14.157668 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f5ae00-3354-4c1e-8da8-df45c23d42e3 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:14.157969 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f5ae00-3354-4c1e-8da8-df45c23d42e3 None admin] PATH_INFO: `/v3/projects/9471b834f91741fbaf409539b1810552` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:14.159153 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f5ae00-3354-4c1e-8da8-df45c23d42e3 None admin] RBAC: Authorizing `identity:get_project(project_id=9471b834f91741fbaf409539b1810552)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:14.160597 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f5ae00-3354-4c1e-8da8-df45c23d42e3 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:14.162331 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 230/453] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:24:14 2022] GET /identity/v3/projects/9471b834f91741fbaf409539b1810552 => generated 344 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:14.167820 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c77854-c07c-4457-b5a6-437fe9f2872b None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:14.275154 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c77854-c07c-4457-b5a6-437fe9f2872b None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:14.310456 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2744e8-46c3-4b25-98d8-dab4c480b65a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:14.311314 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2744e8-46c3-4b25-98d8-dab4c480b65a None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:14.311651 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2744e8-46c3-4b25-98d8-dab4c480b65a None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:14.311878 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2744e8-46c3-4b25-98d8-dab4c480b65a None admin] PATH_INFO: `/v3/groups/741b95495ffd44ce8e040cc5e23b155e` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:14.313106 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2744e8-46c3-4b25-98d8-dab4c480b65a None admin] RBAC: Authorizing `identity:get_group(group_id=741b95495ffd44ce8e040cc5e23b155e)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:14.314610 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2744e8-46c3-4b25-98d8-dab4c480b65a None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:14.316404 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 224/454] 10.222.0.57 () {56 vars in 1180 bytes} [Thu May 19 14:24:14 2022] GET /identity/v3/groups/741b95495ffd44ce8e040cc5e23b155e => generated 229 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:14.322274 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f5ae00-3354-4c1e-8da8-df45c23d42e3 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:14.422840 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f5ae00-3354-4c1e-8da8-df45c23d42e3 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:14.435195 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:14.456325 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4f72ac-0acd-41a8-b11b-0beff7809aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:14.456973 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4f72ac-0acd-41a8-b11b-0beff7809aef None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:14.457259 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4f72ac-0acd-41a8-b11b-0beff7809aef None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:14.457541 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4f72ac-0acd-41a8-b11b-0beff7809aef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:14.458404 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4f72ac-0acd-41a8-b11b-0beff7809aef None admin] RBAC: Adding query filter params (group.id=741b95495ffd44ce8e040cc5e23b155e, role.id=a3b793754e7b40b0bb21afe74eec3c18, scope.project.id=9471b834f91741fbaf409539b1810552) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:14.458721 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4f72ac-0acd-41a8-b11b-0beff7809aef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:14.460164 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4f72ac-0acd-41a8-b11b-0beff7809aef None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:14.466465 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 231/455] 10.222.0.57 () {56 vars in 1376 bytes} [Thu May 19 14:24:14 2022] GET /identity/v3/role_assignments?group.id=741b95495ffd44ce8e040cc5e23b155e&scope.project.id=9471b834f91741fbaf409539b1810552&role.id=a3b793754e7b40b0bb21afe74eec3c18 => generated 611 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:17.056766 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afc5a1d-8290-4e0d-aa13-032eb94e6a22 None None] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:17.057278 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afc5a1d-8290-4e0d-aa13-032eb94e6a22 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:17.057278 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afc5a1d-8290-4e0d-aa13-032eb94e6a22 None None] PATH_INFO: `/` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:17.059024 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 225/456] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:24:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:24:17.064120 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527d60a6-0ea7-42b5-9c71-87b1856bb7d4 None None] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:17.064974 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527d60a6-0ea7-42b5-9c71-87b1856bb7d4 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:17.065493 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527d60a6-0ea7-42b5-9c71-87b1856bb7d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:17.091433 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.auth.core [None req-527d60a6-0ea7-42b5-9c71-87b1856bb7d4 None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143587) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:24:17.092998 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-527d60a6-0ea7-42b5-9c71-87b1856bb7d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:24:17.185680 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 232/457] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:24:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:24:17.243341 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afc5a1d-8290-4e0d-aa13-032eb94e6a22 None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:17.290506 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:17.391212 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afc5a1d-8290-4e0d-aa13-032eb94e6a22 None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:17.439122 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dac9e4a-8929-44ac-85f8-68e77218d833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:17.439929 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dac9e4a-8929-44ac-85f8-68e77218d833 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:17.440215 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dac9e4a-8929-44ac-85f8-68e77218d833 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:17.440498 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dac9e4a-8929-44ac-85f8-68e77218d833 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:17.452481 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dac9e4a-8929-44ac-85f8-68e77218d833 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:17.453962 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dac9e4a-8929-44ac-85f8-68e77218d833 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:17.459531 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823520ac-6b7e-44d8-93ff-e2decd62eaeb None None] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:17.460079 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823520ac-6b7e-44d8-93ff-e2decd62eaeb None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:17.460572 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823520ac-6b7e-44d8-93ff-e2decd62eaeb None None] PATH_INFO: `/` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:17.462463 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 233/458] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:24:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:24:17.471561 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777c38f0-af8e-4b03-842f-6b8adea30fcc None None] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:17.471855 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777c38f0-af8e-4b03-842f-6b8adea30fcc None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:17.472342 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777c38f0-af8e-4b03-842f-6b8adea30fcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:17.477119 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-7dac9e4a-8929-44ac-85f8-68e77218d833 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:17.477119 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:17.477119 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:17.477119 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:17.477119 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:17.477119 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:17.477119 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:17.477119 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:17.477119 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:17.477119 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:17.477119 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:17.477119 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:17.477119 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:24:17.477119 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:24:17.477119 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:24:17.478322 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:24:17.478322 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:17.478322 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:17.478322 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:17.478322 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:17.478322 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:17.478322 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:17.478322 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:17.478322 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:24:17.478322 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:17.478322 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:17.478322 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:17.478322 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:17.478322 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:17.478322 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:17.478322 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:17.478322 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:17.479489 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:24:17.479489 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:24:17.479489 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:17.479489 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:24:17.479489 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 226/459] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:24:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:17.492011 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dac9e4a-8929-44ac-85f8-68e77218d833 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:17.511746 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.auth.core [None req-777c38f0-af8e-4b03-842f-6b8adea30fcc None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143587) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:24:17.515635 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-777c38f0-af8e-4b03-842f-6b8adea30fcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:24:17.642572 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:17.668470 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 234/460] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:24:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:24:17.675429 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dac9e4a-8929-44ac-85f8-68e77218d833 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:17.712452 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777c38f0-af8e-4b03-842f-6b8adea30fcc None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:17.728189 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc09617-87ef-4af7-8014-1afefcde6db1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:17.729321 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc09617-87ef-4af7-8014-1afefcde6db1 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:17.731205 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc09617-87ef-4af7-8014-1afefcde6db1 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:17.731671 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc09617-87ef-4af7-8014-1afefcde6db1 None admin] PATH_INFO: `/v3/domains` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:17.733387 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc09617-87ef-4af7-8014-1afefcde6db1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:17.734667 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc09617-87ef-4af7-8014-1afefcde6db1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:17.736243 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc09617-87ef-4af7-8014-1afefcde6db1 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:17.757931 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 227/461] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:24:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:24:17.759652 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34e4cf9-abf6-4e5c-a10b-f43f7f7443bd None None] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:17.760113 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34e4cf9-abf6-4e5c-a10b-f43f7f7443bd None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:17.760541 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34e4cf9-abf6-4e5c-a10b-f43f7f7443bd None None] PATH_INFO: `/` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:17.762217 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 228/462] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:24:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:24:17.763750 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34e4cf9-abf6-4e5c-a10b-f43f7f7443bd None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:17.768392 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:17.847479 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777c38f0-af8e-4b03-842f-6b8adea30fcc None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:17.876200 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34e4cf9-abf6-4e5c-a10b-f43f7f7443bd None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:17.885994 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b73425-6f3a-4aa4-8dc9-29df245b7eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:17.886783 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b73425-6f3a-4aa4-8dc9-29df245b7eb9 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:17.887739 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b73425-6f3a-4aa4-8dc9-29df245b7eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:17.887940 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b73425-6f3a-4aa4-8dc9-29df245b7eb9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:17.898947 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b73425-6f3a-4aa4-8dc9-29df245b7eb9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:17.900354 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b73425-6f3a-4aa4-8dc9-29df245b7eb9 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:17.912892 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fcb020-7bfd-4660-968d-f697a0a060bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:17.913563 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fcb020-7bfd-4660-968d-f697a0a060bf None admin] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:17.913871 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fcb020-7bfd-4660-968d-f697a0a060bf None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:17.914149 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fcb020-7bfd-4660-968d-f697a0a060bf None admin] PATH_INFO: `/v3/users` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:17.914850 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fcb020-7bfd-4660-968d-f697a0a060bf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:17.916282 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fcb020-7bfd-4660-968d-f697a0a060bf None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:17.919127 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-06b73425-6f3a-4aa4-8dc9-29df245b7eb9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:17.919127 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:17.919127 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:17.919127 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:17.919127 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:17.919127 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:17.919127 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:17.919127 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:17.919127 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:17.919127 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:17.919127 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:17.919127 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:17.919127 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:24:17.919127 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:24:17.919127 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:24:17.920396 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:24:17.920396 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:17.920396 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:17.920396 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:17.920396 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:17.920396 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:17.920396 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:17.920396 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:17.920396 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:24:17.920396 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:17.920396 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:17.920396 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:17.920396 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:17.920396 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:17.920396 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:17.920396 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:17.920396 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:17.926588 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:24:17.926588 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:24:17.926588 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:17.926588 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:24:17.926588 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 235/463] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:24:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:24:17.926588 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bf5c4a-8496-427b-a3d7-2ecdf9eb15ed None None] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:17.927219 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bf5c4a-8496-427b-a3d7-2ecdf9eb15ed None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:17.927219 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bf5c4a-8496-427b-a3d7-2ecdf9eb15ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:17.948749 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.auth.core [None req-e3bf5c4a-8496-427b-a3d7-2ecdf9eb15ed None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143587) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:24:17.950659 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-e3bf5c4a-8496-427b-a3d7-2ecdf9eb15ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:24:17.962033 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 229/464] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 19 14:24:17 2022] POST /identity/v3/users => generated 246 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:24:17.962033 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d19807-86fa-4f4b-bd51-1bce0254dbd9 None None] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:17.962033 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d19807-86fa-4f4b-bd51-1bce0254dbd9 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:17.962033 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d19807-86fa-4f4b-bd51-1bce0254dbd9 None None] PATH_INFO: `/` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:17.962033 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 230/465] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:24:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:24:17.962033 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d19807-86fa-4f4b-bd51-1bce0254dbd9 None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:17.964757 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:18.050241 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 236/466] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:24:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:24:18.052803 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0732020a-bb44-43b1-9602-fb0d0eff5933 None None] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:18.053080 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0732020a-bb44-43b1-9602-fb0d0eff5933 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:18.053356 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0732020a-bb44-43b1-9602-fb0d0eff5933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:18.088234 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.auth.core [None req-0732020a-bb44-43b1-9602-fb0d0eff5933 None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143587) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:24:18.089496 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-0732020a-bb44-43b1-9602-fb0d0eff5933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:24:18.111742 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:18.124543 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d19807-86fa-4f4b-bd51-1bce0254dbd9 None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:18.166652 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b401f85b-a8a2-494e-9679-ef708505ab52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:18.167683 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b401f85b-a8a2-494e-9679-ef708505ab52 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:18.168073 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b401f85b-a8a2-494e-9679-ef708505ab52 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:18.168354 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b401f85b-a8a2-494e-9679-ef708505ab52 None admin] PATH_INFO: `/v3/domains` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:18.169660 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b401f85b-a8a2-494e-9679-ef708505ab52 None admin] RBAC: Adding query filter params (name=Default) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:18.171650 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b401f85b-a8a2-494e-9679-ef708505ab52 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:18.172042 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b401f85b-a8a2-494e-9679-ef708505ab52 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:18.184674 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 231/467] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:24:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:18.186067 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b401f85b-a8a2-494e-9679-ef708505ab52 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:18.191319 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 237/468] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:24:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:24:18.192025 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0732020a-bb44-43b1-9602-fb0d0eff5933 None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:18.280138 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:18.330974 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b401f85b-a8a2-494e-9679-ef708505ab52 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:18.331545 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0732020a-bb44-43b1-9602-fb0d0eff5933 None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:18.372800 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a065498-4d00-4f5e-8428-55a7d8c590d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:18.374666 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a065498-4d00-4f5e-8428-55a7d8c590d2 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:18.375194 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a065498-4d00-4f5e-8428-55a7d8c590d2 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:18.375906 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a065498-4d00-4f5e-8428-55a7d8c590d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:18.381654 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d899e0-4e0c-4c0a-92bd-dc410c53bce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:18.382371 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d899e0-4e0c-4c0a-92bd-dc410c53bce4 None admin] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:18.382651 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d899e0-4e0c-4c0a-92bd-dc410c53bce4 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:18.382925 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d899e0-4e0c-4c0a-92bd-dc410c53bce4 None admin] PATH_INFO: `/v3/users` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:18.383644 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d899e0-4e0c-4c0a-92bd-dc410c53bce4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:18.385100 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d899e0-4e0c-4c0a-92bd-dc410c53bce4 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:18.390534 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a065498-4d00-4f5e-8428-55a7d8c590d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:18.392269 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a065498-4d00-4f5e-8428-55a7d8c590d2 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:18.413631 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 238/469] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 19 14:24:18 2022] POST /identity/v3/users => generated 248 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:24:18.414503 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d899e0-4e0c-4c0a-92bd-dc410c53bce4 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:18.417811 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-1a065498-4d00-4f5e-8428-55a7d8c590d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:18.417811 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:18.417811 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:18.417811 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:18.417811 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:18.417811 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:18.417811 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:18.417811 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:18.417811 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:18.417811 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:18.417811 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:18.417811 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:18.417811 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:24:18.417811 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:24:18.417811 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:24:18.418880 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:24:18.418880 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:18.418880 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:18.418880 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:18.418880 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:18.418880 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:18.418880 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:18.418880 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:18.418880 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:24:18.418880 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:18.418880 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:18.418880 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:18.418880 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:18.418880 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:18.418880 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:18.418880 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:18.418880 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:18.419964 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:24:18.419964 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:24:18.419964 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:18.419964 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:24:18.420207 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:18.422512 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 232/470] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:24:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:18.429366 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a065498-4d00-4f5e-8428-55a7d8c590d2 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:18.573894 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d899e0-4e0c-4c0a-92bd-dc410c53bce4 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:18.614054 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa43672c-39a3-4ea3-81b8-da790f9939d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:18.615026 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa43672c-39a3-4ea3-81b8-da790f9939d4 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:18.615314 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa43672c-39a3-4ea3-81b8-da790f9939d4 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:18.615594 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa43672c-39a3-4ea3-81b8-da790f9939d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:18.628985 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa43672c-39a3-4ea3-81b8-da790f9939d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:18.630781 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa43672c-39a3-4ea3-81b8-da790f9939d4 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:18.655941 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-aa43672c-39a3-4ea3-81b8-da790f9939d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:18.655941 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:18.655941 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:18.655941 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:18.655941 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:18.655941 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:18.655941 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:18.655941 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:18.655941 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:18.655941 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:18.655941 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:18.655941 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:18.655941 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:24:18.655941 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:24:18.655941 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:24:18.656998 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:24:18.656998 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:18.656998 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:18.656998 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:18.656998 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:18.656998 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:18.656998 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:18.656998 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:18.656998 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:24:18.656998 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:18.656998 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:18.656998 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:18.656998 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:18.656998 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:18.656998 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:18.656998 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:18.656998 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:18.658019 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:24:18.658019 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:24:18.658019 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:18.658019 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:24:18.658019 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 239/471] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:24:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:18.671024 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa43672c-39a3-4ea3-81b8-da790f9939d4 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:18.674103 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a065498-4d00-4f5e-8428-55a7d8c590d2 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:18.696040 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:18.732685 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326c26df-355b-437c-b385-bdd5ef1ea48a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:18.733651 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326c26df-355b-437c-b385-bdd5ef1ea48a None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:18.734106 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326c26df-355b-437c-b385-bdd5ef1ea48a None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:18.734556 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326c26df-355b-437c-b385-bdd5ef1ea48a None admin] PATH_INFO: `/v3/domains` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:18.735443 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326c26df-355b-437c-b385-bdd5ef1ea48a None admin] RBAC: Adding query filter params (name=Default) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:18.735966 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326c26df-355b-437c-b385-bdd5ef1ea48a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:18.737499 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326c26df-355b-437c-b385-bdd5ef1ea48a None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:18.750268 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:18.753384 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 233/472] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:24:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:24:18.759261 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326c26df-355b-437c-b385-bdd5ef1ea48a None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:18.801691 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa43672c-39a3-4ea3-81b8-da790f9939d4 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:18.842643 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe99a46c-a454-4934-aa6b-427c1e8b25c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:18.843564 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe99a46c-a454-4934-aa6b-427c1e8b25c6 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:18.844037 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe99a46c-a454-4934-aa6b-427c1e8b25c6 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:18.844476 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe99a46c-a454-4934-aa6b-427c1e8b25c6 None admin] PATH_INFO: `/v3/domains` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:18.846064 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe99a46c-a454-4934-aa6b-427c1e8b25c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:18.846560 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe99a46c-a454-4934-aa6b-427c1e8b25c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:18.848286 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe99a46c-a454-4934-aa6b-427c1e8b25c6 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:18.868067 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 240/473] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:24:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:18.869640 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2049763b-27b8-48b8-87b4-a5a6ec24a627 None None] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:18.877971 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2049763b-27b8-48b8-87b4-a5a6ec24a627 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:18.877971 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2049763b-27b8-48b8-87b4-a5a6ec24a627 None None] PATH_INFO: `/` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:18.877971 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 241/474] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:24:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:24:18.880718 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2049763b-27b8-48b8-87b4-a5a6ec24a627 None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:18.894397 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326c26df-355b-437c-b385-bdd5ef1ea48a None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:18.930337 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d0ea06-29cf-421d-90f2-97925bcc28ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:18.931032 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d0ea06-29cf-421d-90f2-97925bcc28ab None admin] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:18.931391 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d0ea06-29cf-421d-90f2-97925bcc28ab None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:18.931741 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d0ea06-29cf-421d-90f2-97925bcc28ab None admin] PATH_INFO: `/v3/users` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:18.932524 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d0ea06-29cf-421d-90f2-97925bcc28ab None admin] RBAC: Authorizing `identity:create_user()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:18.934053 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d0ea06-29cf-421d-90f2-97925bcc28ab None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:18.966775 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 234/475] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 19 14:24:18 2022] POST /identity/v3/users => generated 248 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:24:18.969246 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279d3655-354e-432f-bfc6-792528b7d61c None None] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:18.969546 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279d3655-354e-432f-bfc6-792528b7d61c None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:18.969833 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279d3655-354e-432f-bfc6-792528b7d61c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:18.994164 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.auth.core [None req-279d3655-354e-432f-bfc6-792528b7d61c None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143586) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:24:18.995536 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.fernet_utils [None req-279d3655-354e-432f-bfc6-792528b7d61c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143586) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:24:19.004719 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2049763b-27b8-48b8-87b4-a5a6ec24a627 None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:19.023958 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:19.028247 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:19.045793 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8485e6f0-65c5-4c48-bd14-d648710f1e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:19.046621 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8485e6f0-65c5-4c48-bd14-d648710f1e2b None admin] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:19.047589 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8485e6f0-65c5-4c48-bd14-d648710f1e2b None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:19.047589 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8485e6f0-65c5-4c48-bd14-d648710f1e2b None admin] PATH_INFO: `/v3/users` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:19.048094 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8485e6f0-65c5-4c48-bd14-d648710f1e2b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:19.050335 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8485e6f0-65c5-4c48-bd14-d648710f1e2b None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:19.074396 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 242/476] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 19 14:24:18 2022] POST /identity/v3/users => generated 249 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:24:19.088936 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 235/477] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:24:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:24:19.113651 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8485e6f0-65c5-4c48-bd14-d648710f1e2b None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:19.232514 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8485e6f0-65c5-4c48-bd14-d648710f1e2b None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:19.274435 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2174de-e80b-431f-9193-0d57bcf94338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:19.275134 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2174de-e80b-431f-9193-0d57bcf94338 None admin] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:19.275419 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2174de-e80b-431f-9193-0d57bcf94338 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:19.275699 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2174de-e80b-431f-9193-0d57bcf94338 None admin] PATH_INFO: `/v3/roles` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:19.276485 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2174de-e80b-431f-9193-0d57bcf94338 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:19.278034 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2174de-e80b-431f-9193-0d57bcf94338 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:19.286797 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.sql.core [None req-8e2174de-e80b-431f-9193-0d57bcf94338 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 19 14:24:19.286797 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 19 14:24:19.286797 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [parameters: {'id': 'f52dd0da65c84478be2cbd64252fdeb6', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 19 14:24:19.286797 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=143587) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 19 14:24:19.304715 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-8e2174de-e80b-431f-9193-0d57bcf94338 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 19 14:24:19.304715 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:19.304715 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 14:24:19.304715 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 14:24:19.304715 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 14:24:19.304715 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 14:24:19.304715 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 14:24:19.304715 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application result = self._query(query) May 19 14:24:19.304715 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 14:24:19.304715 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application conn.query(q) May 19 14:24:19.304715 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 14:24:19.304715 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 14:24:19.304715 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 14:24:19.304715 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application result.read() May 19 14:24:19.304715 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 14:24:19.305814 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 14:24:19.305814 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 14:24:19.305814 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 14:24:19.305814 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 14:24:19.305814 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 14:24:19.305814 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 14:24:19.305814 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 14:24:19.305814 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 19 14:24:19.305814 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:24:19.305814 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 19 14:24:19.305814 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:24:19.305814 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:19.305814 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 19 14:24:19.305814 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 19 14:24:19.305814 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role May 19 14:24:19.305814 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return ref.to_dict() May 19 14:24:19.305814 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 14:24:19.305814 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application next(self.gen) May 19 14:24:19.306904 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 19 14:24:19.306904 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application yield resource May 19 14:24:19.306904 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 14:24:19.306904 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application next(self.gen) May 19 14:24:19.306904 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 19 14:24:19.306904 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application self.session.rollback() May 19 14:24:19.306904 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 19 14:24:19.306904 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application self.force_reraise() May 19 14:24:19.306904 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 19 14:24:19.306904 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise self.value May 19 14:24:19.306904 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 19 14:24:19.306904 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 19 14:24:19.306904 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 19 14:24:19.306904 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application session.commit() May 19 14:24:19.306904 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 19 14:24:19.306904 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 19 14:24:19.306904 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 19 14:24:19.307891 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application self._prepare_impl() May 19 14:24:19.307891 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 19 14:24:19.307891 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application self.session.flush() May 19 14:24:19.307891 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 19 14:24:19.307891 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application self._flush(objects) May 19 14:24:19.307891 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 19 14:24:19.307891 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 19 14:24:19.307891 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 19 14:24:19.307891 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application compat.raise_( May 19 14:24:19.307891 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 14:24:19.307891 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception May 19 14:24:19.307891 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 19 14:24:19.307891 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application flush_context.execute() May 19 14:24:19.307891 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 19 14:24:19.307891 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rec.execute(self) May 19 14:24:19.307891 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 19 14:24:19.307891 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 19 14:24:19.308924 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 19 14:24:19.308924 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application _emit_insert_statements( May 19 14:24:19.308924 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 19 14:24:19.308924 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application c = connection._execute_20( May 19 14:24:19.308924 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 19 14:24:19.308924 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 19 14:24:19.308924 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 19 14:24:19.308924 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 19 14:24:19.308924 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 19 14:24:19.308924 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application ret = self._execute_context( May 19 14:24:19.308924 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 19 14:24:19.308924 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 19 14:24:19.308924 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 19 14:24:19.308924 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 19 14:24:19.308924 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 14:24:19.308924 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception May 19 14:24:19.312091 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 14:24:19.312091 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 14:24:19.312091 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 14:24:19.312091 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 14:24:19.312091 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 14:24:19.312091 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application result = self._query(query) May 19 14:24:19.312091 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 14:24:19.312091 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application conn.query(q) May 19 14:24:19.312091 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 14:24:19.312091 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 14:24:19.312091 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 14:24:19.312091 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application result.read() May 19 14:24:19.312091 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 14:24:19.312091 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 14:24:19.312091 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 14:24:19.312091 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 14:24:19.318471 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 14:24:19.318471 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 14:24:19.318471 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 14:24:19.318471 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 14:24:19.318471 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 19 14:24:19.318471 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 19 14:24:19.318471 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [parameters: {'id': 'f52dd0da65c84478be2cbd64252fdeb6', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 19 14:24:19.318471 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 19 14:24:19.318471 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:24:19.318471 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:24:19.318471 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:24:19.318471 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:19.318471 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:19.318471 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:19.318471 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:19.318471 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:19.321877 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:19.321877 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:19.321877 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:19.321877 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:19.321877 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:19.321877 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:19.321877 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post May 19 14:24:19.321877 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( May 19 14:24:19.321877 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:19.321877 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:19.321877 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role May 19 14:24:19.321877 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) May 19 14:24:19.321877 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 19 14:24:19.321877 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 19 14:24:19.321877 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 19 14:24:19.321877 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:24:19.324066 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 243/478] 10.222.0.57 () {60 vars in 1131 bytes} [Thu May 19 14:24:19 2022] POST /identity/v3/roles => generated 144 bytes in 193 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:19.324224 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279d3655-354e-432f-bfc6-792528b7d61c None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:19.439593 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279d3655-354e-432f-bfc6-792528b7d61c None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:19.478398 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de0c11a-93af-4df4-b252-7c95217bb20d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:19.479218 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de0c11a-93af-4df4-b252-7c95217bb20d None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:19.479490 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de0c11a-93af-4df4-b252-7c95217bb20d None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:19.479768 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de0c11a-93af-4df4-b252-7c95217bb20d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:19.493687 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de0c11a-93af-4df4-b252-7c95217bb20d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:24:19.502845 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-9de0c11a-93af-4df4-b252-7c95217bb20d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:24:19.502845 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:19.502845 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:19.502845 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:19.502845 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:19.502845 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:19.502845 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:19.502845 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:19.502845 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:19.502845 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:19.502845 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:19.502845 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:19.502845 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:24:19.502845 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:24:19.502845 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:24:19.502845 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:24:19.503947 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:24:19.503947 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:24:19.503947 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:24:19.503947 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:24:19.503947 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:19.503947 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:19.503947 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:19.503947 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:19.503947 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:19.503947 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:19.503947 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:19.503947 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:24:19.503947 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:19.503947 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:19.503947 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:19.503947 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:19.504874 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:19.504874 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:19.504874 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:19.504874 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:19.504874 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:24:19.504874 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:24:19.504874 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:24:19.504874 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:24:19.504874 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:24:19.504874 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:24:19.504874 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:24:19.504874 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:24:19.507845 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 236/479] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 19 14:24:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:19.513400 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2174de-e80b-431f-9193-0d57bcf94338 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:19.569717 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:19.655679 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2174de-e80b-431f-9193-0d57bcf94338 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:19.700715 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7041fdf-74ae-4cff-bd64-972d6c8561ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:19.701413 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7041fdf-74ae-4cff-bd64-972d6c8561ca None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:19.701703 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7041fdf-74ae-4cff-bd64-972d6c8561ca None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:19.702006 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7041fdf-74ae-4cff-bd64-972d6c8561ca None admin] PATH_INFO: `/v3/roles` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:19.702778 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7041fdf-74ae-4cff-bd64-972d6c8561ca None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:19.703092 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7041fdf-74ae-4cff-bd64-972d6c8561ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:19.704483 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7041fdf-74ae-4cff-bd64-972d6c8561ca None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:19.717880 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 244/480] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:24:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:20.899494 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93ea4d2-9568-49fb-98ec-e3c961ac5c8e None None] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:20.900613 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93ea4d2-9568-49fb-98ec-e3c961ac5c8e None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:20.901139 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93ea4d2-9568-49fb-98ec-e3c961ac5c8e None None] PATH_INFO: `/` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:20.903633 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 237/481] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:24:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:24:20.908697 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4898cd93-f4be-48ae-8a14-65bde2c5818a None None] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:20.909519 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4898cd93-f4be-48ae-8a14-65bde2c5818a None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:20.910089 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4898cd93-f4be-48ae-8a14-65bde2c5818a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:20.929503 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.auth.core [None req-4898cd93-f4be-48ae-8a14-65bde2c5818a None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143587) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:24:20.931078 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-4898cd93-f4be-48ae-8a14-65bde2c5818a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:24:21.013341 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 245/482] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:24:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:24:21.048321 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93ea4d2-9568-49fb-98ec-e3c961ac5c8e None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:21.078283 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:21.196454 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93ea4d2-9568-49fb-98ec-e3c961ac5c8e None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:21.241064 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fbc7bd-a12a-45c0-b7b9-c55e440393ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:21.241891 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fbc7bd-a12a-45c0-b7b9-c55e440393ea None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:21.242182 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fbc7bd-a12a-45c0-b7b9-c55e440393ea None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:21.242461 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fbc7bd-a12a-45c0-b7b9-c55e440393ea None admin] PATH_INFO: `/v3/roles/service` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:21.257963 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fbc7bd-a12a-45c0-b7b9-c55e440393ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:24:21.270333 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-f1fbc7bd-a12a-45c0-b7b9-c55e440393ea None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:24:21.270333 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:21.270333 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:21.270333 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:21.270333 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:21.270333 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:21.270333 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:21.270333 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:21.270333 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:21.270333 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:21.270333 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:21.270333 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:21.270333 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:24:21.270333 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:24:21.270333 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:24:21.270333 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:24:21.271458 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:24:21.271458 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:24:21.271458 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:24:21.271458 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:24:21.271458 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:21.271458 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:21.271458 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:21.271458 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:21.271458 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:21.271458 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:21.271458 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:21.271458 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:24:21.271458 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:21.271458 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:21.271458 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:21.271458 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:21.273118 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:21.273118 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:21.273118 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:21.273118 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:21.273118 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:24:21.273118 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:24:21.273118 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:24:21.273118 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:24:21.273118 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:24:21.273118 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:24:21.273118 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:24:21.273118 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:24:21.273118 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 238/483] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 14:24:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:21.277294 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4898cd93-f4be-48ae-8a14-65bde2c5818a None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:21.314276 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:21.413746 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018b7721-4f15-4133-814c-7f4326359817 None None] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:21.414126 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018b7721-4f15-4133-814c-7f4326359817 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:21.414339 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018b7721-4f15-4133-814c-7f4326359817 None None] PATH_INFO: `/` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:21.415976 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 239/484] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:24:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:24:21.416490 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4898cd93-f4be-48ae-8a14-65bde2c5818a None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:21.421750 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bd2676-fe0d-48b2-9e9d-ec607392a17c None None] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:21.422073 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bd2676-fe0d-48b2-9e9d-ec607392a17c None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:21.422352 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bd2676-fe0d-48b2-9e9d-ec607392a17c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:21.453193 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a57e4e-4def-4a54-8413-a3d3ca8d40f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:21.453923 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a57e4e-4def-4a54-8413-a3d3ca8d40f1 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:21.454254 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a57e4e-4def-4a54-8413-a3d3ca8d40f1 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:21.457492 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a57e4e-4def-4a54-8413-a3d3ca8d40f1 None admin] PATH_INFO: `/v3/roles` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:21.458289 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a57e4e-4def-4a54-8413-a3d3ca8d40f1 None admin] RBAC: Adding query filter params (name=service) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:21.458603 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a57e4e-4def-4a54-8413-a3d3ca8d40f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:21.459998 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a57e4e-4def-4a54-8413-a3d3ca8d40f1 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:21.462302 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.auth.core [None req-11bd2676-fe0d-48b2-9e9d-ec607392a17c None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143586) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:24:21.463509 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.fernet_utils [None req-11bd2676-fe0d-48b2-9e9d-ec607392a17c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143586) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:24:21.476744 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 246/485] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 19 14:24:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:21.481733 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a57e4e-4def-4a54-8413-a3d3ca8d40f1 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:21.583315 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 240/486] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:24:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:24:21.613852 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a57e4e-4def-4a54-8413-a3d3ca8d40f1 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:21.616094 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bd2676-fe0d-48b2-9e9d-ec607392a17c None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:21.618583 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:21.630275 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:21.735914 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bd2676-fe0d-48b2-9e9d-ec607392a17c None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:21.750558 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50873ef-a3ad-4650-8260-bdf233bb6a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:21.751841 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50873ef-a3ad-4650-8260-bdf233bb6a12 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:21.752333 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50873ef-a3ad-4650-8260-bdf233bb6a12 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:21.752810 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50873ef-a3ad-4650-8260-bdf233bb6a12 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:21.772069 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252549de-ad1a-4c25-940a-b51fee45cdfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:21.772874 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252549de-ad1a-4c25-940a-b51fee45cdfd None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:21.773158 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252549de-ad1a-4c25-940a-b51fee45cdfd None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:21.773435 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252549de-ad1a-4c25-940a-b51fee45cdfd None admin] PATH_INFO: `/v3/roles/service` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:21.786046 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252549de-ad1a-4c25-940a-b51fee45cdfd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:24:21.790702 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50873ef-a3ad-4650-8260-bdf233bb6a12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:21.792429 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50873ef-a3ad-4650-8260-bdf233bb6a12 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:21.794742 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-252549de-ad1a-4c25-940a-b51fee45cdfd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:24:21.794742 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:21.794742 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:21.794742 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:21.794742 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:21.794742 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:21.794742 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:21.794742 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:21.794742 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:21.794742 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:21.794742 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:21.794742 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:21.794742 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:24:21.794742 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:24:21.794742 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:24:21.794742 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:24:21.795757 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:24:21.795757 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:24:21.795757 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:24:21.795757 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:24:21.795757 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:21.795757 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:21.795757 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:21.795757 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:21.795757 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:21.795757 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:21.795757 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:21.795757 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:24:21.795757 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:21.795757 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:21.795757 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:21.795757 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:21.796660 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:21.796660 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:21.796660 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:21.796660 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:21.796660 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:24:21.796660 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:24:21.796660 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:24:21.796660 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:24:21.796660 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:24:21.796660 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:24:21.796660 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:24:21.796660 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:24:21.796660 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 241/487] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 14:24:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:24:21.801714 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252549de-ad1a-4c25-940a-b51fee45cdfd None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:21.835380 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-f50873ef-a3ad-4650-8260-bdf233bb6a12 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:21.835380 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:21.835380 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:21.835380 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:21.835380 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:21.835380 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:21.835380 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:21.835380 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:21.835380 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:21.835380 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:21.835380 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:21.835380 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:21.835380 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:24:21.835380 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:24:21.835380 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:24:21.836522 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:24:21.836522 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:21.836522 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:21.836522 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:21.836522 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:21.836522 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:21.836522 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:21.836522 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:21.836522 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:24:21.836522 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:21.836522 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:21.836522 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:21.836522 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:21.836522 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:21.836522 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:21.836522 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:21.836522 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:21.837684 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:24:21.837684 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:24:21.837684 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:21.837684 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:24:21.839585 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 247/488] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:24:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 358 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:24:21.850810 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50873ef-a3ad-4650-8260-bdf233bb6a12 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:21.882206 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:21.915079 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252549de-ad1a-4c25-940a-b51fee45cdfd None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:21.952304 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050178cc-5a84-47ce-bc13-2c594c0d392c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:21.953084 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050178cc-5a84-47ce-bc13-2c594c0d392c None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:21.953440 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050178cc-5a84-47ce-bc13-2c594c0d392c None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:21.953783 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050178cc-5a84-47ce-bc13-2c594c0d392c None admin] PATH_INFO: `/v3/roles` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:21.954636 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050178cc-5a84-47ce-bc13-2c594c0d392c None admin] RBAC: Adding query filter params (name=service) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:21.955181 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050178cc-5a84-47ce-bc13-2c594c0d392c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:21.956717 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050178cc-5a84-47ce-bc13-2c594c0d392c None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:21.966103 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 242/489] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 19 14:24:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:24:21.967555 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bf3086-a552-4828-99ab-4f8cbc051697 None None] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:21.967968 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bf3086-a552-4828-99ab-4f8cbc051697 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:21.968333 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bf3086-a552-4828-99ab-4f8cbc051697 None None] PATH_INFO: `/` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:21.970060 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 243/490] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:24:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:24:21.972001 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bf3086-a552-4828-99ab-4f8cbc051697 None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:22.038223 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50873ef-a3ad-4650-8260-bdf233bb6a12 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:22.082009 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1312578-9627-492c-870a-40b595668cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:22.082921 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1312578-9627-492c-870a-40b595668cb7 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:22.083306 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1312578-9627-492c-870a-40b595668cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:22.083777 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1312578-9627-492c-870a-40b595668cb7 None admin] PATH_INFO: `/v3/domains` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:22.084603 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1312578-9627-492c-870a-40b595668cb7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:22.085039 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1312578-9627-492c-870a-40b595668cb7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:22.086644 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1312578-9627-492c-870a-40b595668cb7 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:22.100897 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 248/491] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:24:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:24:22.102658 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e182d9c8-5757-44cb-9920-1c1c4ad111d5 None None] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:22.103060 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e182d9c8-5757-44cb-9920-1c1c4ad111d5 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:22.103435 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e182d9c8-5757-44cb-9920-1c1c4ad111d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:22.106749 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bf3086-a552-4828-99ab-4f8cbc051697 None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:22.110682 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:22.123077 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.auth.core [None req-e182d9c8-5757-44cb-9920-1c1c4ad111d5 None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143587) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:24:22.124943 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-e182d9c8-5757-44cb-9920-1c1c4ad111d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:24:22.142298 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:22.149028 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96eb2024-6133-47dd-af22-17e7bd75223f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:22.150111 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eb2024-6133-47dd-af22-17e7bd75223f None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:22.150655 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eb2024-6133-47dd-af22-17e7bd75223f None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:22.151146 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eb2024-6133-47dd-af22-17e7bd75223f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:22.163417 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96eb2024-6133-47dd-af22-17e7bd75223f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:22.165155 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96eb2024-6133-47dd-af22-17e7bd75223f None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:22.185526 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-96eb2024-6133-47dd-af22-17e7bd75223f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:22.185526 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:22.185526 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:22.185526 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:22.185526 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:22.185526 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:22.185526 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:22.185526 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:22.185526 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:22.185526 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:22.185526 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:22.185526 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:22.185526 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:24:22.185526 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:24:22.185526 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:24:22.186729 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:24:22.186729 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:22.186729 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:22.186729 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:22.186729 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:22.186729 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:22.186729 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:22.186729 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:22.186729 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:24:22.186729 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:22.186729 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:22.186729 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:22.186729 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:22.186729 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:22.186729 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:22.186729 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:22.186729 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:22.187729 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:24:22.187729 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:24:22.187729 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:22.187729 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:24:22.194359 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 244/492] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:24:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:22.195462 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96eb2024-6133-47dd-af22-17e7bd75223f None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:22.209898 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 249/493] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:24:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:24:22.209898 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e182d9c8-5757-44cb-9920-1c1c4ad111d5 None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:22.310348 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96eb2024-6133-47dd-af22-17e7bd75223f None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:22.318721 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e182d9c8-5757-44cb-9920-1c1c4ad111d5 None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:22.346063 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f310b44-df43-418a-94c3-4876a2aad414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:22.346857 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f310b44-df43-418a-94c3-4876a2aad414 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:22.347141 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f310b44-df43-418a-94c3-4876a2aad414 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:22.347585 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f310b44-df43-418a-94c3-4876a2aad414 None admin] PATH_INFO: `/v3/users/nova` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:22.350850 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:22.356982 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f614453-a69d-49f7-b19f-5b2b5ba91d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:22.357276 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f310b44-df43-418a-94c3-4876a2aad414 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:22.357659 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f614453-a69d-49f7-b19f-5b2b5ba91d87 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:22.357974 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f614453-a69d-49f7-b19f-5b2b5ba91d87 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:22.358254 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f614453-a69d-49f7-b19f-5b2b5ba91d87 None admin] PATH_INFO: `/v3/domains` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:22.358833 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f310b44-df43-418a-94c3-4876a2aad414 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:22.359004 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f614453-a69d-49f7-b19f-5b2b5ba91d87 None admin] RBAC: Adding query filter params (name=Default) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:22.359294 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f614453-a69d-49f7-b19f-5b2b5ba91d87 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:22.360676 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f614453-a69d-49f7-b19f-5b2b5ba91d87 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:22.373230 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 250/494] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:24:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:22.374635 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7276612-7ca0-463c-af20-7b476cd4ccd9 None None] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:22.374928 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7276612-7ca0-463c-af20-7b476cd4ccd9 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:22.375207 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7276612-7ca0-463c-af20-7b476cd4ccd9 None None] PATH_INFO: `/` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:22.377942 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 251/495] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:24:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:24:22.377942 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7276612-7ca0-463c-af20-7b476cd4ccd9 None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:22.380798 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-2f310b44-df43-418a-94c3-4876a2aad414 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 14:24:22.380798 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:22.380798 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:22.380798 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:22.380798 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:22.380798 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:22.380798 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:22.380798 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:22.380798 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:22.380798 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:22.380798 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:22.380798 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:22.380798 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:24:22.380798 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:24:22.380798 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:24:22.380798 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:24:22.381829 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:22.381829 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:22.381829 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:24:22.381829 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:24:22.381829 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:24:22.381829 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:24:22.381829 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:22.381829 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:22.381829 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:22.381829 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:22.381829 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:22.381829 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:24:22.381829 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:22.381829 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:22.381829 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:22.381829 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:22.381829 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:22.381829 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:22.382938 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:22.382938 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:22.382938 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:24:22.382938 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:24:22.382938 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:24:22.382938 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:24:22.382938 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:24:22.382938 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:24:22.382938 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 14:24:22.382938 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:24:22.382938 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 245/496] 10.222.0.57 () {56 vars in 1093 bytes} [Thu May 19 14:24:22 2022] GET /identity/v3/users/nova => generated 82 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:22.382938 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f310b44-df43-418a-94c3-4876a2aad414 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:22.435176 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:22.494554 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f310b44-df43-418a-94c3-4876a2aad414 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:22.495315 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7276612-7ca0-463c-af20-7b476cd4ccd9 None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:22.529804 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3e362b-9234-4b24-a4c5-6b8d6e696446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:22.530273 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f8bcba-fded-4943-b35d-9c998675e47e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:22.530664 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3e362b-9234-4b24-a4c5-6b8d6e696446 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:22.530946 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3e362b-9234-4b24-a4c5-6b8d6e696446 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:22.531051 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f8bcba-fded-4943-b35d-9c998675e47e None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:22.531224 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3e362b-9234-4b24-a4c5-6b8d6e696446 None admin] PATH_INFO: `/v3/roles/service` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:22.531328 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f8bcba-fded-4943-b35d-9c998675e47e None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:22.531581 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f8bcba-fded-4943-b35d-9c998675e47e None admin] PATH_INFO: `/v3/users/glance` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:22.540388 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f8bcba-fded-4943-b35d-9c998675e47e None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:22.541895 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f8bcba-fded-4943-b35d-9c998675e47e None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:22.544159 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3e362b-9234-4b24-a4c5-6b8d6e696446 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:24:22.552226 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-5d3e362b-9234-4b24-a4c5-6b8d6e696446 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:24:22.552226 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:22.552226 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:22.552226 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:22.552226 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:22.552226 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:22.552226 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:22.552226 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:22.552226 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:22.552226 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:22.552226 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:22.552226 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:22.552226 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:24:22.552226 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:24:22.552226 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:24:22.552226 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:24:22.553346 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:24:22.553346 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:24:22.553346 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:24:22.553346 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:24:22.553346 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:22.553346 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:22.553346 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:22.553346 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:22.553346 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:22.553346 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:22.553346 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:22.553346 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:24:22.553346 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:22.553346 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:22.553346 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:22.553346 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:22.554357 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:22.554357 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:22.554357 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:22.554357 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:22.554357 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:24:22.554357 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:24:22.554357 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:24:22.554357 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:24:22.554357 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:24:22.554357 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:24:22.554357 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:24:22.554357 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:24:22.561486 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 252/497] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 14:24:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:22.561655 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-b0f8bcba-fded-4943-b35d-9c998675e47e None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 19 14:24:22.561655 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:22.561655 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:22.561655 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:22.561655 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:22.561655 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:22.561655 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:22.561655 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:22.561655 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:22.561655 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:22.561655 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:22.561655 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:22.561655 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:24:22.561655 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:24:22.561655 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:24:22.561655 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:24:22.562646 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:22.562646 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:22.562646 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:24:22.562646 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:24:22.562646 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:24:22.562646 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:24:22.562646 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:22.562646 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:22.562646 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:22.562646 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:22.562646 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:22.562646 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:24:22.562646 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:22.562646 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:22.562646 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:22.562646 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:22.562646 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:22.562646 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:22.563699 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:22.563699 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:22.563699 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:24:22.563699 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:24:22.563699 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:24:22.563699 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:24:22.563699 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:24:22.563699 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:24:22.563699 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 19 14:24:22.563699 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:24:22.564307 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469b4fd4-e1bd-4d04-b00c-093fe4a4fc18 None None] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:22.564307 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469b4fd4-e1bd-4d04-b00c-093fe4a4fc18 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:22.564461 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 246/498] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:24:22 2022] GET /identity/v3/users/glance => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:22.564537 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469b4fd4-e1bd-4d04-b00c-093fe4a4fc18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:22.564625 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f8bcba-fded-4943-b35d-9c998675e47e None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:22.584779 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.auth.core [None req-469b4fd4-e1bd-4d04-b00c-093fe4a4fc18 None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143587) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:24:22.586038 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-469b4fd4-e1bd-4d04-b00c-093fe4a4fc18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:24:22.638670 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:22.662774 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 253/499] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:24:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:24:22.663891 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469b4fd4-e1bd-4d04-b00c-093fe4a4fc18 None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:22.681346 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f8bcba-fded-4943-b35d-9c998675e47e None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:22.684686 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:22.722438 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08ed94c-c43b-406f-b155-eead3e469f7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:22.723301 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08ed94c-c43b-406f-b155-eead3e469f7c None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:22.724248 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08ed94c-c43b-406f-b155-eead3e469f7c None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:22.724539 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08ed94c-c43b-406f-b155-eead3e469f7c None admin] PATH_INFO: `/v3/users` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:22.725549 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08ed94c-c43b-406f-b155-eead3e469f7c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:22.725886 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08ed94c-c43b-406f-b155-eead3e469f7c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:22.727338 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08ed94c-c43b-406f-b155-eead3e469f7c None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:22.765166 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 247/500] 10.222.0.57 () {56 vars in 1133 bytes} [Thu May 19 14:24:22 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:22.766055 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08ed94c-c43b-406f-b155-eead3e469f7c None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:22.781253 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469b4fd4-e1bd-4d04-b00c-093fe4a4fc18 None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:22.816396 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03721f6-1d7f-4003-8438-04d7f4ac7489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:22.816987 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03721f6-1d7f-4003-8438-04d7f4ac7489 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:22.817270 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03721f6-1d7f-4003-8438-04d7f4ac7489 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:22.817550 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03721f6-1d7f-4003-8438-04d7f4ac7489 None admin] PATH_INFO: `/v3/roles` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:22.818331 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03721f6-1d7f-4003-8438-04d7f4ac7489 None admin] RBAC: Adding query filter params (name=service) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:22.818644 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03721f6-1d7f-4003-8438-04d7f4ac7489 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:22.820018 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03721f6-1d7f-4003-8438-04d7f4ac7489 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:22.828838 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 254/501] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 19 14:24:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:22.829636 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03721f6-1d7f-4003-8438-04d7f4ac7489 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:22.869743 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08ed94c-c43b-406f-b155-eead3e469f7c None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:22.875235 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:22.908831 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5622af7d-998d-44e5-999f-d7cfd29a0b46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:22.909501 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5622af7d-998d-44e5-999f-d7cfd29a0b46 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:22.909792 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5622af7d-998d-44e5-999f-d7cfd29a0b46 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:22.910102 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5622af7d-998d-44e5-999f-d7cfd29a0b46 None admin] PATH_INFO: `/v3/users` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:22.910944 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5622af7d-998d-44e5-999f-d7cfd29a0b46 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:22.911256 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5622af7d-998d-44e5-999f-d7cfd29a0b46 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:22.912697 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5622af7d-998d-44e5-999f-d7cfd29a0b46 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:22.929462 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 248/502] 10.222.0.57 () {56 vars in 1137 bytes} [Thu May 19 14:24:22 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:22.931023 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77694a1-2696-414a-b49e-7e0d4d06d6f3 None None] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:22.931318 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77694a1-2696-414a-b49e-7e0d4d06d6f3 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:22.931592 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77694a1-2696-414a-b49e-7e0d4d06d6f3 None None] PATH_INFO: `/` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:22.936943 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 249/503] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:24:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:24:22.936943 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77694a1-2696-414a-b49e-7e0d4d06d6f3 None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:22.944365 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03721f6-1d7f-4003-8438-04d7f4ac7489 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:22.956294 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:22.980227 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b60ffe-f67b-4412-8263-6c8aac89b77e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:22.981049 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b60ffe-f67b-4412-8263-6c8aac89b77e None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:22.981325 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b60ffe-f67b-4412-8263-6c8aac89b77e None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:22.981692 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b60ffe-f67b-4412-8263-6c8aac89b77e None admin] PATH_INFO: `/v3/roles/service` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:22.991801 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b60ffe-f67b-4412-8263-6c8aac89b77e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:24:22.999485 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-c0b60ffe-f67b-4412-8263-6c8aac89b77e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:24:22.999485 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:22.999485 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:22.999485 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:22.999485 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:22.999485 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:22.999485 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:22.999485 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:22.999485 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:22.999485 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:22.999485 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:22.999485 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:22.999485 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:24:22.999485 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:24:22.999485 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:24:22.999485 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:24:23.000663 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:24:23.000663 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:24:23.000663 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:24:23.000663 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:24:23.000663 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:23.000663 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:23.000663 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:23.000663 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:23.000663 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:23.000663 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:23.000663 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:23.000663 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:24:23.000663 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:23.000663 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:23.000663 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:23.000663 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:23.001646 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:23.001646 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:23.001646 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:23.001646 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:23.001646 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:24:23.001646 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:24:23.001646 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:24:23.001646 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:24:23.001646 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:24:23.001646 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:24:23.001646 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:24:23.001646 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:24:23.001646 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 255/504] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 14:24:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:23.002439 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b60ffe-f67b-4412-8263-6c8aac89b77e None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:23.040219 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77694a1-2696-414a-b49e-7e0d4d06d6f3 None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:23.075284 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a467fdb0-e2dc-4458-bb64-de2f10a47a5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:23.076096 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a467fdb0-e2dc-4458-bb64-de2f10a47a5a None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:23.076396 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a467fdb0-e2dc-4458-bb64-de2f10a47a5a None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:23.076669 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a467fdb0-e2dc-4458-bb64-de2f10a47a5a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:23.101318 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b60ffe-f67b-4412-8263-6c8aac89b77e None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:23.103630 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:23.134658 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db412c88-062c-4fd2-a6c6-74f4494334a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:23.135450 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db412c88-062c-4fd2-a6c6-74f4494334a9 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:23.135734 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db412c88-062c-4fd2-a6c6-74f4494334a9 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:23.136014 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db412c88-062c-4fd2-a6c6-74f4494334a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:23.145183 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a467fdb0-e2dc-4458-bb64-de2f10a47a5a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:23.146692 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a467fdb0-e2dc-4458-bb64-de2f10a47a5a None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:23.147157 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db412c88-062c-4fd2-a6c6-74f4494334a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:23.148592 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db412c88-062c-4fd2-a6c6-74f4494334a9 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:23.166161 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-db412c88-062c-4fd2-a6c6-74f4494334a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:23.166161 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:23.166161 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:23.166161 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:23.166161 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:23.166161 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:23.166161 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:23.166161 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:23.166161 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:23.166161 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:23.166161 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:23.166161 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:23.166161 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:24:23.166161 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:24:23.166161 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:24:23.167333 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:24:23.167333 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:23.167333 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:23.167333 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:23.167333 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:23.167333 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:23.167333 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:23.167333 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:23.167333 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:24:23.167333 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:23.167333 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:23.167333 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:23.167333 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:23.167333 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:23.167333 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:23.167333 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:23.167333 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:23.168322 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:24:23.168322 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:24:23.168322 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:23.168322 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:24:23.168322 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 256/505] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:24:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:23.168322 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db412c88-062c-4fd2-a6c6-74f4494334a9 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:23.172957 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:23.252697 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 250/506] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:24:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:24:23.254065 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2374136-d5a9-44fb-aded-6a763b8976ac None None] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:23.254363 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2374136-d5a9-44fb-aded-6a763b8976ac None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:23.254639 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2374136-d5a9-44fb-aded-6a763b8976ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:23.271659 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.auth.core [None req-c2374136-d5a9-44fb-aded-6a763b8976ac None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143586) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:24:23.272558 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db412c88-062c-4fd2-a6c6-74f4494334a9 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:23.272855 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.fernet_utils [None req-c2374136-d5a9-44fb-aded-6a763b8976ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143586) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:24:23.306283 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b977bf-08f3-45f5-8b74-c46002e2781c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:23.306928 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b977bf-08f3-45f5-8b74-c46002e2781c None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:23.307206 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b977bf-08f3-45f5-8b74-c46002e2781c None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:23.307487 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b977bf-08f3-45f5-8b74-c46002e2781c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:23.332906 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:23.338490 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 251/507] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:24:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:24:23.339330 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2374136-d5a9-44fb-aded-6a763b8976ac None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:23.379360 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b977bf-08f3-45f5-8b74-c46002e2781c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:23.380931 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b977bf-08f3-45f5-8b74-c46002e2781c None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:23.404973 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:23.455951 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2374136-d5a9-44fb-aded-6a763b8976ac None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:23.482824 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 257/508] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:24:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:24:23.483882 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b977bf-08f3-45f5-8b74-c46002e2781c None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:23.490937 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4223f1-17f9-4c46-8efc-aca6bfeeda5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:23.491672 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4223f1-17f9-4c46-8efc-aca6bfeeda5e None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:23.491962 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4223f1-17f9-4c46-8efc-aca6bfeeda5e None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:23.492237 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4223f1-17f9-4c46-8efc-aca6bfeeda5e None admin] PATH_INFO: `/v3/roles` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:23.492984 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4223f1-17f9-4c46-8efc-aca6bfeeda5e None admin] RBAC: Adding query filter params (name=service) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:23.493286 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4223f1-17f9-4c46-8efc-aca6bfeeda5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:23.494680 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4223f1-17f9-4c46-8efc-aca6bfeeda5e None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:23.502713 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 252/509] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 19 14:24:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:23.503491 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4223f1-17f9-4c46-8efc-aca6bfeeda5e None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:23.542122 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:23.587112 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b977bf-08f3-45f5-8b74-c46002e2781c None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:23.603286 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4223f1-17f9-4c46-8efc-aca6bfeeda5e None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:23.616241 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:23.622408 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91719730-0834-4f12-a1b8-2e8f77e9ba8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:23.623072 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91719730-0834-4f12-a1b8-2e8f77e9ba8a None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:23.623350 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91719730-0834-4f12-a1b8-2e8f77e9ba8a None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:23.624051 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91719730-0834-4f12-a1b8-2e8f77e9ba8a None admin] PATH_INFO: `/v3/domains` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:23.624777 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91719730-0834-4f12-a1b8-2e8f77e9ba8a None admin] RBAC: Adding query filter params (name=Default) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:23.625086 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91719730-0834-4f12-a1b8-2e8f77e9ba8a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:23.626486 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91719730-0834-4f12-a1b8-2e8f77e9ba8a None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:23.637809 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d49bc74-4458-44ea-a11d-bb04ef937030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:23.638532 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 258/510] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:24:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:23.638786 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d49bc74-4458-44ea-a11d-bb04ef937030 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:23.638956 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d49bc74-4458-44ea-a11d-bb04ef937030 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:23.639175 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91719730-0834-4f12-a1b8-2e8f77e9ba8a None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:23.639268 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d49bc74-4458-44ea-a11d-bb04ef937030 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:23.655383 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d49bc74-4458-44ea-a11d-bb04ef937030 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:23.657481 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d49bc74-4458-44ea-a11d-bb04ef937030 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:23.675162 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-0d49bc74-4458-44ea-a11d-bb04ef937030 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:23.675162 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:23.675162 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:23.675162 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:23.675162 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:23.675162 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:23.675162 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:23.675162 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:23.675162 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:23.675162 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:23.675162 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:23.675162 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:23.675162 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:24:23.675162 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:24:23.675162 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:24:23.676429 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:24:23.676429 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:23.676429 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:23.676429 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:23.676429 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:23.676429 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:23.676429 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:23.676429 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:23.676429 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:24:23.676429 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:23.676429 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:23.676429 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:23.676429 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:23.676429 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:23.676429 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:23.676429 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:23.676429 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:23.677447 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:24:23.677447 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:24:23.677447 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:23.677447 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:24:23.677447 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 253/511] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:24:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:23.677447 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d49bc74-4458-44ea-a11d-bb04ef937030 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:23.752466 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91719730-0834-4f12-a1b8-2e8f77e9ba8a None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:23.768135 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:23.778427 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d49bc74-4458-44ea-a11d-bb04ef937030 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:23.786706 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07acad06-bd08-4faa-bd65-277abe8931b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:23.787497 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07acad06-bd08-4faa-bd65-277abe8931b0 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:23.787777 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07acad06-bd08-4faa-bd65-277abe8931b0 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:23.788053 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07acad06-bd08-4faa-bd65-277abe8931b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:23.799373 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07acad06-bd08-4faa-bd65-277abe8931b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:23.800824 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07acad06-bd08-4faa-bd65-277abe8931b0 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:23.812361 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94ab990-8e53-4ec8-8939-9ff424b07add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:23.813153 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94ab990-8e53-4ec8-8939-9ff424b07add None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:23.813443 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94ab990-8e53-4ec8-8939-9ff424b07add None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:23.813724 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94ab990-8e53-4ec8-8939-9ff424b07add None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:23.818597 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-07acad06-bd08-4faa-bd65-277abe8931b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:23.818597 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:23.818597 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:23.818597 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:23.818597 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:23.818597 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:23.818597 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:23.818597 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:23.818597 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:23.818597 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:23.818597 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:23.818597 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:23.818597 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:24:23.818597 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:24:23.818597 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:24:23.819695 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:24:23.819695 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:23.819695 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:23.819695 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:23.819695 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:23.819695 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:23.819695 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:23.819695 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:23.819695 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:24:23.819695 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:23.819695 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:23.819695 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:23.819695 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:23.819695 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:23.819695 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:23.819695 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:23.819695 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:23.820699 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:24:23.820699 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:24:23.820699 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:23.820699 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:24:23.822135 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 259/512] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:24:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:23.822830 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07acad06-bd08-4faa-bd65-277abe8931b0 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:23.825588 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94ab990-8e53-4ec8-8939-9ff424b07add None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:23.827074 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94ab990-8e53-4ec8-8939-9ff424b07add None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:23.844235 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-f94ab990-8e53-4ec8-8939-9ff424b07add None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:23.844235 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:23.844235 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:23.844235 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:23.844235 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:23.844235 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:23.844235 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:23.844235 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:23.844235 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:23.844235 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:23.844235 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:23.844235 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:23.844235 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:24:23.844235 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:24:23.844235 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:24:23.845263 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:24:23.845263 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:23.845263 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:23.845263 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:23.845263 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:23.845263 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:23.845263 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:23.845263 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:23.845263 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:24:23.845263 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:23.845263 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:23.845263 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:23.845263 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:23.845263 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:23.845263 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:23.845263 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:23.845263 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:23.846299 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:24:23.846299 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:24:23.846299 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:23.846299 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:24:23.846299 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 254/513] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:24:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:23.846299 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94ab990-8e53-4ec8-8939-9ff424b07add None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:23.864855 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:23.931826 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07acad06-bd08-4faa-bd65-277abe8931b0 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:23.946721 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94ab990-8e53-4ec8-8939-9ff424b07add None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:23.964740 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73409a9-030c-43b6-9f92-48ed4efcded3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:23.966194 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73409a9-030c-43b6-9f92-48ed4efcded3 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:23.966477 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73409a9-030c-43b6-9f92-48ed4efcded3 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:23.966752 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73409a9-030c-43b6-9f92-48ed4efcded3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:23.977336 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73409a9-030c-43b6-9f92-48ed4efcded3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:23.978810 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73409a9-030c-43b6-9f92-48ed4efcded3 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:23.979627 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4ae881-09f5-4619-a85e-8244609f4a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:23.980413 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4ae881-09f5-4619-a85e-8244609f4a17 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:23.980693 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4ae881-09f5-4619-a85e-8244609f4a17 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:23.981585 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4ae881-09f5-4619-a85e-8244609f4a17 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:23.989332 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4ae881-09f5-4619-a85e-8244609f4a17 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:23.990783 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4ae881-09f5-4619-a85e-8244609f4a17 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:23.996260 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-d73409a9-030c-43b6-9f92-48ed4efcded3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:23.996260 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:23.996260 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:23.996260 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:23.996260 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:23.996260 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:23.996260 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:23.996260 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:23.996260 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:23.996260 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:23.996260 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:23.996260 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:23.996260 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:24:23.996260 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:24:23.996260 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:24:23.997278 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:24:23.997278 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:23.997278 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:23.997278 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:23.997278 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:23.997278 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:23.997278 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:23.997278 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:23.997278 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:24:23.997278 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:23.997278 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:23.997278 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:23.997278 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:23.997278 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:23.997278 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:23.997278 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:23.997278 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:23.998291 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:24:23.998291 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:24:23.998291 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:23.998291 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:24:23.998291 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 260/514] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:24:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:23.998609 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73409a9-030c-43b6-9f92-48ed4efcded3 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:24.004747 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-6f4ae881-09f5-4619-a85e-8244609f4a17 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 14:24:24.004747 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:24.004747 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:24.004747 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:24.004747 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:24.004747 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:24.004747 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:24.004747 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:24.004747 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:24.004747 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:24.004747 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:24.004747 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:24.004747 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:24:24.004747 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:24:24.004747 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:24:24.004747 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:24:24.005835 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:24.005835 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:24.005835 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:24:24.005835 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:24:24.005835 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:24:24.005835 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:24:24.005835 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:24.005835 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:24.005835 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:24.005835 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:24.005835 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:24.005835 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:24:24.005835 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:24.005835 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:24.005835 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:24.005835 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:24.005835 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:24.005835 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:24.007079 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:24.007079 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:24.007079 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:24:24.007079 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:24:24.007079 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:24:24.007079 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:24:24.007079 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:24:24.007079 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:24:24.007079 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 14:24:24.007079 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:24:24.007079 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 255/515] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:24:23 2022] GET /identity/v3/users/cinder => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:24.007079 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4ae881-09f5-4619-a85e-8244609f4a17 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:24.026756 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:24.083677 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:24.106942 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73409a9-030c-43b6-9f92-48ed4efcded3 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:24.108303 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4ae881-09f5-4619-a85e-8244609f4a17 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:24.141560 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128193c0-9b8d-4839-a1b3-a2a6116f0390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:24.141782 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d3b471-808f-43dc-a792-d4fa3cd4e77a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:24.142265 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128193c0-9b8d-4839-a1b3-a2a6116f0390 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:24.142462 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d3b471-808f-43dc-a792-d4fa3cd4e77a None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:24.142576 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128193c0-9b8d-4839-a1b3-a2a6116f0390 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:24.142748 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d3b471-808f-43dc-a792-d4fa3cd4e77a None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:24.142845 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128193c0-9b8d-4839-a1b3-a2a6116f0390 None admin] PATH_INFO: `/v3/domains` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:24.143028 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d3b471-808f-43dc-a792-d4fa3cd4e77a None admin] PATH_INFO: `/v3/domains` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:24.143529 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128193c0-9b8d-4839-a1b3-a2a6116f0390 None admin] RBAC: Adding query filter params (name=Default) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:24.143750 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d3b471-808f-43dc-a792-d4fa3cd4e77a None admin] RBAC: Adding query filter params (name=Default) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:24.143857 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128193c0-9b8d-4839-a1b3-a2a6116f0390 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:24.144062 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d3b471-808f-43dc-a792-d4fa3cd4e77a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:24.145219 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128193c0-9b8d-4839-a1b3-a2a6116f0390 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:24.145415 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d3b471-808f-43dc-a792-d4fa3cd4e77a None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:24.162648 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 256/516] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:24:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:24.162834 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 261/517] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:24:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:24.162966 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d3b471-808f-43dc-a792-d4fa3cd4e77a None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:24.163060 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128193c0-9b8d-4839-a1b3-a2a6116f0390 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:24.240603 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:24.260969 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128193c0-9b8d-4839-a1b3-a2a6116f0390 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:24.263125 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d3b471-808f-43dc-a792-d4fa3cd4e77a None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:24.297263 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0458f9cd-55ea-420d-80ef-0ee0f579ba02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:24.297945 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0458f9cd-55ea-420d-80ef-0ee0f579ba02 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:24.298236 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0458f9cd-55ea-420d-80ef-0ee0f579ba02 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:24.298508 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0458f9cd-55ea-420d-80ef-0ee0f579ba02 None admin] PATH_INFO: `/v3/domains` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:24.299228 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0458f9cd-55ea-420d-80ef-0ee0f579ba02 None admin] RBAC: Adding query filter params (name=Default) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:24.299539 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0458f9cd-55ea-420d-80ef-0ee0f579ba02 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:24.299756 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03cb5a8-8efb-439c-add7-4807b633fcea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:24.300425 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03cb5a8-8efb-439c-add7-4807b633fcea None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:24.300715 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03cb5a8-8efb-439c-add7-4807b633fcea None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:24.300916 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0458f9cd-55ea-420d-80ef-0ee0f579ba02 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:24.301030 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03cb5a8-8efb-439c-add7-4807b633fcea None admin] PATH_INFO: `/v3/domains` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:24.301718 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03cb5a8-8efb-439c-add7-4807b633fcea None admin] RBAC: Adding query filter params (name=Default) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:24.302065 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03cb5a8-8efb-439c-add7-4807b633fcea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:24.304074 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03cb5a8-8efb-439c-add7-4807b633fcea None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:24.306442 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:24.312437 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 262/518] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:24:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:24.313275 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0458f9cd-55ea-420d-80ef-0ee0f579ba02 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:24.322526 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 257/519] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:24:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:24.323385 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03cb5a8-8efb-439c-add7-4807b633fcea None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:24.416153 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0458f9cd-55ea-420d-80ef-0ee0f579ba02 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:24.425329 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03cb5a8-8efb-439c-add7-4807b633fcea None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:24.449476 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05531ba-0c90-4de9-93c5-d4653c612523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:24.450818 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05531ba-0c90-4de9-93c5-d4653c612523 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:24.451097 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05531ba-0c90-4de9-93c5-d4653c612523 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:24.451380 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05531ba-0c90-4de9-93c5-d4653c612523 None admin] PATH_INFO: `/v3/users` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:24.452218 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05531ba-0c90-4de9-93c5-d4653c612523 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:24.452533 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05531ba-0c90-4de9-93c5-d4653c612523 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:24.453996 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05531ba-0c90-4de9-93c5-d4653c612523 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:24.458279 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e56a311-5ae3-4d1c-b26e-8299c47d0939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:24.459070 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e56a311-5ae3-4d1c-b26e-8299c47d0939 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:24.459356 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e56a311-5ae3-4d1c-b26e-8299c47d0939 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:24.459633 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e56a311-5ae3-4d1c-b26e-8299c47d0939 None admin] PATH_INFO: `/v3/projects/service` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:24.466838 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e56a311-5ae3-4d1c-b26e-8299c47d0939 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:24.468278 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e56a311-5ae3-4d1c-b26e-8299c47d0939 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:24.479013 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:24.484158 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-4e56a311-5ae3-4d1c-b26e-8299c47d0939 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:24:24.484158 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:24.484158 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:24.484158 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:24.484158 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:24.484158 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:24.484158 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:24.484158 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:24.484158 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:24.484158 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:24.484158 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:24.484158 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:24.484158 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:24:24.484158 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:24:24.484158 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:24:24.485359 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:24:24.485359 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:24.485359 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:24.485359 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:24.485359 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:24.485359 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:24.485359 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:24.485359 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:24.485359 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:24:24.485359 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:24.485359 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:24.485359 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:24.485359 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:24.485359 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:24.485359 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:24.485359 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:24.485359 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:24.486587 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:24:24.486587 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:24:24.486587 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:24:24.486587 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:24:24.486587 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:24:24.486587 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:24:24.486587 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:24:24.486587 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:24:24.486587 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 258/520] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:24:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:24.486587 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e56a311-5ae3-4d1c-b26e-8299c47d0939 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:24.494434 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 263/521] 10.222.0.57 () {56 vars in 1137 bytes} [Thu May 19 14:24:24 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:24.495656 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05531ba-0c90-4de9-93c5-d4653c612523 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:24.544953 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:24.589088 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e56a311-5ae3-4d1c-b26e-8299c47d0939 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:24.595717 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05531ba-0c90-4de9-93c5-d4653c612523 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:24.625575 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884cf59c-33c6-4ae0-a470-c3303298a6a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:24.626322 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884cf59c-33c6-4ae0-a470-c3303298a6a3 None admin] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:24.626604 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884cf59c-33c6-4ae0-a470-c3303298a6a3 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:24.626877 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884cf59c-33c6-4ae0-a470-c3303298a6a3 None admin] PATH_INFO: `/v3/users` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:24.627573 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884cf59c-33c6-4ae0-a470-c3303298a6a3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:24.629004 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884cf59c-33c6-4ae0-a470-c3303298a6a3 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:24.629592 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c7c828-bcc6-4427-a129-fb9cef394fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:24.630423 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c7c828-bcc6-4427-a129-fb9cef394fd4 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:24.630708 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c7c828-bcc6-4427-a129-fb9cef394fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:24.630987 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c7c828-bcc6-4427-a129-fb9cef394fd4 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:24.638623 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c7c828-bcc6-4427-a129-fb9cef394fd4 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:24.640073 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c7c828-bcc6-4427-a129-fb9cef394fd4 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:24.649041 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 259/522] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 19 14:24:24 2022] POST /identity/v3/users => generated 247 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:24:24.649876 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884cf59c-33c6-4ae0-a470-c3303298a6a3 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:24.657448 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-a7c7c828-bcc6-4427-a129-fb9cef394fd4 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 19 14:24:24.657448 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:24.657448 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:24.657448 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:24.657448 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:24.657448 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:24.657448 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:24.657448 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:24.657448 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:24.657448 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:24.657448 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:24.657448 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:24.657448 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:24:24.657448 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:24:24.657448 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:24:24.657448 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:24:24.658605 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:24.658605 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:24.658605 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:24:24.658605 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:24:24.658605 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:24:24.658605 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:24:24.658605 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:24.658605 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:24.658605 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:24.658605 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:24.658605 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:24.658605 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:24:24.658605 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:24.658605 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:24.658605 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:24.658605 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:24.658605 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:24.658605 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:24.659678 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:24.659678 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:24.659678 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:24:24.659678 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:24:24.659678 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:24:24.659678 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:24:24.659678 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:24:24.659678 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:24:24.659678 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 19 14:24:24.659678 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:24:24.659678 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 264/523] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 14:24:24 2022] GET /identity/v3/users/neutron => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:24.659678 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c7c828-bcc6-4427-a129-fb9cef394fd4 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:24.743733 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:24.778703 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884cf59c-33c6-4ae0-a470-c3303298a6a3 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:24.793348 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c7c828-bcc6-4427-a129-fb9cef394fd4 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:24.794941 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:24.826621 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4b2f18-4fc2-4f1b-9327-28c7858b766b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:24.827432 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4b2f18-4fc2-4f1b-9327-28c7858b766b None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:24.827720 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4b2f18-4fc2-4f1b-9327-28c7858b766b None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:24.827993 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4b2f18-4fc2-4f1b-9327-28c7858b766b None admin] PATH_INFO: `/v3/projects/service` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:24.835431 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ac26f5-b67f-419a-a8a9-ce15397b43c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:24.836304 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ac26f5-b67f-419a-a8a9-ce15397b43c1 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:24.836304 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ac26f5-b67f-419a-a8a9-ce15397b43c1 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:24.836506 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4b2f18-4fc2-4f1b-9327-28c7858b766b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:24.836589 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ac26f5-b67f-419a-a8a9-ce15397b43c1 None admin] PATH_INFO: `/v3/projects` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:24.837868 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ac26f5-b67f-419a-a8a9-ce15397b43c1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:24.837868 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ac26f5-b67f-419a-a8a9-ce15397b43c1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:24.838088 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4b2f18-4fc2-4f1b-9327-28c7858b766b None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:24.839126 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ac26f5-b67f-419a-a8a9-ce15397b43c1 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:24.855036 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-8f4b2f18-4fc2-4f1b-9327-28c7858b766b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:24:24.855036 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:24.855036 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:24.855036 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:24.855036 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:24.855036 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:24.855036 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:24.855036 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:24.855036 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:24.855036 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:24.855036 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:24.855036 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:24.855036 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:24:24.855036 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:24:24.855036 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:24:24.856120 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:24:24.856120 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:24.856120 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:24.856120 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:24.856120 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:24.856120 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:24.856120 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:24.856120 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:24.856120 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:24:24.856120 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:24.856120 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:24.856120 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:24.856120 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:24.856120 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:24.856120 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:24.856120 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:24.856120 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:24.857118 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:24:24.857118 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:24:24.857118 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:24:24.857118 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:24:24.857118 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:24:24.857118 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:24:24.857118 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:24:24.857118 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:24:24.857118 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 260/524] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:24:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:24.858751 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4b2f18-4fc2-4f1b-9327-28c7858b766b None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:24.860064 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 265/525] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:24:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:24.860913 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ac26f5-b67f-419a-a8a9-ce15397b43c1 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:24.972332 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4b2f18-4fc2-4f1b-9327-28c7858b766b None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:24.975105 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ac26f5-b67f-419a-a8a9-ce15397b43c1 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:24.992546 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:25.010191 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31afb40-56f2-4fd0-b06e-5df5051426e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:25.010845 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31afb40-56f2-4fd0-b06e-5df5051426e5 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:25.011133 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31afb40-56f2-4fd0-b06e-5df5051426e5 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:25.011397 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31afb40-56f2-4fd0-b06e-5df5051426e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:25.014746 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca46d0d-26c4-468c-b24b-c1ac68c8760a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:25.016461 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca46d0d-26c4-468c-b24b-c1ac68c8760a None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:25.016939 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca46d0d-26c4-468c-b24b-c1ac68c8760a None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:25.017417 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca46d0d-26c4-468c-b24b-c1ac68c8760a None admin] PATH_INFO: `/v3/users` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:25.018481 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca46d0d-26c4-468c-b24b-c1ac68c8760a None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:25.019012 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca46d0d-26c4-468c-b24b-c1ac68c8760a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:25.021077 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca46d0d-26c4-468c-b24b-c1ac68c8760a None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:25.047111 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 266/526] 10.222.0.57 () {56 vars in 1139 bytes} [Thu May 19 14:24:24 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:25.048482 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca46d0d-26c4-468c-b24b-c1ac68c8760a None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:25.055595 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:25.087279 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31afb40-56f2-4fd0-b06e-5df5051426e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:25.089169 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31afb40-56f2-4fd0-b06e-5df5051426e5 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:25.156324 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca46d0d-26c4-468c-b24b-c1ac68c8760a None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:25.192985 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a9bcdd-834f-4e48-ac54-4ee1262c0117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:25.193651 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a9bcdd-834f-4e48-ac54-4ee1262c0117 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:25.193963 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a9bcdd-834f-4e48-ac54-4ee1262c0117 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:25.194243 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a9bcdd-834f-4e48-ac54-4ee1262c0117 None admin] PATH_INFO: `/v3/projects` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:25.195033 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a9bcdd-834f-4e48-ac54-4ee1262c0117 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:25.195346 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a9bcdd-834f-4e48-ac54-4ee1262c0117 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:25.196840 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a9bcdd-834f-4e48-ac54-4ee1262c0117 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:25.200330 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 261/527] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:24:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:24:25.201392 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31afb40-56f2-4fd0-b06e-5df5051426e5 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:25.210983 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 267/528] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:24:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:25.212052 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a9bcdd-834f-4e48-ac54-4ee1262c0117 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:25.251259 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:25.279373 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:25.313771 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31afb40-56f2-4fd0-b06e-5df5051426e5 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:25.327364 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a9bcdd-834f-4e48-ac54-4ee1262c0117 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:25.351334 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ca2699-6962-4aac-ad6c-2bbaa19f177b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:25.352001 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ca2699-6962-4aac-ad6c-2bbaa19f177b None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:25.352279 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ca2699-6962-4aac-ad6c-2bbaa19f177b None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:25.352560 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ca2699-6962-4aac-ad6c-2bbaa19f177b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:25.353389 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ca2699-6962-4aac-ad6c-2bbaa19f177b None admin] RBAC: Adding query filter params (role.id=b0b74fefb27e4a0ebd895893aa9862d7, scope.project.id=252d9673f08c479a98d04fee3f5856a4, user.id=5f28de22c2ee4c53b6d14d5b0ddf7fe9) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:25.353704 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ca2699-6962-4aac-ad6c-2bbaa19f177b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:25.355178 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ca2699-6962-4aac-ad6c-2bbaa19f177b None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:25.361513 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 262/529] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:24:25 2022] GET /identity/v3/role_assignments?user.id=5f28de22c2ee4c53b6d14d5b0ddf7fe9&scope.project.id=252d9673f08c479a98d04fee3f5856a4&role.id=b0b74fefb27e4a0ebd895893aa9862d7 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:25.363289 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4d3e16-2c9b-4449-86b0-fa85e7d23e6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:25.363945 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4d3e16-2c9b-4449-86b0-fa85e7d23e6d None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:25.364230 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4d3e16-2c9b-4449-86b0-fa85e7d23e6d None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:25.364508 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4d3e16-2c9b-4449-86b0-fa85e7d23e6d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:25.370920 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ca2699-6962-4aac-ad6c-2bbaa19f177b None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:25.462166 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4d3e16-2c9b-4449-86b0-fa85e7d23e6d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:25.464216 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4d3e16-2c9b-4449-86b0-fa85e7d23e6d None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:25.490567 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ca2699-6962-4aac-ad6c-2bbaa19f177b None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:25.507390 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:25.530404 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92927625-878d-479e-8094-6ed14ed3b2e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:25.531251 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92927625-878d-479e-8094-6ed14ed3b2e4 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:25.531540 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92927625-878d-479e-8094-6ed14ed3b2e4 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:25.531825 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92927625-878d-479e-8094-6ed14ed3b2e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:25.535468 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:25.545086 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92927625-878d-479e-8094-6ed14ed3b2e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:25.547166 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92927625-878d-479e-8094-6ed14ed3b2e4 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:25.568236 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-92927625-878d-479e-8094-6ed14ed3b2e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:25.568236 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:25.568236 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:25.568236 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:25.568236 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:25.568236 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:25.568236 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:25.568236 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:25.568236 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:25.568236 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:25.568236 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:25.568236 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:25.568236 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:24:25.568236 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:24:25.568236 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:24:25.569484 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:24:25.569484 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:25.569484 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:25.569484 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:25.569484 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:25.569484 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:25.569484 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:25.569484 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:25.569484 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:24:25.569484 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:25.569484 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:25.569484 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:25.569484 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:25.569484 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:25.569484 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:25.569484 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:25.569484 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:25.570741 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:24:25.570741 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:24:25.570741 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:25.570741 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:24:25.576422 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 263/530] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:24:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:25.576422 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92927625-878d-479e-8094-6ed14ed3b2e4 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:25.610498 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 268/531] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:24:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:24:25.610498 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4d3e16-2c9b-4449-86b0-fa85e7d23e6d None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:25.716529 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92927625-878d-479e-8094-6ed14ed3b2e4 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:25.729458 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4d3e16-2c9b-4449-86b0-fa85e7d23e6d None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:25.760236 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19199bc9-6506-4750-ad5d-174e4899ac75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:25.760817 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19199bc9-6506-4750-ad5d-174e4899ac75 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:25.761097 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19199bc9-6506-4750-ad5d-174e4899ac75 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:25.761375 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19199bc9-6506-4750-ad5d-174e4899ac75 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:25.762527 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19199bc9-6506-4750-ad5d-174e4899ac75 None admin] RBAC: Adding query filter params (role.id=b0b74fefb27e4a0ebd895893aa9862d7, scope.project.id=252d9673f08c479a98d04fee3f5856a4, user.id=afec042821074a889ef9afa75bb8bf40) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:25.762840 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19199bc9-6506-4750-ad5d-174e4899ac75 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:25.764386 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19199bc9-6506-4750-ad5d-174e4899ac75 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:25.771025 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 264/532] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:24:25 2022] GET /identity/v3/role_assignments?user.id=afec042821074a889ef9afa75bb8bf40&scope.project.id=252d9673f08c479a98d04fee3f5856a4&role.id=b0b74fefb27e4a0ebd895893aa9862d7 => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:25.771902 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19199bc9-6506-4750-ad5d-174e4899ac75 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:25.776074 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaab38a-3573-494d-9b6f-bf2227b2c6a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:25.776906 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaab38a-3573-494d-9b6f-bf2227b2c6a8 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:25.777291 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaab38a-3573-494d-9b6f-bf2227b2c6a8 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:25.778231 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaab38a-3573-494d-9b6f-bf2227b2c6a8 None admin] PATH_INFO: `/v3/domains` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:25.779089 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaab38a-3573-494d-9b6f-bf2227b2c6a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:25.779503 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaab38a-3573-494d-9b6f-bf2227b2c6a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:25.781041 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaab38a-3573-494d-9b6f-bf2227b2c6a8 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:25.784063 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:25.795770 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 269/533] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:24:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:25.803326 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaab38a-3573-494d-9b6f-bf2227b2c6a8 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:25.842398 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:25.891006 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19199bc9-6506-4750-ad5d-174e4899ac75 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:25.930159 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cafc35b-4197-4a27-9f2c-d0c580466c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:25.931332 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cafc35b-4197-4a27-9f2c-d0c580466c5b None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:25.931841 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cafc35b-4197-4a27-9f2c-d0c580466c5b None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:25.932320 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cafc35b-4197-4a27-9f2c-d0c580466c5b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:25.944632 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cafc35b-4197-4a27-9f2c-d0c580466c5b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:25.946556 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cafc35b-4197-4a27-9f2c-d0c580466c5b None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:25.966656 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-1cafc35b-4197-4a27-9f2c-d0c580466c5b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:25.966656 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:25.966656 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:25.966656 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:25.966656 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:25.966656 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:25.966656 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:25.966656 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:25.966656 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:25.966656 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:25.966656 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:25.966656 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:25.966656 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:24:25.966656 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:24:25.966656 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:24:25.967695 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:24:25.967695 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:25.967695 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:25.967695 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:25.967695 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:25.967695 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:25.967695 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:25.967695 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:25.967695 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:24:25.967695 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:25.967695 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:25.967695 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:25.967695 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:25.967695 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:25.967695 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:25.967695 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:25.967695 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:25.968667 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:24:25.968667 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:24:25.968667 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:25.968667 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:24:25.971194 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 265/534] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:24:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:25.976664 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cafc35b-4197-4a27-9f2c-d0c580466c5b None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:25.992608 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaab38a-3573-494d-9b6f-bf2227b2c6a8 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:26.078748 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4971f54-8079-4a3c-9b68-5647c58cb9ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:26.079506 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4971f54-8079-4a3c-9b68-5647c58cb9ab None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:26.079793 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4971f54-8079-4a3c-9b68-5647c58cb9ab None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:26.080071 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4971f54-8079-4a3c-9b68-5647c58cb9ab None admin] PATH_INFO: `/v3/projects/service` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:26.092938 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4971f54-8079-4a3c-9b68-5647c58cb9ab None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:26.094503 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4971f54-8079-4a3c-9b68-5647c58cb9ab None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:26.108689 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cafc35b-4197-4a27-9f2c-d0c580466c5b None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:26.112031 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-a4971f54-8079-4a3c-9b68-5647c58cb9ab None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:24:26.112031 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:26.112031 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:26.112031 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:26.112031 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:26.112031 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:26.112031 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:26.112031 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:26.112031 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:26.112031 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:26.112031 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:26.112031 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:26.112031 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:24:26.112031 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:24:26.112031 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:24:26.113137 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:24:26.113137 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:26.113137 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:26.113137 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:26.113137 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:26.113137 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:26.113137 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:26.113137 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:26.113137 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:24:26.113137 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:26.113137 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:26.113137 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:26.113137 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:26.113137 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:26.113137 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:26.113137 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:26.113137 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:26.114311 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:24:26.114311 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:24:26.114311 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:24:26.114311 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:24:26.114311 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:24:26.114311 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:24:26.114311 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:24:26.114311 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:24:26.115466 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 270/535] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:24:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:26.122050 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:26.123955 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4971f54-8079-4a3c-9b68-5647c58cb9ab None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:26.152094 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc363128-13e1-48b5-b54a-86ce2f278186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:26.153935 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc363128-13e1-48b5-b54a-86ce2f278186 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:26.154593 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc363128-13e1-48b5-b54a-86ce2f278186 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:26.155101 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc363128-13e1-48b5-b54a-86ce2f278186 None admin] PATH_INFO: `/v3/domains` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:26.156047 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc363128-13e1-48b5-b54a-86ce2f278186 None admin] RBAC: Adding query filter params (name=Default) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:26.156572 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc363128-13e1-48b5-b54a-86ce2f278186 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:26.158279 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc363128-13e1-48b5-b54a-86ce2f278186 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:26.172514 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 266/536] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:24:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:26.178313 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc363128-13e1-48b5-b54a-86ce2f278186 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:26.229924 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:26.278743 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4971f54-8079-4a3c-9b68-5647c58cb9ab None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:26.317011 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1e5188-f161-4beb-a334-8ba7d7714338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:26.318099 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1e5188-f161-4beb-a334-8ba7d7714338 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:26.318616 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1e5188-f161-4beb-a334-8ba7d7714338 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:26.319093 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1e5188-f161-4beb-a334-8ba7d7714338 None admin] PATH_INFO: `/v3/projects` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:26.320103 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1e5188-f161-4beb-a334-8ba7d7714338 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:26.320627 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1e5188-f161-4beb-a334-8ba7d7714338 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:26.322367 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1e5188-f161-4beb-a334-8ba7d7714338 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:26.353603 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 271/537] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:24:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:26.355329 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1e5188-f161-4beb-a334-8ba7d7714338 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:26.383600 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc363128-13e1-48b5-b54a-86ce2f278186 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:26.454294 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4973587a-2273-4806-8a39-6f98cacb11db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:26.455524 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4973587a-2273-4806-8a39-6f98cacb11db None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:26.457452 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4973587a-2273-4806-8a39-6f98cacb11db None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:26.457990 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4973587a-2273-4806-8a39-6f98cacb11db None admin] PATH_INFO: `/v3/projects/service` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:26.473573 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4973587a-2273-4806-8a39-6f98cacb11db None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:26.475786 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4973587a-2273-4806-8a39-6f98cacb11db None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:26.508428 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-4973587a-2273-4806-8a39-6f98cacb11db None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:24:26.508428 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:26.508428 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:26.508428 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:26.508428 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:26.508428 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:26.508428 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:26.508428 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:26.508428 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:26.508428 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:26.508428 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:26.508428 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:26.508428 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:24:26.508428 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:24:26.508428 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:24:26.509581 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:24:26.509581 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:26.509581 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:26.509581 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:26.509581 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:26.509581 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:26.509581 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:26.509581 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:26.509581 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:24:26.509581 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:26.509581 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:26.509581 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:26.509581 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:26.509581 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:26.509581 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:26.509581 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:26.509581 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:26.510607 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:24:26.510607 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:24:26.510607 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:24:26.510607 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:24:26.510607 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:24:26.510607 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:24:26.510607 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:24:26.510607 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:24:26.510607 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 267/538] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:24:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:24:26.514028 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4973587a-2273-4806-8a39-6f98cacb11db None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:26.552836 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1e5188-f161-4beb-a334-8ba7d7714338 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:26.562198 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:26.592171 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:26.665275 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4973587a-2273-4806-8a39-6f98cacb11db None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:26.674567 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c411f2-a315-49ae-b1b4-62a44b9e9862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:26.675241 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c411f2-a315-49ae-b1b4-62a44b9e9862 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:26.675529 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c411f2-a315-49ae-b1b4-62a44b9e9862 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:26.675808 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c411f2-a315-49ae-b1b4-62a44b9e9862 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:26.676649 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c411f2-a315-49ae-b1b4-62a44b9e9862 None admin] RBAC: Adding query filter params (role.id=b0b74fefb27e4a0ebd895893aa9862d7, scope.project.id=252d9673f08c479a98d04fee3f5856a4, user.id=95d18a2c8d8d4251ba8961a908574ee2) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:26.676968 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c411f2-a315-49ae-b1b4-62a44b9e9862 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:26.678455 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c411f2-a315-49ae-b1b4-62a44b9e9862 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:26.702224 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 272/539] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:24:26 2022] GET /identity/v3/role_assignments?user.id=95d18a2c8d8d4251ba8961a908574ee2&scope.project.id=252d9673f08c479a98d04fee3f5856a4&role.id=b0b74fefb27e4a0ebd895893aa9862d7 => generated 259 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:26.714803 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a0b4cf-e7e3-480c-a99e-b22edb6d8a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:26.716068 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a0b4cf-e7e3-480c-a99e-b22edb6d8a1f None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:26.716357 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a0b4cf-e7e3-480c-a99e-b22edb6d8a1f None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:26.716643 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a0b4cf-e7e3-480c-a99e-b22edb6d8a1f None admin] PATH_INFO: `/v3/projects` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:26.717465 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a0b4cf-e7e3-480c-a99e-b22edb6d8a1f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:26.717779 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a0b4cf-e7e3-480c-a99e-b22edb6d8a1f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:26.719281 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a0b4cf-e7e3-480c-a99e-b22edb6d8a1f None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:26.752568 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 268/540] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:24:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:26.758679 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c411f2-a315-49ae-b1b4-62a44b9e9862 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:26.915144 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c411f2-a315-49ae-b1b4-62a44b9e9862 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:26.964275 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fe5f1d-ccfb-4e79-a126-c453211c4e44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:26.965787 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fe5f1d-ccfb-4e79-a126-c453211c4e44 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:26.966432 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fe5f1d-ccfb-4e79-a126-c453211c4e44 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:26.966947 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fe5f1d-ccfb-4e79-a126-c453211c4e44 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:26.968045 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fe5f1d-ccfb-4e79-a126-c453211c4e44 None admin] RBAC: Adding query filter params (role.id=b0b74fefb27e4a0ebd895893aa9862d7, scope.project.id=252d9673f08c479a98d04fee3f5856a4, user.id=0ed9caca45c9449b84582da44a1262ef) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:26.968599 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fe5f1d-ccfb-4e79-a126-c453211c4e44 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:26.970354 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fe5f1d-ccfb-4e79-a126-c453211c4e44 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:26.981551 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 273/541] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:24:26 2022] GET /identity/v3/role_assignments?user.id=0ed9caca45c9449b84582da44a1262ef&scope.project.id=252d9673f08c479a98d04fee3f5856a4&role.id=b0b74fefb27e4a0ebd895893aa9862d7 => generated 259 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:27.193519 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45426783-99df-4aa7-9f93-6bad818573d7 None None] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:27.194737 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45426783-99df-4aa7-9f93-6bad818573d7 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:27.195253 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45426783-99df-4aa7-9f93-6bad818573d7 None None] PATH_INFO: `/` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:27.197322 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 269/542] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:24:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:24:27.203547 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cfc6a0-3edb-41dc-ac84-f71313217a30 None None] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:27.204209 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cfc6a0-3edb-41dc-ac84-f71313217a30 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:27.204695 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cfc6a0-3edb-41dc-ac84-f71313217a30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:27.233484 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.auth.core [None req-48cfc6a0-3edb-41dc-ac84-f71313217a30 None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143587) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:24:27.233484 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-48cfc6a0-3edb-41dc-ac84-f71313217a30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:24:27.279311 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:27.309747 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 274/543] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:24:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:24:27.343632 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45426783-99df-4aa7-9f93-6bad818573d7 None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:27.453680 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:27.479502 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45426783-99df-4aa7-9f93-6bad818573d7 None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:27.524891 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de58180-3cbd-41d1-9e0e-1b0a36def0c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:27.525698 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de58180-3cbd-41d1-9e0e-1b0a36def0c2 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:27.526015 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de58180-3cbd-41d1-9e0e-1b0a36def0c2 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:27.526295 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de58180-3cbd-41d1-9e0e-1b0a36def0c2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:27.543794 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de58180-3cbd-41d1-9e0e-1b0a36def0c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:24:27.551535 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-5de58180-3cbd-41d1-9e0e-1b0a36def0c2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:24:27.551535 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:27.551535 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:27.551535 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:27.551535 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:27.551535 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:27.551535 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:27.551535 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:27.551535 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:27.551535 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:27.551535 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:27.551535 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:27.551535 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:24:27.551535 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:24:27.551535 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:24:27.551535 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:24:27.552643 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:24:27.552643 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:24:27.552643 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:24:27.552643 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:24:27.552643 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:27.552643 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:27.552643 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:27.552643 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:27.552643 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:27.552643 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:27.552643 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:27.552643 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:24:27.552643 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:27.552643 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:27.552643 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:27.552643 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:27.553660 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:27.553660 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:27.553660 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:27.553660 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:27.553660 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:24:27.553660 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:24:27.553660 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:24:27.553660 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:24:27.553660 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:24:27.553660 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:24:27.553660 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:24:27.553660 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:24:27.557285 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 270/544] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 14:24:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:27.559629 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cfc6a0-3edb-41dc-ac84-f71313217a30 None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:27.734365 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cfc6a0-3edb-41dc-ac84-f71313217a30 None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:27.790838 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ee4fbc-25dc-467a-8e60-0117e13b2a99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:27.791524 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ee4fbc-25dc-467a-8e60-0117e13b2a99 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:27.791807 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ee4fbc-25dc-467a-8e60-0117e13b2a99 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:27.792085 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ee4fbc-25dc-467a-8e60-0117e13b2a99 None admin] PATH_INFO: `/v3/roles` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:27.792845 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ee4fbc-25dc-467a-8e60-0117e13b2a99 None admin] RBAC: Adding query filter params (name=service) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:27.794110 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ee4fbc-25dc-467a-8e60-0117e13b2a99 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:27.795547 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ee4fbc-25dc-467a-8e60-0117e13b2a99 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:27.808603 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 275/545] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 19 14:24:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:27.809886 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de58180-3cbd-41d1-9e0e-1b0a36def0c2 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:27.960673 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de58180-3cbd-41d1-9e0e-1b0a36def0c2 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:28.007845 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f6882c-cdb7-40d4-ad61-d608a62006df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:28.008567 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f6882c-cdb7-40d4-ad61-d608a62006df None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:28.008853 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f6882c-cdb7-40d4-ad61-d608a62006df None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:28.009135 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f6882c-cdb7-40d4-ad61-d608a62006df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:28.027970 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f6882c-cdb7-40d4-ad61-d608a62006df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:28.029452 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f6882c-cdb7-40d4-ad61-d608a62006df None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:28.032313 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:28.058570 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-e6f6882c-cdb7-40d4-ad61-d608a62006df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:28.058570 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:28.058570 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:28.058570 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:28.058570 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:28.058570 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:28.058570 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:28.058570 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:28.058570 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:28.058570 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:28.058570 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:28.058570 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:28.058570 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:24:28.058570 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:24:28.058570 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:24:28.060430 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:24:28.060430 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:28.060430 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:28.060430 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:28.060430 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:28.060430 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:28.060430 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:28.060430 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:28.060430 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:24:28.060430 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:28.060430 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:28.060430 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:28.060430 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:28.060430 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:28.060430 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:28.060430 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:28.060430 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:28.061426 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:24:28.061426 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:24:28.061426 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:28.061426 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:24:28.061426 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 271/546] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:24:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:28.066545 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ee4fbc-25dc-467a-8e60-0117e13b2a99 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:28.103997 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f18c881-cb1c-4b06-84be-fcfa78f1f725 None None] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:28.104350 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f18c881-cb1c-4b06-84be-fcfa78f1f725 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:28.104537 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f18c881-cb1c-4b06-84be-fcfa78f1f725 None None] PATH_INFO: `/` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:28.106895 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 272/547] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:24:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:24:28.116183 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c567c07e-4a23-4a42-9f80-16553aa073fb None None] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:28.116481 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c567c07e-4a23-4a42-9f80-16553aa073fb None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:28.116740 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c567c07e-4a23-4a42-9f80-16553aa073fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:28.120284 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:28.154914 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.auth.core [None req-c567c07e-4a23-4a42-9f80-16553aa073fb None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143586) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:24:28.156143 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.fernet_utils [None req-c567c07e-4a23-4a42-9f80-16553aa073fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143586) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:24:28.212537 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ee4fbc-25dc-467a-8e60-0117e13b2a99 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:28.283271 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9b6c20-962e-49e7-ba63-5ffe23c1179c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:28.284052 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9b6c20-962e-49e7-ba63-5ffe23c1179c None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:28.284230 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9b6c20-962e-49e7-ba63-5ffe23c1179c None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:28.284410 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9b6c20-962e-49e7-ba63-5ffe23c1179c None admin] PATH_INFO: `/v3/domains` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:28.285122 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9b6c20-962e-49e7-ba63-5ffe23c1179c None admin] RBAC: Adding query filter params (name=Default) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:28.285434 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9b6c20-962e-49e7-ba63-5ffe23c1179c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:28.286858 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9b6c20-962e-49e7-ba63-5ffe23c1179c None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:28.296693 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 273/548] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:24:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:24:28.310274 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 276/549] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:24:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:28.315668 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c567c07e-4a23-4a42-9f80-16553aa073fb None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:28.321280 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9b6c20-962e-49e7-ba63-5ffe23c1179c None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:28.505796 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:28.510107 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c567c07e-4a23-4a42-9f80-16553aa073fb None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:28.517703 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9b6c20-962e-49e7-ba63-5ffe23c1179c None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:28.563718 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a80fdb-c674-426b-a517-f69b5e5e9cbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:28.564962 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a80fdb-c674-426b-a517-f69b5e5e9cbd None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:28.565504 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a80fdb-c674-426b-a517-f69b5e5e9cbd None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:28.566004 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a80fdb-c674-426b-a517-f69b5e5e9cbd None admin] PATH_INFO: `/v3/roles/service` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:28.576812 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:28.578266 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a80fdb-c674-426b-a517-f69b5e5e9cbd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:24:28.586555 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-82a80fdb-c674-426b-a517-f69b5e5e9cbd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:24:28.586555 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:28.586555 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:28.586555 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:28.586555 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:28.586555 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:28.586555 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:28.586555 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:28.586555 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:28.586555 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:28.586555 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:28.586555 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:28.586555 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:24:28.586555 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:24:28.586555 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:24:28.586555 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:24:28.587689 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:24:28.587689 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:24:28.587689 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:24:28.587689 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:24:28.587689 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:28.587689 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:28.587689 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:28.587689 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:28.587689 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:28.587689 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:28.587689 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:28.587689 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:24:28.587689 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:28.587689 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:28.587689 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:28.587689 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:28.588618 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:28.588618 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:28.588618 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:28.588618 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:28.588618 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:24:28.588618 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:24:28.588618 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:24:28.588618 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:24:28.588618 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:24:28.588618 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:24:28.588618 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:24:28.588618 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:24:28.592026 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 277/550] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 14:24:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:28.597183 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4424b6-4d35-4b99-a8a9-65491103ee5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:28.598558 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4424b6-4d35-4b99-a8a9-65491103ee5d None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:28.599059 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4424b6-4d35-4b99-a8a9-65491103ee5d None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:28.599477 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a80fdb-c674-426b-a517-f69b5e5e9cbd None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:28.602216 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4424b6-4d35-4b99-a8a9-65491103ee5d None admin] PATH_INFO: `/v3/users/swift` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:28.610667 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4424b6-4d35-4b99-a8a9-65491103ee5d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:28.612130 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4424b6-4d35-4b99-a8a9-65491103ee5d None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:28.644148 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-6a4424b6-4d35-4b99-a8a9-65491103ee5d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 14:24:28.644148 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:28.644148 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:28.644148 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:28.644148 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:28.644148 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:28.644148 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:28.644148 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:28.644148 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:28.644148 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:28.644148 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:28.644148 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:28.644148 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:24:28.644148 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:24:28.644148 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:24:28.644148 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:24:28.645476 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:28.645476 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:28.645476 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:24:28.645476 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:24:28.645476 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:24:28.645476 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:24:28.645476 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:28.645476 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:28.645476 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:28.645476 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:28.645476 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:28.645476 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:24:28.645476 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:28.645476 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:28.645476 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:28.645476 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:28.645476 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:28.645476 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:28.646662 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:28.646662 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:28.646662 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:24:28.646662 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:24:28.646662 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:24:28.646662 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:24:28.646662 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:24:28.646662 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:24:28.646662 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 14:24:28.646662 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:24:28.647734 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 274/551] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:24:28 2022] GET /identity/v3/users/swift => generated 83 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:28.653659 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b302a2-cb7a-488f-a905-b62dfbf884b8 None None] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:28.653659 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b302a2-cb7a-488f-a905-b62dfbf884b8 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:28.653659 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b302a2-cb7a-488f-a905-b62dfbf884b8 None None] PATH_INFO: `/` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:28.662080 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 275/552] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:24:28 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:24:28.663246 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b302a2-cb7a-488f-a905-b62dfbf884b8 None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:28.750254 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a80fdb-c674-426b-a517-f69b5e5e9cbd None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:28.786923 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13847d07-58c6-4cf3-862c-33515c984522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:28.787589 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13847d07-58c6-4cf3-862c-33515c984522 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:28.787876 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13847d07-58c6-4cf3-862c-33515c984522 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:28.788209 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13847d07-58c6-4cf3-862c-33515c984522 None admin] PATH_INFO: `/v3/roles` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:28.788969 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13847d07-58c6-4cf3-862c-33515c984522 None admin] RBAC: Adding query filter params (name=service) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:28.789279 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13847d07-58c6-4cf3-862c-33515c984522 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:28.790857 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13847d07-58c6-4cf3-862c-33515c984522 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:28.799076 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b302a2-cb7a-488f-a905-b62dfbf884b8 None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:28.800914 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 278/553] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 19 14:24:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:24:28.806270 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5553708-4e3f-45ee-9a1a-6611b91f82ce None None] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:28.806624 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5553708-4e3f-45ee-9a1a-6611b91f82ce None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:28.806923 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5553708-4e3f-45ee-9a1a-6611b91f82ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:28.825182 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.auth.core [None req-a5553708-4e3f-45ee-9a1a-6611b91f82ce None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143587) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:24:28.826367 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-a5553708-4e3f-45ee-9a1a-6611b91f82ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:24:28.840653 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa166c1-a04a-4a73-a3b7-81e467ab670f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:28.841761 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:28.843434 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa166c1-a04a-4a73-a3b7-81e467ab670f None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:28.843960 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa166c1-a04a-4a73-a3b7-81e467ab670f None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:28.844465 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa166c1-a04a-4a73-a3b7-81e467ab670f None admin] PATH_INFO: `/v3/users` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:28.845543 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa166c1-a04a-4a73-a3b7-81e467ab670f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:28.846156 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa166c1-a04a-4a73-a3b7-81e467ab670f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:28.847805 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa166c1-a04a-4a73-a3b7-81e467ab670f None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:28.868942 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 276/554] 10.222.0.57 () {56 vars in 1135 bytes} [Thu May 19 14:24:28 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:28.870138 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa166c1-a04a-4a73-a3b7-81e467ab670f None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:28.905888 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 279/555] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:24:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:24:28.906325 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5553708-4e3f-45ee-9a1a-6611b91f82ce None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:28.937438 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:29.015654 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa166c1-a04a-4a73-a3b7-81e467ab670f None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:29.025184 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5553708-4e3f-45ee-9a1a-6611b91f82ce None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:29.055595 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd12ace-1e8d-4a57-9cce-3a6db6b329ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:29.056404 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd12ace-1e8d-4a57-9cce-3a6db6b329ae None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:29.056691 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd12ace-1e8d-4a57-9cce-3a6db6b329ae None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:29.056972 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd12ace-1e8d-4a57-9cce-3a6db6b329ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:29.069344 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd12ace-1e8d-4a57-9cce-3a6db6b329ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:29.070999 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd12ace-1e8d-4a57-9cce-3a6db6b329ae None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:29.072286 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2e85ed-5d7b-48e9-b822-891bbfbf35d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:29.073208 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2e85ed-5d7b-48e9-b822-891bbfbf35d7 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:29.073918 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2e85ed-5d7b-48e9-b822-891bbfbf35d7 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:29.074211 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2e85ed-5d7b-48e9-b822-891bbfbf35d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:29.090811 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-9fd12ace-1e8d-4a57-9cce-3a6db6b329ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:29.090811 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:29.090811 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:29.090811 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:29.090811 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:29.090811 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:29.090811 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:29.090811 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:29.090811 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:29.090811 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:29.090811 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:29.090811 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:29.090811 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:24:29.090811 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:24:29.090811 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:24:29.091820 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:24:29.091820 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:29.091820 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:29.091820 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:29.091820 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:29.091820 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:29.091820 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:29.091820 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:29.091820 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:24:29.091820 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:29.091820 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:29.091820 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:29.091820 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:29.091820 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:29.091820 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:29.091820 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:29.091820 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:29.092817 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:24:29.092817 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:24:29.092817 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:29.092817 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:24:29.094953 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:29.096818 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 277/556] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:24:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:29.097769 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd12ace-1e8d-4a57-9cce-3a6db6b329ae None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:29.165607 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2e85ed-5d7b-48e9-b822-891bbfbf35d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:29.167978 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2e85ed-5d7b-48e9-b822-891bbfbf35d7 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:29.208734 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:29.216667 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd12ace-1e8d-4a57-9cce-3a6db6b329ae None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:29.266609 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76be151-358f-410f-8a3a-9f6261934185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:29.267667 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76be151-358f-410f-8a3a-9f6261934185 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:29.267956 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76be151-358f-410f-8a3a-9f6261934185 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:29.268273 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76be151-358f-410f-8a3a-9f6261934185 None admin] PATH_INFO: `/v3/roles/service` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:29.283552 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76be151-358f-410f-8a3a-9f6261934185 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:24:29.294224 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-d76be151-358f-410f-8a3a-9f6261934185 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:24:29.294224 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:29.294224 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:29.294224 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:29.294224 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:29.294224 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:29.294224 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:29.294224 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:29.294224 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:29.294224 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:29.294224 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:29.294224 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:29.294224 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:24:29.294224 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:24:29.294224 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:24:29.294224 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:24:29.296595 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:24:29.296595 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:24:29.296595 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:24:29.296595 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:24:29.296595 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:29.296595 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:29.296595 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:29.296595 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:29.296595 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:29.296595 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:29.296595 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:29.296595 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:24:29.296595 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:29.296595 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:29.296595 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:29.296595 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:29.299225 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:29.299225 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:29.299225 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:29.299225 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:29.299225 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:24:29.299225 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:24:29.299225 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:24:29.299225 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:24:29.299225 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:24:29.299225 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:24:29.299225 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:24:29.299225 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:24:29.299225 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 278/557] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 14:24:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:29.299225 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76be151-358f-410f-8a3a-9f6261934185 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:29.309567 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 280/558] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:24:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:24:29.315925 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2e85ed-5d7b-48e9-b822-891bbfbf35d7 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:29.375058 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:29.420542 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76be151-358f-410f-8a3a-9f6261934185 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:29.431392 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2e85ed-5d7b-48e9-b822-891bbfbf35d7 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:29.459357 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fabe82a-7ff1-493f-9da7-d64f68147297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:29.460025 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fabe82a-7ff1-493f-9da7-d64f68147297 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:29.460313 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fabe82a-7ff1-493f-9da7-d64f68147297 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:29.460791 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fabe82a-7ff1-493f-9da7-d64f68147297 None admin] PATH_INFO: `/v3/domains` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:29.461315 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fabe82a-7ff1-493f-9da7-d64f68147297 None admin] RBAC: Adding query filter params (name=Default) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:29.461638 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fabe82a-7ff1-493f-9da7-d64f68147297 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:29.463048 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fabe82a-7ff1-493f-9da7-d64f68147297 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:29.469531 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a682be-df23-47f4-852b-9e367c2a6e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:29.470217 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a682be-df23-47f4-852b-9e367c2a6e64 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:29.470502 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a682be-df23-47f4-852b-9e367c2a6e64 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:29.470776 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a682be-df23-47f4-852b-9e367c2a6e64 None admin] PATH_INFO: `/v3/roles` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:29.471515 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a682be-df23-47f4-852b-9e367c2a6e64 None admin] RBAC: Adding query filter params (name=service) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:29.471822 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a682be-df23-47f4-852b-9e367c2a6e64 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:29.473181 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a682be-df23-47f4-852b-9e367c2a6e64 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:29.476906 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 279/559] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:24:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:29.477976 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fabe82a-7ff1-493f-9da7-d64f68147297 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:29.482579 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 281/560] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 19 14:24:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:29.483865 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2b73ca-5337-4596-b21e-d24f97114afb None None] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:29.484152 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2b73ca-5337-4596-b21e-d24f97114afb None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:29.484428 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2b73ca-5337-4596-b21e-d24f97114afb None None] PATH_INFO: `/` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:29.485802 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 282/561] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:24:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:24:29.490319 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2b73ca-5337-4596-b21e-d24f97114afb None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:29.507269 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:29.601529 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fabe82a-7ff1-493f-9da7-d64f68147297 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:29.601982 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2b73ca-5337-4596-b21e-d24f97114afb None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:29.613152 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:29.638327 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ed4a4b-82fa-4195-8110-f5214ebcf269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:29.639057 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ed4a4b-82fa-4195-8110-f5214ebcf269 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:29.639375 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1c7b01-ce04-49e0-86f6-eda5bc0bc7aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:29.639602 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ed4a4b-82fa-4195-8110-f5214ebcf269 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:29.639602 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ed4a4b-82fa-4195-8110-f5214ebcf269 None admin] PATH_INFO: `/v3/users/nova` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:29.640178 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1c7b01-ce04-49e0-86f6-eda5bc0bc7aa None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:29.640364 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1c7b01-ce04-49e0-86f6-eda5bc0bc7aa None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:29.640724 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1c7b01-ce04-49e0-86f6-eda5bc0bc7aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:29.647893 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ed4a4b-82fa-4195-8110-f5214ebcf269 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:29.649336 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ed4a4b-82fa-4195-8110-f5214ebcf269 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:29.651122 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1c7b01-ce04-49e0-86f6-eda5bc0bc7aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:29.655282 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1c7b01-ce04-49e0-86f6-eda5bc0bc7aa None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:29.667960 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-87ed4a4b-82fa-4195-8110-f5214ebcf269 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 14:24:29.667960 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:29.667960 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:29.667960 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:29.667960 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:29.667960 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:29.667960 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:29.667960 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:29.667960 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:29.667960 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:29.667960 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:29.667960 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:29.667960 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:24:29.667960 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:24:29.667960 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:24:29.667960 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:24:29.669148 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:29.669148 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:29.669148 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:24:29.669148 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:24:29.669148 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:24:29.669148 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:24:29.669148 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:29.669148 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:29.669148 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:29.669148 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:29.669148 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:29.669148 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:24:29.669148 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:29.669148 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:29.669148 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:29.669148 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:29.669148 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:29.669148 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:29.670239 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:29.670239 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:29.670239 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:24:29.670239 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:24:29.670239 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:24:29.670239 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:24:29.670239 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:24:29.670239 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:24:29.670239 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 14:24:29.670239 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:24:29.674224 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 283/562] 10.222.0.57 () {56 vars in 1093 bytes} [Thu May 19 14:24:29 2022] GET /identity/v3/users/nova => generated 82 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:29.675127 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ed4a4b-82fa-4195-8110-f5214ebcf269 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:29.679547 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-ab1c7b01-ce04-49e0-86f6-eda5bc0bc7aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:29.679547 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:29.679547 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:29.679547 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:29.679547 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:29.679547 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:29.679547 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:29.679547 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:29.679547 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:29.679547 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:29.679547 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:29.679547 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:29.679547 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:24:29.679547 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:24:29.679547 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:24:29.680493 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:24:29.680493 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:29.680493 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:29.680493 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:29.680493 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:29.680493 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:29.680493 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:29.680493 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:29.680493 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:24:29.680493 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:29.680493 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:29.680493 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:29.680493 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:29.680493 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:29.680493 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:29.680493 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:29.680493 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:29.681524 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:24:29.681524 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:24:29.681524 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:29.681524 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:24:29.681524 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 280/563] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:24:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:29.682300 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce5a626-9743-427e-ba09-b6af81ebd06f None None] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:29.682587 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce5a626-9743-427e-ba09-b6af81ebd06f None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:29.682866 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce5a626-9743-427e-ba09-b6af81ebd06f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:29.700516 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.auth.core [None req-7ce5a626-9743-427e-ba09-b6af81ebd06f None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143586) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:24:29.701715 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.fernet_utils [None req-7ce5a626-9743-427e-ba09-b6af81ebd06f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143586) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:24:29.765775 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:29.771803 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 281/564] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:24:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:24:29.772843 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce5a626-9743-427e-ba09-b6af81ebd06f None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:29.787821 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ed4a4b-82fa-4195-8110-f5214ebcf269 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:29.824963 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562f505f-f0f4-4f9c-81e0-a2b92b127e3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:29.825686 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562f505f-f0f4-4f9c-81e0-a2b92b127e3b None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:29.826009 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562f505f-f0f4-4f9c-81e0-a2b92b127e3b None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:29.826290 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562f505f-f0f4-4f9c-81e0-a2b92b127e3b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:29.836644 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562f505f-f0f4-4f9c-81e0-a2b92b127e3b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:29.838137 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562f505f-f0f4-4f9c-81e0-a2b92b127e3b None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:29.855294 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-562f505f-f0f4-4f9c-81e0-a2b92b127e3b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:29.855294 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:29.855294 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:29.855294 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:29.855294 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:29.855294 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:29.855294 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:29.855294 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:29.855294 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:29.855294 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:29.855294 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:29.855294 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:29.855294 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:24:29.855294 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:24:29.855294 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:24:29.856935 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:24:29.856935 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:29.856935 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:29.856935 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:29.856935 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:29.856935 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:29.856935 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:29.856935 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:29.856935 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:24:29.856935 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:29.856935 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:29.856935 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:29.856935 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:29.856935 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:29.856935 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:29.856935 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:29.856935 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:29.857972 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:24:29.857972 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:24:29.857972 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:29.857972 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:24:29.857972 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 284/565] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:24:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:29.858297 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562f505f-f0f4-4f9c-81e0-a2b92b127e3b None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:29.877481 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce5a626-9743-427e-ba09-b6af81ebd06f None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:29.884417 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:29.911867 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d6e1cf-eed3-4bb3-9298-27d4d22259de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:29.912518 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d6e1cf-eed3-4bb3-9298-27d4d22259de None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:29.912798 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d6e1cf-eed3-4bb3-9298-27d4d22259de None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:29.913100 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d6e1cf-eed3-4bb3-9298-27d4d22259de None admin] PATH_INFO: `/v3/users` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:29.913969 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d6e1cf-eed3-4bb3-9298-27d4d22259de None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:29.914287 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d6e1cf-eed3-4bb3-9298-27d4d22259de None admin] RBAC: Authorizing `identity:list_users()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:29.915733 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d6e1cf-eed3-4bb3-9298-27d4d22259de None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:29.933040 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 282/566] 10.222.0.57 () {56 vars in 1133 bytes} [Thu May 19 14:24:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:29.934604 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67059187-659d-43eb-b059-aeda728913d9 None None] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:29.934895 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67059187-659d-43eb-b059-aeda728913d9 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:29.935172 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67059187-659d-43eb-b059-aeda728913d9 None None] PATH_INFO: `/` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:29.937881 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 283/567] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:24:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:24:29.937881 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67059187-659d-43eb-b059-aeda728913d9 None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:29.963528 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562f505f-f0f4-4f9c-81e0-a2b92b127e3b None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:29.994747 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:29.999015 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe784ee9-404b-4bd2-92db-07e49ab58207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:29.999678 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe784ee9-404b-4bd2-92db-07e49ab58207 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:29.999966 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe784ee9-404b-4bd2-92db-07e49ab58207 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:30.000246 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe784ee9-404b-4bd2-92db-07e49ab58207 None admin] PATH_INFO: `/v3/domains` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:30.000972 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe784ee9-404b-4bd2-92db-07e49ab58207 None admin] RBAC: Adding query filter params (name=Default) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:30.001286 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe784ee9-404b-4bd2-92db-07e49ab58207 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:30.002804 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe784ee9-404b-4bd2-92db-07e49ab58207 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:30.014803 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 285/568] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:24:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:30.015884 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe784ee9-404b-4bd2-92db-07e49ab58207 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:30.040929 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67059187-659d-43eb-b059-aeda728913d9 None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:30.076500 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9c3681-d7cc-42da-a80c-3020c2349cd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:30.077287 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9c3681-d7cc-42da-a80c-3020c2349cd7 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:30.077565 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9c3681-d7cc-42da-a80c-3020c2349cd7 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:30.077862 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9c3681-d7cc-42da-a80c-3020c2349cd7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:30.087724 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9c3681-d7cc-42da-a80c-3020c2349cd7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:24:30.096350 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-cc9c3681-d7cc-42da-a80c-3020c2349cd7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:24:30.096350 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:30.096350 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:30.096350 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:30.096350 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:30.096350 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:30.096350 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:30.096350 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:30.096350 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:30.096350 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:30.096350 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:30.096350 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:30.096350 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:24:30.096350 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:24:30.096350 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:24:30.096350 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:24:30.097638 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:24:30.097638 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:24:30.097638 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:24:30.097638 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:24:30.097638 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:30.097638 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:30.097638 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:30.097638 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:30.097638 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:30.097638 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:30.097638 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:30.097638 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:24:30.097638 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:30.097638 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:30.097638 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:30.097638 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:30.098623 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:30.098623 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:30.098623 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:30.098623 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:30.098623 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:24:30.098623 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:24:30.098623 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:24:30.098623 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:24:30.098623 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:24:30.098623 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:24:30.098623 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:24:30.098623 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:24:30.098623 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 284/569] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 14:24:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:30.098623 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9c3681-d7cc-42da-a80c-3020c2349cd7 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:30.101674 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:30.121882 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe784ee9-404b-4bd2-92db-07e49ab58207 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:30.156090 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aa6268-34f9-46aa-af22-0bb0bca9d4d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:30.156756 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aa6268-34f9-46aa-af22-0bb0bca9d4d1 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:30.157048 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aa6268-34f9-46aa-af22-0bb0bca9d4d1 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:30.157323 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aa6268-34f9-46aa-af22-0bb0bca9d4d1 None admin] PATH_INFO: `/v3/domains` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:30.158082 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50aa6268-34f9-46aa-af22-0bb0bca9d4d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:30.158394 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50aa6268-34f9-46aa-af22-0bb0bca9d4d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:30.159777 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50aa6268-34f9-46aa-af22-0bb0bca9d4d1 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:30.171881 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 286/570] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:24:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:30.173035 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601dc2d9-ded3-47a1-94c5-399e7c987314 None None] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:30.173335 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601dc2d9-ded3-47a1-94c5-399e7c987314 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:30.173618 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601dc2d9-ded3-47a1-94c5-399e7c987314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:30.190688 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.auth.core [None req-601dc2d9-ded3-47a1-94c5-399e7c987314 None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143587) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:24:30.191882 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-601dc2d9-ded3-47a1-94c5-399e7c987314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:24:30.202165 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9c3681-d7cc-42da-a80c-3020c2349cd7 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:30.218268 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:30.236681 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da16be58-1325-4186-ae5e-bbb91d3c3796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:30.237474 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da16be58-1325-4186-ae5e-bbb91d3c3796 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:30.237757 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da16be58-1325-4186-ae5e-bbb91d3c3796 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:30.238065 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da16be58-1325-4186-ae5e-bbb91d3c3796 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:30.249164 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da16be58-1325-4186-ae5e-bbb91d3c3796 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:30.250641 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da16be58-1325-4186-ae5e-bbb91d3c3796 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:30.259323 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 287/571] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:24:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:24:30.260170 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601dc2d9-ded3-47a1-94c5-399e7c987314 None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:30.268117 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-da16be58-1325-4186-ae5e-bbb91d3c3796 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:30.268117 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:30.268117 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:30.268117 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:30.268117 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:30.268117 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:30.268117 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:30.268117 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:30.268117 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:30.268117 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:30.268117 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:30.268117 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:30.268117 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:24:30.268117 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:24:30.268117 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:24:30.269104 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:24:30.269104 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:30.269104 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:30.269104 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:30.269104 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:30.269104 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:30.269104 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:30.269104 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:30.269104 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:24:30.269104 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:30.269104 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:30.269104 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:30.269104 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:30.269104 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:30.269104 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:30.269104 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:30.269104 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:30.270193 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:24:30.270193 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:24:30.270193 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:30.270193 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:24:30.270193 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 285/572] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:24:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:30.270193 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da16be58-1325-4186-ae5e-bbb91d3c3796 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:30.344072 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:30.374871 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da16be58-1325-4186-ae5e-bbb91d3c3796 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:30.377327 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601dc2d9-ded3-47a1-94c5-399e7c987314 None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:30.408154 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ff17f1-8efe-4545-ab46-6ab88666fa37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:30.408824 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ff17f1-8efe-4545-ab46-6ab88666fa37 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:30.409113 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ff17f1-8efe-4545-ab46-6ab88666fa37 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:30.409402 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ff17f1-8efe-4545-ab46-6ab88666fa37 None admin] PATH_INFO: `/v3/roles` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:30.410182 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ff17f1-8efe-4545-ab46-6ab88666fa37 None admin] RBAC: Adding query filter params (name=service) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:30.410494 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ff17f1-8efe-4545-ab46-6ab88666fa37 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:30.411033 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c190e50-379d-49b0-bfad-5a80985d694a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:30.411853 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c190e50-379d-49b0-bfad-5a80985d694a None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:30.411956 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ff17f1-8efe-4545-ab46-6ab88666fa37 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:30.412135 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c190e50-379d-49b0-bfad-5a80985d694a None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:30.412407 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c190e50-379d-49b0-bfad-5a80985d694a None admin] PATH_INFO: `/v3/projects/service` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:30.420057 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c190e50-379d-49b0-bfad-5a80985d694a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:30.420701 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 286/573] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 19 14:24:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:30.421503 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ff17f1-8efe-4545-ab46-6ab88666fa37 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:30.421876 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c190e50-379d-49b0-bfad-5a80985d694a None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:30.437382 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-6c190e50-379d-49b0-bfad-5a80985d694a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:24:30.437382 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:30.437382 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:30.437382 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:30.437382 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:30.437382 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:30.437382 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:30.437382 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:30.437382 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:30.437382 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:30.437382 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:30.437382 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:30.437382 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:24:30.437382 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:24:30.437382 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:24:30.438552 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:24:30.438552 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:30.438552 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:30.438552 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:30.438552 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:30.438552 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:30.438552 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:30.438552 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:30.438552 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:24:30.438552 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:30.438552 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:30.438552 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:30.438552 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:30.438552 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:30.438552 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:30.438552 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:30.438552 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:30.439746 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:24:30.439746 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:24:30.439746 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:24:30.439746 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:24:30.439746 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:24:30.439746 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:24:30.439746 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:24:30.439746 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:24:30.439746 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 288/574] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:24:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:30.439746 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c190e50-379d-49b0-bfad-5a80985d694a None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:30.458574 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:30.527633 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ff17f1-8efe-4545-ab46-6ab88666fa37 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:30.539456 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c190e50-379d-49b0-bfad-5a80985d694a None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:30.561546 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645da491-22e4-4572-ae79-2dd3e78f4289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:30.562370 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645da491-22e4-4572-ae79-2dd3e78f4289 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:30.562660 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645da491-22e4-4572-ae79-2dd3e78f4289 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:30.562944 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645da491-22e4-4572-ae79-2dd3e78f4289 None admin] PATH_INFO: `/v3/users/glance` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:30.569583 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:30.570399 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645da491-22e4-4572-ae79-2dd3e78f4289 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:30.572235 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645da491-22e4-4572-ae79-2dd3e78f4289 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:30.575252 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcd122f-d041-42fa-9c41-b10666d36de7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:30.575835 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcd122f-d041-42fa-9c41-b10666d36de7 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:30.576121 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcd122f-d041-42fa-9c41-b10666d36de7 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:30.576399 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcd122f-d041-42fa-9c41-b10666d36de7 None admin] PATH_INFO: `/v3/domains` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:30.577125 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcd122f-d041-42fa-9c41-b10666d36de7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:30.577434 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcd122f-d041-42fa-9c41-b10666d36de7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:30.578835 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcd122f-d041-42fa-9c41-b10666d36de7 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:30.586591 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-645da491-22e4-4572-ae79-2dd3e78f4289 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 19 14:24:30.586591 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:30.586591 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:30.586591 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:30.586591 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:30.586591 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:30.586591 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:30.586591 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:30.586591 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:30.586591 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:30.586591 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:30.586591 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:30.586591 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:24:30.586591 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:24:30.586591 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:24:30.586591 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:24:30.587732 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:30.587732 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:30.587732 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:24:30.587732 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:24:30.587732 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:24:30.587732 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:24:30.587732 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:30.587732 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:30.587732 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:30.587732 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:30.587732 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:30.587732 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:24:30.587732 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:30.587732 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:30.587732 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:30.587732 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:30.587732 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:30.587732 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:30.588785 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:30.588785 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:30.588785 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:24:30.588785 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:24:30.588785 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:24:30.588785 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:24:30.588785 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:24:30.588785 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:24:30.588785 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 19 14:24:30.588785 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:24:30.588785 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 287/575] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:24:30 2022] GET /identity/v3/users/glance => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:30.588785 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645da491-22e4-4572-ae79-2dd3e78f4289 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:30.600066 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 289/576] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:24:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:30.600066 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcd122f-d041-42fa-9c41-b10666d36de7 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:30.699956 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:30.709379 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645da491-22e4-4572-ae79-2dd3e78f4289 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:30.720714 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcd122f-d041-42fa-9c41-b10666d36de7 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:30.743565 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb04a33-c1ff-466f-827e-74e6824ef684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:30.744356 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb04a33-c1ff-466f-827e-74e6824ef684 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:30.744638 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb04a33-c1ff-466f-827e-74e6824ef684 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:30.744916 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb04a33-c1ff-466f-827e-74e6824ef684 None admin] PATH_INFO: `/v3/roles/service` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:30.755359 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb04a33-c1ff-466f-827e-74e6824ef684 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:24:30.755520 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862512dc-d7ad-45ae-8ba8-ee10c48a257a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:30.756183 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862512dc-d7ad-45ae-8ba8-ee10c48a257a None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:30.756466 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862512dc-d7ad-45ae-8ba8-ee10c48a257a None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:30.756741 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862512dc-d7ad-45ae-8ba8-ee10c48a257a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:30.763114 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-4bb04a33-c1ff-466f-827e-74e6824ef684 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:24:30.763114 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:30.763114 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:30.763114 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:30.763114 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:30.763114 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:30.763114 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:30.763114 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:30.763114 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:30.763114 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:30.763114 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:30.763114 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:30.763114 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:24:30.763114 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:24:30.763114 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:24:30.763114 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:24:30.764141 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:24:30.764141 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:24:30.764141 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:24:30.764141 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:24:30.764141 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:30.764141 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:30.764141 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:30.764141 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:30.764141 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:30.764141 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:30.764141 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:30.764141 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:24:30.764141 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:30.764141 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:30.764141 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:30.764141 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:30.765094 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:30.765094 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:30.765094 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:30.765094 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:30.765094 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:24:30.765094 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:24:30.765094 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:24:30.765094 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:24:30.765094 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:24:30.765094 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:24:30.765094 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:24:30.765094 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:24:30.765094 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 288/577] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 14:24:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:30.765094 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb04a33-c1ff-466f-827e-74e6824ef684 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:30.771681 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862512dc-d7ad-45ae-8ba8-ee10c48a257a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:30.773157 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862512dc-d7ad-45ae-8ba8-ee10c48a257a None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:30.791311 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-862512dc-d7ad-45ae-8ba8-ee10c48a257a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:30.791311 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:30.791311 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:30.791311 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:30.791311 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:30.791311 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:30.791311 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:30.791311 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:30.791311 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:30.791311 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:30.791311 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:30.791311 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:30.791311 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:24:30.791311 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:24:30.791311 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:24:30.792435 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:24:30.792435 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:30.792435 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:30.792435 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:30.792435 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:30.792435 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:30.792435 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:30.792435 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:30.792435 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:24:30.792435 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:30.792435 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:30.792435 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:30.792435 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:30.792435 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:30.792435 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:30.792435 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:30.792435 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:30.794227 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:24:30.794227 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:24:30.794227 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:30.794227 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:24:30.797639 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 290/578] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:24:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:30.798772 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862512dc-d7ad-45ae-8ba8-ee10c48a257a None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:30.843685 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:30.866115 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb04a33-c1ff-466f-827e-74e6824ef684 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:30.898691 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32124d8a-286f-4a98-80ca-5b3643d63af7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:30.899354 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32124d8a-286f-4a98-80ca-5b3643d63af7 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:30.899640 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32124d8a-286f-4a98-80ca-5b3643d63af7 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:30.899766 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862512dc-d7ad-45ae-8ba8-ee10c48a257a None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:30.899917 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32124d8a-286f-4a98-80ca-5b3643d63af7 None admin] PATH_INFO: `/v3/projects` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:30.900705 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32124d8a-286f-4a98-80ca-5b3643d63af7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:30.901172 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32124d8a-286f-4a98-80ca-5b3643d63af7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:30.903186 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32124d8a-286f-4a98-80ca-5b3643d63af7 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:30.915682 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 289/579] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:24:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:30.916480 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32124d8a-286f-4a98-80ca-5b3643d63af7 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:30.928387 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:30.939700 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2347762-0132-4d46-8668-3e1d943b5b6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:30.940386 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2347762-0132-4d46-8668-3e1d943b5b6b None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:30.940673 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2347762-0132-4d46-8668-3e1d943b5b6b None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:30.940953 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2347762-0132-4d46-8668-3e1d943b5b6b None admin] PATH_INFO: `/v3/users` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:30.941795 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2347762-0132-4d46-8668-3e1d943b5b6b None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:30.942143 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2347762-0132-4d46-8668-3e1d943b5b6b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:30.943595 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2347762-0132-4d46-8668-3e1d943b5b6b None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:30.961355 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 291/580] 10.222.0.57 () {56 vars in 1137 bytes} [Thu May 19 14:24:30 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:30.962124 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2347762-0132-4d46-8668-3e1d943b5b6b None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:31.022174 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32124d8a-286f-4a98-80ca-5b3643d63af7 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:31.057159 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9946e0e-fdba-4759-8152-c1f376357cb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:31.057983 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9946e0e-fdba-4759-8152-c1f376357cb9 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:31.058266 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9946e0e-fdba-4759-8152-c1f376357cb9 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:31.058924 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9946e0e-fdba-4759-8152-c1f376357cb9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:31.062711 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2347762-0132-4d46-8668-3e1d943b5b6b None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:31.064234 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:31.065631 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9946e0e-fdba-4759-8152-c1f376357cb9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:31.067199 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9946e0e-fdba-4759-8152-c1f376357cb9 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:31.081367 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-d9946e0e-fdba-4759-8152-c1f376357cb9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:24:31.081367 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:31.081367 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:31.081367 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:31.081367 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:31.081367 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:31.081367 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:31.081367 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:31.081367 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:31.081367 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:31.081367 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:31.081367 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:31.081367 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:24:31.081367 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:24:31.081367 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:24:31.082501 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:24:31.082501 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:31.082501 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:31.082501 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:31.082501 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:31.082501 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:31.082501 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:31.082501 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:31.082501 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:24:31.082501 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:31.082501 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:31.082501 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:31.082501 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:31.082501 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:31.082501 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:31.082501 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:31.082501 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:31.083506 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:24:31.083506 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:24:31.083506 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:24:31.083506 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:24:31.083506 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:24:31.083506 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:24:31.083506 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:24:31.083506 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:24:31.083506 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 290/581] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:24:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:31.083506 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9946e0e-fdba-4759-8152-c1f376357cb9 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:31.102221 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69af04f1-df55-41d2-8df5-97d7d75b065c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:31.102892 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69af04f1-df55-41d2-8df5-97d7d75b065c None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:31.103178 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69af04f1-df55-41d2-8df5-97d7d75b065c None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:31.103458 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69af04f1-df55-41d2-8df5-97d7d75b065c None admin] PATH_INFO: `/v3/roles` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:31.104210 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69af04f1-df55-41d2-8df5-97d7d75b065c None admin] RBAC: Adding query filter params (name=service) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:31.104521 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69af04f1-df55-41d2-8df5-97d7d75b065c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:31.106023 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69af04f1-df55-41d2-8df5-97d7d75b065c None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:31.114280 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 292/582] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 19 14:24:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:31.115088 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69af04f1-df55-41d2-8df5-97d7d75b065c None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:31.158461 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:31.190338 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9946e0e-fdba-4759-8152-c1f376357cb9 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:31.213321 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69af04f1-df55-41d2-8df5-97d7d75b065c None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:31.225086 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf9cac8-8d90-4b2d-93e8-51c8aa26240c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:31.225743 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf9cac8-8d90-4b2d-93e8-51c8aa26240c None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:31.226052 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf9cac8-8d90-4b2d-93e8-51c8aa26240c None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:31.226327 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf9cac8-8d90-4b2d-93e8-51c8aa26240c None admin] PATH_INFO: `/v3/domains` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:31.227040 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf9cac8-8d90-4b2d-93e8-51c8aa26240c None admin] RBAC: Adding query filter params (name=Default) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:31.227344 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf9cac8-8d90-4b2d-93e8-51c8aa26240c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:31.228696 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf9cac8-8d90-4b2d-93e8-51c8aa26240c None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:31.240385 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 291/583] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:24:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:31.241367 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf9cac8-8d90-4b2d-93e8-51c8aa26240c None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:31.247944 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03efc35-20fb-4305-b171-fbb57f186617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:31.248599 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03efc35-20fb-4305-b171-fbb57f186617 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:31.248888 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03efc35-20fb-4305-b171-fbb57f186617 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:31.249165 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03efc35-20fb-4305-b171-fbb57f186617 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:31.250040 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03efc35-20fb-4305-b171-fbb57f186617 None admin] RBAC: Adding query filter params (role.id=b0b74fefb27e4a0ebd895893aa9862d7, scope.project.id=252d9673f08c479a98d04fee3f5856a4, user.id=d3e544b4989e49a19536b40de5069bcf) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:31.250354 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03efc35-20fb-4305-b171-fbb57f186617 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:31.251822 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03efc35-20fb-4305-b171-fbb57f186617 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:31.257274 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 293/584] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:24:31 2022] GET /identity/v3/role_assignments?user.id=d3e544b4989e49a19536b40de5069bcf&scope.project.id=252d9673f08c479a98d04fee3f5856a4&role.id=b0b74fefb27e4a0ebd895893aa9862d7 => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:31.258074 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03efc35-20fb-4305-b171-fbb57f186617 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:31.292306 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:31.351855 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf9cac8-8d90-4b2d-93e8-51c8aa26240c None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:31.378041 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03efc35-20fb-4305-b171-fbb57f186617 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:31.384371 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:31.393957 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bb8936-4365-4f5d-9bc2-49616f55f77f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:31.394785 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bb8936-4365-4f5d-9bc2-49616f55f77f None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:31.395071 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bb8936-4365-4f5d-9bc2-49616f55f77f None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:31.395349 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bb8936-4365-4f5d-9bc2-49616f55f77f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:31.407231 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bb8936-4365-4f5d-9bc2-49616f55f77f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:31.408704 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bb8936-4365-4f5d-9bc2-49616f55f77f None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:31.422150 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4de3e5-65b9-40ef-b1fc-4e095140ae37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:31.423133 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4de3e5-65b9-40ef-b1fc-4e095140ae37 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:31.423588 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4de3e5-65b9-40ef-b1fc-4e095140ae37 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:31.424045 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4de3e5-65b9-40ef-b1fc-4e095140ae37 None admin] PATH_INFO: `/v3/projects` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:31.425003 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4de3e5-65b9-40ef-b1fc-4e095140ae37 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:31.425475 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4de3e5-65b9-40ef-b1fc-4e095140ae37 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:31.427146 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4de3e5-65b9-40ef-b1fc-4e095140ae37 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:31.428321 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-c2bb8936-4365-4f5d-9bc2-49616f55f77f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:31.428321 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:31.428321 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:31.428321 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:31.428321 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:31.428321 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:31.428321 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:31.428321 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:31.428321 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:31.428321 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:31.428321 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:31.428321 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:31.428321 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:24:31.428321 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:24:31.428321 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:24:31.429468 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:24:31.429468 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:31.429468 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:31.429468 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:31.429468 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:31.429468 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:31.429468 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:31.429468 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:31.429468 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:24:31.429468 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:31.429468 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:31.429468 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:31.429468 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:31.429468 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:31.429468 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:31.429468 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:31.429468 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:31.430530 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:24:31.430530 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:24:31.430530 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:31.430530 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:24:31.430530 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 292/585] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:24:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:31.430530 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bb8936-4365-4f5d-9bc2-49616f55f77f None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:31.448218 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 294/586] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:24:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:31.449076 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4de3e5-65b9-40ef-b1fc-4e095140ae37 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:31.546922 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:31.554487 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bb8936-4365-4f5d-9bc2-49616f55f77f None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:31.563262 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4de3e5-65b9-40ef-b1fc-4e095140ae37 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:31.590721 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01701cf8-b3f6-4371-bb35-98b2fc68cc61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:31.591860 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01701cf8-b3f6-4371-bb35-98b2fc68cc61 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:31.592422 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01701cf8-b3f6-4371-bb35-98b2fc68cc61 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:31.592706 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01701cf8-b3f6-4371-bb35-98b2fc68cc61 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:31.601746 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdd0afd-bced-47b0-94a6-6b2f15616d93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:31.602656 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdd0afd-bced-47b0-94a6-6b2f15616d93 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:31.602943 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdd0afd-bced-47b0-94a6-6b2f15616d93 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:31.603221 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdd0afd-bced-47b0-94a6-6b2f15616d93 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:31.605303 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01701cf8-b3f6-4371-bb35-98b2fc68cc61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:31.607308 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01701cf8-b3f6-4371-bb35-98b2fc68cc61 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:31.610704 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdd0afd-bced-47b0-94a6-6b2f15616d93 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:31.612157 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdd0afd-bced-47b0-94a6-6b2f15616d93 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:31.627236 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-01701cf8-b3f6-4371-bb35-98b2fc68cc61 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:31.627236 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:31.627236 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:31.627236 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:31.627236 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:31.627236 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:31.627236 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:31.627236 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:31.627236 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:31.627236 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:31.627236 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:31.627236 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:31.627236 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:24:31.627236 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:24:31.627236 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:24:31.628250 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:24:31.628250 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:31.628250 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:31.628250 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:31.628250 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:31.628250 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:31.628250 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:31.628250 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:31.628250 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:24:31.628250 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:31.628250 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:31.628250 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:31.628250 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:31.628250 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:31.628250 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:31.628250 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:31.628250 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:31.629256 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:24:31.629256 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:24:31.629256 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:31.629256 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:24:31.629499 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-1fdd0afd-bced-47b0-94a6-6b2f15616d93 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 14:24:31.629499 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:31.629499 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:31.629499 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:31.629499 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:31.629499 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:31.629499 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:31.629499 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:31.629499 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:31.629499 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:31.629499 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:31.629499 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:31.629499 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:24:31.629499 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:24:31.629499 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:24:31.629499 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:24:31.631084 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:31.631084 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:31.631084 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:24:31.631084 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:24:31.631084 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:24:31.631084 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:24:31.631084 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:31.631084 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:31.631084 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:31.631084 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:31.631084 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:31.631084 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:24:31.631084 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:31.631084 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:31.631084 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:31.631084 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:31.631084 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:31.631084 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:31.632123 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:31.632123 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:31.632123 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:24:31.632123 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:24:31.632123 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:24:31.632123 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:24:31.632123 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:24:31.632123 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:24:31.632123 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 14:24:31.632123 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:24:31.633402 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 293/587] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:24:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:31.633402 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01701cf8-b3f6-4371-bb35-98b2fc68cc61 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:31.633712 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 295/588] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:24:31 2022] GET /identity/v3/users/cinder => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:31.639120 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdd0afd-bced-47b0-94a6-6b2f15616d93 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:31.671241 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:31.746870 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01701cf8-b3f6-4371-bb35-98b2fc68cc61 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:31.756913 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdd0afd-bced-47b0-94a6-6b2f15616d93 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:31.784807 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012365b2-9b1f-4288-8bb6-516e55d83c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:31.785471 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012365b2-9b1f-4288-8bb6-516e55d83c7b None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:31.785759 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012365b2-9b1f-4288-8bb6-516e55d83c7b None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:31.786075 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012365b2-9b1f-4288-8bb6-516e55d83c7b None admin] PATH_INFO: `/v3/domains` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:31.786795 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012365b2-9b1f-4288-8bb6-516e55d83c7b None admin] RBAC: Adding query filter params (name=Default) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:31.787098 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012365b2-9b1f-4288-8bb6-516e55d83c7b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:31.788478 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012365b2-9b1f-4288-8bb6-516e55d83c7b None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:31.795721 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11745cb-8b4a-4ccb-b6fa-379f9b8f0d82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:31.796619 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11745cb-8b4a-4ccb-b6fa-379f9b8f0d82 None admin] REQUEST_METHOD: `PUT` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:31.796907 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11745cb-8b4a-4ccb-b6fa-379f9b8f0d82 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:31.797237 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11745cb-8b4a-4ccb-b6fa-379f9b8f0d82 None admin] PATH_INFO: `/v3/projects/252d9673f08c479a98d04fee3f5856a4/users/5f28de22c2ee4c53b6d14d5b0ddf7fe9/roles/b0b74fefb27e4a0ebd895893aa9862d7` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:31.811978 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 294/589] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:24:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:31.811978 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012365b2-9b1f-4288-8bb6-516e55d83c7b None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:31.818382 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11745cb-8b4a-4ccb-b6fa-379f9b8f0d82 None admin] RBAC: Authorizing `identity:create_grant(project_id=252d9673f08c479a98d04fee3f5856a4, user_id=5f28de22c2ee4c53b6d14d5b0ddf7fe9, role_id=b0b74fefb27e4a0ebd895893aa9862d7)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:31.819985 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11745cb-8b4a-4ccb-b6fa-379f9b8f0d82 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:31.828954 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:31.845941 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 296/590] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 14:24:31 2022] PUT /identity/v3/projects/252d9673f08c479a98d04fee3f5856a4/users/5f28de22c2ee4c53b6d14d5b0ddf7fe9/roles/b0b74fefb27e4a0ebd895893aa9862d7 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:24:31.847097 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11745cb-8b4a-4ccb-b6fa-379f9b8f0d82 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:31.920259 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:31.930512 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012365b2-9b1f-4288-8bb6-516e55d83c7b None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:31.973956 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67da4e1d-3fee-4eb6-928d-4d0ec363adb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:31.975070 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67da4e1d-3fee-4eb6-928d-4d0ec363adb0 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:31.975635 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67da4e1d-3fee-4eb6-928d-4d0ec363adb0 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:31.976230 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67da4e1d-3fee-4eb6-928d-4d0ec363adb0 None admin] PATH_INFO: `/v3/domains` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:31.977298 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67da4e1d-3fee-4eb6-928d-4d0ec363adb0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:31.977825 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67da4e1d-3fee-4eb6-928d-4d0ec363adb0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:31.979522 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67da4e1d-3fee-4eb6-928d-4d0ec363adb0 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:31.999522 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11745cb-8b4a-4ccb-b6fa-379f9b8f0d82 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:32.000504 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 295/591] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:24:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:32.001706 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67da4e1d-3fee-4eb6-928d-4d0ec363adb0 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:32.045476 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea4b9c6-8551-4be2-8983-d163925fa5d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:32.046198 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea4b9c6-8551-4be2-8983-d163925fa5d0 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:32.046482 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea4b9c6-8551-4be2-8983-d163925fa5d0 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:32.046762 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea4b9c6-8551-4be2-8983-d163925fa5d0 None admin] PATH_INFO: `/v3/users` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:32.047665 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea4b9c6-8551-4be2-8983-d163925fa5d0 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:32.047918 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea4b9c6-8551-4be2-8983-d163925fa5d0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:32.049392 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea4b9c6-8551-4be2-8983-d163925fa5d0 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:32.071152 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 297/592] 10.222.0.57 () {56 vars in 1137 bytes} [Thu May 19 14:24:31 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:32.071593 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea4b9c6-8551-4be2-8983-d163925fa5d0 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:32.123467 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67da4e1d-3fee-4eb6-928d-4d0ec363adb0 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:32.147802 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:32.172295 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb499596-b8bd-47ce-96e7-8f4694d49609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:32.173936 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb499596-b8bd-47ce-96e7-8f4694d49609 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:32.174736 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb499596-b8bd-47ce-96e7-8f4694d49609 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:32.175254 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb499596-b8bd-47ce-96e7-8f4694d49609 None admin] PATH_INFO: `/v3/projects/service` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:32.185664 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb499596-b8bd-47ce-96e7-8f4694d49609 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:32.188466 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb499596-b8bd-47ce-96e7-8f4694d49609 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:32.196252 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea4b9c6-8551-4be2-8983-d163925fa5d0 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:32.207524 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-bb499596-b8bd-47ce-96e7-8f4694d49609 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:24:32.207524 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:32.207524 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:32.207524 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:32.207524 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:32.207524 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:32.207524 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:32.207524 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:32.207524 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:32.207524 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:32.207524 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:32.207524 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:32.207524 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:24:32.207524 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:24:32.207524 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:24:32.208527 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:24:32.208527 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:32.208527 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:32.208527 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:32.208527 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:32.208527 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:32.208527 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:32.208527 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:32.208527 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:24:32.208527 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:32.208527 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:32.208527 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:32.208527 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:32.208527 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:32.208527 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:32.208527 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:32.208527 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:32.209525 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:24:32.209525 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:24:32.209525 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:24:32.209525 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:24:32.209525 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:24:32.209525 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:24:32.209525 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:24:32.209525 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:24:32.211797 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 296/593] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:24:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:32.217127 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb499596-b8bd-47ce-96e7-8f4694d49609 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:32.233877 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:32.254063 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319651d1-983c-4a67-87cb-17df0074c725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:32.255388 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319651d1-983c-4a67-87cb-17df0074c725 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:32.256751 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319651d1-983c-4a67-87cb-17df0074c725 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:32.257196 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319651d1-983c-4a67-87cb-17df0074c725 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:32.267392 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319651d1-983c-4a67-87cb-17df0074c725 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:32.269974 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319651d1-983c-4a67-87cb-17df0074c725 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:32.292600 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-319651d1-983c-4a67-87cb-17df0074c725 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 19 14:24:32.292600 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:32.292600 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:32.292600 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:32.292600 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:32.292600 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:32.292600 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:32.292600 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:32.292600 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:32.292600 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:32.292600 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:32.292600 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:32.292600 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:24:32.292600 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:24:32.292600 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:24:32.292600 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:24:32.294592 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:32.294592 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:32.294592 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:24:32.294592 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:24:32.294592 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:24:32.294592 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:24:32.294592 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:32.294592 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:32.294592 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:32.294592 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:32.294592 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:32.294592 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:24:32.294592 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:32.294592 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:32.294592 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:32.294592 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:32.294592 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:32.294592 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:32.295854 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:32.295854 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:32.295854 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:24:32.295854 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:24:32.295854 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:24:32.295854 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:24:32.295854 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:24:32.295854 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:24:32.295854 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 19 14:24:32.295854 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:24:32.304121 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 298/594] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 14:24:32 2022] GET /identity/v3/users/neutron => generated 85 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:32.305309 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319651d1-983c-4a67-87cb-17df0074c725 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:32.332400 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb499596-b8bd-47ce-96e7-8f4694d49609 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:32.379804 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d294b88e-41f5-41b3-b18b-ea96a9bd8c16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:32.381375 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d294b88e-41f5-41b3-b18b-ea96a9bd8c16 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:32.381982 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d294b88e-41f5-41b3-b18b-ea96a9bd8c16 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:32.382440 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d294b88e-41f5-41b3-b18b-ea96a9bd8c16 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:32.398945 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d294b88e-41f5-41b3-b18b-ea96a9bd8c16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:32.401163 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d294b88e-41f5-41b3-b18b-ea96a9bd8c16 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:32.426316 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-d294b88e-41f5-41b3-b18b-ea96a9bd8c16 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:32.426316 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:32.426316 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:32.426316 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:32.426316 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:32.426316 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:32.426316 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:32.426316 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:32.426316 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:32.426316 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:32.426316 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:32.426316 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:32.426316 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:24:32.426316 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:24:32.426316 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:24:32.427548 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:24:32.427548 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:32.427548 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:32.427548 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:32.427548 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:32.427548 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:32.427548 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:32.427548 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:32.427548 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:24:32.427548 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:32.427548 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:32.427548 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:32.427548 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:32.427548 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:32.427548 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:32.427548 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:32.427548 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:32.428734 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:24:32.428734 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:24:32.428734 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:32.428734 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:24:32.431355 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 297/595] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:24:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:32.432327 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d294b88e-41f5-41b3-b18b-ea96a9bd8c16 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:32.435654 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319651d1-983c-4a67-87cb-17df0074c725 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:32.462271 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:32.478567 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f875a69-a551-4653-935f-f4d6266b98f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:32.479084 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f875a69-a551-4653-935f-f4d6266b98f8 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:32.479435 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f875a69-a551-4653-935f-f4d6266b98f8 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:32.479815 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f875a69-a551-4653-935f-f4d6266b98f8 None admin] PATH_INFO: `/v3/projects` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:32.480990 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f875a69-a551-4653-935f-f4d6266b98f8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:32.481385 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f875a69-a551-4653-935f-f4d6266b98f8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:32.482947 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f875a69-a551-4653-935f-f4d6266b98f8 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:32.501014 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 299/596] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:24:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:32.501014 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f875a69-a551-4653-935f-f4d6266b98f8 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:32.518117 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:32.555513 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d294b88e-41f5-41b3-b18b-ea96a9bd8c16 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:32.592892 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ded29d3-3c29-470e-98a1-356cfc17c7a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:32.593987 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ded29d3-3c29-470e-98a1-356cfc17c7a5 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:32.594491 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ded29d3-3c29-470e-98a1-356cfc17c7a5 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:32.594986 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ded29d3-3c29-470e-98a1-356cfc17c7a5 None admin] PATH_INFO: `/v3/users` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:32.596017 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ded29d3-3c29-470e-98a1-356cfc17c7a5 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:32.596532 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ded29d3-3c29-470e-98a1-356cfc17c7a5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:32.598202 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ded29d3-3c29-470e-98a1-356cfc17c7a5 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:32.608326 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f875a69-a551-4653-935f-f4d6266b98f8 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:32.623561 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 298/597] 10.222.0.57 () {56 vars in 1139 bytes} [Thu May 19 14:24:32 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:32.624741 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ded29d3-3c29-470e-98a1-356cfc17c7a5 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:32.649018 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba61f179-2413-4aee-bdd1-8cd16597d89c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:32.649688 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba61f179-2413-4aee-bdd1-8cd16597d89c None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:32.650066 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba61f179-2413-4aee-bdd1-8cd16597d89c None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:32.650284 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba61f179-2413-4aee-bdd1-8cd16597d89c None admin] PATH_INFO: `/v3/domains` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:32.651017 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba61f179-2413-4aee-bdd1-8cd16597d89c None admin] RBAC: Adding query filter params (name=Default) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:32.651335 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba61f179-2413-4aee-bdd1-8cd16597d89c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:32.653908 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba61f179-2413-4aee-bdd1-8cd16597d89c None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:32.667116 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 300/598] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:24:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:32.668126 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba61f179-2413-4aee-bdd1-8cd16597d89c None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:32.706787 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:32.735473 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ded29d3-3c29-470e-98a1-356cfc17c7a5 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:32.760202 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:32.772607 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f3951c-efd8-4401-b11f-22857e5a66eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:32.773643 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f3951c-efd8-4401-b11f-22857e5a66eb None admin] REQUEST_METHOD: `PUT` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:32.774156 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f3951c-efd8-4401-b11f-22857e5a66eb None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:32.774523 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f3951c-efd8-4401-b11f-22857e5a66eb None admin] PATH_INFO: `/v3/projects/252d9673f08c479a98d04fee3f5856a4/users/afec042821074a889ef9afa75bb8bf40/roles/b0b74fefb27e4a0ebd895893aa9862d7` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:32.783539 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba61f179-2413-4aee-bdd1-8cd16597d89c None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:32.790672 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f3951c-efd8-4401-b11f-22857e5a66eb None admin] RBAC: Authorizing `identity:create_grant(project_id=252d9673f08c479a98d04fee3f5856a4, user_id=afec042821074a889ef9afa75bb8bf40, role_id=b0b74fefb27e4a0ebd895893aa9862d7)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:32.792250 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f3951c-efd8-4401-b11f-22857e5a66eb None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:32.818124 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 299/599] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 14:24:32 2022] PUT /identity/v3/projects/252d9673f08c479a98d04fee3f5856a4/users/afec042821074a889ef9afa75bb8bf40/roles/b0b74fefb27e4a0ebd895893aa9862d7 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:24:32.819293 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f3951c-efd8-4401-b11f-22857e5a66eb None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:32.822837 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b048e85-3831-4532-a726-d616b5dab5c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:32.823869 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b048e85-3831-4532-a726-d616b5dab5c3 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:32.824943 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b048e85-3831-4532-a726-d616b5dab5c3 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:32.825458 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b048e85-3831-4532-a726-d616b5dab5c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:32.837393 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b048e85-3831-4532-a726-d616b5dab5c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:32.839225 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b048e85-3831-4532-a726-d616b5dab5c3 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:32.862357 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-1b048e85-3831-4532-a726-d616b5dab5c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:32.862357 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:32.862357 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:32.862357 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:32.862357 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:32.862357 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:32.862357 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:32.862357 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:32.862357 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:32.862357 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:32.862357 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:32.862357 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:32.862357 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:24:32.862357 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:24:32.862357 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:24:32.863454 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:24:32.863454 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:32.863454 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:32.863454 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:32.863454 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:32.863454 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:32.863454 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:32.863454 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:32.863454 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:24:32.863454 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:32.863454 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:32.863454 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:32.863454 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:32.863454 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:32.863454 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:32.863454 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:32.863454 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:32.864475 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:24:32.864475 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:24:32.864475 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:32.864475 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:24:32.867117 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 301/600] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:24:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:32.879217 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b048e85-3831-4532-a726-d616b5dab5c3 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:32.961904 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f3951c-efd8-4401-b11f-22857e5a66eb None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:32.994519 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d96b2b8-1fce-4baf-a09e-1ec002c3e805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:32.995377 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d96b2b8-1fce-4baf-a09e-1ec002c3e805 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:32.996475 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d96b2b8-1fce-4baf-a09e-1ec002c3e805 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:32.996847 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d96b2b8-1fce-4baf-a09e-1ec002c3e805 None admin] PATH_INFO: `/v3/projects/service` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:33.005121 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d96b2b8-1fce-4baf-a09e-1ec002c3e805 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:33.007073 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d96b2b8-1fce-4baf-a09e-1ec002c3e805 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:33.022561 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-8d96b2b8-1fce-4baf-a09e-1ec002c3e805 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:24:33.022561 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:33.022561 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:33.022561 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:33.022561 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:33.022561 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:33.022561 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:33.022561 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:33.022561 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:33.022561 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:33.022561 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:33.022561 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:33.022561 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:24:33.022561 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:24:33.022561 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:24:33.023794 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:24:33.023794 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:33.023794 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:33.023794 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:33.023794 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:33.023794 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:33.023794 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:33.023794 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:33.023794 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:24:33.023794 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:33.023794 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:33.023794 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:33.023794 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:33.023794 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:33.023794 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:33.023794 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:33.023794 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:33.024828 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:24:33.024828 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:24:33.024828 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:24:33.024828 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:24:33.024828 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:24:33.024828 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:24:33.024828 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:24:33.024828 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:24:33.026100 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 300/601] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:24:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:33.033876 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d96b2b8-1fce-4baf-a09e-1ec002c3e805 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:33.059850 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:33.073669 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b048e85-3831-4532-a726-d616b5dab5c3 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:33.128380 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:33.129238 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3a5002-ab99-4f29-8115-fb13f1fd65c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:33.129958 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3a5002-ab99-4f29-8115-fb13f1fd65c6 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:33.130253 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3a5002-ab99-4f29-8115-fb13f1fd65c6 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:33.130559 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3a5002-ab99-4f29-8115-fb13f1fd65c6 None admin] PATH_INFO: `/v3/domains` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:33.131331 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3a5002-ab99-4f29-8115-fb13f1fd65c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:33.131653 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3a5002-ab99-4f29-8115-fb13f1fd65c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:33.133113 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3a5002-ab99-4f29-8115-fb13f1fd65c6 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:33.166217 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 302/602] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:24:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:24:33.174183 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3a5002-ab99-4f29-8115-fb13f1fd65c6 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:33.229392 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d96b2b8-1fce-4baf-a09e-1ec002c3e805 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:33.268104 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5696febd-a6a7-4050-ba1c-4bbddeb50d88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:33.268778 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5696febd-a6a7-4050-ba1c-4bbddeb50d88 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:33.269070 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5696febd-a6a7-4050-ba1c-4bbddeb50d88 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:33.269352 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5696febd-a6a7-4050-ba1c-4bbddeb50d88 None admin] PATH_INFO: `/v3/projects` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:33.271775 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5696febd-a6a7-4050-ba1c-4bbddeb50d88 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:33.272098 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5696febd-a6a7-4050-ba1c-4bbddeb50d88 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:33.273649 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5696febd-a6a7-4050-ba1c-4bbddeb50d88 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:33.287798 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 301/603] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:24:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:24:33.290449 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3a5002-ab99-4f29-8115-fb13f1fd65c6 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:33.293681 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5696febd-a6a7-4050-ba1c-4bbddeb50d88 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:33.330193 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0407ca-481b-497b-9487-6a4b621949e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:33.330738 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0407ca-481b-497b-9487-6a4b621949e8 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:33.331240 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0407ca-481b-497b-9487-6a4b621949e8 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:33.331757 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0407ca-481b-497b-9487-6a4b621949e8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:33.340296 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0407ca-481b-497b-9487-6a4b621949e8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:33.342338 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0407ca-481b-497b-9487-6a4b621949e8 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:33.359203 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-ff0407ca-481b-497b-9487-6a4b621949e8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:24:33.359203 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:33.359203 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:33.359203 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:33.359203 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:33.359203 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:33.359203 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:33.359203 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:33.359203 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:33.359203 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:33.359203 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:33.359203 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:33.359203 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:24:33.359203 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:24:33.359203 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:24:33.360439 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:24:33.360439 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:33.360439 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:33.360439 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:33.360439 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:33.360439 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:33.360439 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:33.360439 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:33.360439 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:24:33.360439 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:33.360439 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:33.360439 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:33.360439 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:33.360439 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:33.360439 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:33.360439 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:33.360439 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:33.361958 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:24:33.361958 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:24:33.361958 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:24:33.361958 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:24:33.361958 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:24:33.361958 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:24:33.361958 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:24:33.361958 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:24:33.368879 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 303/604] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:24:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:24:33.370506 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0407ca-481b-497b-9487-6a4b621949e8 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:33.385582 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:33.515289 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0407ca-481b-497b-9487-6a4b621949e8 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:33.551717 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:33.560815 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b140f67c-2c86-4c27-aadc-d2a40746a8f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:33.561720 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b140f67c-2c86-4c27-aadc-d2a40746a8f6 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:33.562192 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b140f67c-2c86-4c27-aadc-d2a40746a8f6 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:33.563087 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b140f67c-2c86-4c27-aadc-d2a40746a8f6 None admin] PATH_INFO: `/v3/projects` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:33.564231 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b140f67c-2c86-4c27-aadc-d2a40746a8f6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:33.564720 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b140f67c-2c86-4c27-aadc-d2a40746a8f6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:33.566382 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b140f67c-2c86-4c27-aadc-d2a40746a8f6 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:33.568084 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5696febd-a6a7-4050-ba1c-4bbddeb50d88 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:33.582288 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 304/605] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:24:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:33.589494 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b140f67c-2c86-4c27-aadc-d2a40746a8f6 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:33.636252 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd39ee0-52d3-4775-821f-d92b7ec62f19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:33.638184 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd39ee0-52d3-4775-821f-d92b7ec62f19 None admin] REQUEST_METHOD: `PUT` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:33.638815 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd39ee0-52d3-4775-821f-d92b7ec62f19 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:33.640462 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd39ee0-52d3-4775-821f-d92b7ec62f19 None admin] PATH_INFO: `/v3/projects/252d9673f08c479a98d04fee3f5856a4/users/95d18a2c8d8d4251ba8961a908574ee2/roles/b0b74fefb27e4a0ebd895893aa9862d7` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:33.677305 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd39ee0-52d3-4775-821f-d92b7ec62f19 None admin] RBAC: Authorizing `identity:create_grant(project_id=252d9673f08c479a98d04fee3f5856a4, user_id=95d18a2c8d8d4251ba8961a908574ee2, role_id=b0b74fefb27e4a0ebd895893aa9862d7)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:33.678793 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd39ee0-52d3-4775-821f-d92b7ec62f19 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:33.701716 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 302/606] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 14:24:33 2022] PUT /identity/v3/projects/252d9673f08c479a98d04fee3f5856a4/users/95d18a2c8d8d4251ba8961a908574ee2/roles/b0b74fefb27e4a0ebd895893aa9862d7 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:24:33.704161 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8d1235-6cda-42db-b197-f2f24b702c63 None None] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:33.704454 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8d1235-6cda-42db-b197-f2f24b702c63 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:33.705955 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8d1235-6cda-42db-b197-f2f24b702c63 None None] PATH_INFO: `/` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:33.707915 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 303/607] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:24:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:24:33.714525 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115c3bb0-0ed4-4da5-8188-fcb48db69021 None None] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:33.715524 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115c3bb0-0ed4-4da5-8188-fcb48db69021 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:33.715952 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115c3bb0-0ed4-4da5-8188-fcb48db69021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:33.728026 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:33.740140 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.auth.core [None req-115c3bb0-0ed4-4da5-8188-fcb48db69021 None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143586) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:24:33.742238 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.fernet_utils [None req-115c3bb0-0ed4-4da5-8188-fcb48db69021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143586) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:24:33.796469 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b140f67c-2c86-4c27-aadc-d2a40746a8f6 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:33.885733 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdedfb9-8787-49aa-88b4-55161ca301fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:33.886723 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdedfb9-8787-49aa-88b4-55161ca301fc None admin] REQUEST_METHOD: `PUT` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:33.887009 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdedfb9-8787-49aa-88b4-55161ca301fc None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:33.887290 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdedfb9-8787-49aa-88b4-55161ca301fc None admin] PATH_INFO: `/v3/projects/252d9673f08c479a98d04fee3f5856a4/users/0ed9caca45c9449b84582da44a1262ef/roles/b0b74fefb27e4a0ebd895893aa9862d7` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:33.890171 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 304/608] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:24:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:24:33.907812 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdedfb9-8787-49aa-88b4-55161ca301fc None admin] RBAC: Authorizing `identity:create_grant(project_id=252d9673f08c479a98d04fee3f5856a4, user_id=0ed9caca45c9449b84582da44a1262ef, role_id=b0b74fefb27e4a0ebd895893aa9862d7)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:33.909335 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdedfb9-8787-49aa-88b4-55161ca301fc None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:33.919830 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115c3bb0-0ed4-4da5-8188-fcb48db69021 None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:33.938198 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 305/609] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 14:24:33 2022] PUT /identity/v3/projects/252d9673f08c479a98d04fee3f5856a4/users/0ed9caca45c9449b84582da44a1262ef/roles/b0b74fefb27e4a0ebd895893aa9862d7 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 14:24:34.014881 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:34.053300 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115c3bb0-0ed4-4da5-8188-fcb48db69021 None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:34.101378 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7f3069-3213-4e77-be98-3206f758a201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:34.102297 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7f3069-3213-4e77-be98-3206f758a201 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:34.102582 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7f3069-3213-4e77-be98-3206f758a201 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:34.102856 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7f3069-3213-4e77-be98-3206f758a201 None admin] PATH_INFO: `/v3/roles/service` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:34.124580 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7f3069-3213-4e77-be98-3206f758a201 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:24:34.132879 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-fa7f3069-3213-4e77-be98-3206f758a201 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:24:34.132879 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:34.132879 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:34.132879 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:34.132879 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:34.132879 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:34.132879 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:34.132879 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:34.132879 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:34.132879 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:34.132879 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:34.132879 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:34.132879 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:24:34.132879 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:24:34.132879 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:24:34.132879 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:24:34.135813 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:24:34.135813 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:24:34.135813 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:24:34.135813 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:24:34.135813 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:34.135813 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:34.135813 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:34.135813 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:34.135813 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:34.135813 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:34.135813 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:34.135813 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:24:34.135813 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:34.135813 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:34.135813 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:34.135813 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:34.138542 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:34.138542 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:34.138542 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:34.138542 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:34.138542 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:24:34.138542 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:24:34.138542 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:24:34.138542 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:24:34.138542 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:24:34.138542 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:24:34.138542 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:24:34.138542 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:24:34.138542 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 305/610] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 14:24:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:34.145872 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdedfb9-8787-49aa-88b4-55161ca301fc None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:34.256318 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdedfb9-8787-49aa-88b4-55161ca301fc None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:34.274051 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:34.305569 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ee3079-a18f-4698-9fee-5e336b3d61f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:34.306330 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ee3079-a18f-4698-9fee-5e336b3d61f5 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:34.306618 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ee3079-a18f-4698-9fee-5e336b3d61f5 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:34.306899 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ee3079-a18f-4698-9fee-5e336b3d61f5 None admin] PATH_INFO: `/v3/roles` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:34.307739 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ee3079-a18f-4698-9fee-5e336b3d61f5 None admin] RBAC: Adding query filter params (name=service) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:34.308054 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ee3079-a18f-4698-9fee-5e336b3d61f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:34.310501 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ee3079-a18f-4698-9fee-5e336b3d61f5 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:34.319775 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b97c5c5-199a-47a5-a633-a4d4f74d648b None None] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:34.319775 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b97c5c5-199a-47a5-a633-a4d4f74d648b None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:34.319775 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b97c5c5-199a-47a5-a633-a4d4f74d648b None None] PATH_INFO: `/` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:34.320370 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 306/611] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 19 14:24:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:34.321704 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 306/612] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:24:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:24:34.327797 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ee3079-a18f-4698-9fee-5e336b3d61f5 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:34.332920 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd08b36d-5663-4687-b9a6-6a1ab120cb44 None None] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:34.333202 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd08b36d-5663-4687-b9a6-6a1ab120cb44 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:34.333485 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd08b36d-5663-4687-b9a6-6a1ab120cb44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:34.368685 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.auth.core [None req-cd08b36d-5663-4687-b9a6-6a1ab120cb44 None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143586) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:24:34.369920 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.fernet_utils [None req-cd08b36d-5663-4687-b9a6-6a1ab120cb44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143586) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:24:34.477376 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ee3079-a18f-4698-9fee-5e336b3d61f5 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:34.519271 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7074784-a47b-40c2-b84d-ed2d2f92dd19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:34.520790 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7074784-a47b-40c2-b84d-ed2d2f92dd19 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:34.521078 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7074784-a47b-40c2-b84d-ed2d2f92dd19 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:34.521355 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7074784-a47b-40c2-b84d-ed2d2f92dd19 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:34.535071 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 307/613] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:24:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:24:34.540284 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7074784-a47b-40c2-b84d-ed2d2f92dd19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:34.541908 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7074784-a47b-40c2-b84d-ed2d2f92dd19 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:34.560010 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd08b36d-5663-4687-b9a6-6a1ab120cb44 None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:34.565600 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-c7074784-a47b-40c2-b84d-ed2d2f92dd19 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:34.565600 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:34.565600 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:34.565600 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:34.565600 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:34.565600 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:34.565600 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:34.565600 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:34.565600 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:34.565600 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:34.565600 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:34.565600 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:34.565600 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:24:34.565600 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:24:34.565600 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:24:34.566781 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:24:34.566781 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:34.566781 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:34.566781 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:34.566781 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:34.566781 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:34.566781 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:34.566781 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:34.566781 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:24:34.566781 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:34.566781 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:34.566781 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:34.566781 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:34.566781 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:34.566781 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:34.566781 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:34.566781 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:34.567790 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:24:34.567790 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:24:34.567790 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:34.567790 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:24:34.567790 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 307/614] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:24:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:34.577274 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7074784-a47b-40c2-b84d-ed2d2f92dd19 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:34.637093 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:34.665537 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:34.729271 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd08b36d-5663-4687-b9a6-6a1ab120cb44 None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:34.763831 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7074784-a47b-40c2-b84d-ed2d2f92dd19 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:34.795261 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f57da6-0154-4648-a802-dd0b801299c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:34.796067 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f57da6-0154-4648-a802-dd0b801299c7 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:34.796350 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f57da6-0154-4648-a802-dd0b801299c7 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:34.796627 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f57da6-0154-4648-a802-dd0b801299c7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:34.814394 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f57da6-0154-4648-a802-dd0b801299c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:24:34.825653 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-f3f57da6-0154-4648-a802-dd0b801299c7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:24:34.825653 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:34.825653 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:34.825653 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:34.825653 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:34.825653 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:34.825653 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:34.825653 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:34.825653 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:34.825653 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:34.825653 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:34.825653 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:34.825653 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:24:34.825653 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:24:34.825653 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:24:34.825653 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:24:34.826755 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:24:34.826755 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:24:34.826755 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:24:34.826755 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:24:34.826755 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:34.826755 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:34.826755 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:34.826755 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:34.826755 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:34.826755 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:34.826755 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:34.826755 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:24:34.826755 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:34.826755 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:34.826755 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:34.826755 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:34.827698 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:34.827698 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:34.827698 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:34.827698 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:34.827698 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:24:34.827698 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:24:34.827698 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:24:34.827698 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:24:34.827698 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:24:34.827698 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:24:34.827698 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:24:34.827698 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:24:34.828966 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 308/615] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 14:24:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:24:34.834021 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd228f32-2a1e-4ed0-8407-cd81acdc8c1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:34.834709 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd228f32-2a1e-4ed0-8407-cd81acdc8c1c None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:34.834997 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd228f32-2a1e-4ed0-8407-cd81acdc8c1c None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:34.835272 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd228f32-2a1e-4ed0-8407-cd81acdc8c1c None admin] PATH_INFO: `/v3/domains` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:34.836009 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd228f32-2a1e-4ed0-8407-cd81acdc8c1c None admin] RBAC: Adding query filter params (name=Default) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:34.836320 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd228f32-2a1e-4ed0-8407-cd81acdc8c1c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:34.837716 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd228f32-2a1e-4ed0-8407-cd81acdc8c1c None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:34.840844 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f57da6-0154-4648-a802-dd0b801299c7 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:34.863234 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 308/616] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:24:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:24:34.869233 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd228f32-2a1e-4ed0-8407-cd81acdc8c1c None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:35.029508 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f57da6-0154-4648-a802-dd0b801299c7 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:35.030298 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:35.044302 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd228f32-2a1e-4ed0-8407-cd81acdc8c1c None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:35.071086 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:35.091744 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e85646-b4f4-4669-813e-e369bf5c858f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:35.092417 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e85646-b4f4-4669-813e-e369bf5c858f None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:35.092701 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e85646-b4f4-4669-813e-e369bf5c858f None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:35.092977 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e85646-b4f4-4669-813e-e369bf5c858f None admin] PATH_INFO: `/v3/roles` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:35.093727 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e85646-b4f4-4669-813e-e369bf5c858f None admin] RBAC: Adding query filter params (name=service) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:35.094057 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e85646-b4f4-4669-813e-e369bf5c858f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:35.095426 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e85646-b4f4-4669-813e-e369bf5c858f None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:35.107667 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 309/617] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 19 14:24:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:35.108206 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba46000-4a3b-45d3-a9c3-c7fdad8f5c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:35.109266 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba46000-4a3b-45d3-a9c3-c7fdad8f5c55 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:35.109802 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba46000-4a3b-45d3-a9c3-c7fdad8f5c55 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:35.110326 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba46000-4a3b-45d3-a9c3-c7fdad8f5c55 None admin] PATH_INFO: `/v3/users/swift` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:35.117864 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e85646-b4f4-4669-813e-e369bf5c858f None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:35.123506 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba46000-4a3b-45d3-a9c3-c7fdad8f5c55 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:35.124975 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba46000-4a3b-45d3-a9c3-c7fdad8f5c55 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:35.148043 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-cba46000-4a3b-45d3-a9c3-c7fdad8f5c55 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 14:24:35.148043 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:35.148043 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:35.148043 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:35.148043 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:35.148043 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:35.148043 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:35.148043 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:35.148043 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:35.148043 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:35.148043 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:35.148043 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:35.148043 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:24:35.148043 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:24:35.148043 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:24:35.148043 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:24:35.149148 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:35.149148 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:35.149148 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:24:35.149148 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:24:35.149148 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:24:35.149148 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:24:35.149148 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:35.149148 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:35.149148 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:35.149148 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:35.149148 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:35.149148 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:24:35.149148 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:35.149148 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:35.149148 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:35.149148 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:35.149148 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:35.149148 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:35.150262 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:35.150262 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:35.150262 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:24:35.150262 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:24:35.150262 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:24:35.150262 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:24:35.150262 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:24:35.150262 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:24:35.150262 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 14:24:35.150262 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:24:35.157973 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 309/618] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:24:34 2022] GET /identity/v3/users/swift => generated 83 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:24:35.163816 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba46000-4a3b-45d3-a9c3-c7fdad8f5c55 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:35.304710 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e85646-b4f4-4669-813e-e369bf5c858f None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:35.342525 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba46000-4a3b-45d3-a9c3-c7fdad8f5c55 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:35.380593 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ee46bc-fead-4390-8d4e-2bf3a495c995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:35.381274 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ee46bc-fead-4390-8d4e-2bf3a495c995 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:35.381575 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ee46bc-fead-4390-8d4e-2bf3a495c995 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:35.381829 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ee46bc-fead-4390-8d4e-2bf3a495c995 None admin] PATH_INFO: `/v3/users` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:35.382703 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ee46bc-fead-4390-8d4e-2bf3a495c995 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:35.383014 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ee46bc-fead-4390-8d4e-2bf3a495c995 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:35.384461 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ee46bc-fead-4390-8d4e-2bf3a495c995 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:35.403211 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdb184a-8f77-4974-98e6-0b716bb8a58f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:35.404026 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdb184a-8f77-4974-98e6-0b716bb8a58f None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:35.404309 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdb184a-8f77-4974-98e6-0b716bb8a58f None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:35.404640 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdb184a-8f77-4974-98e6-0b716bb8a58f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:35.404885 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 310/619] 10.222.0.57 () {56 vars in 1135 bytes} [Thu May 19 14:24:35 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:35.413289 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ee46bc-fead-4390-8d4e-2bf3a495c995 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:35.428162 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdb184a-8f77-4974-98e6-0b716bb8a58f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:35.430152 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdb184a-8f77-4974-98e6-0b716bb8a58f None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:35.431836 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:35.457620 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-1bdb184a-8f77-4974-98e6-0b716bb8a58f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:35.457620 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:35.457620 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:35.457620 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:35.457620 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:35.457620 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:35.457620 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:35.457620 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:35.457620 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:35.457620 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:35.457620 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:35.457620 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:35.457620 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:24:35.457620 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:24:35.457620 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:24:35.458792 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:24:35.458792 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:35.458792 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:35.458792 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:35.458792 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:35.458792 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:35.458792 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:35.458792 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:35.458792 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:24:35.458792 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:35.458792 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:35.458792 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:35.458792 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:35.458792 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:35.458792 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:35.458792 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:35.458792 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:35.459791 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:24:35.459791 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:24:35.459791 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:35.459791 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:24:35.471572 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 310/620] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:24:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 358 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:24:35.479745 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdb184a-8f77-4974-98e6-0b716bb8a58f None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:35.554248 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:35.579325 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ee46bc-fead-4390-8d4e-2bf3a495c995 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:35.615748 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da424e3e-d0ec-405a-903d-9a7151176189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:35.616943 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da424e3e-d0ec-405a-903d-9a7151176189 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:35.617460 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da424e3e-d0ec-405a-903d-9a7151176189 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:35.618019 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da424e3e-d0ec-405a-903d-9a7151176189 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:35.629604 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da424e3e-d0ec-405a-903d-9a7151176189 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:35.631573 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da424e3e-d0ec-405a-903d-9a7151176189 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:35.664172 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-da424e3e-d0ec-405a-903d-9a7151176189 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:35.664172 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:35.664172 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:35.664172 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:35.664172 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:35.664172 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:35.664172 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:35.664172 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:35.664172 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:35.664172 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:35.664172 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:35.664172 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:35.664172 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:24:35.664172 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:24:35.664172 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:24:35.665174 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:24:35.665174 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:35.665174 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:35.665174 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:35.665174 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:35.665174 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:35.665174 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:35.665174 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:35.665174 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:24:35.665174 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:35.665174 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:35.665174 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:35.665174 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:35.665174 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:35.665174 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:35.665174 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:35.665174 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:35.666232 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:24:35.666232 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:24:35.666232 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:35.666232 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:24:35.670028 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 311/621] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:24:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:35.675061 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da424e3e-d0ec-405a-903d-9a7151176189 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:35.724425 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdb184a-8f77-4974-98e6-0b716bb8a58f None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:35.764384 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19deae6c-735f-46f5-9cf2-843bcb51aac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:35.765052 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19deae6c-735f-46f5-9cf2-843bcb51aac0 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:35.765332 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19deae6c-735f-46f5-9cf2-843bcb51aac0 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:35.765606 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19deae6c-735f-46f5-9cf2-843bcb51aac0 None admin] PATH_INFO: `/v3/domains` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:35.766365 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19deae6c-735f-46f5-9cf2-843bcb51aac0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:35.766670 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19deae6c-735f-46f5-9cf2-843bcb51aac0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:35.768035 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19deae6c-735f-46f5-9cf2-843bcb51aac0 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:35.781137 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 311/622] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:24:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:35.790851 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19deae6c-735f-46f5-9cf2-843bcb51aac0 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:35.800888 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:35.861712 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da424e3e-d0ec-405a-903d-9a7151176189 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:35.923150 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14da385-533a-4a8b-98ba-05ea63a6c5e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:35.924176 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14da385-533a-4a8b-98ba-05ea63a6c5e4 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:35.924649 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14da385-533a-4a8b-98ba-05ea63a6c5e4 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:35.925654 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14da385-533a-4a8b-98ba-05ea63a6c5e4 None admin] PATH_INFO: `/v3/domains` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:35.926616 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14da385-533a-4a8b-98ba-05ea63a6c5e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:35.927116 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14da385-533a-4a8b-98ba-05ea63a6c5e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:35.929663 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14da385-533a-4a8b-98ba-05ea63a6c5e4 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:35.942984 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 312/623] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:24:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:35.945928 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7399c6-f43e-492e-8745-a25522b7c6f1 None None] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:35.945928 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7399c6-f43e-492e-8745-a25522b7c6f1 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:35.945928 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7399c6-f43e-492e-8745-a25522b7c6f1 None None] PATH_INFO: `/` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:35.950884 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 313/624] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:24:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:24:35.952459 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7399c6-f43e-492e-8745-a25522b7c6f1 None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:35.954535 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:35.959140 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19deae6c-735f-46f5-9cf2-843bcb51aac0 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:35.997873 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ef8d0b-fea1-48cd-a7e0-e211123a59cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:35.999090 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ef8d0b-fea1-48cd-a7e0-e211123a59cb None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:35.999585 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ef8d0b-fea1-48cd-a7e0-e211123a59cb None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:36.000093 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ef8d0b-fea1-48cd-a7e0-e211123a59cb None admin] PATH_INFO: `/v3/users/nova` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:36.008693 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ef8d0b-fea1-48cd-a7e0-e211123a59cb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:36.010477 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ef8d0b-fea1-48cd-a7e0-e211123a59cb None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:36.026248 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-36ef8d0b-fea1-48cd-a7e0-e211123a59cb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 14:24:36.026248 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:36.026248 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:36.026248 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:36.026248 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:36.026248 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:36.026248 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:36.026248 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:36.026248 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:36.026248 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:36.026248 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:36.026248 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:36.026248 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:24:36.026248 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:24:36.026248 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:24:36.026248 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:24:36.027907 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:36.027907 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:36.027907 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:24:36.027907 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:24:36.027907 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:24:36.027907 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:24:36.027907 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:36.027907 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:36.027907 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:36.027907 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:36.027907 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:36.027907 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:24:36.027907 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:36.027907 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:36.027907 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:36.027907 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:36.027907 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:36.027907 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:36.029001 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:36.029001 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:36.029001 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:24:36.029001 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:24:36.029001 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:24:36.029001 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:24:36.029001 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:24:36.029001 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:24:36.029001 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 14:24:36.029001 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:24:36.036071 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 312/625] 10.222.0.57 () {56 vars in 1093 bytes} [Thu May 19 14:24:35 2022] GET /identity/v3/users/nova => generated 82 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:24:36.037795 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d5ca5d-889f-4fca-964d-b6e8929afca9 None None] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:36.038347 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d5ca5d-889f-4fca-964d-b6e8929afca9 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:36.038839 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d5ca5d-889f-4fca-964d-b6e8929afca9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:36.057019 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.auth.core [None req-54d5ca5d-889f-4fca-964d-b6e8929afca9 None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143586) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:24:36.058730 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.fernet_utils [None req-54d5ca5d-889f-4fca-964d-b6e8929afca9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143586) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:24:36.086580 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7399c6-f43e-492e-8745-a25522b7c6f1 None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:36.117781 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:36.133888 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6781eb-4a3b-4c86-a86e-80bacb40ddc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:36.134836 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6781eb-4a3b-4c86-a86e-80bacb40ddc7 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:36.135054 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6781eb-4a3b-4c86-a86e-80bacb40ddc7 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:36.135276 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6781eb-4a3b-4c86-a86e-80bacb40ddc7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:36.135384 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 313/626] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:24:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:24:36.136427 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d5ca5d-889f-4fca-964d-b6e8929afca9 None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:36.155026 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6781eb-4a3b-4c86-a86e-80bacb40ddc7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:36.156514 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6781eb-4a3b-4c86-a86e-80bacb40ddc7 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:36.183901 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-2a6781eb-4a3b-4c86-a86e-80bacb40ddc7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:24:36.183901 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:36.183901 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:36.183901 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:36.183901 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:36.183901 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:36.183901 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:36.183901 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:36.183901 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:36.183901 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:36.183901 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:36.183901 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:36.183901 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:24:36.183901 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:24:36.183901 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:24:36.185003 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:24:36.185003 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:36.185003 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:36.185003 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:36.185003 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:36.185003 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:36.185003 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:36.185003 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:36.185003 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:24:36.185003 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:36.185003 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:36.185003 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:36.185003 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:36.185003 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:36.185003 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:36.185003 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:36.185003 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:36.186044 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:24:36.186044 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:24:36.186044 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:24:36.186044 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:24:36.186044 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:24:36.186044 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:24:36.186044 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:24:36.186044 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:24:36.189931 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 314/627] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:24:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:36.190761 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6781eb-4a3b-4c86-a86e-80bacb40ddc7 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:36.241747 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:36.257567 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d5ca5d-889f-4fca-964d-b6e8929afca9 None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:36.296863 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6a81bc-444c-4d98-b7b9-0b526f9793d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:36.297960 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6a81bc-444c-4d98-b7b9-0b526f9793d1 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:36.298223 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6781eb-4a3b-4c86-a86e-80bacb40ddc7 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:36.298691 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6a81bc-444c-4d98-b7b9-0b526f9793d1 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:36.299197 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6a81bc-444c-4d98-b7b9-0b526f9793d1 None admin] PATH_INFO: `/v3/users` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:36.300459 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6a81bc-444c-4d98-b7b9-0b526f9793d1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:36.301003 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6a81bc-444c-4d98-b7b9-0b526f9793d1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:36.302867 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6a81bc-444c-4d98-b7b9-0b526f9793d1 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:36.330876 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 314/628] 10.222.0.57 () {56 vars in 1133 bytes} [Thu May 19 14:24:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:36.332454 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6a81bc-444c-4d98-b7b9-0b526f9793d1 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:36.339790 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ebf3f2-a495-4083-91e4-0de95beaf0a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:36.341023 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ebf3f2-a495-4083-91e4-0de95beaf0a2 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:36.341313 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ebf3f2-a495-4083-91e4-0de95beaf0a2 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:36.341601 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ebf3f2-a495-4083-91e4-0de95beaf0a2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:36.354374 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ebf3f2-a495-4083-91e4-0de95beaf0a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:24:36.363890 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-c4ebf3f2-a495-4083-91e4-0de95beaf0a2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:24:36.363890 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:36.363890 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:36.363890 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:36.363890 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:36.363890 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:36.363890 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:36.363890 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:36.363890 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:36.363890 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:36.363890 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:36.363890 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:36.363890 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:24:36.363890 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:24:36.363890 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:24:36.363890 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:24:36.365021 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:24:36.365021 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:24:36.365021 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:24:36.365021 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:24:36.365021 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:36.365021 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:36.365021 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:36.365021 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:36.365021 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:36.365021 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:36.365021 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:36.365021 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:24:36.365021 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:36.365021 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:36.365021 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:36.365021 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:36.366128 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:36.366128 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:36.366128 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:36.366128 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:36.366128 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:24:36.366128 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:24:36.366128 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:24:36.366128 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:24:36.366128 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:24:36.366128 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:24:36.366128 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:24:36.366128 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:24:36.370084 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 315/629] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 14:24:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:36.371595 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ebf3f2-a495-4083-91e4-0de95beaf0a2 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:36.422527 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:36.460861 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6a81bc-444c-4d98-b7b9-0b526f9793d1 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:36.484606 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ebf3f2-a495-4083-91e4-0de95beaf0a2 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:36.505239 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682981ff-6093-4c34-9fd1-273e6be4fb92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:36.505929 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682981ff-6093-4c34-9fd1-273e6be4fb92 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:36.506214 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682981ff-6093-4c34-9fd1-273e6be4fb92 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:36.506487 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682981ff-6093-4c34-9fd1-273e6be4fb92 None admin] PATH_INFO: `/v3/projects` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:36.507344 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682981ff-6093-4c34-9fd1-273e6be4fb92 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:36.507653 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682981ff-6093-4c34-9fd1-273e6be4fb92 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:36.509089 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682981ff-6093-4c34-9fd1-273e6be4fb92 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:36.522079 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 315/630] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:24:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:36.522929 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682981ff-6093-4c34-9fd1-273e6be4fb92 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:36.526312 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef4acd2-3e58-4577-8696-07783b63ea7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:36.527052 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef4acd2-3e58-4577-8696-07783b63ea7c None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:36.527424 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef4acd2-3e58-4577-8696-07783b63ea7c None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:36.528288 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef4acd2-3e58-4577-8696-07783b63ea7c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:36.533319 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:36.599467 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef4acd2-3e58-4577-8696-07783b63ea7c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:36.600926 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef4acd2-3e58-4577-8696-07783b63ea7c None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:36.626282 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682981ff-6093-4c34-9fd1-273e6be4fb92 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:36.643643 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:36.662203 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc4e04f-90ff-402a-807c-424562bc5e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:36.662883 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc4e04f-90ff-402a-807c-424562bc5e12 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:36.663165 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc4e04f-90ff-402a-807c-424562bc5e12 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:36.663444 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc4e04f-90ff-402a-807c-424562bc5e12 None admin] PATH_INFO: `/v3/roles` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:36.664201 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc4e04f-90ff-402a-807c-424562bc5e12 None admin] RBAC: Adding query filter params (name=service) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:36.664512 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc4e04f-90ff-402a-807c-424562bc5e12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:36.665898 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc4e04f-90ff-402a-807c-424562bc5e12 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:36.674247 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 316/631] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 19 14:24:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:36.675471 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab2979d-0bcc-4f73-88f4-91bff44c77a6 None None] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:36.675767 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab2979d-0bcc-4f73-88f4-91bff44c77a6 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:36.676044 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab2979d-0bcc-4f73-88f4-91bff44c77a6 None None] PATH_INFO: `/` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:36.677430 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 317/632] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:24:36 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:24:36.678581 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab2979d-0bcc-4f73-88f4-91bff44c77a6 None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:36.708719 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 316/633] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:24:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:24:36.710372 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fc633f-8f97-4f43-82f7-083c0ea4e950 None None] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:36.710663 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fc633f-8f97-4f43-82f7-083c0ea4e950 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:36.710942 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fc633f-8f97-4f43-82f7-083c0ea4e950 None None] PATH_INFO: `/` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:36.712216 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 317/634] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:24:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:24:36.712911 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fc633f-8f97-4f43-82f7-083c0ea4e950 None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:36.754437 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:36.789588 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab2979d-0bcc-4f73-88f4-91bff44c77a6 None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:36.816335 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fc633f-8f97-4f43-82f7-083c0ea4e950 None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:36.822467 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7ed996-5d73-4afa-b2d7-e4779eda9275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:36.823352 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7ed996-5d73-4afa-b2d7-e4779eda9275 None admin] REQUEST_METHOD: `PUT` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:36.823634 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7ed996-5d73-4afa-b2d7-e4779eda9275 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:36.823913 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7ed996-5d73-4afa-b2d7-e4779eda9275 None admin] PATH_INFO: `/v3/projects/252d9673f08c479a98d04fee3f5856a4/users/d3e544b4989e49a19536b40de5069bcf/roles/b0b74fefb27e4a0ebd895893aa9862d7` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:36.839232 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7ed996-5d73-4afa-b2d7-e4779eda9275 None admin] RBAC: Authorizing `identity:create_grant(project_id=252d9673f08c479a98d04fee3f5856a4, user_id=d3e544b4989e49a19536b40de5069bcf, role_id=b0b74fefb27e4a0ebd895893aa9862d7)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:36.840843 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7ed996-5d73-4afa-b2d7-e4779eda9275 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:36.851958 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634dc5a9-be53-4736-9925-a77de2414086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:36.852746 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634dc5a9-be53-4736-9925-a77de2414086 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:36.853027 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634dc5a9-be53-4736-9925-a77de2414086 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:36.853311 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634dc5a9-be53-4736-9925-a77de2414086 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:36.854342 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 318/635] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 14:24:36 2022] PUT /identity/v3/projects/252d9673f08c479a98d04fee3f5856a4/users/d3e544b4989e49a19536b40de5069bcf/roles/b0b74fefb27e4a0ebd895893aa9862d7 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:24:36.856001 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ffb631-7f88-4953-a281-3edf829d3d2b None None] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:36.856721 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ffb631-7f88-4953-a281-3edf829d3d2b None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:36.857012 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ffb631-7f88-4953-a281-3edf829d3d2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:36.864801 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634dc5a9-be53-4736-9925-a77de2414086 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:36.866335 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634dc5a9-be53-4736-9925-a77de2414086 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:36.868450 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:36.878615 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.auth.core [None req-21ffb631-7f88-4953-a281-3edf829d3d2b None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143586) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:24:36.879760 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.fernet_utils [None req-21ffb631-7f88-4953-a281-3edf829d3d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143586) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:24:36.888037 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-634dc5a9-be53-4736-9925-a77de2414086 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:36.888037 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:36.888037 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:36.888037 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:36.888037 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:36.888037 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:36.888037 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:36.888037 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:36.888037 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:36.888037 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:36.888037 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:36.888037 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:36.888037 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:24:36.888037 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:24:36.888037 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:24:36.889264 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:24:36.889264 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:36.889264 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:36.889264 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:36.889264 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:36.889264 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:36.889264 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:36.889264 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:36.889264 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:24:36.889264 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:36.889264 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:36.889264 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:36.889264 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:36.889264 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:36.889264 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:36.889264 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:36.889264 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:36.890353 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:24:36.890353 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:24:36.890353 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:36.890353 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:24:36.891316 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 318/636] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:24:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:36.892189 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634dc5a9-be53-4736-9925-a77de2414086 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:36.964454 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 319/637] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:24:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:24:36.967936 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdf1b26-a21f-46ae-81a8-c33df90885d0 None None] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:36.968931 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdf1b26-a21f-46ae-81a8-c33df90885d0 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:36.969531 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdf1b26-a21f-46ae-81a8-c33df90885d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:36.990125 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.auth.core [None req-1fdf1b26-a21f-46ae-81a8-c33df90885d0 None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143586) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:24:36.991660 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.fernet_utils [None req-1fdf1b26-a21f-46ae-81a8-c33df90885d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143586) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:24:37.012919 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634dc5a9-be53-4736-9925-a77de2414086 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:37.047400 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:37.057742 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893ddb9b-78ee-43ff-b3f8-369762d7e799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:37.059803 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893ddb9b-78ee-43ff-b3f8-369762d7e799 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:37.060099 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893ddb9b-78ee-43ff-b3f8-369762d7e799 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:37.060360 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893ddb9b-78ee-43ff-b3f8-369762d7e799 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:37.071656 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893ddb9b-78ee-43ff-b3f8-369762d7e799 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:37.073112 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893ddb9b-78ee-43ff-b3f8-369762d7e799 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:37.075283 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 320/638] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:24:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:24:37.076154 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdf1b26-a21f-46ae-81a8-c33df90885d0 None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:37.091724 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-893ddb9b-78ee-43ff-b3f8-369762d7e799 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:37.091724 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:37.091724 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:37.091724 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:37.091724 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:37.091724 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:37.091724 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:37.091724 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:37.091724 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:37.091724 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:37.091724 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:37.091724 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:37.091724 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:24:37.091724 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:24:37.091724 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:24:37.092942 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:24:37.092942 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:37.092942 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:37.092942 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:37.092942 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:37.092942 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:37.092942 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:37.092942 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:37.092942 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:24:37.092942 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:37.092942 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:37.092942 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:37.092942 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:37.092942 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:37.092942 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:37.092942 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:37.092942 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:37.094036 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:24:37.094036 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:24:37.094036 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:37.094036 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:24:37.095927 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 319/639] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:24:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:37.096890 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893ddb9b-78ee-43ff-b3f8-369762d7e799 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:37.146055 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:37.194876 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdf1b26-a21f-46ae-81a8-c33df90885d0 None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:37.200651 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893ddb9b-78ee-43ff-b3f8-369762d7e799 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:37.232329 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5316066-feff-4b55-9517-e7c544a39df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:37.233097 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5316066-feff-4b55-9517-e7c544a39df6 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:37.233380 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5316066-feff-4b55-9517-e7c544a39df6 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:37.233660 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5316066-feff-4b55-9517-e7c544a39df6 None admin] PATH_INFO: `/v3/domains` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:37.234413 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5316066-feff-4b55-9517-e7c544a39df6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:37.234814 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5316066-feff-4b55-9517-e7c544a39df6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:37.236199 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5316066-feff-4b55-9517-e7c544a39df6 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:37.238061 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afc096d-da39-46d5-b20f-5c19de13f9f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:37.238866 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afc096d-da39-46d5-b20f-5c19de13f9f4 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:37.239146 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afc096d-da39-46d5-b20f-5c19de13f9f4 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:37.239423 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afc096d-da39-46d5-b20f-5c19de13f9f4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:37.250045 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 321/640] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:24:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:37.250280 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afc096d-da39-46d5-b20f-5c19de13f9f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:24:37.251283 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5316066-feff-4b55-9517-e7c544a39df6 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:37.258035 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-1afc096d-da39-46d5-b20f-5c19de13f9f4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:24:37.258035 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:37.258035 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:37.258035 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:37.258035 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:37.258035 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:37.258035 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:37.258035 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:37.258035 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:37.258035 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:37.258035 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:37.258035 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:37.258035 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:24:37.258035 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:24:37.258035 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:24:37.258035 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:24:37.259079 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:24:37.259079 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:24:37.259079 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:24:37.259079 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:24:37.259079 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:37.259079 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:37.259079 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:37.259079 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:37.259079 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:37.259079 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:37.259079 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:37.259079 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:24:37.259079 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:37.259079 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:37.259079 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:37.259079 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:37.260184 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:37.260184 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:37.260184 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:37.260184 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:37.260184 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:24:37.260184 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:24:37.260184 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:24:37.260184 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:24:37.260184 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:24:37.260184 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:24:37.260184 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:24:37.260184 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:24:37.260184 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 320/641] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 14:24:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:37.260184 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afc096d-da39-46d5-b20f-5c19de13f9f4 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:37.300742 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:37.364964 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afc096d-da39-46d5-b20f-5c19de13f9f4 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:37.367948 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5316066-feff-4b55-9517-e7c544a39df6 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:37.372766 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:37.405703 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95d9c85-a9b1-482e-9e6f-6fe29ca519e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:37.406096 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c9c83a-7b9c-4a37-b3b9-aecb872b76b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:37.406335 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95d9c85-a9b1-482e-9e6f-6fe29ca519e2 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:37.406622 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95d9c85-a9b1-482e-9e6f-6fe29ca519e2 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:37.406884 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c9c83a-7b9c-4a37-b3b9-aecb872b76b9 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:37.407007 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95d9c85-a9b1-482e-9e6f-6fe29ca519e2 None admin] PATH_INFO: `/v3/domains` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:37.407176 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c9c83a-7b9c-4a37-b3b9-aecb872b76b9 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:37.407445 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c9c83a-7b9c-4a37-b3b9-aecb872b76b9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:37.407619 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95d9c85-a9b1-482e-9e6f-6fe29ca519e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:37.407935 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95d9c85-a9b1-482e-9e6f-6fe29ca519e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:37.409319 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95d9c85-a9b1-482e-9e6f-6fe29ca519e2 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:37.420026 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c9c83a-7b9c-4a37-b3b9-aecb872b76b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:24:37.425983 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 321/642] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:24:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:37.427096 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95d9c85-a9b1-482e-9e6f-6fe29ca519e2 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:37.427889 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-51c9c83a-7b9c-4a37-b3b9-aecb872b76b9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:24:37.427889 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:37.427889 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:37.427889 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:37.427889 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:37.427889 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:37.427889 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:37.427889 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:37.427889 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:37.427889 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:37.427889 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:37.427889 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:37.427889 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:24:37.427889 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:24:37.427889 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:24:37.427889 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:24:37.428952 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:24:37.428952 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:24:37.428952 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:24:37.428952 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:24:37.428952 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:37.428952 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:37.428952 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:37.428952 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:37.428952 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:37.428952 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:37.428952 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:37.428952 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:24:37.428952 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:37.428952 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:37.428952 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:37.428952 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:37.429928 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:37.429928 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:37.429928 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:37.429928 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:37.429928 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:24:37.429928 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:24:37.429928 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:24:37.429928 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:24:37.429928 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:24:37.429928 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:24:37.429928 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:24:37.429928 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:24:37.429928 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 322/643] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 14:24:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:37.430765 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c9c83a-7b9c-4a37-b3b9-aecb872b76b9 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:37.536139 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95d9c85-a9b1-482e-9e6f-6fe29ca519e2 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:37.538187 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:37.542080 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c9c83a-7b9c-4a37-b3b9-aecb872b76b9 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:37.575470 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffebacf-a6f4-443f-9e2a-0930ba9e835c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:37.576270 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffebacf-a6f4-443f-9e2a-0930ba9e835c None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:37.576554 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffebacf-a6f4-443f-9e2a-0930ba9e835c None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:37.576825 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffebacf-a6f4-443f-9e2a-0930ba9e835c None admin] PATH_INFO: `/v3/users/glance` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:37.579177 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6900e932-0fd2-4cef-9e80-1d96e69593fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:37.579856 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6900e932-0fd2-4cef-9e80-1d96e69593fc None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:37.580139 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6900e932-0fd2-4cef-9e80-1d96e69593fc None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:37.580420 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6900e932-0fd2-4cef-9e80-1d96e69593fc None admin] PATH_INFO: `/v3/roles` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:37.581179 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6900e932-0fd2-4cef-9e80-1d96e69593fc None admin] RBAC: Adding query filter params (name=service) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:37.581493 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6900e932-0fd2-4cef-9e80-1d96e69593fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:37.582901 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6900e932-0fd2-4cef-9e80-1d96e69593fc None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:37.584357 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffebacf-a6f4-443f-9e2a-0930ba9e835c None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:37.585802 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffebacf-a6f4-443f-9e2a-0930ba9e835c None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:37.593184 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 323/644] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 19 14:24:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:37.594111 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6900e932-0fd2-4cef-9e80-1d96e69593fc None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:37.606142 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-5ffebacf-a6f4-443f-9e2a-0930ba9e835c None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 19 14:24:37.606142 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:37.606142 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:37.606142 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:37.606142 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:37.606142 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:37.606142 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:37.606142 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:37.606142 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:37.606142 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:37.606142 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:37.606142 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:37.606142 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:24:37.606142 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:24:37.606142 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:24:37.606142 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:24:37.607719 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:37.607719 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:37.607719 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:24:37.607719 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:24:37.607719 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:24:37.607719 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:24:37.607719 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:37.607719 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:37.607719 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:37.607719 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:37.607719 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:37.607719 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:24:37.607719 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:37.607719 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:37.607719 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:37.607719 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:37.607719 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:37.607719 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:37.608788 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:37.608788 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:37.608788 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:24:37.608788 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:24:37.608788 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:24:37.608788 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:24:37.608788 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:24:37.608788 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:24:37.608788 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 19 14:24:37.608788 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:24:37.608788 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 322/645] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:24:37 2022] GET /identity/v3/users/glance => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:37.608788 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffebacf-a6f4-443f-9e2a-0930ba9e835c None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:37.644397 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:37.717869 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6900e932-0fd2-4cef-9e80-1d96e69593fc None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:37.733091 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffebacf-a6f4-443f-9e2a-0930ba9e835c None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:37.753680 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5608fa3-ae5f-476d-b582-a81d9a994285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:37.754595 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5608fa3-ae5f-476d-b582-a81d9a994285 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:37.755164 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5608fa3-ae5f-476d-b582-a81d9a994285 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:37.755164 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5608fa3-ae5f-476d-b582-a81d9a994285 None admin] PATH_INFO: `/v3/projects/service` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:37.762920 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5608fa3-ae5f-476d-b582-a81d9a994285 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:37.764459 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5608fa3-ae5f-476d-b582-a81d9a994285 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:37.770471 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e2bb5c-e898-411c-b1d3-65ece3345622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:37.771143 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e2bb5c-e898-411c-b1d3-65ece3345622 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:37.771429 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e2bb5c-e898-411c-b1d3-65ece3345622 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:37.771709 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e2bb5c-e898-411c-b1d3-65ece3345622 None admin] PATH_INFO: `/v3/roles` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:37.772459 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e2bb5c-e898-411c-b1d3-65ece3345622 None admin] RBAC: Adding query filter params (name=service) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:37.772772 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e2bb5c-e898-411c-b1d3-65ece3345622 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:37.774223 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e2bb5c-e898-411c-b1d3-65ece3345622 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:37.779486 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-e5608fa3-ae5f-476d-b582-a81d9a994285 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:24:37.779486 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:37.779486 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:37.779486 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:37.779486 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:37.779486 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:37.779486 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:37.779486 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:37.779486 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:37.779486 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:37.779486 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:37.779486 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:37.779486 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:24:37.779486 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:24:37.779486 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:24:37.780491 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:24:37.780491 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:37.780491 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:37.780491 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:37.780491 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:37.780491 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:37.780491 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:37.780491 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:37.780491 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:24:37.780491 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:37.780491 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:37.780491 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:37.780491 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:37.780491 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:37.780491 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:37.780491 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:37.780491 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:37.781492 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:24:37.781492 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:24:37.781492 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:24:37.781492 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:24:37.781492 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:24:37.781492 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:24:37.781492 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:24:37.781492 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:24:37.785295 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 324/646] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:24:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:37.786306 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5608fa3-ae5f-476d-b582-a81d9a994285 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:37.786421 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 323/647] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 19 14:24:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:37.787081 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e2bb5c-e898-411c-b1d3-65ece3345622 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:37.812950 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:37.887656 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:37.903138 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e2bb5c-e898-411c-b1d3-65ece3345622 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:37.908933 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5608fa3-ae5f-476d-b582-a81d9a994285 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:37.939438 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215feb36-4ecf-42ce-bf7b-42a116ceb130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:37.940111 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215feb36-4ecf-42ce-bf7b-42a116ceb130 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:37.940403 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215feb36-4ecf-42ce-bf7b-42a116ceb130 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:37.940685 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215feb36-4ecf-42ce-bf7b-42a116ceb130 None admin] PATH_INFO: `/v3/users` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:37.941523 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215feb36-4ecf-42ce-bf7b-42a116ceb130 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:37.941871 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215feb36-4ecf-42ce-bf7b-42a116ceb130 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:37.943309 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215feb36-4ecf-42ce-bf7b-42a116ceb130 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:37.947094 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2af31a-1d13-4059-8eb2-bc1ce0da20e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:37.948074 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2af31a-1d13-4059-8eb2-bc1ce0da20e5 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:37.948512 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2af31a-1d13-4059-8eb2-bc1ce0da20e5 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:37.949656 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2af31a-1d13-4059-8eb2-bc1ce0da20e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:37.960359 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2af31a-1d13-4059-8eb2-bc1ce0da20e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:37.961419 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 324/648] 10.222.0.57 () {56 vars in 1137 bytes} [Thu May 19 14:24:37 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:37.962245 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2af31a-1d13-4059-8eb2-bc1ce0da20e5 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:37.962377 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215feb36-4ecf-42ce-bf7b-42a116ceb130 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:37.984635 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-5f2af31a-1d13-4059-8eb2-bc1ce0da20e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:37.984635 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:37.984635 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:37.984635 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:37.984635 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:37.984635 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:37.984635 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:37.984635 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:37.984635 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:37.984635 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:37.984635 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:37.984635 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:37.984635 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:24:37.984635 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:24:37.984635 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:24:37.986019 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:24:37.986019 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:37.986019 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:37.986019 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:37.986019 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:37.986019 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:37.986019 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:37.986019 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:37.986019 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:24:37.986019 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:37.986019 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:37.986019 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:37.986019 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:37.986019 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:37.986019 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:37.986019 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:37.986019 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:37.987174 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:24:37.987174 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:24:37.987174 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:37.987174 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:24:37.988740 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 325/649] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:24:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:37.989666 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2af31a-1d13-4059-8eb2-bc1ce0da20e5 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:38.068191 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:38.084658 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215feb36-4ecf-42ce-bf7b-42a116ceb130 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:38.102147 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2af31a-1d13-4059-8eb2-bc1ce0da20e5 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:38.121956 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a239d37-7e40-421e-9fbe-4ba8a2ba1ef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:38.122620 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a239d37-7e40-421e-9fbe-4ba8a2ba1ef6 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:38.122904 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a239d37-7e40-421e-9fbe-4ba8a2ba1ef6 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:38.123183 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a239d37-7e40-421e-9fbe-4ba8a2ba1ef6 None admin] PATH_INFO: `/v3/projects` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:38.124424 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a239d37-7e40-421e-9fbe-4ba8a2ba1ef6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:38.124740 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a239d37-7e40-421e-9fbe-4ba8a2ba1ef6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:38.126224 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a239d37-7e40-421e-9fbe-4ba8a2ba1ef6 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:38.138962 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d45082-5029-46f9-9830-c47880910c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:38.144042 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 325/650] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:24:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:38.144196 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d45082-5029-46f9-9830-c47880910c7b None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:38.144196 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d45082-5029-46f9-9830-c47880910c7b None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:38.144196 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d45082-5029-46f9-9830-c47880910c7b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:38.144471 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a239d37-7e40-421e-9fbe-4ba8a2ba1ef6 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:38.154886 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:38.156441 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d45082-5029-46f9-9830-c47880910c7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:38.158416 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d45082-5029-46f9-9830-c47880910c7b None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:38.177019 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-c3d45082-5029-46f9-9830-c47880910c7b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:38.177019 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:38.177019 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:38.177019 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:38.177019 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:38.177019 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:38.177019 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:38.177019 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:38.177019 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:38.177019 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:38.177019 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:38.177019 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:38.177019 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:24:38.177019 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:24:38.177019 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:24:38.178099 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:24:38.178099 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:38.178099 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:38.178099 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:38.178099 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:38.178099 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:38.178099 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:38.178099 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:38.178099 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:24:38.178099 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:38.178099 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:38.178099 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:38.178099 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:38.178099 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:38.178099 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:38.178099 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:38.178099 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:38.179122 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:24:38.179122 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:24:38.179122 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:38.179122 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:24:38.179122 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 326/651] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:24:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:38.179122 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d45082-5029-46f9-9830-c47880910c7b None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:38.260723 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a239d37-7e40-421e-9fbe-4ba8a2ba1ef6 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:38.288687 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d45082-5029-46f9-9830-c47880910c7b None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:38.300550 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b1c5d3-9b18-4b56-8ccc-4ad6a3ad59d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:38.301495 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b1c5d3-9b18-4b56-8ccc-4ad6a3ad59d4 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:38.301788 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b1c5d3-9b18-4b56-8ccc-4ad6a3ad59d4 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:38.302097 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b1c5d3-9b18-4b56-8ccc-4ad6a3ad59d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:38.314161 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:38.327585 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511446f0-0d02-4779-ad40-a71a8aaecf25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:38.328255 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511446f0-0d02-4779-ad40-a71a8aaecf25 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:38.328707 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511446f0-0d02-4779-ad40-a71a8aaecf25 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:38.328991 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511446f0-0d02-4779-ad40-a71a8aaecf25 None admin] PATH_INFO: `/v3/domains` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:38.330357 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511446f0-0d02-4779-ad40-a71a8aaecf25 None admin] RBAC: Adding query filter params (name=Default) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:38.331630 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511446f0-0d02-4779-ad40-a71a8aaecf25 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:38.332887 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511446f0-0d02-4779-ad40-a71a8aaecf25 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:38.350281 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 327/652] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:24:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:38.351550 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511446f0-0d02-4779-ad40-a71a8aaecf25 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:38.381343 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b1c5d3-9b18-4b56-8ccc-4ad6a3ad59d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:38.383098 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b1c5d3-9b18-4b56-8ccc-4ad6a3ad59d4 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:38.392094 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:38.462618 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511446f0-0d02-4779-ad40-a71a8aaecf25 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:38.500431 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 326/653] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:24:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:24:38.500653 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a805c053-86d9-4c37-9a41-8c1786f28726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:38.501219 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a805c053-86d9-4c37-9a41-8c1786f28726 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:38.501507 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a805c053-86d9-4c37-9a41-8c1786f28726 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:38.501790 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a805c053-86d9-4c37-9a41-8c1786f28726 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:38.502665 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a805c053-86d9-4c37-9a41-8c1786f28726 None admin] RBAC: Adding query filter params (role.id=b0b74fefb27e4a0ebd895893aa9862d7, scope.project.id=252d9673f08c479a98d04fee3f5856a4, user.id=5f28de22c2ee4c53b6d14d5b0ddf7fe9) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:38.502990 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a805c053-86d9-4c37-9a41-8c1786f28726 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:38.504429 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a805c053-86d9-4c37-9a41-8c1786f28726 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:38.505528 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b1c5d3-9b18-4b56-8ccc-4ad6a3ad59d4 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:38.512401 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 328/654] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:24:38 2022] GET /identity/v3/role_assignments?user.id=5f28de22c2ee4c53b6d14d5b0ddf7fe9&scope.project.id=252d9673f08c479a98d04fee3f5856a4&role.id=b0b74fefb27e4a0ebd895893aa9862d7 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:38.513155 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a805c053-86d9-4c37-9a41-8c1786f28726 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:38.582209 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:38.653567 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a805c053-86d9-4c37-9a41-8c1786f28726 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:38.664763 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b1c5d3-9b18-4b56-8ccc-4ad6a3ad59d4 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:38.674151 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:38.703351 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c236963-b5cc-41a4-b023-0c83bd131e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:38.704711 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c236963-b5cc-41a4-b023-0c83bd131e8b None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:38.705276 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c236963-b5cc-41a4-b023-0c83bd131e8b None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:38.705809 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c236963-b5cc-41a4-b023-0c83bd131e8b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:38.707318 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7058b91b-3bfe-4105-8704-12c3a96cf3f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:38.708093 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7058b91b-3bfe-4105-8704-12c3a96cf3f2 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:38.708498 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7058b91b-3bfe-4105-8704-12c3a96cf3f2 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:38.708892 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7058b91b-3bfe-4105-8704-12c3a96cf3f2 None admin] PATH_INFO: `/v3/domains` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:38.709700 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7058b91b-3bfe-4105-8704-12c3a96cf3f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:38.710542 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7058b91b-3bfe-4105-8704-12c3a96cf3f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:38.711534 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7058b91b-3bfe-4105-8704-12c3a96cf3f2 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:38.717430 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c236963-b5cc-41a4-b023-0c83bd131e8b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:38.719516 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c236963-b5cc-41a4-b023-0c83bd131e8b None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:38.727316 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 327/655] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:24:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:38.728114 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7058b91b-3bfe-4105-8704-12c3a96cf3f2 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:38.739202 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-9c236963-b5cc-41a4-b023-0c83bd131e8b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 14:24:38.739202 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:38.739202 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:38.739202 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:38.739202 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:38.739202 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:38.739202 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:38.739202 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:38.739202 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:38.739202 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:38.739202 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:38.739202 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:38.739202 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:24:38.739202 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:24:38.739202 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:24:38.739202 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:24:38.740294 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:38.740294 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:38.740294 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:24:38.740294 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:24:38.740294 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:24:38.740294 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:24:38.740294 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:38.740294 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:38.740294 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:38.740294 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:38.740294 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:38.740294 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:24:38.740294 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:38.740294 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:38.740294 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:38.740294 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:38.740294 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:38.740294 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:38.741381 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:38.741381 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:38.741381 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:24:38.741381 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:24:38.741381 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:24:38.741381 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:24:38.741381 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:24:38.741381 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:24:38.741381 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 14:24:38.741381 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:24:38.741381 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 329/656] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:24:38 2022] GET /identity/v3/users/cinder => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:38.742120 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c236963-b5cc-41a4-b023-0c83bd131e8b None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:38.863377 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7058b91b-3bfe-4105-8704-12c3a96cf3f2 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:38.884004 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c236963-b5cc-41a4-b023-0c83bd131e8b None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:38.896963 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:38.905397 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3174f7c6-cedf-4bbd-b751-00ee5971d4a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:38.906323 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3174f7c6-cedf-4bbd-b751-00ee5971d4a2 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:38.906609 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3174f7c6-cedf-4bbd-b751-00ee5971d4a2 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:38.906895 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3174f7c6-cedf-4bbd-b751-00ee5971d4a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:38.919255 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3174f7c6-cedf-4bbd-b751-00ee5971d4a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:38.920956 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3174f7c6-cedf-4bbd-b751-00ee5971d4a2 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:38.930208 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7df1aa7-17ff-4bad-81aa-de70bc29fe1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:38.931296 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7df1aa7-17ff-4bad-81aa-de70bc29fe1a None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:38.931584 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7df1aa7-17ff-4bad-81aa-de70bc29fe1a None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:38.931863 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7df1aa7-17ff-4bad-81aa-de70bc29fe1a None admin] PATH_INFO: `/v3/users/neutron` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:38.940141 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-3174f7c6-cedf-4bbd-b751-00ee5971d4a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:38.940141 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:38.940141 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:38.940141 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:38.940141 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:38.940141 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:38.940141 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:38.940141 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:38.940141 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:38.940141 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:38.940141 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:38.940141 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:38.940141 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:24:38.940141 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:24:38.940141 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:24:38.941155 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:24:38.941155 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:38.941155 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:38.941155 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:38.941155 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:38.941155 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:38.941155 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:38.941155 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:38.941155 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:24:38.941155 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:38.941155 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:38.941155 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:38.941155 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:38.941155 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:38.941155 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:38.941155 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:38.941155 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:38.942188 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:24:38.942188 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:24:38.942188 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:38.942188 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:24:38.942454 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7df1aa7-17ff-4bad-81aa-de70bc29fe1a None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:38.942535 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 328/657] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:24:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:38.942610 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7df1aa7-17ff-4bad-81aa-de70bc29fe1a None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:38.947031 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3174f7c6-cedf-4bbd-b751-00ee5971d4a2 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:38.961417 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-e7df1aa7-17ff-4bad-81aa-de70bc29fe1a None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 19 14:24:38.961417 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:38.961417 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:38.961417 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:38.961417 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:38.961417 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:38.961417 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:38.961417 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:38.961417 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:38.961417 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:38.961417 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:38.961417 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:38.961417 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:24:38.961417 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:24:38.961417 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:24:38.961417 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:24:38.963394 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:38.963394 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:38.963394 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:24:38.963394 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:24:38.963394 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:24:38.963394 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:24:38.963394 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:38.963394 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:38.963394 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:38.963394 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:38.963394 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:38.963394 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:24:38.963394 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:38.963394 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:38.963394 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:38.963394 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:38.963394 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:38.963394 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:38.964709 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:38.964709 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:38.964709 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:24:38.964709 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:24:38.964709 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:24:38.964709 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:24:38.964709 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:24:38.964709 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:24:38.964709 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 19 14:24:38.964709 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:24:38.964709 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 330/658] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 14:24:38 2022] GET /identity/v3/users/neutron => generated 85 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:38.964709 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7df1aa7-17ff-4bad-81aa-de70bc29fe1a None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:38.976774 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:39.073910 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3174f7c6-cedf-4bbd-b751-00ee5971d4a2 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:39.105081 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7df1aa7-17ff-4bad-81aa-de70bc29fe1a None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:39.117560 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c119de-0fc4-40ba-9793-c8bce71e317d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:39.119073 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c119de-0fc4-40ba-9793-c8bce71e317d None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:39.119517 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c119de-0fc4-40ba-9793-c8bce71e317d None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:39.119949 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c119de-0fc4-40ba-9793-c8bce71e317d None admin] PATH_INFO: `/v3/users` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:39.122097 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c119de-0fc4-40ba-9793-c8bce71e317d None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:39.122602 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c119de-0fc4-40ba-9793-c8bce71e317d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:39.124663 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c119de-0fc4-40ba-9793-c8bce71e317d None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:39.144551 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68ad67e-2d37-42c2-8a9d-1b257fe69b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:39.145472 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68ad67e-2d37-42c2-8a9d-1b257fe69b14 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:39.145966 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68ad67e-2d37-42c2-8a9d-1b257fe69b14 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:39.146415 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68ad67e-2d37-42c2-8a9d-1b257fe69b14 None admin] PATH_INFO: `/v3/domains` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:39.147300 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68ad67e-2d37-42c2-8a9d-1b257fe69b14 None admin] RBAC: Adding query filter params (name=Default) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:39.147752 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68ad67e-2d37-42c2-8a9d-1b257fe69b14 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:39.149307 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68ad67e-2d37-42c2-8a9d-1b257fe69b14 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:39.150005 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 329/659] 10.222.0.57 () {56 vars in 1137 bytes} [Thu May 19 14:24:38 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:39.151421 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c119de-0fc4-40ba-9793-c8bce71e317d None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:39.164563 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 331/660] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:24:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:39.169630 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fa19b3-45e4-4ff9-b9f7-377b053d8467 None None] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:39.169630 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fa19b3-45e4-4ff9-b9f7-377b053d8467 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:39.169630 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fa19b3-45e4-4ff9-b9f7-377b053d8467 None None] PATH_INFO: `/` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:39.169630 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 332/661] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:24:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:24:39.170447 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fa19b3-45e4-4ff9-b9f7-377b053d8467 None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:39.201388 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:39.239083 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:39.263760 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c119de-0fc4-40ba-9793-c8bce71e317d None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:39.283869 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fa19b3-45e4-4ff9-b9f7-377b053d8467 None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:39.299391 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976fa3b7-2ec9-431a-98eb-13a6ecff204d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:39.300070 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976fa3b7-2ec9-431a-98eb-13a6ecff204d None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:39.300355 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976fa3b7-2ec9-431a-98eb-13a6ecff204d None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:39.300635 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976fa3b7-2ec9-431a-98eb-13a6ecff204d None admin] PATH_INFO: `/v3/users` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:39.301473 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976fa3b7-2ec9-431a-98eb-13a6ecff204d None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:39.301786 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976fa3b7-2ec9-431a-98eb-13a6ecff204d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:39.303260 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976fa3b7-2ec9-431a-98eb-13a6ecff204d None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:39.320813 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185c3bd8-beea-46af-b0f7-fc6e836d7aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:39.321102 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 330/662] 10.222.0.57 () {56 vars in 1139 bytes} [Thu May 19 14:24:39 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:39.321906 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976fa3b7-2ec9-431a-98eb-13a6ecff204d None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:39.322276 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185c3bd8-beea-46af-b0f7-fc6e836d7aac None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:39.322565 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185c3bd8-beea-46af-b0f7-fc6e836d7aac None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:39.322840 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185c3bd8-beea-46af-b0f7-fc6e836d7aac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:39.400753 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185c3bd8-beea-46af-b0f7-fc6e836d7aac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:39.402834 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185c3bd8-beea-46af-b0f7-fc6e836d7aac None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:39.432060 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:39.432650 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976fa3b7-2ec9-431a-98eb-13a6ecff204d None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:39.470212 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f77b5b-ce79-4f20-9226-68feca343ae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:39.471264 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f77b5b-ce79-4f20-9226-68feca343ae4 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:39.471705 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f77b5b-ce79-4f20-9226-68feca343ae4 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:39.472159 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f77b5b-ce79-4f20-9226-68feca343ae4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:39.474553 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:39.480905 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f77b5b-ce79-4f20-9226-68feca343ae4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:39.482407 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f77b5b-ce79-4f20-9226-68feca343ae4 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:39.497440 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-d3f77b5b-ce79-4f20-9226-68feca343ae4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:24:39.497440 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:39.497440 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:39.497440 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:39.497440 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:39.497440 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:39.497440 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:39.497440 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:39.497440 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:39.497440 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:39.497440 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:39.497440 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:39.497440 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:24:39.497440 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:24:39.497440 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:24:39.498655 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:24:39.498655 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:39.498655 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:39.498655 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:39.498655 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:39.498655 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:39.498655 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:39.498655 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:39.498655 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:24:39.498655 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:39.498655 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:39.498655 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:39.498655 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:39.498655 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:39.498655 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:39.498655 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:39.498655 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:39.499841 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:24:39.499841 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:24:39.499841 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:24:39.499841 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:24:39.499841 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:24:39.499841 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:24:39.499841 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:24:39.499841 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:24:39.503441 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 331/663] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:24:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:39.504766 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c7d93d-4394-4b9b-8e34-1dd85f8b8070 None None] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:39.505061 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c7d93d-4394-4b9b-8e34-1dd85f8b8070 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:39.505369 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c7d93d-4394-4b9b-8e34-1dd85f8b8070 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:39.517209 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 333/664] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:24:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:24:39.519072 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185c3bd8-beea-46af-b0f7-fc6e836d7aac None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:39.522813 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.auth.core [None req-72c7d93d-4394-4b9b-8e34-1dd85f8b8070 None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143587) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:24:39.524008 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-72c7d93d-4394-4b9b-8e34-1dd85f8b8070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:24:39.594205 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 332/665] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:24:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:24:39.595054 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c7d93d-4394-4b9b-8e34-1dd85f8b8070 None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:39.634564 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185c3bd8-beea-46af-b0f7-fc6e836d7aac None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:39.663833 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:39.674041 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2006d1e2-97ba-4b0b-b561-2c73600e6d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:39.674802 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2006d1e2-97ba-4b0b-b561-2c73600e6d71 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:39.675160 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2006d1e2-97ba-4b0b-b561-2c73600e6d71 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:39.675512 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2006d1e2-97ba-4b0b-b561-2c73600e6d71 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:39.711053 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c7d93d-4394-4b9b-8e34-1dd85f8b8070 None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:39.742261 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:39.748782 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2006d1e2-97ba-4b0b-b561-2c73600e6d71 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:39.750192 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2006d1e2-97ba-4b0b-b561-2c73600e6d71 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:39.756954 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa4695e-7827-4bec-a0d9-8b677be2f414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:39.757623 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa4695e-7827-4bec-a0d9-8b677be2f414 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:39.757947 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa4695e-7827-4bec-a0d9-8b677be2f414 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:39.758229 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa4695e-7827-4bec-a0d9-8b677be2f414 None admin] PATH_INFO: `/v3/projects` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:39.759024 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa4695e-7827-4bec-a0d9-8b677be2f414 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:39.759339 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa4695e-7827-4bec-a0d9-8b677be2f414 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:39.760777 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa4695e-7827-4bec-a0d9-8b677be2f414 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:39.779331 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 333/666] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:24:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:39.779331 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa4695e-7827-4bec-a0d9-8b677be2f414 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:39.869911 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 334/667] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:24:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:24:39.871372 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2006d1e2-97ba-4b0b-b561-2c73600e6d71 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:39.889896 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa4695e-7827-4bec-a0d9-8b677be2f414 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:39.899049 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:39.930556 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7456b2eb-26a8-4729-8d67-c76d6e203420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:39.931349 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7456b2eb-26a8-4729-8d67-c76d6e203420 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:39.931746 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7456b2eb-26a8-4729-8d67-c76d6e203420 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:39.932023 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7456b2eb-26a8-4729-8d67-c76d6e203420 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:39.943710 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7456b2eb-26a8-4729-8d67-c76d6e203420 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:39.945158 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7456b2eb-26a8-4729-8d67-c76d6e203420 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:39.963738 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-7456b2eb-26a8-4729-8d67-c76d6e203420 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:39.963738 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:39.963738 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:39.963738 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:39.963738 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:39.963738 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:39.963738 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:39.963738 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:39.963738 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:39.963738 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:39.963738 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:39.963738 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:39.963738 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:24:39.963738 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:24:39.963738 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:24:39.965276 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:24:39.965276 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:39.965276 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:39.965276 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:39.965276 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:39.965276 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:39.965276 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:39.965276 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:39.965276 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:24:39.965276 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:39.965276 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:39.965276 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:39.965276 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:39.965276 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:39.965276 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:39.965276 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:39.965276 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:39.966861 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:24:39.966861 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:24:39.966861 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:39.966861 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:24:39.966861 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 334/668] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:24:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:39.966861 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7456b2eb-26a8-4729-8d67-c76d6e203420 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:39.979735 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2006d1e2-97ba-4b0b-b561-2c73600e6d71 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:40.007538 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:40.017727 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c704bdd-3c1a-46a5-97ba-0ffcd63c562d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:40.018747 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c704bdd-3c1a-46a5-97ba-0ffcd63c562d None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:40.019126 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c704bdd-3c1a-46a5-97ba-0ffcd63c562d None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:40.019508 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c704bdd-3c1a-46a5-97ba-0ffcd63c562d None admin] PATH_INFO: `/v3/roles/service` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:40.031864 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c704bdd-3c1a-46a5-97ba-0ffcd63c562d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:24:40.039926 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-7c704bdd-3c1a-46a5-97ba-0ffcd63c562d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:24:40.039926 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:40.039926 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:40.039926 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:40.039926 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:40.039926 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:40.039926 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:40.039926 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:40.039926 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:40.039926 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:40.039926 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:40.039926 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:40.039926 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:24:40.039926 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:24:40.039926 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:24:40.039926 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:24:40.041266 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:24:40.041266 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:24:40.041266 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:24:40.041266 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:24:40.041266 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:40.041266 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:40.041266 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:40.041266 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:40.041266 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:40.041266 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:40.041266 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:40.041266 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:24:40.041266 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:40.041266 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:40.041266 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:40.041266 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:40.043617 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:40.043617 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:40.043617 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:40.043617 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:40.043617 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:24:40.043617 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:24:40.043617 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:24:40.043617 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:24:40.043617 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:24:40.043617 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:24:40.043617 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:24:40.043617 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:24:40.043617 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 335/669] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 14:24:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:40.044704 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c704bdd-3c1a-46a5-97ba-0ffcd63c562d None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:40.081375 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7456b2eb-26a8-4729-8d67-c76d6e203420 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:40.117957 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64b38bc-f0a3-413f-8901-729ac2ce2f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:40.118623 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64b38bc-f0a3-413f-8901-729ac2ce2f06 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:40.118908 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64b38bc-f0a3-413f-8901-729ac2ce2f06 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:40.119188 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64b38bc-f0a3-413f-8901-729ac2ce2f06 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:40.120032 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64b38bc-f0a3-413f-8901-729ac2ce2f06 None admin] RBAC: Adding query filter params (role.id=b0b74fefb27e4a0ebd895893aa9862d7, scope.project.id=252d9673f08c479a98d04fee3f5856a4, user.id=afec042821074a889ef9afa75bb8bf40) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:40.120346 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64b38bc-f0a3-413f-8901-729ac2ce2f06 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:40.122106 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64b38bc-f0a3-413f-8901-729ac2ce2f06 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:40.128198 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 335/670] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:24:39 2022] GET /identity/v3/role_assignments?user.id=afec042821074a889ef9afa75bb8bf40&scope.project.id=252d9673f08c479a98d04fee3f5856a4&role.id=b0b74fefb27e4a0ebd895893aa9862d7 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:40.128989 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64b38bc-f0a3-413f-8901-729ac2ce2f06 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:40.158128 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:40.179990 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c704bdd-3c1a-46a5-97ba-0ffcd63c562d None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:40.233411 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f3dfc3-faf8-4c93-93ef-707943ddbde8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:40.234848 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f3dfc3-faf8-4c93-93ef-707943ddbde8 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:40.235673 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f3dfc3-faf8-4c93-93ef-707943ddbde8 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:40.236110 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f3dfc3-faf8-4c93-93ef-707943ddbde8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:40.250063 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:40.253733 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f3dfc3-faf8-4c93-93ef-707943ddbde8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:40.253977 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64b38bc-f0a3-413f-8901-729ac2ce2f06 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:40.255821 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f3dfc3-faf8-4c93-93ef-707943ddbde8 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:40.285660 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-64f3dfc3-faf8-4c93-93ef-707943ddbde8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:40.285660 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:40.285660 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:40.285660 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:40.285660 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:40.285660 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:40.285660 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:40.285660 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:40.285660 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:40.285660 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:40.285660 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:40.285660 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:40.285660 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:24:40.285660 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:24:40.285660 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:24:40.286896 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:24:40.286896 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:40.286896 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:40.286896 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:40.286896 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:40.286896 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:40.286896 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:40.286896 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:40.286896 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:24:40.286896 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:40.286896 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:40.286896 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:40.286896 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:40.286896 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:40.286896 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:40.286896 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:40.286896 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:40.288084 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:24:40.288084 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:24:40.288084 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:40.288084 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:24:40.295248 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 336/671] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:24:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:40.295248 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f3dfc3-faf8-4c93-93ef-707943ddbde8 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:40.306230 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3626245-6297-49b1-bf2c-67ae7d8f6a82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:40.307925 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3626245-6297-49b1-bf2c-67ae7d8f6a82 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:40.309197 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3626245-6297-49b1-bf2c-67ae7d8f6a82 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:40.309693 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3626245-6297-49b1-bf2c-67ae7d8f6a82 None admin] PATH_INFO: `/v3/domains` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:40.311136 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3626245-6297-49b1-bf2c-67ae7d8f6a82 None admin] RBAC: Adding query filter params (name=Default) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:40.311669 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3626245-6297-49b1-bf2c-67ae7d8f6a82 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:40.314128 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3626245-6297-49b1-bf2c-67ae7d8f6a82 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:40.331745 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 336/672] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:24:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:40.332928 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3626245-6297-49b1-bf2c-67ae7d8f6a82 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:40.428016 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f3dfc3-faf8-4c93-93ef-707943ddbde8 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:40.462800 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3626245-6297-49b1-bf2c-67ae7d8f6a82 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:40.472932 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74df9dd-a26a-4f8c-9318-8329aa7ffee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:40.473912 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74df9dd-a26a-4f8c-9318-8329aa7ffee5 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:40.474184 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74df9dd-a26a-4f8c-9318-8329aa7ffee5 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:40.474464 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74df9dd-a26a-4f8c-9318-8329aa7ffee5 None admin] PATH_INFO: `/v3/roles` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:40.475221 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74df9dd-a26a-4f8c-9318-8329aa7ffee5 None admin] RBAC: Adding query filter params (name=service) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:40.475532 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74df9dd-a26a-4f8c-9318-8329aa7ffee5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:40.476913 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74df9dd-a26a-4f8c-9318-8329aa7ffee5 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:40.487405 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 337/673] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 19 14:24:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:40.488391 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74df9dd-a26a-4f8c-9318-8329aa7ffee5 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:40.498363 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:40.503801 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbfa5ed-2778-4060-8fd0-1f9d2129e08c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:40.504476 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbfa5ed-2778-4060-8fd0-1f9d2129e08c None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:40.504762 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbfa5ed-2778-4060-8fd0-1f9d2129e08c None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:40.505045 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbfa5ed-2778-4060-8fd0-1f9d2129e08c None admin] PATH_INFO: `/v3/domains` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:40.505780 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbfa5ed-2778-4060-8fd0-1f9d2129e08c None admin] RBAC: Adding query filter params (name=Default) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:40.506124 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbfa5ed-2778-4060-8fd0-1f9d2129e08c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:40.507512 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbfa5ed-2778-4060-8fd0-1f9d2129e08c None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:40.525046 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 337/674] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:24:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:40.525046 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbfa5ed-2778-4060-8fd0-1f9d2129e08c None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:40.551275 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:40.638321 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74df9dd-a26a-4f8c-9318-8329aa7ffee5 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:40.641537 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbfa5ed-2778-4060-8fd0-1f9d2129e08c None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:40.681148 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6dec95-ad21-4e2a-809b-d881a05c0902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:40.681653 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78862a0b-edaa-472a-ac06-b07dc5800103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:40.681970 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6dec95-ad21-4e2a-809b-d881a05c0902 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:40.682254 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6dec95-ad21-4e2a-809b-d881a05c0902 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:40.682527 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6dec95-ad21-4e2a-809b-d881a05c0902 None admin] PATH_INFO: `/v3/projects/service` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:40.683788 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78862a0b-edaa-472a-ac06-b07dc5800103 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:40.684235 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78862a0b-edaa-472a-ac06-b07dc5800103 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:40.684854 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78862a0b-edaa-472a-ac06-b07dc5800103 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:40.691472 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6dec95-ad21-4e2a-809b-d881a05c0902 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:40.693413 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6dec95-ad21-4e2a-809b-d881a05c0902 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:40.697131 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78862a0b-edaa-472a-ac06-b07dc5800103 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:40.699585 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78862a0b-edaa-472a-ac06-b07dc5800103 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:40.711078 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-2e6dec95-ad21-4e2a-809b-d881a05c0902 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:24:40.711078 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:40.711078 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:40.711078 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:40.711078 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:40.711078 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:40.711078 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:40.711078 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:40.711078 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:40.711078 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:40.711078 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:40.711078 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:40.711078 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:24:40.711078 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:24:40.711078 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:24:40.712133 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:24:40.712133 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:40.712133 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:40.712133 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:40.712133 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:40.712133 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:40.712133 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:40.712133 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:40.712133 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:24:40.712133 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:40.712133 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:40.712133 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:40.712133 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:40.712133 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:40.712133 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:40.712133 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:40.712133 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:40.713151 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:24:40.713151 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:24:40.713151 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:24:40.713151 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:24:40.713151 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:24:40.713151 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:24:40.713151 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:24:40.713151 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:24:40.713151 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 338/675] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:24:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:40.713151 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6dec95-ad21-4e2a-809b-d881a05c0902 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:40.725868 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-78862a0b-edaa-472a-ac06-b07dc5800103 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:40.725868 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:40.725868 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:40.725868 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:40.725868 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:40.725868 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:40.725868 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:40.725868 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:40.725868 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:40.725868 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:40.725868 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:40.725868 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:40.725868 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:24:40.725868 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:24:40.725868 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:24:40.727199 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:24:40.727199 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:40.727199 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:40.727199 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:40.727199 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:40.727199 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:40.727199 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:40.727199 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:40.727199 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:24:40.727199 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:40.727199 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:40.727199 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:40.727199 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:40.727199 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:40.727199 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:40.727199 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:40.727199 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:40.728216 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:24:40.728216 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:24:40.728216 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:40.728216 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:24:40.731031 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 338/676] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:24:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:40.732160 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78862a0b-edaa-472a-ac06-b07dc5800103 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:40.789254 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:40.832171 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:40.837940 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6dec95-ad21-4e2a-809b-d881a05c0902 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:40.870161 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78862a0b-edaa-472a-ac06-b07dc5800103 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:40.885407 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4f4e91-04a7-44c4-b4af-28fbc6722ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:40.887424 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4f4e91-04a7-44c4-b4af-28fbc6722ebe None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:40.887941 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4f4e91-04a7-44c4-b4af-28fbc6722ebe None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:40.888698 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4f4e91-04a7-44c4-b4af-28fbc6722ebe None admin] PATH_INFO: `/v3/projects/service` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:40.898623 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4f4e91-04a7-44c4-b4af-28fbc6722ebe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:40.901049 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4f4e91-04a7-44c4-b4af-28fbc6722ebe None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:40.909091 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf32954d-434f-4cc8-b890-9d1d431f84dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:40.909895 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf32954d-434f-4cc8-b890-9d1d431f84dc None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:40.910290 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf32954d-434f-4cc8-b890-9d1d431f84dc None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:40.910671 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf32954d-434f-4cc8-b890-9d1d431f84dc None admin] PATH_INFO: `/v3/projects` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:40.911567 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf32954d-434f-4cc8-b890-9d1d431f84dc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:40.911982 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf32954d-434f-4cc8-b890-9d1d431f84dc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:40.913536 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf32954d-434f-4cc8-b890-9d1d431f84dc None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:40.920522 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-6b4f4e91-04a7-44c4-b4af-28fbc6722ebe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:24:40.920522 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:40.920522 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:40.920522 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:40.920522 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:40.920522 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:40.920522 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:40.920522 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:40.920522 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:40.920522 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:40.920522 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:40.920522 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:40.920522 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:24:40.920522 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:24:40.920522 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:24:40.921693 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:24:40.921693 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:40.921693 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:40.921693 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:40.921693 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:40.921693 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:40.921693 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:40.921693 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:40.921693 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:24:40.921693 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:40.921693 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:40.921693 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:40.921693 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:40.921693 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:40.921693 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:40.921693 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:40.921693 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:40.922957 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:24:40.922957 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:24:40.922957 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:24:40.922957 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:24:40.922957 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:24:40.922957 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:24:40.922957 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:24:40.922957 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:24:40.930555 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 339/677] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:24:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:40.931949 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4f4e91-04a7-44c4-b4af-28fbc6722ebe None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:40.934666 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 339/678] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:24:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:40.935856 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf32954d-434f-4cc8-b890-9d1d431f84dc None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:41.046785 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4f4e91-04a7-44c4-b4af-28fbc6722ebe None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:41.047682 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf32954d-434f-4cc8-b890-9d1d431f84dc None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:41.071510 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:41.084454 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9115447-fb2e-4b84-b005-f35f55cbe179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:41.084905 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:41.085036 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9115447-fb2e-4b84-b005-f35f55cbe179 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:41.085309 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9115447-fb2e-4b84-b005-f35f55cbe179 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:41.085597 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9115447-fb2e-4b84-b005-f35f55cbe179 None admin] PATH_INFO: `/v3/domains` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:41.086332 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9115447-fb2e-4b84-b005-f35f55cbe179 None admin] RBAC: Adding query filter params (name=Default) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:41.086643 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9115447-fb2e-4b84-b005-f35f55cbe179 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:41.088058 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9115447-fb2e-4b84-b005-f35f55cbe179 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:41.091927 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f0f4fe-88d4-46f8-a04f-ab35d869561a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:41.092940 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f0f4fe-88d4-46f8-a04f-ab35d869561a None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:41.093584 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f0f4fe-88d4-46f8-a04f-ab35d869561a None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:41.093913 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f0f4fe-88d4-46f8-a04f-ab35d869561a None admin] PATH_INFO: `/v3/projects` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:41.094713 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f0f4fe-88d4-46f8-a04f-ab35d869561a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:41.095029 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f0f4fe-88d4-46f8-a04f-ab35d869561a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:41.096480 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f0f4fe-88d4-46f8-a04f-ab35d869561a None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:41.105818 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 340/679] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:24:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:41.105818 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9115447-fb2e-4b84-b005-f35f55cbe179 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:41.110837 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 340/680] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:24:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:41.112248 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3675c7bb-add3-4a71-a9ad-8ac0d49260dd None None] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:41.112547 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3675c7bb-add3-4a71-a9ad-8ac0d49260dd None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:41.112827 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3675c7bb-add3-4a71-a9ad-8ac0d49260dd None None] PATH_INFO: `/` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:41.114208 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 341/681] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:24:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:24:41.115011 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3675c7bb-add3-4a71-a9ad-8ac0d49260dd None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:41.228208 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9115447-fb2e-4b84-b005-f35f55cbe179 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:41.265958 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82071777-5226-4768-9867-7bbdd46f2442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:41.266619 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82071777-5226-4768-9867-7bbdd46f2442 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:41.266902 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82071777-5226-4768-9867-7bbdd46f2442 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:41.267188 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82071777-5226-4768-9867-7bbdd46f2442 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:41.268014 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82071777-5226-4768-9867-7bbdd46f2442 None admin] RBAC: Adding query filter params (role.id=b0b74fefb27e4a0ebd895893aa9862d7, scope.project.id=252d9673f08c479a98d04fee3f5856a4, user.id=95d18a2c8d8d4251ba8961a908574ee2) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:41.268338 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82071777-5226-4768-9867-7bbdd46f2442 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:41.269751 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82071777-5226-4768-9867-7bbdd46f2442 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:41.277883 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 341/682] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:24:41 2022] GET /identity/v3/role_assignments?user.id=95d18a2c8d8d4251ba8961a908574ee2&scope.project.id=252d9673f08c479a98d04fee3f5856a4&role.id=b0b74fefb27e4a0ebd895893aa9862d7 => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:41.277883 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82071777-5226-4768-9867-7bbdd46f2442 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:41.352211 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:41.362628 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3675c7bb-add3-4a71-a9ad-8ac0d49260dd None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:41.403420 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46a0973-af53-4893-9bd9-1b4d76646112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:41.404554 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46a0973-af53-4893-9bd9-1b4d76646112 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:41.405158 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46a0973-af53-4893-9bd9-1b4d76646112 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:41.405636 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46a0973-af53-4893-9bd9-1b4d76646112 None admin] PATH_INFO: `/v3/users/swift` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:41.414584 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46a0973-af53-4893-9bd9-1b4d76646112 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:41.416593 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46a0973-af53-4893-9bd9-1b4d76646112 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:41.431000 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82071777-5226-4768-9867-7bbdd46f2442 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:41.436873 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-a46a0973-af53-4893-9bd9-1b4d76646112 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 14:24:41.436873 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:41.436873 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:41.436873 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:41.436873 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:41.436873 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:41.436873 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:41.436873 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:41.436873 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:41.436873 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:41.436873 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:41.436873 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:41.436873 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:24:41.436873 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:24:41.436873 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:24:41.436873 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:24:41.438028 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:41.438028 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:41.438028 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:24:41.438028 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:24:41.438028 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:24:41.438028 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:24:41.438028 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:41.438028 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:41.438028 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:41.438028 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:41.438028 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:41.438028 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:24:41.438028 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:41.438028 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:41.438028 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:41.438028 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:41.438028 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:41.438028 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:41.439122 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:41.439122 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:41.439122 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:24:41.439122 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:24:41.439122 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:24:41.439122 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:24:41.439122 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:24:41.439122 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:24:41.439122 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 14:24:41.439122 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:24:41.442328 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 342/683] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:24:41 2022] GET /identity/v3/users/swift => generated 83 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:41.443870 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb94afa9-3ecc-45db-81c7-9e720f10409c None None] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:41.444151 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb94afa9-3ecc-45db-81c7-9e720f10409c None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:41.444433 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb94afa9-3ecc-45db-81c7-9e720f10409c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:41.468857 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.auth.core [None req-eb94afa9-3ecc-45db-81c7-9e720f10409c None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143587) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:24:41.470684 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-eb94afa9-3ecc-45db-81c7-9e720f10409c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:24:41.482019 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bcf58c-ac16-4336-a6a1-e22462ddca07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:41.483907 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bcf58c-ac16-4336-a6a1-e22462ddca07 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:41.484191 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bcf58c-ac16-4336-a6a1-e22462ddca07 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:41.484464 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bcf58c-ac16-4336-a6a1-e22462ddca07 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:41.485303 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bcf58c-ac16-4336-a6a1-e22462ddca07 None admin] RBAC: Adding query filter params (role.id=b0b74fefb27e4a0ebd895893aa9862d7, scope.project.id=252d9673f08c479a98d04fee3f5856a4, user.id=0ed9caca45c9449b84582da44a1262ef) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:41.485622 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bcf58c-ac16-4336-a6a1-e22462ddca07 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:41.487699 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bcf58c-ac16-4336-a6a1-e22462ddca07 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:41.497541 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 342/684] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:24:41 2022] GET /identity/v3/role_assignments?user.id=0ed9caca45c9449b84582da44a1262ef&scope.project.id=252d9673f08c479a98d04fee3f5856a4&role.id=b0b74fefb27e4a0ebd895893aa9862d7 => generated 608 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:41.498443 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bcf58c-ac16-4336-a6a1-e22462ddca07 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:41.546175 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:41.578838 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 343/685] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:24:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:24:41.612921 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb94afa9-3ecc-45db-81c7-9e720f10409c None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:41.656330 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bcf58c-ac16-4336-a6a1-e22462ddca07 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:41.690345 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:41.739073 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e33860a-6b6a-48d8-b269-559285405443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:41.739761 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e33860a-6b6a-48d8-b269-559285405443 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:41.740044 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e33860a-6b6a-48d8-b269-559285405443 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:41.740322 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e33860a-6b6a-48d8-b269-559285405443 None admin] PATH_INFO: `/v3/users` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:41.741181 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e33860a-6b6a-48d8-b269-559285405443 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:41.741496 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e33860a-6b6a-48d8-b269-559285405443 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:41.743079 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e33860a-6b6a-48d8-b269-559285405443 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:41.767102 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 343/686] 10.222.0.57 () {56 vars in 1135 bytes} [Thu May 19 14:24:41 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:41.772757 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e33860a-6b6a-48d8-b269-559285405443 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:41.801244 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb94afa9-3ecc-45db-81c7-9e720f10409c None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:41.851299 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7710c07b-5c2e-4dd9-bd4e-4d4641926a0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:41.852616 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7710c07b-5c2e-4dd9-bd4e-4d4641926a0a None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:41.853970 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7710c07b-5c2e-4dd9-bd4e-4d4641926a0a None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:41.854475 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7710c07b-5c2e-4dd9-bd4e-4d4641926a0a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:41.877193 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7710c07b-5c2e-4dd9-bd4e-4d4641926a0a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:24:41.893314 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-7710c07b-5c2e-4dd9-bd4e-4d4641926a0a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:24:41.893314 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:41.893314 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:41.893314 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:41.893314 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:41.893314 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:41.893314 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:41.893314 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:41.893314 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:41.893314 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:41.893314 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:41.893314 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:41.893314 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:24:41.893314 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:24:41.893314 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:24:41.893314 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:24:41.894460 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:24:41.894460 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:24:41.894460 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:24:41.894460 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:24:41.894460 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:41.894460 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:41.894460 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:41.894460 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:41.894460 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:41.894460 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:41.894460 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:41.894460 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:24:41.894460 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:41.894460 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:41.894460 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:41.894460 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:41.895512 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:41.895512 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:41.895512 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:41.895512 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:41.895512 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:24:41.895512 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:24:41.895512 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:24:41.895512 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:24:41.895512 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:24:41.895512 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:24:41.895512 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:24:41.895512 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:24:41.902538 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 344/687] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:24:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:41.906421 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7710c07b-5c2e-4dd9-bd4e-4d4641926a0a None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:41.915520 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e33860a-6b6a-48d8-b269-559285405443 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:41.960795 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab8e83e-59c5-4adb-8374-7382e32e5a74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:41.961616 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab8e83e-59c5-4adb-8374-7382e32e5a74 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:41.962052 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab8e83e-59c5-4adb-8374-7382e32e5a74 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:41.962431 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab8e83e-59c5-4adb-8374-7382e32e5a74 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:41.974286 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:42.030939 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:42.078365 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7710c07b-5c2e-4dd9-bd4e-4d4641926a0a None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:42.097244 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab8e83e-59c5-4adb-8374-7382e32e5a74 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:42.098988 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab8e83e-59c5-4adb-8374-7382e32e5a74 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:42.151843 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff47de37-8429-4012-a105-d12ce0ec6de5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:42.152426 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff47de37-8429-4012-a105-d12ce0ec6de5 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:42.152718 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff47de37-8429-4012-a105-d12ce0ec6de5 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:42.152996 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff47de37-8429-4012-a105-d12ce0ec6de5 None admin] PATH_INFO: `/v3/roles` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:42.153761 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff47de37-8429-4012-a105-d12ce0ec6de5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:42.154095 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff47de37-8429-4012-a105-d12ce0ec6de5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:42.155500 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff47de37-8429-4012-a105-d12ce0ec6de5 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:42.177299 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 345/688] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:24:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:42.178211 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff47de37-8429-4012-a105-d12ce0ec6de5 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:42.306195 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 344/689] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:24:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 534 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:24:42.309426 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab8e83e-59c5-4adb-8374-7382e32e5a74 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:42.392683 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff47de37-8429-4012-a105-d12ce0ec6de5 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:42.395171 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:42.430427 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c32082-9b58-4457-86b1-8fffa46377ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:42.431136 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c32082-9b58-4457-86b1-8fffa46377ec None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:42.431413 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c32082-9b58-4457-86b1-8fffa46377ec None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:42.431683 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c32082-9b58-4457-86b1-8fffa46377ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:42.445252 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c32082-9b58-4457-86b1-8fffa46377ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:42.446780 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c32082-9b58-4457-86b1-8fffa46377ec None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:42.467753 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-15c32082-9b58-4457-86b1-8fffa46377ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:42.467753 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:42.467753 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:42.467753 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:42.467753 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:42.467753 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:42.467753 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:42.467753 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:42.467753 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:42.467753 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:42.467753 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:42.467753 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:42.467753 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:24:42.467753 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:24:42.467753 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:24:42.468847 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:24:42.468847 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:42.468847 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:42.468847 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:42.468847 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:42.468847 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:42.468847 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:42.468847 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:42.468847 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:24:42.468847 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:42.468847 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:42.468847 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:42.468847 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:42.468847 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:42.468847 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:42.468847 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:42.468847 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:42.470164 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:24:42.470164 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:24:42.470164 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:42.470164 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:24:42.470164 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 346/690] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:24:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:42.475957 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c32082-9b58-4457-86b1-8fffa46377ec None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:42.481174 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:42.576447 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab8e83e-59c5-4adb-8374-7382e32e5a74 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:42.587667 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c32082-9b58-4457-86b1-8fffa46377ec None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:42.627197 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb936382-8ea4-457d-8631-1ff33e7a77df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:42.627866 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb936382-8ea4-457d-8631-1ff33e7a77df None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:42.628145 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb936382-8ea4-457d-8631-1ff33e7a77df None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:42.628417 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb936382-8ea4-457d-8631-1ff33e7a77df None admin] PATH_INFO: `/v3/domains` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:42.629134 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb936382-8ea4-457d-8631-1ff33e7a77df None admin] RBAC: Adding query filter params (name=Default) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:42.629436 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb936382-8ea4-457d-8631-1ff33e7a77df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:42.630844 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb936382-8ea4-457d-8631-1ff33e7a77df None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:42.656097 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 347/691] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:24:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:24:42.662867 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb936382-8ea4-457d-8631-1ff33e7a77df None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:42.669869 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadd4bda-4c74-4440-b211-902d2d40e72a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:42.669869 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadd4bda-4c74-4440-b211-902d2d40e72a None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:42.669869 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadd4bda-4c74-4440-b211-902d2d40e72a None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:42.670207 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadd4bda-4c74-4440-b211-902d2d40e72a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:42.682617 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadd4bda-4c74-4440-b211-902d2d40e72a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:42.684689 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadd4bda-4c74-4440-b211-902d2d40e72a None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:42.705539 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-fadd4bda-4c74-4440-b211-902d2d40e72a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:42.705539 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:42.705539 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:42.705539 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:42.705539 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:42.705539 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:42.705539 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:42.705539 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:42.705539 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:42.705539 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:42.705539 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:42.705539 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:42.705539 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:24:42.705539 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:24:42.705539 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:24:42.706631 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:24:42.706631 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:42.706631 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:42.706631 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:42.706631 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:42.706631 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:42.706631 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:42.706631 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:42.706631 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:24:42.706631 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:42.706631 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:42.706631 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:42.706631 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:42.706631 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:42.706631 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:42.706631 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:42.706631 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:42.707634 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:24:42.707634 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:24:42.707634 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:42.707634 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:24:42.707891 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:42.708427 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 345/692] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:24:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 399 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:42.713737 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05595de-6f05-4f16-9212-661c0af0960f None None] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:42.714871 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05595de-6f05-4f16-9212-661c0af0960f None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:42.715454 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05595de-6f05-4f16-9212-661c0af0960f None None] PATH_INFO: `/` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:42.718039 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 346/693] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:24:42 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:24:42.718039 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05595de-6f05-4f16-9212-661c0af0960f None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:42.823219 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb936382-8ea4-457d-8631-1ff33e7a77df None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:42.867936 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05595de-6f05-4f16-9212-661c0af0960f None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:42.868941 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f2b0f2-8f9e-48b9-8f92-ca307b6f030e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:42.869740 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f2b0f2-8f9e-48b9-8f92-ca307b6f030e None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:42.870052 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f2b0f2-8f9e-48b9-8f92-ca307b6f030e None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:42.870325 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f2b0f2-8f9e-48b9-8f92-ca307b6f030e None admin] PATH_INFO: `/v3/users/nova` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:42.879746 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f2b0f2-8f9e-48b9-8f92-ca307b6f030e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:42.881225 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f2b0f2-8f9e-48b9-8f92-ca307b6f030e None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:42.894699 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:42.900021 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-b5f2b0f2-8f9e-48b9-8f92-ca307b6f030e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 14:24:42.900021 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:42.900021 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:42.900021 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:42.900021 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:42.900021 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:42.900021 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:42.900021 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:42.900021 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:42.900021 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:42.900021 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:42.900021 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:42.900021 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:24:42.900021 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:24:42.900021 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:24:42.900021 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:24:42.901041 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:42.901041 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:42.901041 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:24:42.901041 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:24:42.901041 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:24:42.901041 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:24:42.901041 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:42.901041 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:42.901041 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:42.901041 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:42.901041 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:42.901041 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:24:42.901041 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:42.901041 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:42.901041 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:42.901041 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:42.901041 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:42.901041 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:42.902928 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:42.902928 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:42.902928 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:24:42.902928 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:24:42.902928 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:24:42.902928 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:24:42.902928 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:24:42.902928 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:24:42.902928 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 14:24:42.902928 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:24:42.906761 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 348/694] 10.222.0.57 () {56 vars in 1093 bytes} [Thu May 19 14:24:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:42.908490 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8157fac8-42b7-433f-8989-cb65b01e2be1 None None] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:42.908784 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8157fac8-42b7-433f-8989-cb65b01e2be1 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:42.909058 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8157fac8-42b7-433f-8989-cb65b01e2be1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:42.921174 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9152fd36-c08a-4810-8295-1b8d220fc304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:42.922576 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9152fd36-c08a-4810-8295-1b8d220fc304 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:42.923082 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9152fd36-c08a-4810-8295-1b8d220fc304 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:42.923593 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9152fd36-c08a-4810-8295-1b8d220fc304 None admin] PATH_INFO: `/v3/domains` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:42.924595 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9152fd36-c08a-4810-8295-1b8d220fc304 None admin] RBAC: Adding query filter params (name=Default) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:42.925128 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9152fd36-c08a-4810-8295-1b8d220fc304 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:42.926814 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9152fd36-c08a-4810-8295-1b8d220fc304 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:42.927771 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.auth.core [None req-8157fac8-42b7-433f-8989-cb65b01e2be1 None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143587) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:24:42.929098 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-8157fac8-42b7-433f-8989-cb65b01e2be1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:24:42.941468 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 347/695] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:24:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:42.942773 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9152fd36-c08a-4810-8295-1b8d220fc304 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:43.022265 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 349/696] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:24:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:24:43.023863 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8157fac8-42b7-433f-8989-cb65b01e2be1 None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:43.037606 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:43.051308 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9152fd36-c08a-4810-8295-1b8d220fc304 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:43.088907 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a623fd-42b6-429a-9d17-b4fb8c979c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:43.090551 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a623fd-42b6-429a-9d17-b4fb8c979c36 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:43.091056 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a623fd-42b6-429a-9d17-b4fb8c979c36 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:43.091621 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a623fd-42b6-429a-9d17-b4fb8c979c36 None admin] PATH_INFO: `/v3/users` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:43.092689 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a623fd-42b6-429a-9d17-b4fb8c979c36 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:43.093232 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a623fd-42b6-429a-9d17-b4fb8c979c36 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:43.095070 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a623fd-42b6-429a-9d17-b4fb8c979c36 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:43.115436 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 348/697] 10.222.0.57 () {56 vars in 1133 bytes} [Thu May 19 14:24:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:43.120928 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a623fd-42b6-429a-9d17-b4fb8c979c36 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:43.157611 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8157fac8-42b7-433f-8989-cb65b01e2be1 None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:43.166643 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:43.197134 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ff55c4-054f-4075-abdc-1ac2e3926f05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:43.198187 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ff55c4-054f-4075-abdc-1ac2e3926f05 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:43.199124 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ff55c4-054f-4075-abdc-1ac2e3926f05 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:43.199670 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ff55c4-054f-4075-abdc-1ac2e3926f05 None admin] PATH_INFO: `/v3/projects/service` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:43.208046 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ff55c4-054f-4075-abdc-1ac2e3926f05 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:43.209885 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ff55c4-054f-4075-abdc-1ac2e3926f05 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:43.226117 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-a6ff55c4-054f-4075-abdc-1ac2e3926f05 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:24:43.226117 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:43.226117 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:43.226117 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:43.226117 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:43.226117 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:43.226117 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:43.226117 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:43.226117 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:43.226117 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:43.226117 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:43.226117 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:43.226117 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:24:43.226117 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:24:43.226117 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:24:43.227144 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:24:43.227144 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:43.227144 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:43.227144 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:43.227144 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:43.227144 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:43.227144 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:43.227144 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:43.227144 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:24:43.227144 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:43.227144 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:43.227144 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:43.227144 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:43.227144 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:43.227144 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:43.227144 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:43.227144 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:43.228130 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:24:43.228130 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:24:43.228130 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:24:43.228130 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:24:43.228130 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:24:43.228130 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:24:43.228130 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:24:43.228130 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:24:43.235838 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a623fd-42b6-429a-9d17-b4fb8c979c36 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:43.236182 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 350/698] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:24:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:43.237433 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ff55c4-054f-4075-abdc-1ac2e3926f05 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:43.272058 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86abe36-78c5-4381-9b82-f908b740e32b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:43.272893 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86abe36-78c5-4381-9b82-f908b740e32b None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:43.273165 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86abe36-78c5-4381-9b82-f908b740e32b None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:43.273461 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86abe36-78c5-4381-9b82-f908b740e32b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:43.284436 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86abe36-78c5-4381-9b82-f908b740e32b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:43.286041 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86abe36-78c5-4381-9b82-f908b740e32b None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:43.306967 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-a86abe36-78c5-4381-9b82-f908b740e32b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:43.306967 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:43.306967 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:43.306967 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:43.306967 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:43.306967 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:43.306967 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:43.306967 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:43.306967 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:43.306967 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:43.306967 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:43.306967 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:43.306967 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:24:43.306967 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:24:43.306967 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:24:43.308718 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:24:43.308718 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:43.308718 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:43.308718 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:43.308718 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:43.308718 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:43.308718 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:43.308718 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:43.308718 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:24:43.308718 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:43.308718 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:43.308718 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:43.308718 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:43.308718 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:43.308718 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:43.308718 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:43.308718 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:43.310338 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:24:43.310338 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:24:43.310338 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:43.310338 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:24:43.314231 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 349/699] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:24:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:43.315109 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86abe36-78c5-4381-9b82-f908b740e32b None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:43.316735 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:43.358016 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ff55c4-054f-4075-abdc-1ac2e3926f05 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:43.402442 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ca0083-407b-411f-8cd9-af8d54640fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:43.402442 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ca0083-407b-411f-8cd9-af8d54640fed None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:43.402442 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ca0083-407b-411f-8cd9-af8d54640fed None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:43.402442 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ca0083-407b-411f-8cd9-af8d54640fed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:43.418841 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:43.430676 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86abe36-78c5-4381-9b82-f908b740e32b None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:43.470608 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a04303-2840-4085-b981-b901ce4a3a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:43.471272 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a04303-2840-4085-b981-b901ce4a3a6d None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:43.471559 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a04303-2840-4085-b981-b901ce4a3a6d None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:43.471837 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a04303-2840-4085-b981-b901ce4a3a6d None admin] PATH_INFO: `/v3/projects` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:43.472682 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a04303-2840-4085-b981-b901ce4a3a6d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:43.472998 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a04303-2840-4085-b981-b901ce4a3a6d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:43.474720 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a04303-2840-4085-b981-b901ce4a3a6d None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:43.488147 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 350/700] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:24:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:43.489075 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a04303-2840-4085-b981-b901ce4a3a6d None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:43.490530 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ca0083-407b-411f-8cd9-af8d54640fed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:43.492169 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ca0083-407b-411f-8cd9-af8d54640fed None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:43.581441 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:43.601278 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a04303-2840-4085-b981-b901ce4a3a6d None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:43.634192 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 351/701] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:24:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:24:43.635138 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ca0083-407b-411f-8cd9-af8d54640fed None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:43.642624 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d268c30-d771-4055-9120-f4dfe0b81830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:43.643306 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d268c30-d771-4055-9120-f4dfe0b81830 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:43.643592 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d268c30-d771-4055-9120-f4dfe0b81830 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:43.643875 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d268c30-d771-4055-9120-f4dfe0b81830 None admin] PATH_INFO: `/v3/domains` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:43.644608 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d268c30-d771-4055-9120-f4dfe0b81830 None admin] RBAC: Adding query filter params (name=Default) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:43.644920 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d268c30-d771-4055-9120-f4dfe0b81830 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:43.646328 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d268c30-d771-4055-9120-f4dfe0b81830 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:43.669983 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 351/702] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:24:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:43.669983 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d268c30-d771-4055-9120-f4dfe0b81830 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:43.679418 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:43.764900 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ca0083-407b-411f-8cd9-af8d54640fed None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:43.802490 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d1b31b-1793-45fd-8dd5-c73df1f9015f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:43.803297 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d1b31b-1793-45fd-8dd5-c73df1f9015f None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:43.803437 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d1b31b-1793-45fd-8dd5-c73df1f9015f None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:43.803697 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d1b31b-1793-45fd-8dd5-c73df1f9015f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:43.804541 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d1b31b-1793-45fd-8dd5-c73df1f9015f None admin] RBAC: Adding query filter params (role.id=b0b74fefb27e4a0ebd895893aa9862d7, scope.project.id=252d9673f08c479a98d04fee3f5856a4, user.id=d3e544b4989e49a19536b40de5069bcf) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:43.804855 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d1b31b-1793-45fd-8dd5-c73df1f9015f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:43.806416 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d1b31b-1793-45fd-8dd5-c73df1f9015f None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:43.811135 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d268c30-d771-4055-9120-f4dfe0b81830 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:43.813166 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 352/703] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:24:43 2022] GET /identity/v3/role_assignments?user.id=d3e544b4989e49a19536b40de5069bcf&scope.project.id=252d9673f08c479a98d04fee3f5856a4&role.id=b0b74fefb27e4a0ebd895893aa9862d7 => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:43.814834 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d1b31b-1793-45fd-8dd5-c73df1f9015f None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:43.873592 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9ebc8f-0534-480c-95aa-b5b79e4e150c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:43.874736 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9ebc8f-0534-480c-95aa-b5b79e4e150c None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:43.877385 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9ebc8f-0534-480c-95aa-b5b79e4e150c None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:43.877908 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9ebc8f-0534-480c-95aa-b5b79e4e150c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:43.899417 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:43.913027 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9ebc8f-0534-480c-95aa-b5b79e4e150c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:43.914544 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9ebc8f-0534-480c-95aa-b5b79e4e150c None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:43.945467 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-df9ebc8f-0534-480c-95aa-b5b79e4e150c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:43.945467 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:43.945467 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:43.945467 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:43.945467 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:43.945467 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:43.945467 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:43.945467 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:43.945467 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:43.945467 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:43.945467 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:43.945467 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:43.945467 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:24:43.945467 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:24:43.945467 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:24:43.947002 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:24:43.947002 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:43.947002 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:43.947002 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:43.947002 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:43.947002 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:43.947002 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:43.947002 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:43.947002 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:24:43.947002 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:43.947002 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:43.947002 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:43.947002 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:43.947002 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:43.947002 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:43.947002 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:43.947002 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:43.949584 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:24:43.949584 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:24:43.949584 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:43.949584 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:24:43.949584 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 352/704] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:24:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:43.958846 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cc1d4b-5832-4007-9ef6-020c9390311e None None] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:43.959183 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cc1d4b-5832-4007-9ef6-020c9390311e None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:43.959419 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cc1d4b-5832-4007-9ef6-020c9390311e None None] PATH_INFO: `/` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:43.970219 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 353/705] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:24:43 2022] GET /identity => generated 270 bytes in 20 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:24:43.970808 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cc1d4b-5832-4007-9ef6-020c9390311e None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:44.072004 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d1b31b-1793-45fd-8dd5-c73df1f9015f None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:44.095184 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:44.126840 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7e422b-3cae-4341-8982-6a041d2fbfee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:44.127136 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7e422b-3cae-4341-8982-6a041d2fbfee None admin] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:44.128017 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7e422b-3cae-4341-8982-6a041d2fbfee None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:44.128017 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7e422b-3cae-4341-8982-6a041d2fbfee None admin] PATH_INFO: `/v3/users` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:44.128528 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7e422b-3cae-4341-8982-6a041d2fbfee None admin] RBAC: Authorizing `identity:create_user()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:44.131897 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7e422b-3cae-4341-8982-6a041d2fbfee None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:44.162193 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cc1d4b-5832-4007-9ef6-020c9390311e None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:44.162728 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 353/706] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 19 14:24:43 2022] POST /identity/v3/users => generated 254 bytes in 349 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:24:44.165124 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eadca16-ed17-4cf8-a4a0-630a6ce1b548 None None] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:44.165124 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eadca16-ed17-4cf8-a4a0-630a6ce1b548 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:44.165877 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eadca16-ed17-4cf8-a4a0-630a6ce1b548 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:44.196064 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.auth.core [None req-4eadca16-ed17-4cf8-a4a0-630a6ce1b548 None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143587) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:24:44.197477 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-4eadca16-ed17-4cf8-a4a0-630a6ce1b548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:24:44.230719 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9301f40-9042-45e4-837f-e8ba28ae883e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:44.232207 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9301f40-9042-45e4-837f-e8ba28ae883e None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:44.233086 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9301f40-9042-45e4-837f-e8ba28ae883e None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:44.233746 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9301f40-9042-45e4-837f-e8ba28ae883e None admin] PATH_INFO: `/v3/domains` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:44.234961 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9301f40-9042-45e4-837f-e8ba28ae883e None admin] RBAC: Adding query filter params (name=Default) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:44.235847 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9301f40-9042-45e4-837f-e8ba28ae883e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:44.237639 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9301f40-9042-45e4-837f-e8ba28ae883e None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:44.267320 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 354/707] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:24:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:44.269126 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d01796-144d-4256-8160-0642eb3dd9e0 None None] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:44.271390 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d01796-144d-4256-8160-0642eb3dd9e0 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:44.272257 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d01796-144d-4256-8160-0642eb3dd9e0 None None] PATH_INFO: `/` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:44.276422 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 355/708] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:24:44 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:24:44.277376 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d01796-144d-4256-8160-0642eb3dd9e0 None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:44.291262 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 354/709] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:24:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:24:44.299180 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee13d743-36b8-4682-8e42-fb5d2c6d1d54 None None] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:44.299848 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee13d743-36b8-4682-8e42-fb5d2c6d1d54 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:44.300454 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee13d743-36b8-4682-8e42-fb5d2c6d1d54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:44.342673 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.auth.core [None req-ee13d743-36b8-4682-8e42-fb5d2c6d1d54 None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143587) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:24:44.344689 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-ee13d743-36b8-4682-8e42-fb5d2c6d1d54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:24:44.366673 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:44.455530 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d01796-144d-4256-8160-0642eb3dd9e0 None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:44.456404 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 355/710] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:24:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:24:44.463226 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee13d743-36b8-4682-8e42-fb5d2c6d1d54 None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:44.501556 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:44.516181 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1de612-750c-475f-b667-51b59fdecc1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:44.517240 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1de612-750c-475f-b667-51b59fdecc1f None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:44.517524 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1de612-750c-475f-b667-51b59fdecc1f None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:44.517801 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1de612-750c-475f-b667-51b59fdecc1f None admin] PATH_INFO: `/v3/projects/service` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:44.525858 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1de612-750c-475f-b667-51b59fdecc1f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:44.528040 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1de612-750c-475f-b667-51b59fdecc1f None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:44.544306 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-3b1de612-750c-475f-b667-51b59fdecc1f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:24:44.544306 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:44.544306 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:44.544306 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:44.544306 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:44.544306 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:44.544306 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:44.544306 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:44.544306 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:44.544306 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:44.544306 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:44.544306 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:44.544306 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:24:44.544306 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:24:44.544306 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:24:44.545450 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:24:44.545450 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:44.545450 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:44.545450 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:44.545450 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:44.545450 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:44.545450 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:44.545450 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:44.545450 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:24:44.545450 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:44.545450 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:44.545450 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:44.545450 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:44.545450 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:44.545450 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:44.545450 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:44.545450 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:44.546489 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:24:44.546489 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:24:44.546489 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:24:44.546489 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:24:44.546489 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:24:44.546489 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:24:44.546489 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:24:44.546489 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:24:44.546489 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 356/711] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:24:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:44.546489 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1de612-750c-475f-b667-51b59fdecc1f None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:44.588496 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee13d743-36b8-4682-8e42-fb5d2c6d1d54 None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:44.630891 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07e0f02-3dc4-48e1-a18a-81be341e65a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:44.631703 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07e0f02-3dc4-48e1-a18a-81be341e65a3 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:44.631982 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07e0f02-3dc4-48e1-a18a-81be341e65a3 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:44.632258 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07e0f02-3dc4-48e1-a18a-81be341e65a3 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:44.632916 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07e0f02-3dc4-48e1-a18a-81be341e65a3 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:44.634450 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07e0f02-3dc4-48e1-a18a-81be341e65a3 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:44.651353 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-a07e0f02-3dc4-48e1-a18a-81be341e65a3 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 14:24:44.651353 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:44.651353 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:24:44.651353 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:24:44.651353 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:24:44.651353 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:24:44.651353 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:24:44.651353 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:24:44.651353 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 14:24:44.651353 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:24:44.651353 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:24:44.651353 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:24:44.651353 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:44.651353 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:44.651353 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:44.651353 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:44.653345 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:44.653345 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:44.653345 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:44.653345 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:44.653345 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:44.653345 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:44.653345 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:44.653345 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:24:44.653345 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:24:44.653345 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:24:44.653345 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:24:44.653345 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:44.653345 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:44.653345 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:44.653345 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:44.653345 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:44.654442 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:44.654442 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:44.654442 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:24:44.654442 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:44.654442 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:44.654442 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:44.654442 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:44.654442 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:44.654442 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:44.654442 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:44.654442 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:44.654442 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:24:44.654442 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:24:44.654442 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 14:24:44.654442 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:24:44.654442 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 356/712] 10.222.0.57 () {56 vars in 1129 bytes} [Thu May 19 14:24:44 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:44.661741 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07e0f02-3dc4-48e1-a18a-81be341e65a3 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:44.674006 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:44.687277 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1de612-750c-475f-b667-51b59fdecc1f None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:44.725352 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf32d90-7807-4db6-a763-6a0235e0b90b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:44.726700 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf32d90-7807-4db6-a763-6a0235e0b90b None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:44.727137 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf32d90-7807-4db6-a763-6a0235e0b90b None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:44.727566 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf32d90-7807-4db6-a763-6a0235e0b90b None admin] PATH_INFO: `/v3/services/network` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:44.728382 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf32d90-7807-4db6-a763-6a0235e0b90b None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:44.730768 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf32d90-7807-4db6-a763-6a0235e0b90b None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:44.746217 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-9cf32d90-7807-4db6-a763-6a0235e0b90b None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 19 14:24:44.746217 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:44.746217 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:24:44.746217 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:24:44.746217 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:24:44.746217 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:24:44.746217 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:24:44.746217 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:24:44.746217 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 14:24:44.746217 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:24:44.746217 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:24:44.746217 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:24:44.746217 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:44.746217 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:44.746217 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:44.746217 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:44.746217 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:44.747724 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:44.747724 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:44.747724 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:44.747724 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:44.747724 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:44.747724 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:44.747724 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:24:44.747724 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:24:44.747724 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:24:44.747724 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:24:44.747724 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:44.747724 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:44.747724 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:44.747724 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:44.747724 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:44.747724 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:44.748834 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:44.748834 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:24:44.748834 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:44.748834 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:44.748834 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:44.748834 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:44.748834 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:44.748834 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:44.748834 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:44.748834 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:44.748834 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:24:44.748834 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:24:44.748834 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 14:24:44.748834 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:24:44.751495 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 357/713] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:24:44 2022] GET /identity/v3/services/network => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:44.752507 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf32d90-7807-4db6-a763-6a0235e0b90b None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:44.778160 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07e0f02-3dc4-48e1-a18a-81be341e65a3 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:44.817966 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:44.825218 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba6d224-8e31-4b2b-98b5-44abccb9fd5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:44.826159 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba6d224-8e31-4b2b-98b5-44abccb9fd5c None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:44.826624 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba6d224-8e31-4b2b-98b5-44abccb9fd5c None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:44.827052 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba6d224-8e31-4b2b-98b5-44abccb9fd5c None admin] PATH_INFO: `/v3/projects` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:44.828555 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba6d224-8e31-4b2b-98b5-44abccb9fd5c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:44.829307 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba6d224-8e31-4b2b-98b5-44abccb9fd5c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:44.831509 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba6d224-8e31-4b2b-98b5-44abccb9fd5c None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:44.849665 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 357/714] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:24:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:44.850822 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba6d224-8e31-4b2b-98b5-44abccb9fd5c None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:44.886644 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf32d90-7807-4db6-a763-6a0235e0b90b None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:44.925092 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd7a1b1-6186-4df2-b92a-11a0d9d40f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:44.926207 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd7a1b1-6186-4df2-b92a-11a0d9d40f37 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:44.926684 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd7a1b1-6186-4df2-b92a-11a0d9d40f37 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:44.927125 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd7a1b1-6186-4df2-b92a-11a0d9d40f37 None admin] PATH_INFO: `/v3/services` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:44.927998 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd7a1b1-6186-4df2-b92a-11a0d9d40f37 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:44.928554 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd7a1b1-6186-4df2-b92a-11a0d9d40f37 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:44.930247 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd7a1b1-6186-4df2-b92a-11a0d9d40f37 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:44.936230 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 358/715] 10.222.0.57 () {56 vars in 1124 bytes} [Thu May 19 14:24:44 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:44.937270 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd7a1b1-6186-4df2-b92a-11a0d9d40f37 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:44.942485 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:44.982703 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba6d224-8e31-4b2b-98b5-44abccb9fd5c None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:45.025763 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a351da0-3d65-4af4-a28c-3a3c6226a8d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:45.028526 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a351da0-3d65-4af4-a28c-3a3c6226a8d4 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:45.029228 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a351da0-3d65-4af4-a28c-3a3c6226a8d4 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:45.029904 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a351da0-3d65-4af4-a28c-3a3c6226a8d4 None admin] PATH_INFO: `/v3/services` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:45.031367 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a351da0-3d65-4af4-a28c-3a3c6226a8d4 None admin] RBAC: Adding query filter params (name=network) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:45.032136 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a351da0-3d65-4af4-a28c-3a3c6226a8d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:45.034840 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a351da0-3d65-4af4-a28c-3a3c6226a8d4 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:45.045874 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 358/716] 10.222.0.57 () {56 vars in 1112 bytes} [Thu May 19 14:24:44 2022] GET /identity/v3/services?name=network => generated 124 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:45.047498 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a351da0-3d65-4af4-a28c-3a3c6226a8d4 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:45.091261 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd7a1b1-6186-4df2-b92a-11a0d9d40f37 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:45.115847 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:45.162380 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32634cf9-1e6f-467b-a372-357269040b6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:45.164386 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32634cf9-1e6f-467b-a372-357269040b6d None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:45.165063 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32634cf9-1e6f-467b-a372-357269040b6d None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:45.165731 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32634cf9-1e6f-467b-a372-357269040b6d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:45.168141 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32634cf9-1e6f-467b-a372-357269040b6d None admin] RBAC: Adding query filter params (role.id=a3b793754e7b40b0bb21afe74eec3c18, scope.project.id=252d9673f08c479a98d04fee3f5856a4, user.id=5f28de22c2ee4c53b6d14d5b0ddf7fe9) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:45.168899 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32634cf9-1e6f-467b-a372-357269040b6d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:45.172362 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32634cf9-1e6f-467b-a372-357269040b6d None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:45.184028 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 359/717] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:24:44 2022] GET /identity/v3/role_assignments?user.id=5f28de22c2ee4c53b6d14d5b0ddf7fe9&scope.project.id=252d9673f08c479a98d04fee3f5856a4&role.id=a3b793754e7b40b0bb21afe74eec3c18 => generated 259 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:45.185621 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32634cf9-1e6f-467b-a372-357269040b6d None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:45.281113 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a351da0-3d65-4af4-a28c-3a3c6226a8d4 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:45.317504 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32634cf9-1e6f-467b-a372-357269040b6d None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:45.319179 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a2e54b-3b38-4124-9d19-927551c0c121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:45.319955 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a2e54b-3b38-4124-9d19-927551c0c121 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:45.320241 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a2e54b-3b38-4124-9d19-927551c0c121 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:45.320518 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a2e54b-3b38-4124-9d19-927551c0c121 None admin] PATH_INFO: `/v3/services` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:45.321746 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a2e54b-3b38-4124-9d19-927551c0c121 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:45.321746 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a2e54b-3b38-4124-9d19-927551c0c121 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:45.323054 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a2e54b-3b38-4124-9d19-927551c0c121 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:45.330197 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:45.334446 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 359/718] 10.222.0.57 () {56 vars in 1124 bytes} [Thu May 19 14:24:45 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:45.366946 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fad128-1754-4310-9258-258c650aad8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:45.367970 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fad128-1754-4310-9258-258c650aad8f None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:45.368423 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fad128-1754-4310-9258-258c650aad8f None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:45.368861 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fad128-1754-4310-9258-258c650aad8f None admin] PATH_INFO: `/v3/services` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:45.369912 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fad128-1754-4310-9258-258c650aad8f None admin] RBAC: Adding query filter params (type=network) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:45.370395 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fad128-1754-4310-9258-258c650aad8f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:45.372614 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fad128-1754-4310-9258-258c650aad8f None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:45.380144 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 360/719] 10.222.0.57 () {56 vars in 1112 bytes} [Thu May 19 14:24:45 2022] GET /identity/v3/services?type=network => generated 124 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:46.767774 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9107051f-7c42-43a4-90ef-d7219ac44df3 None None] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:46.768328 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9107051f-7c42-43a4-90ef-d7219ac44df3 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:46.768328 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9107051f-7c42-43a4-90ef-d7219ac44df3 None None] PATH_INFO: `/` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:46.770160 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 360/720] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:24:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:24:46.775145 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c482fe87-1d25-48f1-90fd-94bc03c7e987 None None] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:46.775965 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c482fe87-1d25-48f1-90fd-94bc03c7e987 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:46.776464 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c482fe87-1d25-48f1-90fd-94bc03c7e987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:46.799317 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.auth.core [None req-c482fe87-1d25-48f1-90fd-94bc03c7e987 None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143586) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:24:46.800858 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.fernet_utils [None req-c482fe87-1d25-48f1-90fd-94bc03c7e987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143586) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:24:46.840713 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:46.924936 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23571124-69bc-491a-a9b2-24aff96f4f9d None None] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:46.925580 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23571124-69bc-491a-a9b2-24aff96f4f9d None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:46.926122 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23571124-69bc-491a-a9b2-24aff96f4f9d None None] PATH_INFO: `/` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:46.929402 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 361/721] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:24:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:24:46.934544 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d10ba84-ac56-4cdd-8818-0651bbe0eb94 None None] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:46.935210 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d10ba84-ac56-4cdd-8818-0651bbe0eb94 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:46.935711 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d10ba84-ac56-4cdd-8818-0651bbe0eb94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:46.983089 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.auth.core [None req-0d10ba84-ac56-4cdd-8818-0651bbe0eb94 None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143587) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:24:46.984720 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-0d10ba84-ac56-4cdd-8818-0651bbe0eb94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:24:47.018150 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 361/722] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:24:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:24:47.059035 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c482fe87-1d25-48f1-90fd-94bc03c7e987 None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:47.081946 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 362/723] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:24:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:24:47.099195 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d10ba84-ac56-4cdd-8818-0651bbe0eb94 None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:47.205264 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d10ba84-ac56-4cdd-8818-0651bbe0eb94 None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:47.234821 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:47.235060 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c482fe87-1d25-48f1-90fd-94bc03c7e987 None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:47.244915 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8103ab6-f47b-4976-95cc-370843efa767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:47.246610 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8103ab6-f47b-4976-95cc-370843efa767 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:47.247358 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8103ab6-f47b-4976-95cc-370843efa767 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:47.247874 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8103ab6-f47b-4976-95cc-370843efa767 None admin] PATH_INFO: `/v3/roles/service` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:47.259267 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8103ab6-f47b-4976-95cc-370843efa767 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:24:47.267882 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-c8103ab6-f47b-4976-95cc-370843efa767 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:24:47.267882 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:47.267882 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:47.267882 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:47.267882 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:47.267882 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:47.267882 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:47.267882 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:47.267882 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:47.267882 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:47.267882 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:47.267882 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:47.267882 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:24:47.267882 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:24:47.267882 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:24:47.267882 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:24:47.268938 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:24:47.268938 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:24:47.268938 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:24:47.268938 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:24:47.268938 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:47.268938 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:47.268938 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:47.268938 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:47.268938 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:47.268938 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:47.268938 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:47.268938 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:24:47.268938 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:47.268938 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:47.268938 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:47.268938 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:47.269929 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:47.269929 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:47.269929 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:47.269929 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:47.269929 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:24:47.269929 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:24:47.269929 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:24:47.269929 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:24:47.269929 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:24:47.269929 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:24:47.269929 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:24:47.269929 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:24:47.277322 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 363/724] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 14:24:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:24:47.278806 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8103ab6-f47b-4976-95cc-370843efa767 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:47.338086 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044b6d5c-88c2-4423-abc2-6b5738224a08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:47.339316 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044b6d5c-88c2-4423-abc2-6b5738224a08 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:47.339725 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044b6d5c-88c2-4423-abc2-6b5738224a08 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:47.340093 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044b6d5c-88c2-4423-abc2-6b5738224a08 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:47.363666 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044b6d5c-88c2-4423-abc2-6b5738224a08 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:24:47.383797 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-044b6d5c-88c2-4423-abc2-6b5738224a08 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:24:47.383797 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:47.383797 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:47.383797 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:47.383797 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:47.383797 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:47.383797 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:47.383797 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:47.383797 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:47.383797 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:47.383797 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:47.383797 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:47.383797 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:24:47.383797 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:24:47.383797 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:24:47.383797 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:24:47.384885 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:24:47.384885 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:24:47.384885 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:24:47.384885 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:24:47.384885 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:47.384885 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:47.384885 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:47.384885 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:47.384885 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:47.384885 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:47.384885 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:47.384885 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:24:47.384885 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:47.384885 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:47.384885 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:47.384885 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:47.387183 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:47.387183 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:47.387183 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:47.387183 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:47.387183 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:24:47.387183 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:24:47.387183 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:24:47.387183 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:24:47.387183 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:24:47.387183 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:24:47.387183 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:24:47.387183 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:24:47.387183 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 362/725] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:24:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:24:47.391286 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8103ab6-f47b-4976-95cc-370843efa767 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:47.394822 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044b6d5c-88c2-4423-abc2-6b5738224a08 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:47.427346 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9e8694-9999-48e2-93db-359814172c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:47.428407 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9e8694-9999-48e2-93db-359814172c5b None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:47.428927 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9e8694-9999-48e2-93db-359814172c5b None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:47.429433 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9e8694-9999-48e2-93db-359814172c5b None admin] PATH_INFO: `/v3/roles` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:47.430432 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9e8694-9999-48e2-93db-359814172c5b None admin] RBAC: Adding query filter params (name=service) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:47.430974 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9e8694-9999-48e2-93db-359814172c5b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:47.432568 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9e8694-9999-48e2-93db-359814172c5b None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:47.445943 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 364/726] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 19 14:24:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:47.447933 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9e8694-9999-48e2-93db-359814172c5b None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:47.449646 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:47.502767 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:47.558140 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9e8694-9999-48e2-93db-359814172c5b None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:47.560794 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044b6d5c-88c2-4423-abc2-6b5738224a08 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:47.597583 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fd5bf7-1c44-4d69-922f-95dbc6168453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:47.598792 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fd5bf7-1c44-4d69-922f-95dbc6168453 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:47.599142 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fd5bf7-1c44-4d69-922f-95dbc6168453 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:47.599490 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fd5bf7-1c44-4d69-922f-95dbc6168453 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:47.600275 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9f3a9a-1254-42b9-b762-fdf728ede157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:47.601175 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9f3a9a-1254-42b9-b762-fdf728ede157 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:47.601672 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9f3a9a-1254-42b9-b762-fdf728ede157 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:47.602178 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9f3a9a-1254-42b9-b762-fdf728ede157 None admin] PATH_INFO: `/v3/roles` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:47.603154 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9f3a9a-1254-42b9-b762-fdf728ede157 None admin] RBAC: Adding query filter params (name=admin) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:47.603696 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9f3a9a-1254-42b9-b762-fdf728ede157 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:47.605690 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9f3a9a-1254-42b9-b762-fdf728ede157 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:47.612566 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fd5bf7-1c44-4d69-922f-95dbc6168453 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:47.614061 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fd5bf7-1c44-4d69-922f-95dbc6168453 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:47.616204 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 363/727] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:24:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:47.622570 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9f3a9a-1254-42b9-b762-fdf728ede157 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:47.632825 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-c3fd5bf7-1c44-4d69-922f-95dbc6168453 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:47.632825 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:47.632825 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:47.632825 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:47.632825 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:47.632825 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:47.632825 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:47.632825 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:47.632825 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:47.632825 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:47.632825 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:47.632825 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:47.632825 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:24:47.632825 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:24:47.632825 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:24:47.633902 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:24:47.633902 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:47.633902 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:47.633902 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:47.633902 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:47.633902 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:47.633902 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:47.633902 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:47.633902 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:24:47.633902 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:47.633902 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:47.633902 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:47.633902 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:47.633902 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:47.633902 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:47.633902 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:47.633902 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:47.634913 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:24:47.634913 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:24:47.634913 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:47.634913 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:24:47.638731 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 365/728] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:24:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:24:47.640855 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fd5bf7-1c44-4d69-922f-95dbc6168453 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:47.784494 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:47.798469 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9f3a9a-1254-42b9-b762-fdf728ede157 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:47.818439 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fd5bf7-1c44-4d69-922f-95dbc6168453 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:47.819976 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:47.843836 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1453b8f-2802-4ed6-a05e-653a98e12d9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:47.845119 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1453b8f-2802-4ed6-a05e-653a98e12d9a None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:47.845699 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1453b8f-2802-4ed6-a05e-653a98e12d9a None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:47.846274 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1453b8f-2802-4ed6-a05e-653a98e12d9a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:47.857791 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bce842-a332-499c-a57f-746436e73106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:47.858588 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bce842-a332-499c-a57f-746436e73106 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:47.858780 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bce842-a332-499c-a57f-746436e73106 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:47.859047 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bce842-a332-499c-a57f-746436e73106 None admin] PATH_INFO: `/v3/domains` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:47.859770 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bce842-a332-499c-a57f-746436e73106 None admin] RBAC: Adding query filter params (name=Default) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:47.860080 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bce842-a332-499c-a57f-746436e73106 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:47.860664 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1453b8f-2802-4ed6-a05e-653a98e12d9a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:47.861458 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bce842-a332-499c-a57f-746436e73106 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:47.862652 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1453b8f-2802-4ed6-a05e-653a98e12d9a None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:47.875903 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 366/729] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:24:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:47.877337 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf7f79e-73e4-4d8c-89c0-936ed1e1d3e8 None None] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:47.877625 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf7f79e-73e4-4d8c-89c0-936ed1e1d3e8 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:47.877925 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf7f79e-73e4-4d8c-89c0-936ed1e1d3e8 None None] PATH_INFO: `/` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:47.882756 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-e1453b8f-2802-4ed6-a05e-653a98e12d9a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:47.882756 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:47.882756 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:47.882756 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:47.882756 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:47.882756 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:47.882756 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:47.882756 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:47.882756 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:47.882756 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:47.882756 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:47.882756 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:47.882756 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:24:47.882756 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:24:47.882756 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:24:47.883802 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:24:47.883802 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:47.883802 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:47.883802 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:47.883802 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:47.883802 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:47.883802 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:47.883802 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:47.883802 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:24:47.883802 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:47.883802 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:47.883802 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:47.883802 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:47.883802 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:47.883802 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:47.883802 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:47.883802 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:47.884823 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:24:47.884823 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:24:47.884823 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:47.884823 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:24:47.885068 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 367/730] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:24:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:24:47.885068 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf7f79e-73e4-4d8c-89c0-936ed1e1d3e8 None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:47.887376 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 364/731] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:24:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:47.893499 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fac554-320b-4434-842a-12ee84a8c760 None None] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:47.894089 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fac554-320b-4434-842a-12ee84a8c760 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:47.894588 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fac554-320b-4434-842a-12ee84a8c760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:47.913995 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.auth.core [None req-30fac554-320b-4434-842a-12ee84a8c760 None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143586) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:24:47.915730 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.fernet_utils [None req-30fac554-320b-4434-842a-12ee84a8c760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143586) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:24:47.993501 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 365/732] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:24:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:24:47.994773 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fac554-320b-4434-842a-12ee84a8c760 None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:48.001109 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf7f79e-73e4-4d8c-89c0-936ed1e1d3e8 None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:48.058970 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11713383-5692-46f9-a30b-139e192d4312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:48.059791 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11713383-5692-46f9-a30b-139e192d4312 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:48.060067 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11713383-5692-46f9-a30b-139e192d4312 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:48.060356 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11713383-5692-46f9-a30b-139e192d4312 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:48.068114 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11713383-5692-46f9-a30b-139e192d4312 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:48.069655 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11713383-5692-46f9-a30b-139e192d4312 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:48.076084 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:48.086854 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-11713383-5692-46f9-a30b-139e192d4312 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 14:24:48.086854 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:48.086854 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:48.086854 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:48.086854 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:48.086854 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:48.086854 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:48.086854 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:48.086854 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:48.086854 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:48.086854 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:48.086854 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:48.086854 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:24:48.086854 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:24:48.086854 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:24:48.086854 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:24:48.088613 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:48.088613 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:48.088613 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:24:48.088613 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:24:48.088613 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:24:48.088613 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:24:48.088613 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:48.088613 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:48.088613 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:48.088613 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:48.088613 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:48.088613 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:24:48.088613 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:48.088613 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:48.088613 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:48.088613 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:48.088613 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:48.088613 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:48.090512 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:48.090512 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:48.090512 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:24:48.090512 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:24:48.090512 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:24:48.090512 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:24:48.090512 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:24:48.090512 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:24:48.090512 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 14:24:48.090512 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:24:48.090512 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 368/733] 10.222.0.57 () {56 vars in 1117 bytes} [Thu May 19 14:24:47 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:48.090512 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11713383-5692-46f9-a30b-139e192d4312 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:48.115622 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fac554-320b-4434-842a-12ee84a8c760 None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:48.119164 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:48.163015 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03be679b-015c-4d3e-a4b8-6a7bf63847d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:48.164186 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03be679b-015c-4d3e-a4b8-6a7bf63847d9 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:48.164754 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03be679b-015c-4d3e-a4b8-6a7bf63847d9 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:48.165269 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03be679b-015c-4d3e-a4b8-6a7bf63847d9 None admin] PATH_INFO: `/v3/domains` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:48.166307 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03be679b-015c-4d3e-a4b8-6a7bf63847d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:48.166934 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03be679b-015c-4d3e-a4b8-6a7bf63847d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:48.168669 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03be679b-015c-4d3e-a4b8-6a7bf63847d9 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:48.186745 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 366/734] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:24:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:48.188057 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03be679b-015c-4d3e-a4b8-6a7bf63847d9 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:48.208099 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11713383-5692-46f9-a30b-139e192d4312 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:48.245565 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aeb7f4-168b-4722-b0aa-37c51e524ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:48.246462 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aeb7f4-168b-4722-b0aa-37c51e524ee4 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:48.246901 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aeb7f4-168b-4722-b0aa-37c51e524ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:48.247239 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aeb7f4-168b-4722-b0aa-37c51e524ee4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:48.258650 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5aeb7f4-168b-4722-b0aa-37c51e524ee4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:24:48.266936 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-d5aeb7f4-168b-4722-b0aa-37c51e524ee4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:24:48.266936 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:48.266936 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:48.266936 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:48.266936 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:48.266936 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:48.266936 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:48.266936 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:48.266936 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:48.266936 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:48.266936 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:48.266936 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:48.266936 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:24:48.266936 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:24:48.266936 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:24:48.266936 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:24:48.268010 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:24:48.268010 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:24:48.268010 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:24:48.268010 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:24:48.268010 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:48.268010 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:48.268010 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:48.268010 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:48.268010 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:48.268010 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:48.268010 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:48.268010 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:24:48.268010 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:48.268010 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:48.268010 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:48.268010 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:48.268955 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:48.268955 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:48.268955 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:48.268955 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:48.268955 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:24:48.268955 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:24:48.268955 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:24:48.268955 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:24:48.268955 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:24:48.268955 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:24:48.268955 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:24:48.268955 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:24:48.272969 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 369/735] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:24:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:48.273869 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aeb7f4-168b-4722-b0aa-37c51e524ee4 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:48.301325 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03be679b-015c-4d3e-a4b8-6a7bf63847d9 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:48.334473 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:48.340796 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa50ba2-a9c0-4b7e-818d-cbaf16467530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:48.341480 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa50ba2-a9c0-4b7e-818d-cbaf16467530 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:48.341764 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa50ba2-a9c0-4b7e-818d-cbaf16467530 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:48.342078 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa50ba2-a9c0-4b7e-818d-cbaf16467530 None admin] PATH_INFO: `/v3/users` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:48.342929 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa50ba2-a9c0-4b7e-818d-cbaf16467530 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:48.343244 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa50ba2-a9c0-4b7e-818d-cbaf16467530 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:48.344671 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa50ba2-a9c0-4b7e-818d-cbaf16467530 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:48.362451 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 367/736] 10.222.0.57 () {56 vars in 1149 bytes} [Thu May 19 14:24:48 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:48.363274 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa50ba2-a9c0-4b7e-818d-cbaf16467530 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:48.364289 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:48.382729 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aeb7f4-168b-4722-b0aa-37c51e524ee4 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:48.417122 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92b6079-5804-44a7-9790-15a7cf8fe3cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:48.417992 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92b6079-5804-44a7-9790-15a7cf8fe3cd None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:48.418281 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92b6079-5804-44a7-9790-15a7cf8fe3cd None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:48.418557 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92b6079-5804-44a7-9790-15a7cf8fe3cd None admin] PATH_INFO: `/v3/users/swift` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:48.426137 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92b6079-5804-44a7-9790-15a7cf8fe3cd None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:48.427576 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92b6079-5804-44a7-9790-15a7cf8fe3cd None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:48.441257 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-d92b6079-5804-44a7-9790-15a7cf8fe3cd None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 14:24:48.441257 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:48.441257 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:48.441257 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:48.441257 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:48.441257 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:48.441257 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:48.441257 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:48.441257 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:48.441257 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:48.441257 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:48.441257 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:48.441257 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:24:48.441257 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:24:48.441257 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:24:48.441257 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:24:48.442408 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:48.442408 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:48.442408 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:24:48.442408 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:24:48.442408 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:24:48.442408 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:24:48.442408 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:48.442408 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:48.442408 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:48.442408 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:48.442408 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:48.442408 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:24:48.442408 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:48.442408 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:48.442408 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:48.442408 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:48.442408 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:48.442408 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:48.443467 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:48.443467 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:48.443467 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:24:48.443467 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:24:48.443467 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:24:48.443467 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:24:48.443467 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:24:48.443467 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:24:48.443467 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 14:24:48.443467 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:24:48.443467 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 370/737] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:24:48 2022] GET /identity/v3/users/swift => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:48.444293 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc2d135-f4d2-4ad8-8e1a-7b723776c5a2 None None] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:48.444293 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc2d135-f4d2-4ad8-8e1a-7b723776c5a2 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:48.444293 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc2d135-f4d2-4ad8-8e1a-7b723776c5a2 None None] PATH_INFO: `/` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:48.445386 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 371/738] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:24:48 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:24:48.446459 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697e2094-6cba-41b6-8f4e-e691cd6ab88d None None] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:48.447213 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697e2094-6cba-41b6-8f4e-e691cd6ab88d None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:48.447213 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697e2094-6cba-41b6-8f4e-e691cd6ab88d None None] PATH_INFO: `/` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:48.448191 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 372/739] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:24:48 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:24:48.449577 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697e2094-6cba-41b6-8f4e-e691cd6ab88d None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:48.472187 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa50ba2-a9c0-4b7e-818d-cbaf16467530 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:48.506982 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93680eda-df7f-4bd6-bbaf-283b29be424d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:48.507663 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93680eda-df7f-4bd6-bbaf-283b29be424d None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:48.507955 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93680eda-df7f-4bd6-bbaf-283b29be424d None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:48.508230 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93680eda-df7f-4bd6-bbaf-283b29be424d None admin] PATH_INFO: `/v3/roles` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:48.508976 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93680eda-df7f-4bd6-bbaf-283b29be424d None admin] RBAC: Adding query filter params (name=admin) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:48.509286 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93680eda-df7f-4bd6-bbaf-283b29be424d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:48.510694 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93680eda-df7f-4bd6-bbaf-283b29be424d None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:48.519510 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 368/740] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:24:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:48.520307 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93680eda-df7f-4bd6-bbaf-283b29be424d None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:48.561494 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697e2094-6cba-41b6-8f4e-e691cd6ab88d None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:48.571801 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:48.597466 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556a5477-7a1b-416a-a9fc-c2bd0becef76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:48.598076 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556a5477-7a1b-416a-a9fc-c2bd0becef76 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:48.598611 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556a5477-7a1b-416a-a9fc-c2bd0becef76 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:48.598887 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556a5477-7a1b-416a-a9fc-c2bd0becef76 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:48.610758 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:48.620594 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93680eda-df7f-4bd6-bbaf-283b29be424d None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:48.654638 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a0fa44-04f1-4bd1-94b4-80d80a98e6ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:48.655316 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a0fa44-04f1-4bd1-94b4-80d80a98e6ca None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:48.655601 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a0fa44-04f1-4bd1-94b4-80d80a98e6ca None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:48.655874 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a0fa44-04f1-4bd1-94b4-80d80a98e6ca None admin] PATH_INFO: `/v3/users` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:48.656721 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a0fa44-04f1-4bd1-94b4-80d80a98e6ca None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:48.657041 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a0fa44-04f1-4bd1-94b4-80d80a98e6ca None admin] RBAC: Authorizing `identity:list_users()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:48.658640 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a0fa44-04f1-4bd1-94b4-80d80a98e6ca None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:48.672172 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556a5477-7a1b-416a-a9fc-c2bd0becef76 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:48.673626 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556a5477-7a1b-416a-a9fc-c2bd0becef76 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:48.675965 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 369/741] 10.222.0.57 () {56 vars in 1135 bytes} [Thu May 19 14:24:48 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:48.677229 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a381c5ab-c9aa-4cb5-a4d5-0be17abf1b6f None None] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:48.677523 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a381c5ab-c9aa-4cb5-a4d5-0be17abf1b6f None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:48.677805 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a381c5ab-c9aa-4cb5-a4d5-0be17abf1b6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:48.694311 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.auth.core [None req-a381c5ab-c9aa-4cb5-a4d5-0be17abf1b6f None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143586) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:24:48.695506 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.fernet_utils [None req-a381c5ab-c9aa-4cb5-a4d5-0be17abf1b6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143586) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:24:48.762070 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 370/742] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:24:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:24:48.763281 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23864915-a2dc-49cd-8a58-600852faca1e None None] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:48.763570 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23864915-a2dc-49cd-8a58-600852faca1e None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:48.763848 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23864915-a2dc-49cd-8a58-600852faca1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:48.774504 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 373/743] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:24:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:24:48.775494 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556a5477-7a1b-416a-a9fc-c2bd0becef76 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:48.781732 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.auth.core [None req-23864915-a2dc-49cd-8a58-600852faca1e None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143586) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:24:48.782939 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.fernet_utils [None req-23864915-a2dc-49cd-8a58-600852faca1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143586) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:24:48.818071 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:48.819791 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:48.852667 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 371/744] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:24:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:24:48.853437 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23864915-a2dc-49cd-8a58-600852faca1e None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:48.879446 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556a5477-7a1b-416a-a9fc-c2bd0becef76 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:48.914796 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998f3f5d-0216-4cd7-9dc4-c43a9b62665c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:48.915589 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998f3f5d-0216-4cd7-9dc4-c43a9b62665c None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:48.915873 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998f3f5d-0216-4cd7-9dc4-c43a9b62665c None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:48.916149 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998f3f5d-0216-4cd7-9dc4-c43a9b62665c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:48.927684 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998f3f5d-0216-4cd7-9dc4-c43a9b62665c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:48.929128 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998f3f5d-0216-4cd7-9dc4-c43a9b62665c None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:48.947774 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-998f3f5d-0216-4cd7-9dc4-c43a9b62665c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:48.947774 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:48.947774 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:48.947774 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:48.947774 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:48.947774 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:48.947774 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:48.947774 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:48.947774 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:48.947774 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:48.947774 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:48.947774 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:48.947774 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:24:48.947774 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:24:48.947774 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:24:48.949437 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:24:48.949437 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:48.949437 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:48.949437 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:48.949437 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:48.949437 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:48.949437 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:48.949437 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:48.949437 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:24:48.949437 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:48.949437 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:48.949437 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:48.949437 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:48.949437 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:48.949437 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:48.949437 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:48.949437 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:48.951217 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:24:48.951217 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:24:48.951217 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:48.951217 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:24:48.951217 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 374/745] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:24:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:48.951217 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998f3f5d-0216-4cd7-9dc4-c43a9b62665c None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:48.967406 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23864915-a2dc-49cd-8a58-600852faca1e None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:49.005703 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691f3d92-39d0-403c-afa2-a21c855786a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:49.006378 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691f3d92-39d0-403c-afa2-a21c855786a3 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:49.006660 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691f3d92-39d0-403c-afa2-a21c855786a3 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:49.006934 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691f3d92-39d0-403c-afa2-a21c855786a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:49.043401 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:49.047438 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:49.050447 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998f3f5d-0216-4cd7-9dc4-c43a9b62665c None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:49.077920 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691f3d92-39d0-403c-afa2-a21c855786a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:49.079307 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691f3d92-39d0-403c-afa2-a21c855786a3 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:49.084557 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae0c839-8270-4550-8822-f888e58bf467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:49.085345 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae0c839-8270-4550-8822-f888e58bf467 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:49.085624 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae0c839-8270-4550-8822-f888e58bf467 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:49.085941 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae0c839-8270-4550-8822-f888e58bf467 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:49.096769 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae0c839-8270-4550-8822-f888e58bf467 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:49.098494 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae0c839-8270-4550-8822-f888e58bf467 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:49.117020 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-dae0c839-8270-4550-8822-f888e58bf467 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:49.117020 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:49.117020 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:49.117020 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:49.117020 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:49.117020 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:49.117020 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:49.117020 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:49.117020 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:49.117020 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:49.117020 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:49.117020 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:49.117020 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:24:49.117020 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:24:49.117020 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:24:49.118191 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:24:49.118191 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:49.118191 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:49.118191 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:49.118191 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:49.118191 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:49.118191 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:49.118191 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:49.118191 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:24:49.118191 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:49.118191 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:49.118191 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:49.118191 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:49.118191 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:49.118191 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:49.118191 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:49.118191 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:49.119239 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:24:49.119239 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:24:49.119239 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:49.119239 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:24:49.119239 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 375/746] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:24:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:49.119554 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae0c839-8270-4550-8822-f888e58bf467 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:49.179176 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 372/747] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:24:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:24:49.180003 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691f3d92-39d0-403c-afa2-a21c855786a3 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:49.219508 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae0c839-8270-4550-8822-f888e58bf467 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:49.248794 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:49.255514 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b338ca9c-975e-415e-9f53-2199461f6805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:49.256178 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b338ca9c-975e-415e-9f53-2199461f6805 None admin] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:49.256462 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b338ca9c-975e-415e-9f53-2199461f6805 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:49.256741 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b338ca9c-975e-415e-9f53-2199461f6805 None admin] PATH_INFO: `/v3/services` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:49.257465 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b338ca9c-975e-415e-9f53-2199461f6805 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:49.258871 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b338ca9c-975e-415e-9f53-2199461f6805 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:49.277688 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 376/748] 10.222.0.57 () {60 vars in 1141 bytes} [Thu May 19 14:24:49 2022] POST /identity/v3/services => generated 240 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:24:49.278450 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b338ca9c-975e-415e-9f53-2199461f6805 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:49.281727 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691f3d92-39d0-403c-afa2-a21c855786a3 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:49.288101 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:49.335426 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ebd3e8-e2d1-45dd-b639-3701fce71ef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:49.336114 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ebd3e8-e2d1-45dd-b639-3701fce71ef9 None admin] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:49.336404 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ebd3e8-e2d1-45dd-b639-3701fce71ef9 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:49.336726 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ebd3e8-e2d1-45dd-b639-3701fce71ef9 None admin] PATH_INFO: `/v3/services` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:49.337474 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ebd3e8-e2d1-45dd-b639-3701fce71ef9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:49.338914 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ebd3e8-e2d1-45dd-b639-3701fce71ef9 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:49.357275 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 373/749] 10.222.0.57 () {60 vars in 1141 bytes} [Thu May 19 14:24:49 2022] POST /identity/v3/services => generated 251 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:24:49.358232 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ebd3e8-e2d1-45dd-b639-3701fce71ef9 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:49.415104 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b338ca9c-975e-415e-9f53-2199461f6805 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:49.487834 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c8496b-b77b-4f05-8e1b-3522d02c8034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:49.488623 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c8496b-b77b-4f05-8e1b-3522d02c8034 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:49.489217 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c8496b-b77b-4f05-8e1b-3522d02c8034 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:49.489505 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c8496b-b77b-4f05-8e1b-3522d02c8034 None admin] PATH_INFO: `/v3/domains` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:49.490473 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c8496b-b77b-4f05-8e1b-3522d02c8034 None admin] RBAC: Adding query filter params (name=Default) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:49.490809 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c8496b-b77b-4f05-8e1b-3522d02c8034 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:49.493130 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c8496b-b77b-4f05-8e1b-3522d02c8034 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:49.504538 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ebd3e8-e2d1-45dd-b639-3701fce71ef9 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:49.514292 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 377/750] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:24:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:49.515431 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c8496b-b77b-4f05-8e1b-3522d02c8034 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:49.543332 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:49.555874 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b419202-1c6c-4473-a913-c89ccaba9fd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:49.556562 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b419202-1c6c-4473-a913-c89ccaba9fd1 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:49.556850 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b419202-1c6c-4473-a913-c89ccaba9fd1 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:49.557126 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b419202-1c6c-4473-a913-c89ccaba9fd1 None admin] PATH_INFO: `/v3/domains` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:49.558446 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b419202-1c6c-4473-a913-c89ccaba9fd1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:49.558754 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b419202-1c6c-4473-a913-c89ccaba9fd1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:49.560163 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b419202-1c6c-4473-a913-c89ccaba9fd1 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:49.582951 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 374/751] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:24:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:49.582951 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b419202-1c6c-4473-a913-c89ccaba9fd1 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:49.609582 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:49.670140 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c8496b-b77b-4f05-8e1b-3522d02c8034 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:49.709663 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d2d364-0693-4bb9-b6ee-9c5e9888dd3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:49.710271 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b419202-1c6c-4473-a913-c89ccaba9fd1 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:49.711092 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d2d364-0693-4bb9-b6ee-9c5e9888dd3a None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:49.711623 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d2d364-0693-4bb9-b6ee-9c5e9888dd3a None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:49.712289 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d2d364-0693-4bb9-b6ee-9c5e9888dd3a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:49.726074 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d2d364-0693-4bb9-b6ee-9c5e9888dd3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:49.728820 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d2d364-0693-4bb9-b6ee-9c5e9888dd3a None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:49.753289 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11faa27-ec66-4cae-9bc9-9ac1ab397b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:49.753712 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-31d2d364-0693-4bb9-b6ee-9c5e9888dd3a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:49.753712 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:49.753712 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:49.753712 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:49.753712 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:49.753712 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:49.753712 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:49.753712 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:49.753712 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:49.753712 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:49.753712 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:49.753712 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:49.753712 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:24:49.753712 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:24:49.753712 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:24:49.754701 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:24:49.754701 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:49.754701 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:49.754701 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:49.754701 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:49.754701 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:49.754701 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:49.754701 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:49.754701 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:24:49.754701 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:49.754701 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:49.754701 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:49.754701 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:49.754701 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:49.754701 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:49.754701 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:49.754701 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:49.755685 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:24:49.755685 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:24:49.755685 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:49.755685 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:24:49.756025 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11faa27-ec66-4cae-9bc9-9ac1ab397b1c None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:49.756025 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11faa27-ec66-4cae-9bc9-9ac1ab397b1c None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:49.756025 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11faa27-ec66-4cae-9bc9-9ac1ab397b1c None admin] PATH_INFO: `/v3/users/nova` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:49.758167 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 378/752] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:24:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:49.760198 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d2d364-0693-4bb9-b6ee-9c5e9888dd3a None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:49.766070 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11faa27-ec66-4cae-9bc9-9ac1ab397b1c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:49.767648 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11faa27-ec66-4cae-9bc9-9ac1ab397b1c None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:49.783070 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-d11faa27-ec66-4cae-9bc9-9ac1ab397b1c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 14:24:49.783070 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:49.783070 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:49.783070 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:49.783070 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:49.783070 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:49.783070 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:49.783070 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:49.783070 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:49.783070 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:49.783070 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:49.783070 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:49.783070 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:24:49.783070 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:24:49.783070 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:24:49.783070 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:24:49.784152 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:49.784152 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:49.784152 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:24:49.784152 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:24:49.784152 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:24:49.784152 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:24:49.784152 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:49.784152 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:49.784152 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:49.784152 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:49.784152 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:49.784152 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:24:49.784152 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:49.784152 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:49.784152 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:49.784152 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:49.784152 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:49.784152 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:49.785225 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:49.785225 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:49.785225 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:24:49.785225 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:24:49.785225 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:24:49.785225 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:24:49.785225 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:24:49.785225 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:24:49.785225 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 14:24:49.785225 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:24:49.789066 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 375/753] 10.222.0.57 () {56 vars in 1093 bytes} [Thu May 19 14:24:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:49.790007 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11faa27-ec66-4cae-9bc9-9ac1ab397b1c None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:49.846495 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:49.882852 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d2d364-0693-4bb9-b6ee-9c5e9888dd3a None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:49.912821 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11faa27-ec66-4cae-9bc9-9ac1ab397b1c None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:49.914221 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:49.922912 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d055cc-6832-451f-86b6-e713947419a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:49.923831 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d055cc-6832-451f-86b6-e713947419a8 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:49.924110 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d055cc-6832-451f-86b6-e713947419a8 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:49.924382 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d055cc-6832-451f-86b6-e713947419a8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:49.932200 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d055cc-6832-451f-86b6-e713947419a8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:49.934794 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d055cc-6832-451f-86b6-e713947419a8 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:49.951392 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-f6d055cc-6832-451f-86b6-e713947419a8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:24:49.951392 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:49.951392 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:49.951392 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:49.951392 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:49.951392 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:49.951392 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:49.951392 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:49.951392 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:49.951392 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:49.951392 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:49.951392 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:49.951392 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:24:49.951392 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:24:49.951392 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:24:49.952432 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:24:49.952432 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:49.952432 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:49.952432 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:49.952432 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:49.952432 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:49.952432 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:49.952432 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:49.952432 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:24:49.952432 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:49.952432 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:49.952432 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:49.952432 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:49.952432 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:49.952432 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:49.952432 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:49.952432 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:49.953661 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:24:49.953661 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:24:49.953661 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:24:49.953661 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:24:49.953661 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:24:49.953661 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:24:49.953661 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:24:49.953661 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:24:49.962480 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 379/754] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:24:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:49.962480 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d055cc-6832-451f-86b6-e713947419a8 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:49.973431 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d389a57a-e43d-4a1d-be41-65393b06bd11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:49.974479 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d389a57a-e43d-4a1d-be41-65393b06bd11 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:49.975014 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d389a57a-e43d-4a1d-be41-65393b06bd11 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:49.975534 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d389a57a-e43d-4a1d-be41-65393b06bd11 None admin] PATH_INFO: `/v3/domains` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:49.976485 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d389a57a-e43d-4a1d-be41-65393b06bd11 None admin] RBAC: Adding query filter params (name=Default) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:49.977014 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d389a57a-e43d-4a1d-be41-65393b06bd11 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:49.978653 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d389a57a-e43d-4a1d-be41-65393b06bd11 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:49.992560 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 376/755] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:24:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:49.993875 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d389a57a-e43d-4a1d-be41-65393b06bd11 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:50.071152 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d055cc-6832-451f-86b6-e713947419a8 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:50.101306 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d389a57a-e43d-4a1d-be41-65393b06bd11 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:50.108467 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef7ba17-6d3c-45e7-ad6a-10cd2d862252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:50.109147 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef7ba17-6d3c-45e7-ad6a-10cd2d862252 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:50.109428 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef7ba17-6d3c-45e7-ad6a-10cd2d862252 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:50.109705 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef7ba17-6d3c-45e7-ad6a-10cd2d862252 None admin] PATH_INFO: `/v3/users` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:50.110573 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef7ba17-6d3c-45e7-ad6a-10cd2d862252 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:50.110882 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef7ba17-6d3c-45e7-ad6a-10cd2d862252 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:50.112022 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:50.113481 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef7ba17-6d3c-45e7-ad6a-10cd2d862252 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:50.134498 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 380/756] 10.222.0.57 () {56 vars in 1133 bytes} [Thu May 19 14:24:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:50.135695 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef7ba17-6d3c-45e7-ad6a-10cd2d862252 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:50.147113 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea2d2db-f3e5-411e-9251-d296ad1dbf74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:50.148155 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea2d2db-f3e5-411e-9251-d296ad1dbf74 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:50.148830 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea2d2db-f3e5-411e-9251-d296ad1dbf74 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:50.149363 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea2d2db-f3e5-411e-9251-d296ad1dbf74 None admin] PATH_INFO: `/v3/projects` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:50.150423 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea2d2db-f3e5-411e-9251-d296ad1dbf74 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:50.150963 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea2d2db-f3e5-411e-9251-d296ad1dbf74 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:50.153393 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea2d2db-f3e5-411e-9251-d296ad1dbf74 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:50.166262 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 377/757] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:24:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:50.168313 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea2d2db-f3e5-411e-9251-d296ad1dbf74 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:50.182308 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:50.255475 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef7ba17-6d3c-45e7-ad6a-10cd2d862252 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:50.277794 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea2d2db-f3e5-411e-9251-d296ad1dbf74 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:50.294826 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024667b8-2f7e-46a0-a9b8-be8255deb6b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:50.295651 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024667b8-2f7e-46a0-a9b8-be8255deb6b6 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:50.296084 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024667b8-2f7e-46a0-a9b8-be8255deb6b6 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:50.296294 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024667b8-2f7e-46a0-a9b8-be8255deb6b6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:50.303913 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024667b8-2f7e-46a0-a9b8-be8255deb6b6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:50.305281 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024667b8-2f7e-46a0-a9b8-be8255deb6b6 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:50.316209 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9b59e4-6579-4a4a-a1d9-1def5eeff4b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:50.317333 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9b59e4-6579-4a4a-a1d9-1def5eeff4b5 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:50.317892 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9b59e4-6579-4a4a-a1d9-1def5eeff4b5 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:50.318484 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9b59e4-6579-4a4a-a1d9-1def5eeff4b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:50.321268 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-024667b8-2f7e-46a0-a9b8-be8255deb6b6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:24:50.321268 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:50.321268 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:50.321268 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:50.321268 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:50.321268 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:50.321268 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:50.321268 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:50.321268 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:50.321268 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:50.321268 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:50.321268 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:50.321268 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:24:50.321268 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:24:50.321268 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:24:50.322271 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:24:50.322271 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:50.322271 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:50.322271 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:50.322271 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:50.322271 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:50.322271 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:50.322271 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:50.322271 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:24:50.322271 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:50.322271 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:50.322271 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:50.322271 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:50.322271 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:50.322271 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:50.322271 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:50.322271 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:50.324636 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:24:50.324636 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:24:50.324636 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:24:50.324636 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:24:50.324636 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:24:50.324636 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:24:50.324636 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:24:50.324636 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:24:50.324636 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 381/758] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:24:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:50.324636 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024667b8-2f7e-46a0-a9b8-be8255deb6b6 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:50.335201 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9b59e4-6579-4a4a-a1d9-1def5eeff4b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:50.337002 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9b59e4-6579-4a4a-a1d9-1def5eeff4b5 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:50.357971 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-7c9b59e4-6579-4a4a-a1d9-1def5eeff4b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:50.357971 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:50.357971 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:50.357971 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:50.357971 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:50.357971 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:50.357971 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:50.357971 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:50.357971 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:50.357971 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:50.357971 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:50.357971 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:50.357971 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:24:50.357971 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:24:50.357971 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:24:50.359001 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:24:50.359001 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:50.359001 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:50.359001 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:50.359001 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:50.359001 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:50.359001 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:50.359001 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:50.359001 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:24:50.359001 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:50.359001 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:50.359001 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:50.359001 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:50.359001 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:50.359001 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:50.359001 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:50.359001 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:50.360039 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:24:50.360039 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:24:50.360039 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:50.360039 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:24:50.367501 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 378/759] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:24:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:50.368602 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9b59e4-6579-4a4a-a1d9-1def5eeff4b5 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:50.389214 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:50.431063 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:50.440274 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024667b8-2f7e-46a0-a9b8-be8255deb6b6 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:50.485898 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dd8c88-690e-463f-a998-cf89bb6e8d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:50.486978 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dd8c88-690e-463f-a998-cf89bb6e8d66 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:50.487264 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dd8c88-690e-463f-a998-cf89bb6e8d66 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:50.487543 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dd8c88-690e-463f-a998-cf89bb6e8d66 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:50.488664 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dd8c88-690e-463f-a998-cf89bb6e8d66 None admin] RBAC: Adding query filter params (role.id=b0b74fefb27e4a0ebd895893aa9862d7, scope.project.id=252d9673f08c479a98d04fee3f5856a4, user.id=7977ff2f29eb435cbfeaa0b61d1d7a5d) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:50.488979 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dd8c88-690e-463f-a998-cf89bb6e8d66 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:50.491130 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dd8c88-690e-463f-a998-cf89bb6e8d66 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:50.497569 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9b59e4-6579-4a4a-a1d9-1def5eeff4b5 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:50.504383 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 382/760] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:24:50 2022] GET /identity/v3/role_assignments?user.id=7977ff2f29eb435cbfeaa0b61d1d7a5d&scope.project.id=252d9673f08c479a98d04fee3f5856a4&role.id=b0b74fefb27e4a0ebd895893aa9862d7 => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:50.504383 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dd8c88-690e-463f-a998-cf89bb6e8d66 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:50.568585 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835333e6-d31c-4863-8875-cf022de92a57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:50.569264 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835333e6-d31c-4863-8875-cf022de92a57 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:50.569549 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835333e6-d31c-4863-8875-cf022de92a57 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:50.569834 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835333e6-d31c-4863-8875-cf022de92a57 None admin] PATH_INFO: `/v3/projects` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:50.570666 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835333e6-d31c-4863-8875-cf022de92a57 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:50.570978 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835333e6-d31c-4863-8875-cf022de92a57 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:50.572440 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835333e6-d31c-4863-8875-cf022de92a57 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:50.587984 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 379/761] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:24:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:50.597157 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835333e6-d31c-4863-8875-cf022de92a57 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:50.731077 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:50.741877 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dd8c88-690e-463f-a998-cf89bb6e8d66 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:50.765261 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835333e6-d31c-4863-8875-cf022de92a57 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:50.795676 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd0b96b-7a30-4039-a442-20f9a05c26ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:50.796473 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd0b96b-7a30-4039-a442-20f9a05c26ca None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:50.796759 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd0b96b-7a30-4039-a442-20f9a05c26ca None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:50.797153 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd0b96b-7a30-4039-a442-20f9a05c26ca None admin] PATH_INFO: `/v3/domains` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:50.797930 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd0b96b-7a30-4039-a442-20f9a05c26ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:50.798242 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd0b96b-7a30-4039-a442-20f9a05c26ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:50.799745 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd0b96b-7a30-4039-a442-20f9a05c26ca None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:50.815698 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50193161-cf51-40dd-aa58-1a65908d62c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:50.816370 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50193161-cf51-40dd-aa58-1a65908d62c4 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:50.816647 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50193161-cf51-40dd-aa58-1a65908d62c4 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:50.816917 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50193161-cf51-40dd-aa58-1a65908d62c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:50.817752 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50193161-cf51-40dd-aa58-1a65908d62c4 None admin] RBAC: Adding query filter params (role.id=a3b793754e7b40b0bb21afe74eec3c18, scope.project.id=252d9673f08c479a98d04fee3f5856a4, user.id=d3e544b4989e49a19536b40de5069bcf) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:50.818082 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50193161-cf51-40dd-aa58-1a65908d62c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:50.821738 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50193161-cf51-40dd-aa58-1a65908d62c4 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:50.827773 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 383/762] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:24:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:50.834820 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd0b96b-7a30-4039-a442-20f9a05c26ca None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:50.837362 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 380/763] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:24:50 2022] GET /identity/v3/role_assignments?user.id=d3e544b4989e49a19536b40de5069bcf&scope.project.id=252d9673f08c479a98d04fee3f5856a4&role.id=a3b793754e7b40b0bb21afe74eec3c18 => generated 259 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:24:50.856013 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:50.961189 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd0b96b-7a30-4039-a442-20f9a05c26ca None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:51.000618 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7207112e-f9a9-421e-821e-9a45074625f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:51.001338 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7207112e-f9a9-421e-821e-9a45074625f3 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:51.001623 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7207112e-f9a9-421e-821e-9a45074625f3 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:51.001922 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7207112e-f9a9-421e-821e-9a45074625f3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:51.009391 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7207112e-f9a9-421e-821e-9a45074625f3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:51.010927 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7207112e-f9a9-421e-821e-9a45074625f3 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:51.025950 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-7207112e-f9a9-421e-821e-9a45074625f3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:24:51.025950 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:51.025950 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:51.025950 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:51.025950 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:51.025950 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:51.025950 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:51.025950 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:51.025950 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:51.025950 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:51.025950 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:51.025950 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:51.025950 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:24:51.025950 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:24:51.025950 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:24:51.027569 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:24:51.027569 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:51.027569 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:51.027569 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:51.027569 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:51.027569 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:51.027569 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:51.027569 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:51.027569 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:24:51.027569 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:51.027569 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:51.027569 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:51.027569 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:51.027569 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:51.027569 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:51.027569 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:51.027569 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:51.028781 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:24:51.028781 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:24:51.028781 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:24:51.028781 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:24:51.028781 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:24:51.028781 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:24:51.028781 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:24:51.028781 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:24:51.033008 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 384/764] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:24:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:51.034515 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50193161-cf51-40dd-aa58-1a65908d62c4 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:51.158315 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50193161-cf51-40dd-aa58-1a65908d62c4 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:51.196297 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471f310e-4c42-4eb8-aaf2-42fac22cd4b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:51.196963 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471f310e-4c42-4eb8-aaf2-42fac22cd4b3 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:51.197247 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471f310e-4c42-4eb8-aaf2-42fac22cd4b3 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:51.197524 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471f310e-4c42-4eb8-aaf2-42fac22cd4b3 None admin] PATH_INFO: `/v3/projects` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:51.198404 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471f310e-4c42-4eb8-aaf2-42fac22cd4b3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:51.199364 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471f310e-4c42-4eb8-aaf2-42fac22cd4b3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:51.201001 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471f310e-4c42-4eb8-aaf2-42fac22cd4b3 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:51.217961 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 381/765] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:24:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:51.219772 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7207112e-f9a9-421e-821e-9a45074625f3 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:51.312244 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:51.336482 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7207112e-f9a9-421e-821e-9a45074625f3 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:51.373697 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14aaf6e-3d4c-4f4a-8688-9052016acd22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:51.374626 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14aaf6e-3d4c-4f4a-8688-9052016acd22 None admin] REQUEST_METHOD: `PUT` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:51.374962 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14aaf6e-3d4c-4f4a-8688-9052016acd22 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:51.375245 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14aaf6e-3d4c-4f4a-8688-9052016acd22 None admin] PATH_INFO: `/v3/projects/252d9673f08c479a98d04fee3f5856a4/users/5f28de22c2ee4c53b6d14d5b0ddf7fe9/roles/a3b793754e7b40b0bb21afe74eec3c18` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:51.377254 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14aaf6e-3d4c-4f4a-8688-9052016acd22 None admin] RBAC: Authorizing `identity:create_grant(project_id=252d9673f08c479a98d04fee3f5856a4, user_id=5f28de22c2ee4c53b6d14d5b0ddf7fe9, role_id=a3b793754e7b40b0bb21afe74eec3c18)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:51.378729 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14aaf6e-3d4c-4f4a-8688-9052016acd22 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:51.401613 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 385/766] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 14:24:51 2022] PUT /identity/v3/projects/252d9673f08c479a98d04fee3f5856a4/users/5f28de22c2ee4c53b6d14d5b0ddf7fe9/roles/a3b793754e7b40b0bb21afe74eec3c18 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:24:51.834967 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f87d4a-1eb7-44fe-bd60-d54046be3fb7 None None] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:51.834967 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f87d4a-1eb7-44fe-bd60-d54046be3fb7 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:51.834967 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f87d4a-1eb7-44fe-bd60-d54046be3fb7 None None] PATH_INFO: `/` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:51.835815 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 382/767] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:24:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:24:51.842477 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989ed760-0e7a-4f1a-87ca-8e89938faa7a None None] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:51.842775 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989ed760-0e7a-4f1a-87ca-8e89938faa7a None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:51.843062 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989ed760-0e7a-4f1a-87ca-8e89938faa7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:51.866516 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.auth.core [None req-989ed760-0e7a-4f1a-87ca-8e89938faa7a None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143587) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:24:51.867700 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-989ed760-0e7a-4f1a-87ca-8e89938faa7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:24:51.928852 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06734031-6b3c-483b-b11a-6a886f072ccf None None] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:51.929053 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06734031-6b3c-483b-b11a-6a886f072ccf None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:51.929328 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06734031-6b3c-483b-b11a-6a886f072ccf None None] PATH_INFO: `/` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:51.931007 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 383/768] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:24:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:24:51.936872 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db5ac20-fac0-41ad-911f-1e92bac46abb None None] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:51.937530 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db5ac20-fac0-41ad-911f-1e92bac46abb None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:51.938063 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db5ac20-fac0-41ad-911f-1e92bac46abb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:51.950194 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:51.964479 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 386/769] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:24:51 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:24:51.977559 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.auth.core [None req-8db5ac20-fac0-41ad-911f-1e92bac46abb None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143586) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:24:51.978823 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.fernet_utils [None req-8db5ac20-fac0-41ad-911f-1e92bac46abb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143586) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:24:51.991653 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989ed760-0e7a-4f1a-87ca-8e89938faa7a None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:52.101944 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:52.117626 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 384/770] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:24:51 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:24:52.147249 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db5ac20-fac0-41ad-911f-1e92bac46abb None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:52.149552 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989ed760-0e7a-4f1a-87ca-8e89938faa7a None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:52.191082 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03868e21-d8e2-48df-81c9-a446e1010e15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:52.191801 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03868e21-d8e2-48df-81c9-a446e1010e15 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:52.192085 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03868e21-d8e2-48df-81c9-a446e1010e15 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:52.192362 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03868e21-d8e2-48df-81c9-a446e1010e15 None admin] PATH_INFO: `/v3/services/network` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:52.193024 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03868e21-d8e2-48df-81c9-a446e1010e15 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:52.194444 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03868e21-d8e2-48df-81c9-a446e1010e15 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:52.219976 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-03868e21-d8e2-48df-81c9-a446e1010e15 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 19 14:24:52.219976 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:52.219976 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:24:52.219976 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:24:52.219976 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:24:52.219976 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:24:52.219976 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:24:52.219976 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:24:52.219976 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 14:24:52.219976 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:24:52.219976 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:24:52.219976 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:24:52.219976 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:52.219976 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:52.219976 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:52.219976 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:52.219976 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:52.221133 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:52.221133 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:52.221133 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:52.221133 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:52.221133 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:52.221133 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:52.221133 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:24:52.221133 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:24:52.221133 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:24:52.221133 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:24:52.221133 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:52.221133 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:52.221133 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:52.221133 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:52.221133 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:52.221133 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:52.222116 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:52.222116 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:24:52.222116 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:52.222116 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:52.222116 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:52.222116 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:52.222116 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:52.222116 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:52.222116 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:52.222116 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:52.222116 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:24:52.222116 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:24:52.222116 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 14:24:52.222116 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:24:52.223774 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 387/771] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:24:51 2022] GET /identity/v3/services/network => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:24:52.231501 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03868e21-d8e2-48df-81c9-a446e1010e15 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:52.300843 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db5ac20-fac0-41ad-911f-1e92bac46abb None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:52.332984 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:52.339247 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fe9e9e-74d7-4d53-a698-6cd411744134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:52.340387 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fe9e9e-74d7-4d53-a698-6cd411744134 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:52.340898 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fe9e9e-74d7-4d53-a698-6cd411744134 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:52.341381 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fe9e9e-74d7-4d53-a698-6cd411744134 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:52.342301 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fe9e9e-74d7-4d53-a698-6cd411744134 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:52.344114 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fe9e9e-74d7-4d53-a698-6cd411744134 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:52.358658 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-78fe9e9e-74d7-4d53-a698-6cd411744134 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 14:24:52.358658 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:52.358658 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:24:52.358658 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:24:52.358658 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:24:52.358658 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:24:52.358658 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:24:52.358658 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:24:52.358658 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 14:24:52.358658 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:24:52.358658 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:24:52.358658 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:24:52.358658 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:52.358658 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:52.358658 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:52.358658 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:52.359783 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:52.359783 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:52.359783 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:52.359783 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:52.359783 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:52.359783 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:52.359783 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:52.359783 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:24:52.359783 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:24:52.359783 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:24:52.359783 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:24:52.359783 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:52.359783 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:52.359783 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:52.359783 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:52.359783 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:52.360806 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:52.360806 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:52.360806 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:24:52.360806 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:52.360806 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:52.360806 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:52.360806 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:52.360806 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:52.360806 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:52.360806 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:52.360806 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:52.360806 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:24:52.360806 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:24:52.360806 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 14:24:52.360806 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:24:52.363560 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 385/772] 10.222.0.57 () {56 vars in 1129 bytes} [Thu May 19 14:24:52 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:52.369469 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fe9e9e-74d7-4d53-a698-6cd411744134 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:52.460748 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03868e21-d8e2-48df-81c9-a446e1010e15 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:52.481015 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fe9e9e-74d7-4d53-a698-6cd411744134 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:52.519874 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:52.521120 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f8c7db-f69c-4cea-81db-0fc9f631d195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:52.522012 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f8c7db-f69c-4cea-81db-0fc9f631d195 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:52.522609 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f8c7db-f69c-4cea-81db-0fc9f631d195 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:52.523069 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f8c7db-f69c-4cea-81db-0fc9f631d195 None admin] PATH_INFO: `/v3/services` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:52.524057 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f8c7db-f69c-4cea-81db-0fc9f631d195 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:52.524526 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f8c7db-f69c-4cea-81db-0fc9f631d195 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:52.526104 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f8c7db-f69c-4cea-81db-0fc9f631d195 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:52.532996 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 386/773] 10.222.0.57 () {56 vars in 1124 bytes} [Thu May 19 14:24:52 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:24:52.537450 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78afc17-638a-4612-a389-c267db13836f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:52.538372 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78afc17-638a-4612-a389-c267db13836f None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:52.538656 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78afc17-638a-4612-a389-c267db13836f None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:52.538932 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78afc17-638a-4612-a389-c267db13836f None admin] PATH_INFO: `/v3/services` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:52.539646 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78afc17-638a-4612-a389-c267db13836f None admin] RBAC: Adding query filter params (name=network) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:52.539928 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f8c7db-f69c-4cea-81db-0fc9f631d195 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:52.540060 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78afc17-638a-4612-a389-c267db13836f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:52.541416 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78afc17-638a-4612-a389-c267db13836f None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:52.547786 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 388/774] 10.222.0.57 () {56 vars in 1112 bytes} [Thu May 19 14:24:52 2022] GET /identity/v3/services?name=network => generated 124 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:52.555570 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78afc17-638a-4612-a389-c267db13836f None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:52.662222 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:52.741963 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f8c7db-f69c-4cea-81db-0fc9f631d195 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:52.754156 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78afc17-638a-4612-a389-c267db13836f None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:52.812296 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6de8dd7-9cae-40d3-a9dd-df4aeb3f30fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:52.813614 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6de8dd7-9cae-40d3-a9dd-df4aeb3f30fd None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:52.814576 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6de8dd7-9cae-40d3-a9dd-df4aeb3f30fd None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:52.815494 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6de8dd7-9cae-40d3-a9dd-df4aeb3f30fd None admin] PATH_INFO: `/v3/services` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:52.816815 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6de8dd7-9cae-40d3-a9dd-df4aeb3f30fd None admin] RBAC: Adding query filter params (type=network) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:52.818170 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6de8dd7-9cae-40d3-a9dd-df4aeb3f30fd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:52.818923 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39494b2c-2072-4503-a6b6-3395c82f3cde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:52.820499 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6de8dd7-9cae-40d3-a9dd-df4aeb3f30fd None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:52.821327 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39494b2c-2072-4503-a6b6-3395c82f3cde None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:52.821764 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39494b2c-2072-4503-a6b6-3395c82f3cde None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:52.824055 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39494b2c-2072-4503-a6b6-3395c82f3cde None admin] PATH_INFO: `/v3/services` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:52.824961 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39494b2c-2072-4503-a6b6-3395c82f3cde None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:52.825450 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39494b2c-2072-4503-a6b6-3395c82f3cde None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:52.828104 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39494b2c-2072-4503-a6b6-3395c82f3cde None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:52.832450 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 389/775] 10.222.0.57 () {56 vars in 1112 bytes} [Thu May 19 14:24:52 2022] GET /identity/v3/services?type=network => generated 350 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:52.837288 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 387/776] 10.222.0.57 () {56 vars in 1124 bytes} [Thu May 19 14:24:52 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:52.846936 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39494b2c-2072-4503-a6b6-3395c82f3cde None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:52.847135 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6de8dd7-9cae-40d3-a9dd-df4aeb3f30fd None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:52.960084 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:53.043693 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6de8dd7-9cae-40d3-a9dd-df4aeb3f30fd None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:53.047674 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39494b2c-2072-4503-a6b6-3395c82f3cde None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:53.096929 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:53.112703 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0ead9f-2e12-434e-8a45-a32b47230ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:53.113362 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0ead9f-2e12-434e-8a45-a32b47230ca8 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:53.113644 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0ead9f-2e12-434e-8a45-a32b47230ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:53.113945 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0ead9f-2e12-434e-8a45-a32b47230ca8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:53.115832 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0ead9f-2e12-434e-8a45-a32b47230ca8 None admin] RBAC: Adding query filter params (interface=public, service_id=8d8d1f1204504f9cb77252a7d9a88d75, region_id=RegionOne) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:53.116158 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0ead9f-2e12-434e-8a45-a32b47230ca8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:53.117538 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0ead9f-2e12-434e-8a45-a32b47230ca8 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:53.120935 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904d6da4-92e3-420f-bc27-0f44883de910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:53.121599 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904d6da4-92e3-420f-bc27-0f44883de910 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:53.121901 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904d6da4-92e3-420f-bc27-0f44883de910 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:53.122181 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904d6da4-92e3-420f-bc27-0f44883de910 None admin] PATH_INFO: `/v3/endpoints` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:53.122973 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904d6da4-92e3-420f-bc27-0f44883de910 None admin] RBAC: Adding query filter params (interface=public, service_id=56a5d5d589ca434f9a83aab5b81bcda0, region_id=RegionOne) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:53.123281 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904d6da4-92e3-420f-bc27-0f44883de910 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:53.124628 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904d6da4-92e3-420f-bc27-0f44883de910 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:53.133493 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 388/777] 10.222.0.57 () {56 vars in 1251 bytes} [Thu May 19 14:24:52 2022] GET /identity/v3/endpoints?service_id=56a5d5d589ca434f9a83aab5b81bcda0&interface=public®ion_id=RegionOne => generated 194 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:53.139223 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904d6da4-92e3-420f-bc27-0f44883de910 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:53.139977 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 390/778] 10.222.0.57 () {56 vars in 1251 bytes} [Thu May 19 14:24:52 2022] GET /identity/v3/endpoints?service_id=8d8d1f1204504f9cb77252a7d9a88d75&interface=public®ion_id=RegionOne => generated 194 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:53.148425 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0ead9f-2e12-434e-8a45-a32b47230ca8 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:53.346076 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904d6da4-92e3-420f-bc27-0f44883de910 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:53.350205 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0ead9f-2e12-434e-8a45-a32b47230ca8 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:53.391458 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:53.399748 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5c887a-5f42-4a7e-813d-2b5f30485417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:53.400965 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5c887a-5f42-4a7e-813d-2b5f30485417 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:53.401585 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5c887a-5f42-4a7e-813d-2b5f30485417 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:53.402656 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5c887a-5f42-4a7e-813d-2b5f30485417 None admin] PATH_INFO: `/v3/services` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:53.404090 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5c887a-5f42-4a7e-813d-2b5f30485417 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:53.404235 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0f571e-2dbd-4223-b424-6d5e2192a1ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:53.404852 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0f571e-2dbd-4223-b424-6d5e2192a1ac None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:53.405418 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0f571e-2dbd-4223-b424-6d5e2192a1ac None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:53.406142 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0f571e-2dbd-4223-b424-6d5e2192a1ac None admin] PATH_INFO: `/v3/services` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:53.406838 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0f571e-2dbd-4223-b424-6d5e2192a1ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:53.407155 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5c887a-5f42-4a7e-813d-2b5f30485417 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:53.408215 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0f571e-2dbd-4223-b424-6d5e2192a1ac None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:53.413709 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 389/779] 10.222.0.57 () {56 vars in 1087 bytes} [Thu May 19 14:24:53 2022] GET /identity/v3/services => generated 772 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:24:53.414213 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 391/780] 10.222.0.57 () {56 vars in 1087 bytes} [Thu May 19 14:24:53 2022] GET /identity/v3/services => generated 772 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:24:53.415562 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14347ddd-6439-471f-aec3-6b50ba04c842 None None] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:53.416089 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14347ddd-6439-471f-aec3-6b50ba04c842 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:53.416583 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14347ddd-6439-471f-aec3-6b50ba04c842 None None] PATH_INFO: `/` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:53.418737 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 390/781] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:24:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:24:53.447987 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c9b290-5095-4f94-8fc9-2e9a8f095057 None None] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:53.448933 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c9b290-5095-4f94-8fc9-2e9a8f095057 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:53.449217 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c9b290-5095-4f94-8fc9-2e9a8f095057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:53.480338 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.auth.core [None req-c6c9b290-5095-4f94-8fc9-2e9a8f095057 None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143587) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:24:53.481657 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-c6c9b290-5095-4f94-8fc9-2e9a8f095057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:24:53.586217 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 392/782] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:24:53 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:24:53.634688 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14347ddd-6439-471f-aec3-6b50ba04c842 None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:53.712172 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:53.774230 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14347ddd-6439-471f-aec3-6b50ba04c842 None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:53.818137 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9864bd62-d61d-4980-b6fc-35a5edb2ad34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:53.818949 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9864bd62-d61d-4980-b6fc-35a5edb2ad34 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:53.819236 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9864bd62-d61d-4980-b6fc-35a5edb2ad34 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:53.819516 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9864bd62-d61d-4980-b6fc-35a5edb2ad34 None admin] PATH_INFO: `/v3/roles/service` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:53.834752 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9864bd62-d61d-4980-b6fc-35a5edb2ad34 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:24:53.838951 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6113522e-a21e-4b0d-b68a-3cff2cceec81 None None] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:53.839792 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6113522e-a21e-4b0d-b68a-3cff2cceec81 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:53.840304 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6113522e-a21e-4b0d-b68a-3cff2cceec81 None None] PATH_INFO: `/` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:53.842936 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 393/783] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:24:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:24:53.844818 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-9864bd62-d61d-4980-b6fc-35a5edb2ad34 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:24:53.844818 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:53.844818 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:53.844818 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:53.844818 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:53.844818 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:53.844818 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:53.844818 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:53.844818 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:53.844818 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:53.844818 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:53.844818 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:53.844818 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:24:53.844818 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:24:53.844818 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:24:53.844818 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:24:53.846194 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:24:53.846194 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:24:53.846194 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:24:53.846194 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:24:53.846194 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:53.846194 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:53.846194 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:53.846194 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:53.846194 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:53.846194 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:53.846194 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:53.846194 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:24:53.846194 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:53.846194 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:53.846194 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:53.846194 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:53.847253 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:53.847253 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:53.847253 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:53.847253 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:53.847253 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:24:53.847253 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:24:53.847253 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:24:53.847253 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:24:53.847253 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:24:53.847253 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:24:53.847253 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:24:53.847253 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:24:53.847253 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 391/784] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 14:24:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:53.853864 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6113522e-a21e-4b0d-b68a-3cff2cceec81 None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:53.854112 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cf9981-3cfa-46a8-8350-b0a0a29ffdc1 None None] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:53.854112 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cf9981-3cfa-46a8-8350-b0a0a29ffdc1 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:53.854112 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cf9981-3cfa-46a8-8350-b0a0a29ffdc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:53.876631 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.auth.core [None req-b4cf9981-3cfa-46a8-8350-b0a0a29ffdc1 None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143586) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:24:53.877867 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.fernet_utils [None req-b4cf9981-3cfa-46a8-8350-b0a0a29ffdc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143586) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:24:54.032326 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6113522e-a21e-4b0d-b68a-3cff2cceec81 None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:54.059803 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:54.071611 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 392/785] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:24:53 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:24:54.080083 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6084d90-918f-4e10-931b-55b40ca07c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:54.080758 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6084d90-918f-4e10-931b-55b40ca07c43 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:54.081041 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6084d90-918f-4e10-931b-55b40ca07c43 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:54.081321 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6084d90-918f-4e10-931b-55b40ca07c43 None admin] PATH_INFO: `/v3/roles` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:54.082112 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6084d90-918f-4e10-931b-55b40ca07c43 None admin] RBAC: Adding query filter params (name=service) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:54.082427 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6084d90-918f-4e10-931b-55b40ca07c43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:54.083870 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6084d90-918f-4e10-931b-55b40ca07c43 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:54.093694 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 394/786] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 19 14:24:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:24:54.099379 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6084d90-918f-4e10-931b-55b40ca07c43 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:54.100333 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cf9981-3cfa-46a8-8350-b0a0a29ffdc1 None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:54.157332 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:54.222528 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cf9981-3cfa-46a8-8350-b0a0a29ffdc1 None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:54.253319 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6084d90-918f-4e10-931b-55b40ca07c43 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:54.259903 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb92198-2ffa-4f9e-ae7b-220315b07548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:54.260754 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb92198-2ffa-4f9e-ae7b-220315b07548 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:54.261094 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb92198-2ffa-4f9e-ae7b-220315b07548 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:54.261458 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb92198-2ffa-4f9e-ae7b-220315b07548 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:54.275123 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb92198-2ffa-4f9e-ae7b-220315b07548 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:24:54.285691 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-bdb92198-2ffa-4f9e-ae7b-220315b07548 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:24:54.285691 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:54.285691 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:54.285691 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:54.285691 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:54.285691 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:54.285691 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:54.285691 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:54.285691 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:54.285691 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:54.285691 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:54.285691 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:54.285691 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:24:54.285691 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:24:54.285691 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:24:54.285691 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:24:54.286937 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:24:54.286937 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:24:54.286937 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:24:54.286937 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:24:54.286937 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:54.286937 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:54.286937 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:54.286937 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:54.286937 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:54.286937 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:54.286937 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:54.286937 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:24:54.286937 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:54.286937 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:54.286937 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:54.286937 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:54.288570 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:54.288570 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:54.288570 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:54.288570 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:54.288570 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:24:54.288570 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:24:54.288570 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:24:54.288570 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:24:54.288570 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:24:54.288570 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:24:54.288570 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:24:54.288570 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:24:54.288570 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 393/787] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:24:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:54.288570 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782b9921-0767-4cc5-8c7a-4b3148ae1497 None None] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:54.289396 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782b9921-0767-4cc5-8c7a-4b3148ae1497 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:54.289396 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782b9921-0767-4cc5-8c7a-4b3148ae1497 None None] PATH_INFO: `/` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:54.295090 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 394/788] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:24:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:24:54.295090 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782b9921-0767-4cc5-8c7a-4b3148ae1497 None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:54.302851 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1360dea-a2c8-400a-a982-c8637095d368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:54.303989 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1360dea-a2c8-400a-a982-c8637095d368 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:54.304512 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1360dea-a2c8-400a-a982-c8637095d368 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:54.305093 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1360dea-a2c8-400a-a982-c8637095d368 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:54.318407 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1360dea-a2c8-400a-a982-c8637095d368 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:54.320314 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1360dea-a2c8-400a-a982-c8637095d368 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:54.341621 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-d1360dea-a2c8-400a-a982-c8637095d368 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:54.341621 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:54.341621 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:54.341621 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:54.341621 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:54.341621 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:54.341621 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:54.341621 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:54.341621 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:54.341621 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:54.341621 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:54.341621 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:54.341621 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:24:54.341621 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:24:54.341621 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:24:54.342742 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:24:54.342742 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:54.342742 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:54.342742 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:54.342742 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:54.342742 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:54.342742 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:54.342742 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:54.342742 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:24:54.342742 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:54.342742 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:54.342742 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:54.342742 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:54.342742 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:54.342742 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:54.342742 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:54.342742 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:54.343741 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:24:54.343741 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:24:54.343741 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:54.343741 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:24:54.347403 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 395/789] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:24:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:54.349298 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e085bdb6-0571-4bbc-94e5-2ef323788f46 None None] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:54.349866 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e085bdb6-0571-4bbc-94e5-2ef323788f46 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:54.350385 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e085bdb6-0571-4bbc-94e5-2ef323788f46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:54.369023 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.auth.core [None req-e085bdb6-0571-4bbc-94e5-2ef323788f46 None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143587) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:24:54.370723 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-e085bdb6-0571-4bbc-94e5-2ef323788f46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:24:54.398869 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:54.428039 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782b9921-0767-4cc5-8c7a-4b3148ae1497 None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:54.430043 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:54.461984 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 396/790] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:24:54 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:24:54.463411 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e085bdb6-0571-4bbc-94e5-2ef323788f46 None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:54.468647 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea9439c-9de1-436d-a314-d8ea193ecaa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:54.469307 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea9439c-9de1-436d-a314-d8ea193ecaa5 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:54.469590 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea9439c-9de1-436d-a314-d8ea193ecaa5 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:54.469884 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea9439c-9de1-436d-a314-d8ea193ecaa5 None admin] PATH_INFO: `/v3/roles` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:54.471588 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea9439c-9de1-436d-a314-d8ea193ecaa5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:54.471905 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea9439c-9de1-436d-a314-d8ea193ecaa5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:54.473526 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea9439c-9de1-436d-a314-d8ea193ecaa5 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:54.484390 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 395/791] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:24:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:24:54.489598 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea9439c-9de1-436d-a314-d8ea193ecaa5 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:54.598717 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e085bdb6-0571-4bbc-94e5-2ef323788f46 None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:54.610069 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea9439c-9de1-436d-a314-d8ea193ecaa5 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:54.639026 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a098d7-fa82-4ee6-9f52-307931b98d39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:54.639697 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a098d7-fa82-4ee6-9f52-307931b98d39 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:54.639983 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a098d7-fa82-4ee6-9f52-307931b98d39 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:54.640376 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a098d7-fa82-4ee6-9f52-307931b98d39 None admin] PATH_INFO: `/v3/domains` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:54.641199 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a098d7-fa82-4ee6-9f52-307931b98d39 None admin] RBAC: Adding query filter params (name=Default) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:54.641515 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a098d7-fa82-4ee6-9f52-307931b98d39 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:54.643176 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a098d7-fa82-4ee6-9f52-307931b98d39 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:54.656549 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27068ff1-7a44-4937-b3ea-36ecec5c7575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:54.657371 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27068ff1-7a44-4937-b3ea-36ecec5c7575 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:54.657660 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27068ff1-7a44-4937-b3ea-36ecec5c7575 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:54.657966 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27068ff1-7a44-4937-b3ea-36ecec5c7575 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:54.665155 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 397/792] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:24:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:54.669984 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a098d7-fa82-4ee6-9f52-307931b98d39 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:54.677894 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27068ff1-7a44-4937-b3ea-36ecec5c7575 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:54.679435 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27068ff1-7a44-4937-b3ea-36ecec5c7575 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:54.697227 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:54.703481 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-27068ff1-7a44-4937-b3ea-36ecec5c7575 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:54.703481 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:54.703481 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:54.703481 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:54.703481 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:54.703481 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:54.703481 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:54.703481 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:54.703481 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:54.703481 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:54.703481 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:54.703481 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:54.703481 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:24:54.703481 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:24:54.703481 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:24:54.704582 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:24:54.704582 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:54.704582 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:54.704582 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:54.704582 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:54.704582 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:54.704582 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:54.704582 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:54.704582 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:24:54.704582 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:54.704582 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:54.704582 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:54.704582 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:54.704582 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:54.704582 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:54.704582 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:54.704582 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:54.705584 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:24:54.705584 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:24:54.705584 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:54.705584 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:24:54.712432 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 396/793] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:24:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:54.712432 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27068ff1-7a44-4937-b3ea-36ecec5c7575 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:54.733343 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:54.821704 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a098d7-fa82-4ee6-9f52-307931b98d39 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:54.822217 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27068ff1-7a44-4937-b3ea-36ecec5c7575 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:54.865920 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bae4c3-7f84-401b-9717-48647763f809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:54.867131 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bae4c3-7f84-401b-9717-48647763f809 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:54.867404 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca9caa3-9d9d-4f9f-b2ae-23928bbcccf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:54.867913 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bae4c3-7f84-401b-9717-48647763f809 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:54.868206 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca9caa3-9d9d-4f9f-b2ae-23928bbcccf2 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:54.868491 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca9caa3-9d9d-4f9f-b2ae-23928bbcccf2 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:54.868773 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca9caa3-9d9d-4f9f-b2ae-23928bbcccf2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:54.869086 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bae4c3-7f84-401b-9717-48647763f809 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:54.878756 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bae4c3-7f84-401b-9717-48647763f809 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:54.880199 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca9caa3-9d9d-4f9f-b2ae-23928bbcccf2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:24:54.881070 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bae4c3-7f84-401b-9717-48647763f809 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:54.888519 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-2ca9caa3-9d9d-4f9f-b2ae-23928bbcccf2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:24:54.888519 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:54.888519 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:54.888519 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:54.888519 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:54.888519 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:54.888519 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:54.888519 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:54.888519 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:54.888519 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:54.888519 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:54.888519 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:54.888519 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:24:54.888519 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:24:54.888519 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:24:54.888519 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:24:54.889588 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:24:54.889588 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:24:54.889588 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:24:54.889588 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:24:54.889588 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:54.889588 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:54.889588 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:54.889588 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:54.889588 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:54.889588 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:54.889588 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:54.889588 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:24:54.889588 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:54.889588 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:54.889588 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:54.889588 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:54.890579 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:54.890579 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:54.890579 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:54.890579 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:54.890579 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:24:54.890579 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:24:54.890579 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:24:54.890579 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:24:54.890579 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:24:54.890579 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:24:54.890579 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:24:54.890579 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:24:54.890579 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 398/794] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:24:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:54.891372 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca9caa3-9d9d-4f9f-b2ae-23928bbcccf2 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:54.904629 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-42bae4c3-7f84-401b-9717-48647763f809 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 14:24:54.904629 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:54.904629 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:54.904629 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:54.904629 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:54.904629 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:54.904629 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:54.904629 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:54.904629 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:54.904629 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:54.904629 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:54.904629 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:54.904629 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:24:54.904629 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:24:54.904629 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:24:54.904629 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:24:54.905743 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:54.905743 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:54.905743 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:24:54.905743 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:24:54.905743 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:24:54.905743 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:24:54.905743 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:54.905743 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:54.905743 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:54.905743 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:54.905743 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:54.905743 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:24:54.905743 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:54.905743 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:54.905743 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:54.905743 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:54.905743 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:54.905743 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:54.906955 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:54.906955 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:54.906955 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:24:54.906955 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:24:54.906955 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:24:54.906955 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:24:54.906955 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:24:54.906955 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:24:54.906955 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 14:24:54.906955 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:24:54.907559 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 397/795] 10.222.0.57 () {56 vars in 1117 bytes} [Thu May 19 14:24:54 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:54.909528 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bae4c3-7f84-401b-9717-48647763f809 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:54.999544 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:55.014848 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:55.023143 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca9caa3-9d9d-4f9f-b2ae-23928bbcccf2 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:55.044751 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bae4c3-7f84-401b-9717-48647763f809 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:55.062454 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2818d4bb-5bfb-46ff-a5f3-01bd58037a3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:55.063723 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2818d4bb-5bfb-46ff-a5f3-01bd58037a3a None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:55.064175 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2818d4bb-5bfb-46ff-a5f3-01bd58037a3a None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:55.064605 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2818d4bb-5bfb-46ff-a5f3-01bd58037a3a None admin] PATH_INFO: `/v3/domains` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:55.065605 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2818d4bb-5bfb-46ff-a5f3-01bd58037a3a None admin] RBAC: Adding query filter params (name=Default) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:55.066238 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2818d4bb-5bfb-46ff-a5f3-01bd58037a3a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:55.067882 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2818d4bb-5bfb-46ff-a5f3-01bd58037a3a None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:55.082073 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 399/796] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:24:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:55.083233 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2818d4bb-5bfb-46ff-a5f3-01bd58037a3a None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:55.092235 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99324dc8-390d-4930-9306-78d30fc5a2e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:55.093004 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99324dc8-390d-4930-9306-78d30fc5a2e2 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:55.093356 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99324dc8-390d-4930-9306-78d30fc5a2e2 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:55.093708 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99324dc8-390d-4930-9306-78d30fc5a2e2 None admin] PATH_INFO: `/v3/roles` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:55.094568 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99324dc8-390d-4930-9306-78d30fc5a2e2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:55.095505 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99324dc8-390d-4930-9306-78d30fc5a2e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:55.096962 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99324dc8-390d-4930-9306-78d30fc5a2e2 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:55.109714 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 398/797] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:24:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:55.110755 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99324dc8-390d-4930-9306-78d30fc5a2e2 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:55.203176 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2818d4bb-5bfb-46ff-a5f3-01bd58037a3a None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:55.239771 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99324dc8-390d-4930-9306-78d30fc5a2e2 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:55.241392 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1789a21-cd4e-4431-b4dd-4f3ccafd8ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:55.242338 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1789a21-cd4e-4431-b4dd-4f3ccafd8ee4 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:55.242824 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1789a21-cd4e-4431-b4dd-4f3ccafd8ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:55.243302 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1789a21-cd4e-4431-b4dd-4f3ccafd8ee4 None admin] PATH_INFO: `/v3/users` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:55.244340 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1789a21-cd4e-4431-b4dd-4f3ccafd8ee4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:55.244843 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1789a21-cd4e-4431-b4dd-4f3ccafd8ee4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:55.247247 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1789a21-cd4e-4431-b4dd-4f3ccafd8ee4 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:55.257336 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:55.272888 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 400/798] 10.222.0.57 () {56 vars in 1149 bytes} [Thu May 19 14:24:55 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:55.275116 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1789a21-cd4e-4431-b4dd-4f3ccafd8ee4 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:55.277687 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f87e5d-48e4-4c31-a603-4c30ebb10cfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:55.278636 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f87e5d-48e4-4c31-a603-4c30ebb10cfe None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:55.279043 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f87e5d-48e4-4c31-a603-4c30ebb10cfe None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:55.279392 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f87e5d-48e4-4c31-a603-4c30ebb10cfe None admin] PATH_INFO: `/v3/users/swift` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:55.287766 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f87e5d-48e4-4c31-a603-4c30ebb10cfe None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:55.289498 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f87e5d-48e4-4c31-a603-4c30ebb10cfe None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:55.291248 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:55.306276 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-c1f87e5d-48e4-4c31-a603-4c30ebb10cfe None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 14:24:55.306276 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:55.306276 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:55.306276 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:55.306276 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:55.306276 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:55.306276 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:55.306276 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:55.306276 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:55.306276 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:55.306276 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:55.306276 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:55.306276 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:24:55.306276 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:24:55.306276 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:24:55.306276 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:24:55.307387 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:55.307387 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:55.307387 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:24:55.307387 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:24:55.307387 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:24:55.307387 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:24:55.307387 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:55.307387 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:55.307387 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:55.307387 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:55.307387 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:55.307387 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:24:55.307387 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:55.307387 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:55.307387 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:55.307387 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:55.307387 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:55.307387 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:55.309220 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:55.309220 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:55.309220 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:24:55.309220 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:24:55.309220 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:24:55.309220 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:24:55.309220 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:24:55.309220 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:24:55.309220 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 14:24:55.309220 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:24:55.313171 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 399/799] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:24:55 2022] GET /identity/v3/users/swift => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:55.314186 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f87e5d-48e4-4c31-a603-4c30ebb10cfe None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:55.393019 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1789a21-cd4e-4431-b4dd-4f3ccafd8ee4 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:55.424079 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f87e5d-48e4-4c31-a603-4c30ebb10cfe None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:55.431663 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc36c7c5-dbe7-47f2-9c40-dfe15e170092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:55.433561 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc36c7c5-dbe7-47f2-9c40-dfe15e170092 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:55.434111 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc36c7c5-dbe7-47f2-9c40-dfe15e170092 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:55.434605 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc36c7c5-dbe7-47f2-9c40-dfe15e170092 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:55.446111 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc36c7c5-dbe7-47f2-9c40-dfe15e170092 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:55.447985 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc36c7c5-dbe7-47f2-9c40-dfe15e170092 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:55.462330 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbb9844-697a-45e4-b3e3-a485f825f7e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:55.463056 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbb9844-697a-45e4-b3e3-a485f825f7e6 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:55.463347 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbb9844-697a-45e4-b3e3-a485f825f7e6 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:55.463627 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbb9844-697a-45e4-b3e3-a485f825f7e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:55.469331 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-cc36c7c5-dbe7-47f2-9c40-dfe15e170092 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:55.469331 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:55.469331 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:55.469331 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:55.469331 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:55.469331 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:55.469331 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:55.469331 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:55.469331 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:55.469331 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:55.469331 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:55.469331 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:55.469331 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:24:55.469331 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:24:55.469331 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:24:55.470351 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:24:55.470351 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:55.470351 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:55.470351 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:55.470351 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:55.470351 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:55.470351 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:55.470351 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:55.470351 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:24:55.470351 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:55.470351 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:55.470351 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:55.470351 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:55.470351 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:55.470351 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:55.470351 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:55.470351 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:55.471389 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:24:55.471389 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:24:55.471389 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:55.471389 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:24:55.473947 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 401/800] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:24:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:55.475057 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc36c7c5-dbe7-47f2-9c40-dfe15e170092 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:55.475372 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbb9844-697a-45e4-b3e3-a485f825f7e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:55.476950 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbb9844-697a-45e4-b3e3-a485f825f7e6 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:55.495928 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-3fbb9844-697a-45e4-b3e3-a485f825f7e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:55.495928 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:55.495928 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:55.495928 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:55.495928 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:55.495928 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:55.495928 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:55.495928 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:55.495928 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:55.495928 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:55.495928 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:55.495928 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:55.495928 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:24:55.495928 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:24:55.495928 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:24:55.497159 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:24:55.497159 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:55.497159 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:55.497159 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:55.497159 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:55.497159 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:55.497159 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:55.497159 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:55.497159 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:24:55.497159 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:55.497159 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:55.497159 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:55.497159 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:55.497159 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:55.497159 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:55.497159 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:55.497159 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:55.498198 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:24:55.498198 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:24:55.498198 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:55.498198 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:24:55.501964 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 400/801] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:24:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:55.502746 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbb9844-697a-45e4-b3e3-a485f825f7e6 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:55.535177 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:55.557157 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:55.592886 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc36c7c5-dbe7-47f2-9c40-dfe15e170092 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:55.619209 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbb9844-697a-45e4-b3e3-a485f825f7e6 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:55.632617 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e40b4a-2c2c-4944-8e47-d3d04cb045d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:55.633682 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e40b4a-2c2c-4944-8e47-d3d04cb045d5 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:55.634232 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e40b4a-2c2c-4944-8e47-d3d04cb045d5 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:55.634726 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e40b4a-2c2c-4944-8e47-d3d04cb045d5 None admin] PATH_INFO: `/v3/users` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:55.636772 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e40b4a-2c2c-4944-8e47-d3d04cb045d5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:55.637336 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e40b4a-2c2c-4944-8e47-d3d04cb045d5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:55.639083 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e40b4a-2c2c-4944-8e47-d3d04cb045d5 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:55.669482 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 402/802] 10.222.0.57 () {56 vars in 1135 bytes} [Thu May 19 14:24:55 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:55.669482 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e40b4a-2c2c-4944-8e47-d3d04cb045d5 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:55.684969 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9054430a-0d7e-4272-99da-5620a80361ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:55.686557 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9054430a-0d7e-4272-99da-5620a80361ad None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:55.687069 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9054430a-0d7e-4272-99da-5620a80361ad None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:55.687995 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9054430a-0d7e-4272-99da-5620a80361ad None admin] PATH_INFO: `/v3/domains` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:55.689276 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9054430a-0d7e-4272-99da-5620a80361ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:55.689827 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9054430a-0d7e-4272-99da-5620a80361ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:55.692268 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9054430a-0d7e-4272-99da-5620a80361ad None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:55.711401 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 401/803] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:24:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:55.712991 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9054430a-0d7e-4272-99da-5620a80361ad None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:55.797216 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e40b4a-2c2c-4944-8e47-d3d04cb045d5 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:55.809948 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:55.841339 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9054430a-0d7e-4272-99da-5620a80361ad None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:55.850425 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a36c9e-a2ae-449c-b2ef-13e3cb1e5cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:55.851105 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a36c9e-a2ae-449c-b2ef-13e3cb1e5cb1 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:55.851394 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a36c9e-a2ae-449c-b2ef-13e3cb1e5cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:55.851677 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a36c9e-a2ae-449c-b2ef-13e3cb1e5cb1 None admin] PATH_INFO: `/v3/domains` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:55.852893 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a36c9e-a2ae-449c-b2ef-13e3cb1e5cb1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:55.853209 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a36c9e-a2ae-449c-b2ef-13e3cb1e5cb1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:55.854163 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:55.855238 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a36c9e-a2ae-449c-b2ef-13e3cb1e5cb1 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:55.872805 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 403/804] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:24:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:55.872805 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a36c9e-a2ae-449c-b2ef-13e3cb1e5cb1 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:55.884982 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26eb234e-7304-4133-9917-ab02172393ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:55.886480 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eb234e-7304-4133-9917-ab02172393ec None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:55.886827 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eb234e-7304-4133-9917-ab02172393ec None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:55.887194 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eb234e-7304-4133-9917-ab02172393ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:55.898517 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26eb234e-7304-4133-9917-ab02172393ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:55.900047 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26eb234e-7304-4133-9917-ab02172393ec None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:55.918638 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-26eb234e-7304-4133-9917-ab02172393ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:55.918638 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:55.918638 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:55.918638 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:55.918638 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:55.918638 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:55.918638 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:55.918638 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:55.918638 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:55.918638 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:55.918638 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:55.918638 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:55.918638 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:24:55.918638 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:24:55.918638 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:24:55.919800 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:24:55.919800 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:55.919800 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:55.919800 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:55.919800 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:55.919800 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:55.919800 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:55.919800 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:55.919800 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:24:55.919800 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:55.919800 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:55.919800 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:55.919800 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:55.919800 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:55.919800 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:55.919800 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:55.919800 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:55.920864 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:24:55.920864 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:24:55.920864 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:55.920864 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:24:55.920864 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 402/805] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:24:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:55.921355 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c735e1-ab29-4252-b000-fb0838135ca3 None None] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:55.921642 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c735e1-ab29-4252-b000-fb0838135ca3 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:55.921930 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c735e1-ab29-4252-b000-fb0838135ca3 None None] PATH_INFO: `/` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:55.923336 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 403/806] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:24:55 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:24:55.927863 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c735e1-ab29-4252-b000-fb0838135ca3 None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:55.989952 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a36c9e-a2ae-449c-b2ef-13e3cb1e5cb1 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:56.028246 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5001a6-027c-47e3-b086-1a4d95856c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:56.029029 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5001a6-027c-47e3-b086-1a4d95856c8d None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:56.029310 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5001a6-027c-47e3-b086-1a4d95856c8d None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:56.029585 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5001a6-027c-47e3-b086-1a4d95856c8d None admin] PATH_INFO: `/v3/users/nova` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:56.037390 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5001a6-027c-47e3-b086-1a4d95856c8d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:56.038875 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5001a6-027c-47e3-b086-1a4d95856c8d None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:56.042655 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c735e1-ab29-4252-b000-fb0838135ca3 None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:56.053956 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-8f5001a6-027c-47e3-b086-1a4d95856c8d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 14:24:56.053956 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:56.053956 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:56.053956 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:56.053956 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:56.053956 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:56.053956 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:56.053956 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:56.053956 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:56.053956 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:56.053956 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:56.053956 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:56.053956 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:24:56.053956 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:24:56.053956 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:24:56.053956 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:24:56.055126 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:56.055126 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:56.055126 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:24:56.055126 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:24:56.055126 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:24:56.055126 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:24:56.055126 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:56.055126 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:56.055126 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:56.055126 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:56.055126 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:56.055126 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:24:56.055126 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:56.055126 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:56.055126 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:56.055126 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:56.055126 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:56.055126 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:56.056150 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:56.056150 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:56.056150 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:24:56.056150 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:24:56.056150 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:24:56.056150 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:24:56.056150 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:24:56.056150 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:24:56.056150 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 14:24:56.056150 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:24:56.056150 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 404/807] 10.222.0.57 () {56 vars in 1093 bytes} [Thu May 19 14:24:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:56.056150 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5001a6-027c-47e3-b086-1a4d95856c8d None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:56.080269 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753b0257-e382-423b-9ce7-c58f5a187e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:56.081064 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753b0257-e382-423b-9ce7-c58f5a187e0c None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:56.081346 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753b0257-e382-423b-9ce7-c58f5a187e0c None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:56.081617 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753b0257-e382-423b-9ce7-c58f5a187e0c None admin] PATH_INFO: `/v3/projects/service` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:56.084568 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:56.088274 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753b0257-e382-423b-9ce7-c58f5a187e0c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:56.089720 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753b0257-e382-423b-9ce7-c58f5a187e0c None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:56.103565 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-753b0257-e382-423b-9ce7-c58f5a187e0c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:24:56.103565 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:56.103565 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:56.103565 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:56.103565 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:56.103565 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:56.103565 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:56.103565 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:56.103565 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:56.103565 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:56.103565 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:56.103565 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:56.103565 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:24:56.103565 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:24:56.103565 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:24:56.105399 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:24:56.105399 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:56.105399 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:56.105399 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:56.105399 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:56.105399 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:56.105399 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:56.105399 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:56.105399 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:24:56.105399 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:56.105399 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:56.105399 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:56.105399 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:56.105399 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:56.105399 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:56.105399 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:56.105399 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:56.106491 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:24:56.106491 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:24:56.106491 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:24:56.106491 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:24:56.106491 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:24:56.106491 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:24:56.106491 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:24:56.106491 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:24:56.106491 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 404/808] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:24:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:56.107345 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa7335e-7dde-4f52-8838-0bff8ad5128b None None] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:56.107636 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa7335e-7dde-4f52-8838-0bff8ad5128b None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:56.107915 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa7335e-7dde-4f52-8838-0bff8ad5128b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:56.125003 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.auth.core [None req-0aa7335e-7dde-4f52-8838-0bff8ad5128b None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143586) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:24:56.126223 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.fernet_utils [None req-0aa7335e-7dde-4f52-8838-0bff8ad5128b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143586) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:24:56.142691 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:56.162451 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5001a6-027c-47e3-b086-1a4d95856c8d None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:56.193077 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 405/809] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:24:56 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:24:56.194248 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a755149b-d864-42f5-9757-dc40c8072a24 None None] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:56.194545 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a755149b-d864-42f5-9757-dc40c8072a24 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:56.194827 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a755149b-d864-42f5-9757-dc40c8072a24 None None] PATH_INFO: `/` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:56.196208 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 406/810] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:24:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:24:56.196782 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a755149b-d864-42f5-9757-dc40c8072a24 None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:56.198961 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7225b361-b043-46dd-a7b4-2d697d1f07a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:56.200878 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7225b361-b043-46dd-a7b4-2d697d1f07a3 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:56.201381 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7225b361-b043-46dd-a7b4-2d697d1f07a3 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:56.201870 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7225b361-b043-46dd-a7b4-2d697d1f07a3 None admin] PATH_INFO: `/v3/domains` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:56.202811 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7225b361-b043-46dd-a7b4-2d697d1f07a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:56.203304 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7225b361-b043-46dd-a7b4-2d697d1f07a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:56.204887 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7225b361-b043-46dd-a7b4-2d697d1f07a3 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:56.219550 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 405/811] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:24:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:56.220604 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7225b361-b043-46dd-a7b4-2d697d1f07a3 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:56.302230 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a755149b-d864-42f5-9757-dc40c8072a24 None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:56.308088 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:56.321507 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7225b361-b043-46dd-a7b4-2d697d1f07a3 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:56.338830 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc28471-84ef-49ab-8d3a-810b17e9c712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:56.339726 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc28471-84ef-49ab-8d3a-810b17e9c712 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:56.340008 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc28471-84ef-49ab-8d3a-810b17e9c712 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:56.340284 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc28471-84ef-49ab-8d3a-810b17e9c712 None admin] PATH_INFO: `/v3/users` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:56.341165 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc28471-84ef-49ab-8d3a-810b17e9c712 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:56.341482 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc28471-84ef-49ab-8d3a-810b17e9c712 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:56.343096 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc28471-84ef-49ab-8d3a-810b17e9c712 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:56.356801 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37430016-1e95-4ae0-a407-66e5647b8b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:56.357467 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37430016-1e95-4ae0-a407-66e5647b8b1a None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:56.357755 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37430016-1e95-4ae0-a407-66e5647b8b1a None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:56.358086 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37430016-1e95-4ae0-a407-66e5647b8b1a None admin] PATH_INFO: `/v3/projects` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:56.358880 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37430016-1e95-4ae0-a407-66e5647b8b1a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:56.359198 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37430016-1e95-4ae0-a407-66e5647b8b1a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:56.360643 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37430016-1e95-4ae0-a407-66e5647b8b1a None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:56.361281 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 407/812] 10.222.0.57 () {56 vars in 1133 bytes} [Thu May 19 14:24:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:56.362641 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f148c9e-35fb-4c11-8486-2e6dad336db1 None None] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:56.362934 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f148c9e-35fb-4c11-8486-2e6dad336db1 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:56.363214 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f148c9e-35fb-4c11-8486-2e6dad336db1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:56.373405 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 406/813] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:24:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:56.374260 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37430016-1e95-4ae0-a407-66e5647b8b1a None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:56.379893 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.auth.core [None req-4f148c9e-35fb-4c11-8486-2e6dad336db1 None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143586) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:24:56.381083 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.fernet_utils [None req-4f148c9e-35fb-4c11-8486-2e6dad336db1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143586) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:24:56.383882 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:56.449600 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 408/814] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:24:56 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:24:56.450321 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f148c9e-35fb-4c11-8486-2e6dad336db1 None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:56.476227 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37430016-1e95-4ae0-a407-66e5647b8b1a None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:56.510521 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f2c156-9e80-4a8f-941a-c49eaad68b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:56.512134 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f2c156-9e80-4a8f-941a-c49eaad68b13 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:56.512425 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f2c156-9e80-4a8f-941a-c49eaad68b13 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:56.512698 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f2c156-9e80-4a8f-941a-c49eaad68b13 None admin] PATH_INFO: `/v3/services/network` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:56.513349 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f2c156-9e80-4a8f-941a-c49eaad68b13 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:56.514771 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f2c156-9e80-4a8f-941a-c49eaad68b13 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:56.527922 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-c4f2c156-9e80-4a8f-941a-c49eaad68b13 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 19 14:24:56.527922 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:56.527922 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:24:56.527922 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:24:56.527922 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:24:56.527922 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:24:56.527922 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:24:56.527922 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:24:56.527922 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 14:24:56.527922 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:24:56.527922 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:24:56.527922 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:24:56.527922 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:56.527922 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:56.527922 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:56.527922 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:56.527922 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:56.534176 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:56.534176 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:56.534176 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:56.534176 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:56.534176 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:56.534176 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:56.534176 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:24:56.534176 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:24:56.534176 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:24:56.534176 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:24:56.534176 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:56.534176 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:56.534176 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:56.534176 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:56.534176 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:56.534176 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:56.535320 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:56.535320 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:24:56.535320 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:56.535320 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:56.535320 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:56.535320 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:56.535320 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:56.535320 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:56.535320 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:56.535320 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:56.535320 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:24:56.535320 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:24:56.535320 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 14:24:56.535320 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:24:56.535320 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 407/815] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:24:56 2022] GET /identity/v3/services/network => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:56.536215 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f2c156-9e80-4a8f-941a-c49eaad68b13 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:56.545512 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:56.558920 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f148c9e-35fb-4c11-8486-2e6dad336db1 None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:56.592486 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ca56db-c266-4a5d-9af4-25cf264705bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:56.593276 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ca56db-c266-4a5d-9af4-25cf264705bb None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:56.593559 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ca56db-c266-4a5d-9af4-25cf264705bb None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:56.593860 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ca56db-c266-4a5d-9af4-25cf264705bb None admin] PATH_INFO: `/v3/projects/service` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:56.600805 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ca56db-c266-4a5d-9af4-25cf264705bb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:56.602282 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ca56db-c266-4a5d-9af4-25cf264705bb None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:56.605301 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:56.618048 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-87ca56db-c266-4a5d-9af4-25cf264705bb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:24:56.618048 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:56.618048 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:56.618048 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:56.618048 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:56.618048 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:56.618048 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:56.618048 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:56.618048 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:56.618048 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:56.618048 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:56.618048 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:56.618048 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:24:56.618048 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:24:56.618048 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:24:56.619098 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:24:56.619098 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:56.619098 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:56.619098 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:56.619098 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:56.619098 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:56.619098 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:56.619098 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:56.619098 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:24:56.619098 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:56.619098 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:56.619098 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:56.619098 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:56.619098 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:56.619098 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:56.619098 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:56.619098 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:56.620749 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:24:56.620749 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:24:56.620749 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:24:56.620749 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:24:56.620749 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:24:56.620749 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:24:56.620749 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:24:56.620749 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:24:56.620749 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 409/816] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:24:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:56.620749 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ca56db-c266-4a5d-9af4-25cf264705bb None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:56.634652 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f2c156-9e80-4a8f-941a-c49eaad68b13 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:56.671263 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fc913c-f619-4dd5-97f2-43554bd73207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:56.671917 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fc913c-f619-4dd5-97f2-43554bd73207 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:56.672200 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fc913c-f619-4dd5-97f2-43554bd73207 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:56.672482 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fc913c-f619-4dd5-97f2-43554bd73207 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:56.720896 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ca56db-c266-4a5d-9af4-25cf264705bb None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:56.740084 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fc913c-f619-4dd5-97f2-43554bd73207 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:56.741530 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fc913c-f619-4dd5-97f2-43554bd73207 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:56.755912 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:56.756072 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d7e2a1-f0c7-4123-a8e3-73bbacc03102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:56.756855 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d7e2a1-f0c7-4123-a8e3-73bbacc03102 None admin] REQUEST_METHOD: `PUT` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:56.757137 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d7e2a1-f0c7-4123-a8e3-73bbacc03102 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:56.757409 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d7e2a1-f0c7-4123-a8e3-73bbacc03102 None admin] PATH_INFO: `/v3/projects/252d9673f08c479a98d04fee3f5856a4/users/7977ff2f29eb435cbfeaa0b61d1d7a5d/roles/b0b74fefb27e4a0ebd895893aa9862d7` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:56.772859 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d7e2a1-f0c7-4123-a8e3-73bbacc03102 None admin] RBAC: Authorizing `identity:create_grant(project_id=252d9673f08c479a98d04fee3f5856a4, user_id=7977ff2f29eb435cbfeaa0b61d1d7a5d, role_id=b0b74fefb27e4a0ebd895893aa9862d7)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:56.774321 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d7e2a1-f0c7-4123-a8e3-73bbacc03102 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:56.788254 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 410/817] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 14:24:56 2022] PUT /identity/v3/projects/252d9673f08c479a98d04fee3f5856a4/users/7977ff2f29eb435cbfeaa0b61d1d7a5d/roles/b0b74fefb27e4a0ebd895893aa9862d7 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:24:56.789079 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d7e2a1-f0c7-4123-a8e3-73bbacc03102 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:56.847623 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:56.866251 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 408/818] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:24:56 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:24:56.867140 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fc913c-f619-4dd5-97f2-43554bd73207 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:56.915143 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d7e2a1-f0c7-4123-a8e3-73bbacc03102 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:56.955267 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f80cecc-5abc-45ab-8a2c-525730794889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:56.956210 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f80cecc-5abc-45ab-8a2c-525730794889 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:56.956601 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f80cecc-5abc-45ab-8a2c-525730794889 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:56.956920 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f80cecc-5abc-45ab-8a2c-525730794889 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:56.957589 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f80cecc-5abc-45ab-8a2c-525730794889 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:56.959030 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f80cecc-5abc-45ab-8a2c-525730794889 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:56.973169 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-1f80cecc-5abc-45ab-8a2c-525730794889 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 14:24:56.973169 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:56.973169 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:24:56.973169 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:24:56.973169 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:24:56.973169 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:24:56.973169 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:24:56.973169 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:24:56.973169 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 14:24:56.973169 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:24:56.973169 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:24:56.973169 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:24:56.973169 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:56.973169 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:56.973169 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:56.973169 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:56.974351 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:56.974351 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:56.974351 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:56.974351 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:56.974351 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:56.974351 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:56.974351 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:56.974351 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:24:56.974351 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:24:56.974351 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:24:56.974351 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:24:56.974351 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:56.974351 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:56.974351 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:56.974351 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:56.974351 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:56.975293 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:56.975293 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:56.975293 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:24:56.975293 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:56.975293 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:56.975293 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:56.975293 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:56.975293 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:56.975293 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:56.975293 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:56.975293 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:56.975293 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:24:56.975293 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:24:56.975293 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 14:24:56.975293 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:24:56.975293 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 411/819] 10.222.0.57 () {56 vars in 1129 bytes} [Thu May 19 14:24:56 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:56.976992 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f80cecc-5abc-45ab-8a2c-525730794889 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:56.983278 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fc913c-f619-4dd5-97f2-43554bd73207 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:57.000079 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:57.023651 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62abf65a-daca-4d48-a357-10da1bb5535b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:57.024672 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62abf65a-daca-4d48-a357-10da1bb5535b None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:57.025026 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62abf65a-daca-4d48-a357-10da1bb5535b None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:57.025312 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62abf65a-daca-4d48-a357-10da1bb5535b None admin] PATH_INFO: `/v3/services` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:57.026057 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62abf65a-daca-4d48-a357-10da1bb5535b None admin] RBAC: Adding query filter params (name=network) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:57.026889 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62abf65a-daca-4d48-a357-10da1bb5535b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:57.028426 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62abf65a-daca-4d48-a357-10da1bb5535b None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:57.036809 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 409/820] 10.222.0.57 () {56 vars in 1112 bytes} [Thu May 19 14:24:56 2022] GET /identity/v3/services?name=network => generated 124 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:57.036809 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62abf65a-daca-4d48-a357-10da1bb5535b None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:57.103627 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f80cecc-5abc-45ab-8a2c-525730794889 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:57.119167 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:57.143095 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62abf65a-daca-4d48-a357-10da1bb5535b None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:57.144319 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48467710-7816-452f-839e-2349c64a6a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:57.145081 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48467710-7816-452f-839e-2349c64a6a12 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:57.145441 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48467710-7816-452f-839e-2349c64a6a12 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:57.145774 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48467710-7816-452f-839e-2349c64a6a12 None admin] PATH_INFO: `/v3/projects` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:57.146691 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48467710-7816-452f-839e-2349c64a6a12 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:57.147006 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48467710-7816-452f-839e-2349c64a6a12 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:57.148429 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48467710-7816-452f-839e-2349c64a6a12 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:57.166091 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 412/821] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:24:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:57.166091 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48467710-7816-452f-839e-2349c64a6a12 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:57.185485 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6c2d9c-6d4c-40fa-ad85-2678b0ffbeed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:57.186349 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6c2d9c-6d4c-40fa-ad85-2678b0ffbeed None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:57.186627 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6c2d9c-6d4c-40fa-ad85-2678b0ffbeed None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:57.186894 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6c2d9c-6d4c-40fa-ad85-2678b0ffbeed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:57.198068 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6c2d9c-6d4c-40fa-ad85-2678b0ffbeed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:57.199533 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6c2d9c-6d4c-40fa-ad85-2678b0ffbeed None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:57.218509 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-0a6c2d9c-6d4c-40fa-ad85-2678b0ffbeed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:57.218509 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:57.218509 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:57.218509 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:57.218509 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:57.218509 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:57.218509 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:57.218509 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:57.218509 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:57.218509 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:57.218509 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:57.218509 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:57.218509 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:24:57.218509 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:24:57.218509 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:24:57.219550 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:24:57.219550 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:57.219550 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:57.219550 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:57.219550 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:57.219550 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:57.219550 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:57.219550 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:57.219550 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:24:57.219550 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:57.219550 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:57.219550 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:57.219550 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:57.219550 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:57.219550 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:57.219550 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:57.219550 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:57.221211 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:24:57.221211 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:24:57.221211 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:57.221211 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:24:57.221211 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 410/822] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:24:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:57.221211 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6c2d9c-6d4c-40fa-ad85-2678b0ffbeed None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:57.252921 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:57.276936 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48467710-7816-452f-839e-2349c64a6a12 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:57.315768 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cad093-1ab9-45c6-a63d-d389e103746b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:57.316689 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cad093-1ab9-45c6-a63d-d389e103746b None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:57.316938 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cad093-1ab9-45c6-a63d-d389e103746b None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:57.317283 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cad093-1ab9-45c6-a63d-d389e103746b None admin] PATH_INFO: `/v3/services` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:57.318111 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cad093-1ab9-45c6-a63d-d389e103746b None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:57.318493 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cad093-1ab9-45c6-a63d-d389e103746b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:57.319960 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cad093-1ab9-45c6-a63d-d389e103746b None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:57.331064 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 413/823] 10.222.0.57 () {56 vars in 1124 bytes} [Thu May 19 14:24:57 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:57.331064 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cad093-1ab9-45c6-a63d-d389e103746b None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:57.333459 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6c2d9c-6d4c-40fa-ad85-2678b0ffbeed None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:57.372596 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90905e77-44c0-42d0-99bc-1d082f3ac31f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:57.373390 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90905e77-44c0-42d0-99bc-1d082f3ac31f None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:57.373694 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90905e77-44c0-42d0-99bc-1d082f3ac31f None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:57.374069 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90905e77-44c0-42d0-99bc-1d082f3ac31f None admin] PATH_INFO: `/v3/services` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:57.374299 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:57.375200 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90905e77-44c0-42d0-99bc-1d082f3ac31f None admin] RBAC: Adding query filter params (type=network) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:57.375329 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90905e77-44c0-42d0-99bc-1d082f3ac31f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:57.376901 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90905e77-44c0-42d0-99bc-1d082f3ac31f None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:57.383150 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 411/824] 10.222.0.57 () {56 vars in 1112 bytes} [Thu May 19 14:24:57 2022] GET /identity/v3/services?type=network => generated 350 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:57.384034 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90905e77-44c0-42d0-99bc-1d082f3ac31f None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:57.440806 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cad093-1ab9-45c6-a63d-d389e103746b None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:57.476773 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291ec63c-cf6a-4814-9105-81324d918adb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:57.477659 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291ec63c-cf6a-4814-9105-81324d918adb None admin] REQUEST_METHOD: `PUT` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:57.477977 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291ec63c-cf6a-4814-9105-81324d918adb None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:57.478258 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291ec63c-cf6a-4814-9105-81324d918adb None admin] PATH_INFO: `/v3/projects/252d9673f08c479a98d04fee3f5856a4/users/d3e544b4989e49a19536b40de5069bcf/roles/a3b793754e7b40b0bb21afe74eec3c18` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:57.480501 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291ec63c-cf6a-4814-9105-81324d918adb None admin] RBAC: Authorizing `identity:create_grant(project_id=252d9673f08c479a98d04fee3f5856a4, user_id=d3e544b4989e49a19536b40de5069bcf, role_id=a3b793754e7b40b0bb21afe74eec3c18)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:57.481875 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:57.482026 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291ec63c-cf6a-4814-9105-81324d918adb None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:57.492574 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90905e77-44c0-42d0-99bc-1d082f3ac31f None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:57.497031 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 414/825] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 14:24:57 2022] PUT /identity/v3/projects/252d9673f08c479a98d04fee3f5856a4/users/d3e544b4989e49a19536b40de5069bcf/roles/a3b793754e7b40b0bb21afe74eec3c18 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:24:57.497617 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291ec63c-cf6a-4814-9105-81324d918adb None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:57.554503 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4893c695-b39e-4145-b5cf-6afcbf01d427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:57.555627 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4893c695-b39e-4145-b5cf-6afcbf01d427 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:57.556280 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4893c695-b39e-4145-b5cf-6afcbf01d427 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:57.556786 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4893c695-b39e-4145-b5cf-6afcbf01d427 None admin] PATH_INFO: `/v3/domains` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:57.558152 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4893c695-b39e-4145-b5cf-6afcbf01d427 None admin] RBAC: Adding query filter params (name=Default) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:57.558720 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4893c695-b39e-4145-b5cf-6afcbf01d427 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:57.560513 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4893c695-b39e-4145-b5cf-6afcbf01d427 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:57.575847 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 412/826] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:24:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:57.577202 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4893c695-b39e-4145-b5cf-6afcbf01d427 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:57.619632 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:57.623845 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291ec63c-cf6a-4814-9105-81324d918adb None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:57.670663 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f173a4-cea0-447e-bcb5-87c8ca594762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:57.670934 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f173a4-cea0-447e-bcb5-87c8ca594762 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:57.671180 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f173a4-cea0-447e-bcb5-87c8ca594762 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:57.671463 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f173a4-cea0-447e-bcb5-87c8ca594762 None admin] PATH_INFO: `/v3/services` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:57.672197 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f173a4-cea0-447e-bcb5-87c8ca594762 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:57.672575 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f173a4-cea0-447e-bcb5-87c8ca594762 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:57.674071 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f173a4-cea0-447e-bcb5-87c8ca594762 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:57.680299 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 415/827] 10.222.0.57 () {56 vars in 1124 bytes} [Thu May 19 14:24:57 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:57.681267 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f173a4-cea0-447e-bcb5-87c8ca594762 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:57.715559 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4893c695-b39e-4145-b5cf-6afcbf01d427 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:57.772785 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ae18f8-3b90-4a5d-933c-a440a6e207ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:57.773509 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ae18f8-3b90-4a5d-933c-a440a6e207ce None admin] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:57.773802 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ae18f8-3b90-4a5d-933c-a440a6e207ce None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:57.774102 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ae18f8-3b90-4a5d-933c-a440a6e207ce None admin] PATH_INFO: `/v3/endpoints` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:57.777106 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ae18f8-3b90-4a5d-933c-a440a6e207ce None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:57.777106 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ae18f8-3b90-4a5d-933c-a440a6e207ce None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:57.811269 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 413/828] 10.222.0.57 () {60 vars in 1144 bytes} [Thu May 19 14:24:57 2022] POST /identity/v3/endpoints => generated 327 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:24:57.813248 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ae18f8-3b90-4a5d-933c-a440a6e207ce None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:57.815305 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f173a4-cea0-447e-bcb5-87c8ca594762 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:57.834095 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:57.881811 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0200515b-c790-4c1e-b6a8-d88ff4b4b613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:57.883656 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0200515b-c790-4c1e-b6a8-d88ff4b4b613 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:57.884150 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0200515b-c790-4c1e-b6a8-d88ff4b4b613 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:57.884620 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0200515b-c790-4c1e-b6a8-d88ff4b4b613 None admin] PATH_INFO: `/v3/projects/service` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:57.895771 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0200515b-c790-4c1e-b6a8-d88ff4b4b613 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:57.897768 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0200515b-c790-4c1e-b6a8-d88ff4b4b613 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:57.920091 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-0200515b-c790-4c1e-b6a8-d88ff4b4b613 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:24:57.920091 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:57.920091 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:57.920091 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:57.920091 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:57.920091 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:57.920091 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:57.920091 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:57.920091 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:57.920091 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:57.920091 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:57.920091 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:57.920091 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:24:57.920091 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:24:57.920091 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:24:57.921179 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:24:57.921179 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:57.921179 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:57.921179 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:57.921179 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:57.921179 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:57.921179 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:57.921179 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:57.921179 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:24:57.921179 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:57.921179 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:57.921179 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:57.921179 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:57.921179 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:57.921179 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:57.921179 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:57.921179 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:57.922253 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:24:57.922253 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:24:57.922253 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:24:57.922253 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:24:57.922253 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:24:57.922253 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:24:57.922253 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:24:57.922253 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:24:57.925173 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 416/829] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:24:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:57.931343 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0200515b-c790-4c1e-b6a8-d88ff4b4b613 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:57.977881 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ae18f8-3b90-4a5d-933c-a440a6e207ce None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:57.981034 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:58.027999 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84703b4-16a6-4ad4-b91c-9bef1af5dc9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:58.028709 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84703b4-16a6-4ad4-b91c-9bef1af5dc9e None admin] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:58.028998 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84703b4-16a6-4ad4-b91c-9bef1af5dc9e None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:58.029280 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84703b4-16a6-4ad4-b91c-9bef1af5dc9e None admin] PATH_INFO: `/v3/endpoints` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:58.030112 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84703b4-16a6-4ad4-b91c-9bef1af5dc9e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:58.031550 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84703b4-16a6-4ad4-b91c-9bef1af5dc9e None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:58.071640 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 414/830] 10.222.0.57 () {60 vars in 1144 bytes} [Thu May 19 14:24:57 2022] POST /identity/v3/endpoints => generated 346 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:24:58.106549 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0200515b-c790-4c1e-b6a8-d88ff4b4b613 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:58.162990 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e151f7-b8d6-4a37-9d61-beb4c843c342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:58.164124 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e151f7-b8d6-4a37-9d61-beb4c843c342 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:58.164651 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e151f7-b8d6-4a37-9d61-beb4c843c342 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:58.165202 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e151f7-b8d6-4a37-9d61-beb4c843c342 None admin] PATH_INFO: `/v3/projects` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:58.166332 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e151f7-b8d6-4a37-9d61-beb4c843c342 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:58.167015 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e151f7-b8d6-4a37-9d61-beb4c843c342 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:58.168810 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e151f7-b8d6-4a37-9d61-beb4c843c342 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:58.184502 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 417/831] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:24:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:58.191866 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84703b4-16a6-4ad4-b91c-9bef1af5dc9e None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:58.275549 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:58.322815 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84703b4-16a6-4ad4-b91c-9bef1af5dc9e None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:58.362344 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae131fe-d6fe-480e-baf0-613f5c58aa9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:58.363012 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae131fe-d6fe-480e-baf0-613f5c58aa9a None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:58.363298 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae131fe-d6fe-480e-baf0-613f5c58aa9a None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:58.363576 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae131fe-d6fe-480e-baf0-613f5c58aa9a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:58.364422 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae131fe-d6fe-480e-baf0-613f5c58aa9a None admin] RBAC: Adding query filter params (role.id=a3b793754e7b40b0bb21afe74eec3c18, scope.project.id=252d9673f08c479a98d04fee3f5856a4, user.id=5f28de22c2ee4c53b6d14d5b0ddf7fe9) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:58.364737 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae131fe-d6fe-480e-baf0-613f5c58aa9a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:58.366216 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae131fe-d6fe-480e-baf0-613f5c58aa9a None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:58.378007 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 415/832] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:24:58 2022] GET /identity/v3/role_assignments?user.id=5f28de22c2ee4c53b6d14d5b0ddf7fe9&scope.project.id=252d9673f08c479a98d04fee3f5856a4&role.id=a3b793754e7b40b0bb21afe74eec3c18 => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:59.241902 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc17de6a-5e23-4811-b2f7-e992cfc3893f None None] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:59.242475 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc17de6a-5e23-4811-b2f7-e992cfc3893f None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:59.242475 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc17de6a-5e23-4811-b2f7-e992cfc3893f None None] PATH_INFO: `/` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:59.244084 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 418/833] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:24:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:24:59.249688 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa145527-87ba-40aa-8b67-a5c48e0b662b None None] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:59.250201 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa145527-87ba-40aa-8b67-a5c48e0b662b None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:59.250498 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa145527-87ba-40aa-8b67-a5c48e0b662b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:59.267940 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.auth.core [None req-fa145527-87ba-40aa-8b67-a5c48e0b662b None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143587) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:24:59.269118 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-fa145527-87ba-40aa-8b67-a5c48e0b662b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:24:59.339444 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 416/834] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:24:59 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:24:59.365164 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc17de6a-5e23-4811-b2f7-e992cfc3893f None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:59.450684 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:59.479926 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc17de6a-5e23-4811-b2f7-e992cfc3893f None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:59.519590 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a59b0cb-aa43-47bb-9a54-20f07a723f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:59.520778 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a59b0cb-aa43-47bb-9a54-20f07a723f95 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:59.521303 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a59b0cb-aa43-47bb-9a54-20f07a723f95 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:59.521809 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a59b0cb-aa43-47bb-9a54-20f07a723f95 None admin] PATH_INFO: `/v3/roles/service` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:59.533682 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a59b0cb-aa43-47bb-9a54-20f07a723f95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:24:59.541799 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-8a59b0cb-aa43-47bb-9a54-20f07a723f95 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:24:59.541799 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:59.541799 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:59.541799 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:59.541799 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:59.541799 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:59.541799 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:59.541799 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:59.541799 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:59.541799 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:59.541799 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:59.541799 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:59.541799 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:24:59.541799 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:24:59.541799 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:24:59.541799 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:24:59.543447 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:24:59.543447 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:24:59.543447 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:24:59.543447 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:24:59.543447 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:59.543447 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:59.543447 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:59.543447 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:59.543447 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:59.543447 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:59.543447 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:59.543447 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:24:59.543447 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:59.543447 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:59.543447 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:59.543447 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:59.544498 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:59.544498 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:59.544498 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:59.544498 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:59.544498 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:24:59.544498 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:24:59.544498 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:24:59.544498 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:24:59.544498 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:24:59.544498 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:24:59.544498 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:24:59.544498 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:24:59.547495 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 419/835] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 14:24:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:59.557885 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa145527-87ba-40aa-8b67-a5c48e0b662b None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:59.622289 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:59.664950 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa145527-87ba-40aa-8b67-a5c48e0b662b None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:59.700016 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dd8caf-658d-4fee-869f-da430fb5915b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:59.700695 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dd8caf-658d-4fee-869f-da430fb5915b None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:59.700978 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dd8caf-658d-4fee-869f-da430fb5915b None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:59.701253 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dd8caf-658d-4fee-869f-da430fb5915b None admin] PATH_INFO: `/v3/roles` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:59.702045 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dd8caf-658d-4fee-869f-da430fb5915b None admin] RBAC: Adding query filter params (name=service) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:59.702352 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dd8caf-658d-4fee-869f-da430fb5915b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:59.703734 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dd8caf-658d-4fee-869f-da430fb5915b None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:59.712597 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 417/836] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 19 14:24:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:59.717953 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a59b0cb-aa43-47bb-9a54-20f07a723f95 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:59.827958 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a59b0cb-aa43-47bb-9a54-20f07a723f95 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:59.867496 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9c12a6-ff2e-433a-8a03-a911686bd529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:59.868699 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9c12a6-ff2e-433a-8a03-a911686bd529 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:59.869229 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9c12a6-ff2e-433a-8a03-a911686bd529 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:59.869729 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9c12a6-ff2e-433a-8a03-a911686bd529 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:59.872255 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:59.882446 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9c12a6-ff2e-433a-8a03-a911686bd529 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:59.884190 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9c12a6-ff2e-433a-8a03-a911686bd529 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:59.903515 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-6a9c12a6-ff2e-433a-8a03-a911686bd529 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:59.903515 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:59.903515 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:59.903515 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:59.903515 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:59.903515 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:59.903515 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:59.903515 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:59.903515 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:59.903515 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:59.903515 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:59.903515 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:59.903515 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:24:59.903515 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:24:59.903515 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:24:59.904559 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:24:59.904559 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:59.904559 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:59.904559 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:59.904559 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:59.904559 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:59.904559 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:59.904559 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:59.904559 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:24:59.904559 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:59.904559 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:59.904559 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:59.904559 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:59.904559 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:59.904559 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:59.904559 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:59.904559 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:59.905594 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:24:59.905594 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:24:59.905594 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:59.905594 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:24:59.907969 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 420/837] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:24:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:59.917917 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dd8caf-658d-4fee-869f-da430fb5915b None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:00.019673 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c946b67-f583-45a1-92e8-9109e0f10cfa None None] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:00.020026 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c946b67-f583-45a1-92e8-9109e0f10cfa None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:00.020236 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c946b67-f583-45a1-92e8-9109e0f10cfa None None] PATH_INFO: `/` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:00.021331 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dd8caf-658d-4fee-869f-da430fb5915b None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:00.021873 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 421/838] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:25:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:25:00.030081 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba752eec-1d9a-4c0d-83cf-8dcb1d7ee0e2 None None] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:00.030497 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba752eec-1d9a-4c0d-83cf-8dcb1d7ee0e2 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:00.030872 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba752eec-1d9a-4c0d-83cf-8dcb1d7ee0e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:00.051107 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.auth.core [None req-ba752eec-1d9a-4c0d-83cf-8dcb1d7ee0e2 None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143586) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:25:00.052500 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.fernet_utils [None req-ba752eec-1d9a-4c0d-83cf-8dcb1d7ee0e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143586) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:25:00.058033 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:00.075365 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e406b4-c7d3-4282-8a82-48c5a17e17ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:00.076457 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e406b4-c7d3-4282-8a82-48c5a17e17ac None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:00.076958 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e406b4-c7d3-4282-8a82-48c5a17e17ac None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:00.077545 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e406b4-c7d3-4282-8a82-48c5a17e17ac None admin] PATH_INFO: `/v3/domains` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:00.078576 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e406b4-c7d3-4282-8a82-48c5a17e17ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:25:00.079104 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e406b4-c7d3-4282-8a82-48c5a17e17ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:00.080816 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e406b4-c7d3-4282-8a82-48c5a17e17ac None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:00.095064 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 418/839] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:24:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:25:00.100932 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e406b4-c7d3-4282-8a82-48c5a17e17ac None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:00.138232 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 422/840] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:25:00 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:25:00.161835 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba752eec-1d9a-4c0d-83cf-8dcb1d7ee0e2 None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:00.212347 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e406b4-c7d3-4282-8a82-48c5a17e17ac None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:00.250689 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd0af60-bb96-4df9-b76c-9afa189707b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:00.252336 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd0af60-bb96-4df9-b76c-9afa189707b5 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:00.252805 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd0af60-bb96-4df9-b76c-9afa189707b5 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:00.253265 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd0af60-bb96-4df9-b76c-9afa189707b5 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:00.262251 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd0af60-bb96-4df9-b76c-9afa189707b5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:00.264015 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd0af60-bb96-4df9-b76c-9afa189707b5 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:00.279739 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba752eec-1d9a-4c0d-83cf-8dcb1d7ee0e2 None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:00.281643 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-8fd0af60-bb96-4df9-b76c-9afa189707b5 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 14:25:00.281643 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:25:00.281643 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:25:00.281643 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:25:00.281643 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:25:00.281643 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:25:00.281643 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:25:00.281643 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:25:00.281643 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:25:00.281643 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:25:00.281643 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:25:00.281643 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:25:00.281643 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:25:00.281643 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:25:00.281643 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:25:00.281643 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:25:00.282870 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:25:00.282870 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:25:00.282870 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:25:00.282870 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:25:00.282870 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:25:00.282870 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:25:00.282870 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:25:00.282870 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:25:00.282870 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:25:00.282870 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:25:00.282870 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:25:00.282870 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:25:00.282870 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:25:00.282870 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:25:00.282870 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:25:00.282870 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:25:00.282870 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:25:00.282870 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:25:00.283965 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:25:00.283965 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:25:00.283965 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:25:00.283965 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:25:00.283965 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:25:00.283965 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:25:00.283965 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:25:00.283965 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:25:00.283965 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 14:25:00.283965 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:25:00.291264 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 419/841] 10.222.0.57 () {56 vars in 1117 bytes} [Thu May 19 14:25:00 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:25:00.293059 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd0af60-bb96-4df9-b76c-9afa189707b5 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:00.300548 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:00.329055 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28212305-607e-4ae7-8321-adfc40815697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:00.330250 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28212305-607e-4ae7-8321-adfc40815697 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:00.331025 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28212305-607e-4ae7-8321-adfc40815697 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:00.331526 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28212305-607e-4ae7-8321-adfc40815697 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:00.340692 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:00.349183 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28212305-607e-4ae7-8321-adfc40815697 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:25:00.359910 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-28212305-607e-4ae7-8321-adfc40815697 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:25:00.359910 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:25:00.359910 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:25:00.359910 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:25:00.359910 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:25:00.359910 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:25:00.359910 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:25:00.359910 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:25:00.359910 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:25:00.359910 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:25:00.359910 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:25:00.359910 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:25:00.359910 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:25:00.359910 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:25:00.359910 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:25:00.359910 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:25:00.361071 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:25:00.361071 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:25:00.361071 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:25:00.361071 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:25:00.361071 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:25:00.361071 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:25:00.361071 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:25:00.361071 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:25:00.361071 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:25:00.361071 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:25:00.361071 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:25:00.361071 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:25:00.361071 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:25:00.361071 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:25:00.361071 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:25:00.361071 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:25:00.362059 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:25:00.362059 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:25:00.362059 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:25:00.362059 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:25:00.362059 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:25:00.362059 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:25:00.362059 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:25:00.362059 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:25:00.362059 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:25:00.362059 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:25:00.362059 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:25:00.362059 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:25:00.368796 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 423/842] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:25:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:25:00.370280 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28212305-607e-4ae7-8321-adfc40815697 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:00.452525 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd0af60-bb96-4df9-b76c-9afa189707b5 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:00.520307 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf6ec91-8b4a-48ba-ac6e-7a1f7671a5c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:00.520907 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf6ec91-8b4a-48ba-ac6e-7a1f7671a5c2 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:00.521188 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf6ec91-8b4a-48ba-ac6e-7a1f7671a5c2 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:00.521467 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf6ec91-8b4a-48ba-ac6e-7a1f7671a5c2 None admin] PATH_INFO: `/v3/users` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:00.522973 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf6ec91-8b4a-48ba-ac6e-7a1f7671a5c2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:25:00.522973 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf6ec91-8b4a-48ba-ac6e-7a1f7671a5c2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:00.524583 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf6ec91-8b4a-48ba-ac6e-7a1f7671a5c2 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:00.545395 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 420/843] 10.222.0.57 () {56 vars in 1149 bytes} [Thu May 19 14:25:00 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:25:00.551846 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf6ec91-8b4a-48ba-ac6e-7a1f7671a5c2 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:00.557095 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28212305-607e-4ae7-8321-adfc40815697 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:00.613211 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09b0ceb-8612-423a-9143-0dcc9de7927e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:00.614171 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09b0ceb-8612-423a-9143-0dcc9de7927e None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:00.614462 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09b0ceb-8612-423a-9143-0dcc9de7927e None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:00.614749 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09b0ceb-8612-423a-9143-0dcc9de7927e None admin] PATH_INFO: `/v3/roles` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:00.615505 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09b0ceb-8612-423a-9143-0dcc9de7927e None admin] RBAC: Adding query filter params (name=admin) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:25:00.615819 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09b0ceb-8612-423a-9143-0dcc9de7927e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:00.617289 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09b0ceb-8612-423a-9143-0dcc9de7927e None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:00.628002 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 424/844] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:25:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:25:00.629274 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13a8f2d-127b-49ad-b078-8447070dbc6f None None] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:00.629519 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13a8f2d-127b-49ad-b078-8447070dbc6f None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:00.629797 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13a8f2d-127b-49ad-b078-8447070dbc6f None None] PATH_INFO: `/` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:00.635575 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 425/845] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:25:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:25:00.635575 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13a8f2d-127b-49ad-b078-8447070dbc6f None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:00.692717 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:00.695702 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:00.698138 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf6ec91-8b4a-48ba-ac6e-7a1f7671a5c2 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:00.747926 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4b606d-0456-40d0-92ae-6a3e00f1b43b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:00.749269 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4b606d-0456-40d0-92ae-6a3e00f1b43b None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:00.749658 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4b606d-0456-40d0-92ae-6a3e00f1b43b None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:00.750073 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4b606d-0456-40d0-92ae-6a3e00f1b43b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:00.781083 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13a8f2d-127b-49ad-b078-8447070dbc6f None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:00.829092 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc63accb-d32a-4c9b-ae53-9120bbf626de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:00.830342 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc63accb-d32a-4c9b-ae53-9120bbf626de None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:00.830633 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc63accb-d32a-4c9b-ae53-9120bbf626de None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:00.830918 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc63accb-d32a-4c9b-ae53-9120bbf626de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:00.836512 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4b606d-0456-40d0-92ae-6a3e00f1b43b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:00.838172 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4b606d-0456-40d0-92ae-6a3e00f1b43b None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:00.842289 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc63accb-d32a-4c9b-ae53-9120bbf626de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:00.843725 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc63accb-d32a-4c9b-ae53-9120bbf626de None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:00.861391 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-dc63accb-d32a-4c9b-ae53-9120bbf626de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:25:00.861391 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:25:00.861391 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:25:00.861391 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:25:00.861391 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:25:00.861391 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:25:00.861391 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:25:00.861391 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:25:00.861391 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:25:00.861391 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:25:00.861391 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:25:00.861391 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:25:00.861391 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:25:00.861391 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:25:00.861391 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:25:00.862462 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:25:00.862462 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:25:00.862462 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:25:00.862462 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:25:00.862462 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:25:00.862462 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:25:00.862462 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:25:00.862462 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:25:00.862462 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:25:00.862462 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:25:00.862462 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:25:00.862462 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:25:00.862462 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:25:00.862462 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:25:00.862462 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:25:00.862462 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:25:00.862462 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:25:00.863798 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:25:00.863798 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:25:00.863798 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:25:00.863798 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:25:00.863798 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 426/846] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:25:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:25:00.864460 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a60430-c781-4bd6-acfe-3e04e17159fd None None] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:00.864460 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a60430-c781-4bd6-acfe-3e04e17159fd None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:00.864460 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a60430-c781-4bd6-acfe-3e04e17159fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:00.882585 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.auth.core [None req-74a60430-c781-4bd6-acfe-3e04e17159fd None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143586) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:25:00.883775 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.fernet_utils [None req-74a60430-c781-4bd6-acfe-3e04e17159fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143586) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:25:00.937304 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:00.942530 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 421/847] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:25:00 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:25:00.944152 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c6bc38-25fe-4cbc-a6e7-34ed85422842 None None] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:00.944621 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c6bc38-25fe-4cbc-a6e7-34ed85422842 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:00.944897 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c6bc38-25fe-4cbc-a6e7-34ed85422842 None None] PATH_INFO: `/` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:00.946194 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 422/848] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:25:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:25:00.947021 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c6bc38-25fe-4cbc-a6e7-34ed85422842 None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:00.957394 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 427/849] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:25:00 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:25:00.958285 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a60430-c781-4bd6-acfe-3e04e17159fd None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:00.980438 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:01.067339 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a60430-c781-4bd6-acfe-3e04e17159fd None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:01.069402 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c6bc38-25fe-4cbc-a6e7-34ed85422842 None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:01.123809 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85a5348-730c-426f-9770-b84fd48cd3bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:01.124869 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85a5348-730c-426f-9770-b84fd48cd3bd None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:01.125159 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85a5348-730c-426f-9770-b84fd48cd3bd None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:01.125440 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85a5348-730c-426f-9770-b84fd48cd3bd None admin] PATH_INFO: `/v3/domains` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:01.125626 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f8b1ce-dac0-4843-9a4c-aefcdbe4fc45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:01.126795 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85a5348-730c-426f-9770-b84fd48cd3bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:25:01.127102 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85a5348-730c-426f-9770-b84fd48cd3bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:01.127221 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f8b1ce-dac0-4843-9a4c-aefcdbe4fc45 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:01.127391 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f8b1ce-dac0-4843-9a4c-aefcdbe4fc45 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:01.127672 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f8b1ce-dac0-4843-9a4c-aefcdbe4fc45 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:01.128512 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85a5348-730c-426f-9770-b84fd48cd3bd None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:01.142050 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 423/850] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:25:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:25:01.143670 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e4a990-e1f2-4aea-93d9-9a51ebca497f None None] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:01.143670 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e4a990-e1f2-4aea-93d9-9a51ebca497f None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:01.143979 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e4a990-e1f2-4aea-93d9-9a51ebca497f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:01.145395 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f8b1ce-dac0-4843-9a4c-aefcdbe4fc45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:01.147172 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f8b1ce-dac0-4843-9a4c-aefcdbe4fc45 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:01.162679 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.auth.core [None req-20e4a990-e1f2-4aea-93d9-9a51ebca497f None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143587) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:25:01.163885 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-20e4a990-e1f2-4aea-93d9-9a51ebca497f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:25:01.167723 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-a5f8b1ce-dac0-4843-9a4c-aefcdbe4fc45 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:25:01.167723 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:25:01.167723 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:25:01.167723 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:25:01.167723 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:25:01.167723 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:25:01.167723 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:25:01.167723 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:25:01.167723 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:25:01.167723 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:25:01.167723 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:25:01.167723 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:25:01.167723 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:25:01.167723 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:25:01.167723 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:25:01.168867 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:25:01.168867 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:25:01.168867 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:25:01.168867 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:25:01.168867 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:25:01.168867 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:25:01.168867 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:25:01.168867 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:25:01.168867 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:25:01.168867 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:25:01.168867 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:25:01.168867 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:25:01.168867 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:25:01.168867 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:25:01.168867 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:25:01.168867 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:25:01.168867 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:25:01.169928 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:25:01.169928 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:25:01.169928 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:25:01.169928 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:25:01.169928 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 428/851] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:25:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:25:01.169928 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f8b1ce-dac0-4843-9a4c-aefcdbe4fc45 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:01.221075 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:01.235938 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 424/852] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:25:01 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:25:01.236781 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e4a990-e1f2-4aea-93d9-9a51ebca497f None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:01.247149 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:01.271590 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f8b1ce-dac0-4843-9a4c-aefcdbe4fc45 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:01.305430 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278436e6-d554-4b96-b636-a4e9b4a0343e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:01.306266 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278436e6-d554-4b96-b636-a4e9b4a0343e None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:01.306553 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278436e6-d554-4b96-b636-a4e9b4a0343e None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:01.306833 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278436e6-d554-4b96-b636-a4e9b4a0343e None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:01.308131 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278436e6-d554-4b96-b636-a4e9b4a0343e None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:01.309495 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278436e6-d554-4b96-b636-a4e9b4a0343e None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:01.321233 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-278436e6-d554-4b96-b636-a4e9b4a0343e None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 14:25:01.321233 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:25:01.321233 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:25:01.321233 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:25:01.321233 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:25:01.321233 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:25:01.321233 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:25:01.321233 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:25:01.321233 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 14:25:01.321233 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:25:01.321233 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:25:01.321233 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:25:01.321233 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:25:01.321233 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:25:01.321233 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:25:01.321233 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:25:01.321233 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:25:01.322565 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:25:01.322565 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:25:01.322565 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:25:01.322565 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:25:01.322565 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:25:01.322565 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:25:01.322565 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:25:01.322565 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:25:01.322565 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:25:01.322565 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:25:01.322565 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:25:01.322565 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:25:01.322565 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:25:01.322565 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:25:01.322565 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:25:01.322565 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:25:01.323758 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:25:01.323758 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:25:01.323758 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:25:01.323758 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:25:01.323758 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:25:01.323758 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:25:01.323758 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:25:01.323758 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:25:01.323758 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:25:01.323758 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:25:01.323758 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:25:01.323758 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:25:01.323758 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 14:25:01.323758 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:25:01.323758 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 429/853] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 19 14:25:01 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:25:01.324663 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278436e6-d554-4b96-b636-a4e9b4a0343e None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:01.348590 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e4a990-e1f2-4aea-93d9-9a51ebca497f None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:01.384166 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151ed24f-d7ed-46cc-bc40-3b6d5f1725c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:01.384884 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151ed24f-d7ed-46cc-bc40-3b6d5f1725c7 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:01.385169 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151ed24f-d7ed-46cc-bc40-3b6d5f1725c7 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:01.386138 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151ed24f-d7ed-46cc-bc40-3b6d5f1725c7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:01.393284 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151ed24f-d7ed-46cc-bc40-3b6d5f1725c7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:01.394758 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151ed24f-d7ed-46cc-bc40-3b6d5f1725c7 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:01.410152 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-151ed24f-d7ed-46cc-bc40-3b6d5f1725c7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 14:25:01.410152 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:25:01.410152 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:25:01.410152 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:25:01.410152 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:25:01.410152 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:25:01.410152 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:25:01.410152 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:25:01.410152 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:25:01.410152 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:25:01.410152 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:25:01.410152 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:25:01.410152 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:25:01.410152 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:25:01.410152 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:25:01.410152 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:25:01.411362 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:25:01.411362 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:25:01.411362 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:25:01.411362 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:25:01.411362 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:25:01.411362 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:25:01.411362 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:25:01.411362 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:25:01.411362 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:25:01.411362 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:25:01.411362 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:25:01.411362 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:25:01.411362 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:25:01.411362 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:25:01.411362 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:25:01.411362 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:25:01.411362 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:25:01.411362 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:25:01.412428 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:25:01.412428 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:25:01.412428 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:25:01.412428 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:25:01.412428 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:25:01.412428 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:25:01.412428 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:25:01.412428 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:25:01.412428 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 14:25:01.412428 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:25:01.412428 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 425/854] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:25:01 2022] GET /identity/v3/users/swift => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:25:01.412428 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151ed24f-d7ed-46cc-bc40-3b6d5f1725c7 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:01.425628 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278436e6-d554-4b96-b636-a4e9b4a0343e None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:01.460363 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3c7497-2929-488e-b950-e97fb8689e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:01.461024 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3c7497-2929-488e-b950-e97fb8689e8a None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:01.461313 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3c7497-2929-488e-b950-e97fb8689e8a None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:01.461463 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:01.461658 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3c7497-2929-488e-b950-e97fb8689e8a None admin] PATH_INFO: `/v3/domains` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:01.462427 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3c7497-2929-488e-b950-e97fb8689e8a None admin] RBAC: Adding query filter params (name=Default) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:25:01.462664 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3c7497-2929-488e-b950-e97fb8689e8a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:01.464596 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3c7497-2929-488e-b950-e97fb8689e8a None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:01.466940 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:01.477605 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 430/855] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:25:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:25:01.478446 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3c7497-2929-488e-b950-e97fb8689e8a None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:01.516940 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151ed24f-d7ed-46cc-bc40-3b6d5f1725c7 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:01.550114 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea17e3f1-9e41-4e6e-9bd1-a4e8c59aa1a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:01.550922 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea17e3f1-9e41-4e6e-9bd1-a4e8c59aa1a6 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:01.551202 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea17e3f1-9e41-4e6e-9bd1-a4e8c59aa1a6 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:01.551474 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea17e3f1-9e41-4e6e-9bd1-a4e8c59aa1a6 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:01.552151 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea17e3f1-9e41-4e6e-9bd1-a4e8c59aa1a6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:01.553525 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea17e3f1-9e41-4e6e-9bd1-a4e8c59aa1a6 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:01.567927 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-ea17e3f1-9e41-4e6e-9bd1-a4e8c59aa1a6 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 14:25:01.567927 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:25:01.567927 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:25:01.567927 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:25:01.567927 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:25:01.567927 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:25:01.567927 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:25:01.567927 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:25:01.567927 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 14:25:01.567927 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:25:01.567927 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:25:01.567927 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:25:01.567927 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:25:01.567927 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:25:01.567927 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:25:01.567927 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:25:01.574848 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:25:01.574848 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:25:01.574848 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:25:01.574848 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:25:01.574848 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:25:01.574848 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:25:01.574848 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:25:01.574848 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:25:01.574848 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:25:01.574848 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:25:01.574848 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:25:01.574848 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:25:01.574848 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:25:01.574848 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:25:01.574848 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:25:01.574848 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:25:01.576047 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:25:01.576047 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:25:01.576047 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:25:01.576047 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:25:01.576047 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:25:01.576047 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:25:01.576047 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:25:01.576047 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:25:01.576047 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:25:01.576047 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:25:01.576047 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:25:01.576047 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:25:01.576047 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:25:01.576047 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 14:25:01.576047 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:25:01.576047 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 426/856] 10.222.0.57 () {56 vars in 1132 bytes} [Thu May 19 14:25:01 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:25:01.576991 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea17e3f1-9e41-4e6e-9bd1-a4e8c59aa1a6 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:01.579217 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3c7497-2929-488e-b950-e97fb8689e8a None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:01.613835 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc63d75e-717f-4553-a976-ab6c13c33ac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:01.614517 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc63d75e-717f-4553-a976-ab6c13c33ac0 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:01.614802 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc63d75e-717f-4553-a976-ab6c13c33ac0 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:01.615076 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc63d75e-717f-4553-a976-ab6c13c33ac0 None admin] PATH_INFO: `/v3/services` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:01.615779 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc63d75e-717f-4553-a976-ab6c13c33ac0 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:25:01.616092 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc63d75e-717f-4553-a976-ab6c13c33ac0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:01.617446 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc63d75e-717f-4553-a976-ab6c13c33ac0 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:01.622729 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 431/857] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 19 14:25:01 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:25:01.623517 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc63d75e-717f-4553-a976-ab6c13c33ac0 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:01.675937 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea17e3f1-9e41-4e6e-9bd1-a4e8c59aa1a6 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:01.681823 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:01.689997 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:01.712924 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ea8a5f-5df6-4ad5-a4ba-716c7da8a343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:01.713619 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ea8a5f-5df6-4ad5-a4ba-716c7da8a343 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:01.714206 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ea8a5f-5df6-4ad5-a4ba-716c7da8a343 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:01.714577 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ea8a5f-5df6-4ad5-a4ba-716c7da8a343 None admin] PATH_INFO: `/v3/users` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:01.715424 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ea8a5f-5df6-4ad5-a4ba-716c7da8a343 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:25:01.715746 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ea8a5f-5df6-4ad5-a4ba-716c7da8a343 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:01.717191 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ea8a5f-5df6-4ad5-a4ba-716c7da8a343 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:01.724882 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc63d75e-717f-4553-a976-ab6c13c33ac0 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:01.734784 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 427/858] 10.222.0.57 () {56 vars in 1135 bytes} [Thu May 19 14:25:01 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:25:01.735604 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ea8a5f-5df6-4ad5-a4ba-716c7da8a343 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:01.764235 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9297a15a-3359-4faf-8139-e6048e9b0ac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:01.765026 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9297a15a-3359-4faf-8139-e6048e9b0ac4 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:01.765507 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9297a15a-3359-4faf-8139-e6048e9b0ac4 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:01.765632 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9297a15a-3359-4faf-8139-e6048e9b0ac4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:01.772323 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9297a15a-3359-4faf-8139-e6048e9b0ac4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:01.773772 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9297a15a-3359-4faf-8139-e6048e9b0ac4 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:01.787743 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-9297a15a-3359-4faf-8139-e6048e9b0ac4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:25:01.787743 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:25:01.787743 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:25:01.787743 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:25:01.787743 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:25:01.787743 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:25:01.787743 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:25:01.787743 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:25:01.787743 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:25:01.787743 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:25:01.787743 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:25:01.787743 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:25:01.787743 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:25:01.787743 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:25:01.787743 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:25:01.788835 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:25:01.788835 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:25:01.788835 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:25:01.788835 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:25:01.788835 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:25:01.788835 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:25:01.788835 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:25:01.788835 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:25:01.788835 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:25:01.788835 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:25:01.788835 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:25:01.788835 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:25:01.788835 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:25:01.788835 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:25:01.788835 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:25:01.788835 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:25:01.788835 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:25:01.789828 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:25:01.789828 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:25:01.789828 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:25:01.789828 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:25:01.789828 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:25:01.789828 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:25:01.789828 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:25:01.789828 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:25:01.789828 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 432/859] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:25:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:25:01.789828 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9297a15a-3359-4faf-8139-e6048e9b0ac4 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:01.840262 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ea8a5f-5df6-4ad5-a4ba-716c7da8a343 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:01.874920 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9df1e9-8966-49e3-b227-10d87e206409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:01.875591 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9df1e9-8966-49e3-b227-10d87e206409 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:01.875876 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9df1e9-8966-49e3-b227-10d87e206409 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:01.876158 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9df1e9-8966-49e3-b227-10d87e206409 None admin] PATH_INFO: `/v3/services` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:01.876866 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9df1e9-8966-49e3-b227-10d87e206409 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:25:01.877186 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9df1e9-8966-49e3-b227-10d87e206409 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:01.878598 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9df1e9-8966-49e3-b227-10d87e206409 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:01.883208 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 428/860] 10.222.0.57 () {56 vars in 1126 bytes} [Thu May 19 14:25:01 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:25:01.902076 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9df1e9-8966-49e3-b227-10d87e206409 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:01.905063 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9297a15a-3359-4faf-8139-e6048e9b0ac4 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:01.927009 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:01.929506 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:01.942156 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9c1674-4953-4820-8211-d23dec5a84e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:01.942828 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9c1674-4953-4820-8211-d23dec5a84e4 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:01.943112 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9c1674-4953-4820-8211-d23dec5a84e4 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:01.943390 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9c1674-4953-4820-8211-d23dec5a84e4 None admin] PATH_INFO: `/v3/services` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:01.944103 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9c1674-4953-4820-8211-d23dec5a84e4 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:25:01.944417 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9c1674-4953-4820-8211-d23dec5a84e4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:01.945782 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9c1674-4953-4820-8211-d23dec5a84e4 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:01.950287 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 433/861] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 19 14:25:01 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:25:01.951134 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9c1674-4953-4820-8211-d23dec5a84e4 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:02.005316 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9df1e9-8966-49e3-b227-10d87e206409 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:02.043618 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d349295-e9e1-4b93-bc0c-d4dc3140ae91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:02.044290 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d349295-e9e1-4b93-bc0c-d4dc3140ae91 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:02.044574 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d349295-e9e1-4b93-bc0c-d4dc3140ae91 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:02.044862 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d349295-e9e1-4b93-bc0c-d4dc3140ae91 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:02.084320 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9c1674-4953-4820-8211-d23dec5a84e4 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:02.122930 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c1c425-5a95-4543-a3c2-d8dc13fa1547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:02.123610 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c1c425-5a95-4543-a3c2-d8dc13fa1547 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:02.123897 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c1c425-5a95-4543-a3c2-d8dc13fa1547 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:02.124228 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c1c425-5a95-4543-a3c2-d8dc13fa1547 None admin] PATH_INFO: `/v3/projects` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:02.125001 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c1c425-5a95-4543-a3c2-d8dc13fa1547 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:25:02.125314 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c1c425-5a95-4543-a3c2-d8dc13fa1547 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:02.126795 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c1c425-5a95-4543-a3c2-d8dc13fa1547 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:02.139934 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 434/862] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:25:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:25:02.140724 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c1c425-5a95-4543-a3c2-d8dc13fa1547 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:02.141760 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d349295-e9e1-4b93-bc0c-d4dc3140ae91 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:02.143383 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d349295-e9e1-4b93-bc0c-d4dc3140ae91 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:02.192810 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:02.194101 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:02.252241 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c1c425-5a95-4543-a3c2-d8dc13fa1547 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:02.266093 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 429/863] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:25:01 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:25:02.266946 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d349295-e9e1-4b93-bc0c-d4dc3140ae91 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:02.293678 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc9ffde-6949-493f-ab10-0c50f254043b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:02.294632 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc9ffde-6949-493f-ab10-0c50f254043b None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:02.295093 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc9ffde-6949-493f-ab10-0c50f254043b None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:02.295371 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc9ffde-6949-493f-ab10-0c50f254043b None admin] PATH_INFO: `/v3/services` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:02.296077 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc9ffde-6949-493f-ab10-0c50f254043b None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:25:02.296379 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc9ffde-6949-493f-ab10-0c50f254043b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:02.297736 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc9ffde-6949-493f-ab10-0c50f254043b None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:02.309882 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 435/864] 10.222.0.57 () {56 vars in 1126 bytes} [Thu May 19 14:25:02 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:25:02.309882 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc9ffde-6949-493f-ab10-0c50f254043b None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:02.403330 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d349295-e9e1-4b93-bc0c-d4dc3140ae91 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:02.443545 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a846afd-c9c2-427d-a16f-0a48ce317430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:02.444320 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a846afd-c9c2-427d-a16f-0a48ce317430 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:02.444702 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a846afd-c9c2-427d-a16f-0a48ce317430 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:02.445054 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a846afd-c9c2-427d-a16f-0a48ce317430 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:02.446007 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a846afd-c9c2-427d-a16f-0a48ce317430 None admin] RBAC: Adding query filter params (role.id=b0b74fefb27e4a0ebd895893aa9862d7, scope.project.id=252d9673f08c479a98d04fee3f5856a4, user.id=7977ff2f29eb435cbfeaa0b61d1d7a5d) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:25:02.446389 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a846afd-c9c2-427d-a16f-0a48ce317430 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:02.448136 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a846afd-c9c2-427d-a16f-0a48ce317430 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:02.450776 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:02.456364 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 430/865] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:25:02 2022] GET /identity/v3/role_assignments?user.id=7977ff2f29eb435cbfeaa0b61d1d7a5d&scope.project.id=252d9673f08c479a98d04fee3f5856a4&role.id=b0b74fefb27e4a0ebd895893aa9862d7 => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:25:02.466222 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc9ffde-6949-493f-ab10-0c50f254043b None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:02.480150 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:02.512992 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bfb6a3-ce97-4ae9-9e33-608cf4c442be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:02.513804 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bfb6a3-ce97-4ae9-9e33-608cf4c442be None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:02.514119 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bfb6a3-ce97-4ae9-9e33-608cf4c442be None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:02.514395 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bfb6a3-ce97-4ae9-9e33-608cf4c442be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:02.526268 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bfb6a3-ce97-4ae9-9e33-608cf4c442be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:02.527861 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bfb6a3-ce97-4ae9-9e33-608cf4c442be None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:02.549053 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-b8bfb6a3-ce97-4ae9-9e33-608cf4c442be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:25:02.549053 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:25:02.549053 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:25:02.549053 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:25:02.549053 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:25:02.549053 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:25:02.549053 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:25:02.549053 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:25:02.549053 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:25:02.549053 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:25:02.549053 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:25:02.549053 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:25:02.549053 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:25:02.549053 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:25:02.549053 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:25:02.549858 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:25:02.549858 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:25:02.549858 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:25:02.549858 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:25:02.549858 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:25:02.549858 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:25:02.549858 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:25:02.549858 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:25:02.549858 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:25:02.549858 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:25:02.549858 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:25:02.549858 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:25:02.549858 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:25:02.549858 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:25:02.549858 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:25:02.549858 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:25:02.549858 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:25:02.550584 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:25:02.550584 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:25:02.550584 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:25:02.550584 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:25:02.550776 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 436/866] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:25:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:25:02.557132 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a846afd-c9c2-427d-a16f-0a48ce317430 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:02.670591 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a846afd-c9c2-427d-a16f-0a48ce317430 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:02.707756 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aca387-89dd-4be0-9379-ccbcfab445d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:02.708054 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aca387-89dd-4be0-9379-ccbcfab445d8 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:02.708237 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aca387-89dd-4be0-9379-ccbcfab445d8 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:02.708524 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aca387-89dd-4be0-9379-ccbcfab445d8 None admin] PATH_INFO: `/v3/domains` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:02.709293 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95aca387-89dd-4be0-9379-ccbcfab445d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:25:02.709568 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95aca387-89dd-4be0-9379-ccbcfab445d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:02.710999 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95aca387-89dd-4be0-9379-ccbcfab445d8 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:02.724209 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 431/867] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:25:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:25:02.730534 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bfb6a3-ce97-4ae9-9e33-608cf4c442be None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:02.838249 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bfb6a3-ce97-4ae9-9e33-608cf4c442be None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:02.876122 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5268a1-4fac-44a2-b070-14b81f44f2fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:02.876983 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5268a1-4fac-44a2-b070-14b81f44f2fa None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:02.877272 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5268a1-4fac-44a2-b070-14b81f44f2fa None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:02.877551 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5268a1-4fac-44a2-b070-14b81f44f2fa None admin] PATH_INFO: `/v3/projects/service` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:02.884899 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5268a1-4fac-44a2-b070-14b81f44f2fa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:02.886496 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5268a1-4fac-44a2-b070-14b81f44f2fa None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:02.900605 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-9a5268a1-4fac-44a2-b070-14b81f44f2fa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:25:02.900605 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:25:02.900605 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:25:02.900605 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:25:02.900605 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:25:02.900605 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:25:02.900605 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:25:02.900605 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:25:02.900605 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:25:02.900605 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:25:02.900605 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:25:02.900605 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:25:02.900605 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:25:02.900605 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:25:02.900605 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:25:02.901895 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:25:02.901895 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:25:02.901895 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:25:02.901895 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:25:02.901895 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:25:02.901895 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:25:02.901895 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:25:02.901895 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:25:02.901895 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:25:02.901895 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:25:02.901895 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:25:02.901895 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:25:02.901895 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:25:02.901895 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:25:02.901895 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:25:02.901895 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:25:02.901895 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:25:02.902632 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:25:02.902632 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:25:02.902632 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:25:02.902632 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:25:02.902632 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:25:02.902632 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:25:02.902632 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:25:02.902632 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:25:02.903003 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 437/868] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:25:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:25:02.909465 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aca387-89dd-4be0-9379-ccbcfab445d8 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:02.982239 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:03.015769 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aca387-89dd-4be0-9379-ccbcfab445d8 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:03.051908 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d00cee-8e1b-4070-ad23-b6a90aaca302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:03.052950 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d00cee-8e1b-4070-ad23-b6a90aaca302 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:03.053429 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d00cee-8e1b-4070-ad23-b6a90aaca302 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:03.053928 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d00cee-8e1b-4070-ad23-b6a90aaca302 None admin] PATH_INFO: `/v3/projects` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:03.054910 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d00cee-8e1b-4070-ad23-b6a90aaca302 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:25:03.055449 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d00cee-8e1b-4070-ad23-b6a90aaca302 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:03.057202 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d00cee-8e1b-4070-ad23-b6a90aaca302 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:03.070417 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 432/869] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:25:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:25:03.077548 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5268a1-4fac-44a2-b070-14b81f44f2fa None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:03.102145 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:03.194143 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5268a1-4fac-44a2-b070-14b81f44f2fa None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:03.232252 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff4cae8-c2e7-496f-a7bd-a85b37e6aca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:03.232969 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff4cae8-c2e7-496f-a7bd-a85b37e6aca1 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:03.233256 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff4cae8-c2e7-496f-a7bd-a85b37e6aca1 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:03.233587 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff4cae8-c2e7-496f-a7bd-a85b37e6aca1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:03.234477 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff4cae8-c2e7-496f-a7bd-a85b37e6aca1 None admin] RBAC: Adding query filter params (role.id=a3b793754e7b40b0bb21afe74eec3c18, scope.project.id=252d9673f08c479a98d04fee3f5856a4, user.id=d3e544b4989e49a19536b40de5069bcf) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:25:03.234796 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff4cae8-c2e7-496f-a7bd-a85b37e6aca1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:03.236246 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff4cae8-c2e7-496f-a7bd-a85b37e6aca1 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:03.243092 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 438/870] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:25:03 2022] GET /identity/v3/role_assignments?user.id=d3e544b4989e49a19536b40de5069bcf&scope.project.id=252d9673f08c479a98d04fee3f5856a4&role.id=a3b793754e7b40b0bb21afe74eec3c18 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:25:04.277558 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8ad3a5-5914-4f1f-b817-9dd6f2880e66 None None] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:04.278098 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8ad3a5-5914-4f1f-b817-9dd6f2880e66 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:04.278167 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8ad3a5-5914-4f1f-b817-9dd6f2880e66 None None] PATH_INFO: `/` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:04.279647 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 433/871] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:25:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:25:04.285377 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3961e488-8daa-4b19-8629-794cdcff8cb6 None None] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:04.285769 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3961e488-8daa-4b19-8629-794cdcff8cb6 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:04.285952 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3961e488-8daa-4b19-8629-794cdcff8cb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:04.303269 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.auth.core [None req-3961e488-8daa-4b19-8629-794cdcff8cb6 None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143586) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:25:04.304453 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.fernet_utils [None req-3961e488-8daa-4b19-8629-794cdcff8cb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143586) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:25:04.379751 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 439/872] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:25:04 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:25:04.403448 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8ad3a5-5914-4f1f-b817-9dd6f2880e66 None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:04.520836 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8ad3a5-5914-4f1f-b817-9dd6f2880e66 None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:04.539412 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:04.557103 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9974ca-654e-4654-8b51-7323a4d9fa1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:04.557934 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9974ca-654e-4654-8b51-7323a4d9fa1a None admin] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:04.558464 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9974ca-654e-4654-8b51-7323a4d9fa1a None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:04.558916 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9974ca-654e-4654-8b51-7323a4d9fa1a None admin] PATH_INFO: `/v3/services` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:04.559652 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9974ca-654e-4654-8b51-7323a4d9fa1a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:04.561055 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9974ca-654e-4654-8b51-7323a4d9fa1a None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:04.578978 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 434/873] 10.222.0.57 () {60 vars in 1141 bytes} [Thu May 19 14:25:04 2022] POST /identity/v3/services => generated 251 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:25:04.692196 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3290a435-3b08-4c44-8b65-17b8f879c95c None None] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:04.692511 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3290a435-3b08-4c44-8b65-17b8f879c95c None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:04.692754 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3290a435-3b08-4c44-8b65-17b8f879c95c None None] PATH_INFO: `/` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:04.694635 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 440/874] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:25:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:25:04.701194 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7921136c-0030-42c2-b73e-e7ab9c559a0e None None] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:04.701491 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7921136c-0030-42c2-b73e-e7ab9c559a0e None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:04.701776 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7921136c-0030-42c2-b73e-e7ab9c559a0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:04.721013 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.auth.core [None req-7921136c-0030-42c2-b73e-e7ab9c559a0e None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143587) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:25:04.722976 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-7921136c-0030-42c2-b73e-e7ab9c559a0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:25:04.811381 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 435/875] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:25:04 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:25:04.835230 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3290a435-3b08-4c44-8b65-17b8f879c95c None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:04.852749 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:04.933146 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d90ecc-09f8-425a-87de-993619f22096 None None] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:04.933427 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d90ecc-09f8-425a-87de-993619f22096 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:04.933687 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d90ecc-09f8-425a-87de-993619f22096 None None] PATH_INFO: `/` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:04.935358 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 436/876] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:25:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:25:04.941188 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552fa57a-4869-4530-9610-35f5d57e99c3 None None] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:04.941688 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552fa57a-4869-4530-9610-35f5d57e99c3 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:04.942186 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552fa57a-4869-4530-9610-35f5d57e99c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:04.947532 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3290a435-3b08-4c44-8b65-17b8f879c95c None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:04.960618 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.auth.core [None req-552fa57a-4869-4530-9610-35f5d57e99c3 None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143587) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:25:04.962135 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-552fa57a-4869-4530-9610-35f5d57e99c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:25:04.989314 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded592b3-427f-4441-bd03-490a26b0cec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:04.990010 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded592b3-427f-4441-bd03-490a26b0cec0 None admin] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:04.990293 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded592b3-427f-4441-bd03-490a26b0cec0 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:04.990566 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded592b3-427f-4441-bd03-490a26b0cec0 None admin] PATH_INFO: `/v3/services` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:04.991303 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded592b3-427f-4441-bd03-490a26b0cec0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:04.992662 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded592b3-427f-4441-bd03-490a26b0cec0 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:05.011366 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 441/877] 10.222.0.57 () {60 vars in 1141 bytes} [Thu May 19 14:25:04 2022] POST /identity/v3/services => generated 269 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:25:05.056539 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 437/878] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:25:04 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:25:05.082952 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded592b3-427f-4441-bd03-490a26b0cec0 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:05.177036 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:05.209602 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded592b3-427f-4441-bd03-490a26b0cec0 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:05.248240 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e207abc-d400-47ff-aa5b-d4ae1903ff4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:05.249064 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e207abc-d400-47ff-aa5b-d4ae1903ff4a None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:05.249352 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e207abc-d400-47ff-aa5b-d4ae1903ff4a None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:05.249630 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e207abc-d400-47ff-aa5b-d4ae1903ff4a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:05.262173 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e207abc-d400-47ff-aa5b-d4ae1903ff4a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:25:05.270523 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-0e207abc-d400-47ff-aa5b-d4ae1903ff4a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 14:25:05.270523 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:25:05.270523 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:25:05.270523 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:25:05.270523 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:25:05.270523 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:25:05.270523 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:25:05.270523 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:25:05.270523 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:25:05.270523 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:25:05.270523 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:25:05.270523 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:25:05.270523 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:25:05.270523 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:25:05.270523 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:25:05.270523 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:25:05.271298 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:25:05.271298 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:25:05.271298 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:25:05.271298 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:25:05.271298 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:25:05.271298 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:25:05.271298 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:25:05.271298 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:25:05.271298 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:25:05.271298 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:25:05.271298 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:25:05.271298 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:25:05.271298 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:25:05.271298 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:25:05.271298 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:25:05.271298 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:25:05.272427 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:25:05.272427 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:25:05.272427 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:25:05.272427 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:25:05.272427 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:25:05.272427 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:25:05.272427 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:25:05.272427 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:25:05.272427 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:25:05.272427 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:25:05.272427 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 14:25:05.272427 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:25:05.272427 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 442/879] 10.222.0.57 () {56 vars in 1120 bytes} [Thu May 19 14:25:05 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:25:05.278140 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552fa57a-4869-4530-9610-35f5d57e99c3 None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:05.312238 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:05.392468 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552fa57a-4869-4530-9610-35f5d57e99c3 None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:05.429254 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e850f5e-84c8-47e9-91ce-09e5f2363432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:05.430295 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e850f5e-84c8-47e9-91ce-09e5f2363432 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:05.430780 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e850f5e-84c8-47e9-91ce-09e5f2363432 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:05.431231 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e850f5e-84c8-47e9-91ce-09e5f2363432 None admin] PATH_INFO: `/v3/roles` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:05.432330 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e850f5e-84c8-47e9-91ce-09e5f2363432 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:25:05.432835 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e850f5e-84c8-47e9-91ce-09e5f2363432 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:05.434519 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e850f5e-84c8-47e9-91ce-09e5f2363432 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:05.444427 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 438/880] 10.222.0.57 () {56 vars in 1115 bytes} [Thu May 19 14:25:05 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:25:05.450939 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e207abc-d400-47ff-aa5b-d4ae1903ff4a None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:05.562910 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e207abc-d400-47ff-aa5b-d4ae1903ff4a None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:05.599429 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bf50f2-af67-46ab-8bfe-bb0ede1f4830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:05.600234 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bf50f2-af67-46ab-8bfe-bb0ede1f4830 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:05.600517 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bf50f2-af67-46ab-8bfe-bb0ede1f4830 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:05.600797 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bf50f2-af67-46ab-8bfe-bb0ede1f4830 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:05.602833 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:05.613080 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bf50f2-af67-46ab-8bfe-bb0ede1f4830 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:05.614588 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bf50f2-af67-46ab-8bfe-bb0ede1f4830 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:05.641214 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-42bf50f2-af67-46ab-8bfe-bb0ede1f4830 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:25:05.641214 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:25:05.641214 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:25:05.641214 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:25:05.641214 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:25:05.641214 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:25:05.641214 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:25:05.641214 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:25:05.641214 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:25:05.641214 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:25:05.641214 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:25:05.641214 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:25:05.641214 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:25:05.641214 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:25:05.641214 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:25:05.642043 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:25:05.642043 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:25:05.642043 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:25:05.642043 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:25:05.642043 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:25:05.642043 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:25:05.642043 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:25:05.642043 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:25:05.642043 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:25:05.642043 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:25:05.642043 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:25:05.642043 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:25:05.642043 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:25:05.642043 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:25:05.642043 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:25:05.642043 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:25:05.642043 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:25:05.642822 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:25:05.642822 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:25:05.642822 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:25:05.642822 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:25:05.643457 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237f85de-c7a2-49ca-93d4-424942208728 None None] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:05.643984 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237f85de-c7a2-49ca-93d4-424942208728 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:05.644467 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237f85de-c7a2-49ca-93d4-424942208728 None None] PATH_INFO: `/` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:05.645083 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 443/881] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:25:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:25:05.646543 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 439/882] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:25:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:25:05.652054 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bf50f2-af67-46ab-8bfe-bb0ede1f4830 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:05.652839 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583297fb-e72e-4697-807a-67e396714c05 None None] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:05.653322 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583297fb-e72e-4697-807a-67e396714c05 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:05.653874 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583297fb-e72e-4697-807a-67e396714c05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:05.673164 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.auth.core [None req-583297fb-e72e-4697-807a-67e396714c05 None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143587) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:25:05.674758 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-583297fb-e72e-4697-807a-67e396714c05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:25:05.765876 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 440/883] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:25:05 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:25:05.776600 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bf50f2-af67-46ab-8bfe-bb0ede1f4830 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:05.785139 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583297fb-e72e-4697-807a-67e396714c05 None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:05.816187 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef65535f-5d4a-42a0-9811-ff5312a62a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:05.817185 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef65535f-5d4a-42a0-9811-ff5312a62a42 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:05.817710 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef65535f-5d4a-42a0-9811-ff5312a62a42 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:05.818132 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:05.818419 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef65535f-5d4a-42a0-9811-ff5312a62a42 None admin] PATH_INFO: `/v3/domains` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:05.819331 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef65535f-5d4a-42a0-9811-ff5312a62a42 None admin] RBAC: Adding query filter params (name=Default) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:25:05.819828 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef65535f-5d4a-42a0-9811-ff5312a62a42 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:05.821485 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef65535f-5d4a-42a0-9811-ff5312a62a42 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:05.835498 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 444/884] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:25:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:25:05.844264 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef65535f-5d4a-42a0-9811-ff5312a62a42 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:05.899204 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:05.923961 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583297fb-e72e-4697-807a-67e396714c05 None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:05.961740 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441ad514-2843-44ea-8be9-550dcf16b6a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:05.962981 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441ad514-2843-44ea-8be9-550dcf16b6a8 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:05.963659 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441ad514-2843-44ea-8be9-550dcf16b6a8 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:05.964351 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441ad514-2843-44ea-8be9-550dcf16b6a8 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:05.965349 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441ad514-2843-44ea-8be9-550dcf16b6a8 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:05.967028 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441ad514-2843-44ea-8be9-550dcf16b6a8 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:05.967894 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef65535f-5d4a-42a0-9811-ff5312a62a42 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:05.980973 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-441ad514-2843-44ea-8be9-550dcf16b6a8 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 14:25:05.980973 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:25:05.980973 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:25:05.980973 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:25:05.980973 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:25:05.980973 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:25:05.980973 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:25:05.980973 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:25:05.980973 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 14:25:05.980973 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:25:05.980973 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:25:05.980973 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:25:05.980973 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:25:05.980973 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:25:05.980973 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:25:05.980973 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:25:05.980973 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:25:05.982247 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:25:05.982247 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:25:05.982247 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:25:05.982247 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:25:05.982247 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:25:05.982247 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:25:05.982247 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:25:05.982247 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:25:05.982247 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:25:05.982247 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:25:05.982247 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:25:05.982247 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:25:05.982247 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:25:05.982247 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:25:05.982247 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:25:05.982247 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:25:05.983010 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:25:05.983010 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:25:05.983010 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:25:05.983010 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:25:05.983010 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:25:05.983010 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:25:05.983010 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:25:05.983010 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:25:05.983010 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:25:05.983010 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:25:05.983010 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:25:05.983010 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:25:05.983010 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 14:25:05.983010 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:25:05.992417 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 441/885] 10.222.0.57 () {56 vars in 1126 bytes} [Thu May 19 14:25:05 2022] GET /identity/v3/services/object-store => generated 93 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:25:05.993738 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441ad514-2843-44ea-8be9-550dcf16b6a8 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:06.010974 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289dcccc-c5f1-4b5f-8de6-ab0f05fca048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:06.011797 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289dcccc-c5f1-4b5f-8de6-ab0f05fca048 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:06.012084 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289dcccc-c5f1-4b5f-8de6-ab0f05fca048 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:06.012357 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289dcccc-c5f1-4b5f-8de6-ab0f05fca048 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:06.021882 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289dcccc-c5f1-4b5f-8de6-ab0f05fca048 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:06.023940 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289dcccc-c5f1-4b5f-8de6-ab0f05fca048 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:06.043204 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-289dcccc-c5f1-4b5f-8de6-ab0f05fca048 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 14:25:06.043204 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:25:06.043204 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:25:06.043204 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:25:06.043204 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:25:06.043204 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:25:06.043204 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:25:06.043204 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:25:06.043204 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:25:06.043204 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:25:06.043204 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:25:06.043204 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:25:06.043204 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:25:06.043204 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:25:06.043204 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:25:06.043204 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:25:06.044065 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:25:06.044065 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:25:06.044065 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:25:06.044065 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:25:06.044065 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:25:06.044065 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:25:06.044065 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:25:06.044065 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:25:06.044065 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:25:06.044065 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:25:06.044065 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:25:06.044065 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:25:06.044065 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:25:06.044065 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:25:06.044065 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:25:06.044065 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:25:06.044065 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:25:06.044065 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:25:06.044887 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:25:06.044887 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:25:06.044887 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:25:06.044887 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:25:06.044887 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:25:06.044887 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:25:06.044887 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:25:06.044887 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:25:06.044887 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 14:25:06.044887 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:25:06.049907 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 445/886] 10.222.0.57 () {56 vars in 1117 bytes} [Thu May 19 14:25:05 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:25:06.060489 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289dcccc-c5f1-4b5f-8de6-ab0f05fca048 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:06.117967 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:06.143345 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441ad514-2843-44ea-8be9-550dcf16b6a8 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:06.179333 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289dcccc-c5f1-4b5f-8de6-ab0f05fca048 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:06.189596 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:06.200525 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956a657c-783c-4c79-bea6-53ffb1e9424e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:06.201201 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956a657c-783c-4c79-bea6-53ffb1e9424e None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:06.202989 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956a657c-783c-4c79-bea6-53ffb1e9424e None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:06.203270 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956a657c-783c-4c79-bea6-53ffb1e9424e None admin] PATH_INFO: `/v3/services` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:06.203992 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956a657c-783c-4c79-bea6-53ffb1e9424e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:25:06.204304 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956a657c-783c-4c79-bea6-53ffb1e9424e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:06.205710 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956a657c-783c-4c79-bea6-53ffb1e9424e None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:06.215675 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 442/887] 10.222.0.57 () {56 vars in 1122 bytes} [Thu May 19 14:25:05 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:25:06.224320 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956a657c-783c-4c79-bea6-53ffb1e9424e None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:06.238015 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51b60a2-d347-448c-a625-e958a81acc66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:06.239083 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51b60a2-d347-448c-a625-e958a81acc66 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:06.239375 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51b60a2-d347-448c-a625-e958a81acc66 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:06.239659 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51b60a2-d347-448c-a625-e958a81acc66 None admin] PATH_INFO: `/v3/users` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:06.240519 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51b60a2-d347-448c-a625-e958a81acc66 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:25:06.240833 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51b60a2-d347-448c-a625-e958a81acc66 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:06.243520 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51b60a2-d347-448c-a625-e958a81acc66 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:06.268828 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 446/888] 10.222.0.57 () {56 vars in 1149 bytes} [Thu May 19 14:25:06 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:25:06.275426 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51b60a2-d347-448c-a625-e958a81acc66 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:06.383185 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956a657c-783c-4c79-bea6-53ffb1e9424e None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:06.417753 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51b60a2-d347-448c-a625-e958a81acc66 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:06.430084 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37b41cf-4002-442e-8a5b-a9af0c4c7300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:06.431110 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37b41cf-4002-442e-8a5b-a9af0c4c7300 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:06.431624 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37b41cf-4002-442e-8a5b-a9af0c4c7300 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:06.432311 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37b41cf-4002-442e-8a5b-a9af0c4c7300 None admin] PATH_INFO: `/v3/services` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:06.433354 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37b41cf-4002-442e-8a5b-a9af0c4c7300 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:25:06.433923 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37b41cf-4002-442e-8a5b-a9af0c4c7300 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:06.435575 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37b41cf-4002-442e-8a5b-a9af0c4c7300 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:06.441559 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 443/889] 10.222.0.57 () {56 vars in 1122 bytes} [Thu May 19 14:25:06 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:25:06.456397 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9576844-3c1a-4605-b038-50a5718f3777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:06.457091 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9576844-3c1a-4605-b038-50a5718f3777 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:06.457371 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9576844-3c1a-4605-b038-50a5718f3777 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:06.457660 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9576844-3c1a-4605-b038-50a5718f3777 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:06.495911 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:06.540003 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9576844-3c1a-4605-b038-50a5718f3777 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:06.542182 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9576844-3c1a-4605-b038-50a5718f3777 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:06.658699 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 447/890] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:25:06 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:25:06.669682 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37b41cf-4002-442e-8a5b-a9af0c4c7300 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:06.690587 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:06.799060 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37b41cf-4002-442e-8a5b-a9af0c4c7300 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:06.840444 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89fddce-f3a3-4c4b-9c95-c92a184c19be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:06.841636 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89fddce-f3a3-4c4b-9c95-c92a184c19be None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:06.842207 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89fddce-f3a3-4c4b-9c95-c92a184c19be None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:06.842688 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89fddce-f3a3-4c4b-9c95-c92a184c19be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:06.856309 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89fddce-f3a3-4c4b-9c95-c92a184c19be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:06.858116 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89fddce-f3a3-4c4b-9c95-c92a184c19be None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:06.878832 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-a89fddce-f3a3-4c4b-9c95-c92a184c19be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:25:06.878832 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:25:06.878832 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:25:06.878832 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:25:06.878832 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:25:06.878832 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:25:06.878832 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:25:06.878832 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:25:06.878832 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:25:06.878832 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:25:06.878832 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:25:06.878832 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:25:06.878832 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:25:06.878832 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:25:06.878832 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:25:06.879668 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:25:06.879668 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:25:06.879668 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:25:06.879668 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:25:06.879668 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:25:06.879668 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:25:06.879668 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:25:06.879668 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:25:06.879668 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:25:06.879668 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:25:06.879668 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:25:06.879668 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:25:06.879668 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:25:06.879668 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:25:06.879668 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:25:06.879668 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:25:06.879668 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:25:06.880427 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:25:06.880427 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:25:06.880427 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:25:06.880427 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:25:06.883071 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 444/891] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:25:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:25:06.890400 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9576844-3c1a-4605-b038-50a5718f3777 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:06.949923 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4420c901-7e38-4f49-b34e-4473f18262fe None None] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:06.950211 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4420c901-7e38-4f49-b34e-4473f18262fe None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:06.950405 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4420c901-7e38-4f49-b34e-4473f18262fe None None] PATH_INFO: `/` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:06.952240 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 445/892] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:25:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:25:06.959144 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9241a9-0155-4abf-9e5d-5b0d24aa2007 None None] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:06.959585 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9241a9-0155-4abf-9e5d-5b0d24aa2007 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:06.959966 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9241a9-0155-4abf-9e5d-5b0d24aa2007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:06.968328 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:06.996742 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.auth.core [None req-ee9241a9-0155-4abf-9e5d-5b0d24aa2007 None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143587) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:25:06.998994 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-ee9241a9-0155-4abf-9e5d-5b0d24aa2007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:25:07.012382 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9576844-3c1a-4605-b038-50a5718f3777 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:07.053486 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a4a565-0311-4c94-912e-b935793d3c4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:07.054207 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a4a565-0311-4c94-912e-b935793d3c4f None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:07.054487 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a4a565-0311-4c94-912e-b935793d3c4f None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:07.054762 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a4a565-0311-4c94-912e-b935793d3c4f None admin] PATH_INFO: `/v3/domains` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:07.055553 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a4a565-0311-4c94-912e-b935793d3c4f None admin] RBAC: Adding query filter params (name=Default) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:25:07.055859 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a4a565-0311-4c94-912e-b935793d3c4f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:07.057391 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a4a565-0311-4c94-912e-b935793d3c4f None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:07.073948 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 448/893] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:25:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:25:07.078876 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a4a565-0311-4c94-912e-b935793d3c4f None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:07.084284 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:07.096019 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 446/894] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:25:06 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:25:07.120620 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9241a9-0155-4abf-9e5d-5b0d24aa2007 None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:07.225413 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a4a565-0311-4c94-912e-b935793d3c4f None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:07.253129 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:07.257125 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9241a9-0155-4abf-9e5d-5b0d24aa2007 None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:07.266535 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e705a4eb-566c-4b36-a3cb-1badf6a3c238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:07.267724 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e705a4eb-566c-4b36-a3cb-1badf6a3c238 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:07.268231 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e705a4eb-566c-4b36-a3cb-1badf6a3c238 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:07.268798 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e705a4eb-566c-4b36-a3cb-1badf6a3c238 None admin] PATH_INFO: `/v3/projects/service` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:07.277485 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e705a4eb-566c-4b36-a3cb-1badf6a3c238 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:07.279387 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e705a4eb-566c-4b36-a3cb-1badf6a3c238 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:07.296750 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-e705a4eb-566c-4b36-a3cb-1badf6a3c238 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:25:07.296750 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:25:07.296750 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:25:07.296750 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:25:07.296750 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:25:07.296750 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:25:07.296750 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:25:07.296750 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:25:07.296750 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:25:07.296750 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:25:07.296750 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:25:07.296750 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:25:07.296750 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:25:07.296750 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:25:07.296750 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:25:07.297586 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:25:07.297586 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:25:07.297586 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:25:07.297586 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:25:07.297586 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:25:07.297586 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:25:07.297586 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:25:07.297586 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:25:07.297586 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:25:07.297586 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:25:07.297586 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:25:07.297586 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:25:07.297586 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:25:07.297586 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:25:07.297586 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:25:07.297586 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:25:07.297586 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:25:07.298489 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:25:07.298489 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:25:07.298489 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:25:07.298489 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:25:07.298489 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:25:07.298489 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:25:07.298489 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:25:07.298489 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:25:07.298854 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e7a3c9-9be8-4b66-ab54-c063e61db7eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:07.298854 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e7a3c9-9be8-4b66-ab54-c063e61db7eb None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:07.298854 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e7a3c9-9be8-4b66-ab54-c063e61db7eb None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:07.298854 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e7a3c9-9be8-4b66-ab54-c063e61db7eb None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:07.299639 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e7a3c9-9be8-4b66-ab54-c063e61db7eb None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:07.300695 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e7a3c9-9be8-4b66-ab54-c063e61db7eb None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:07.301721 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 449/895] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:25:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:25:07.307676 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e705a4eb-566c-4b36-a3cb-1badf6a3c238 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:07.315258 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-02e7a3c9-9be8-4b66-ab54-c063e61db7eb None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 14:25:07.315258 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:25:07.315258 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:25:07.315258 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:25:07.315258 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:25:07.315258 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:25:07.315258 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:25:07.315258 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:25:07.315258 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 14:25:07.315258 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:25:07.315258 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:25:07.315258 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:25:07.315258 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:25:07.315258 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:25:07.315258 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:25:07.315258 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:25:07.315258 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:25:07.316121 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:25:07.316121 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:25:07.316121 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:25:07.316121 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:25:07.316121 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:25:07.316121 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:25:07.316121 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:25:07.316121 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:25:07.316121 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:25:07.316121 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:25:07.316121 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:25:07.316121 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:25:07.316121 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:25:07.316121 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:25:07.316121 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:25:07.316121 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:25:07.316859 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:25:07.316859 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:25:07.316859 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:25:07.316859 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:25:07.316859 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:25:07.316859 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:25:07.316859 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:25:07.316859 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:25:07.316859 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:25:07.316859 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:25:07.316859 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:25:07.316859 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:25:07.316859 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 14:25:07.316859 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:25:07.321971 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 447/896] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 19 14:25:07 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:25:07.323187 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e7a3c9-9be8-4b66-ab54-c063e61db7eb None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:07.415778 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:07.447353 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e705a4eb-566c-4b36-a3cb-1badf6a3c238 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:07.465312 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e7a3c9-9be8-4b66-ab54-c063e61db7eb None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:07.493444 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be4c6c6-100a-4926-bd8e-3c5fc5d12a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:07.495446 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be4c6c6-100a-4926-bd8e-3c5fc5d12a6d None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:07.495875 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be4c6c6-100a-4926-bd8e-3c5fc5d12a6d None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:07.496447 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be4c6c6-100a-4926-bd8e-3c5fc5d12a6d None admin] PATH_INFO: `/v3/projects` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:07.497725 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be4c6c6-100a-4926-bd8e-3c5fc5d12a6d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:25:07.498444 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be4c6c6-100a-4926-bd8e-3c5fc5d12a6d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:07.500033 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be4c6c6-100a-4926-bd8e-3c5fc5d12a6d None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:07.502327 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80029cf-732c-4625-874c-97f0545e757f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:07.503303 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80029cf-732c-4625-874c-97f0545e757f None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:07.503415 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80029cf-732c-4625-874c-97f0545e757f None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:07.503793 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80029cf-732c-4625-874c-97f0545e757f None admin] PATH_INFO: `/v3/services` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:07.504600 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80029cf-732c-4625-874c-97f0545e757f None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:25:07.505006 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80029cf-732c-4625-874c-97f0545e757f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:07.506498 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80029cf-732c-4625-874c-97f0545e757f None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:07.515539 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 450/897] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:25:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:25:07.517133 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 448/898] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 19 14:25:07 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:25:07.517892 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be4c6c6-100a-4926-bd8e-3c5fc5d12a6d None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:07.521028 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80029cf-732c-4625-874c-97f0545e757f None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:07.572044 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:07.629093 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be4c6c6-100a-4926-bd8e-3c5fc5d12a6d None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:07.632793 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80029cf-732c-4625-874c-97f0545e757f None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:07.664634 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:07.672585 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d72e36f-90d1-42ab-bf18-cd2399656699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:07.673541 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d72e36f-90d1-42ab-bf18-cd2399656699 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:07.673963 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d72e36f-90d1-42ab-bf18-cd2399656699 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:07.674252 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d72e36f-90d1-42ab-bf18-cd2399656699 None admin] PATH_INFO: `/v3/services` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:07.675006 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d72e36f-90d1-42ab-bf18-cd2399656699 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:25:07.675377 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d72e36f-90d1-42ab-bf18-cd2399656699 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:07.675985 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79959f4d-d134-4eaa-bb39-9c337b05bde1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:07.676659 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79959f4d-d134-4eaa-bb39-9c337b05bde1 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:07.676808 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d72e36f-90d1-42ab-bf18-cd2399656699 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:07.676969 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79959f4d-d134-4eaa-bb39-9c337b05bde1 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:07.677220 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79959f4d-d134-4eaa-bb39-9c337b05bde1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:07.678130 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79959f4d-d134-4eaa-bb39-9c337b05bde1 None admin] RBAC: Adding query filter params (role.id=6efe7f48768b4033a20e04bc93529e6d, scope.project.id=252d9673f08c479a98d04fee3f5856a4, user.id=7977ff2f29eb435cbfeaa0b61d1d7a5d) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:25:07.678550 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79959f4d-d134-4eaa-bb39-9c337b05bde1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:07.679941 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79959f4d-d134-4eaa-bb39-9c337b05bde1 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:07.689349 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 451/899] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 19 14:25:07 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:25:07.689349 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666dd108-696d-4c87-83ce-054ad6b708dc None None] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:07.689349 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666dd108-696d-4c87-83ce-054ad6b708dc None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:07.689661 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 449/900] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:25:07 2022] GET /identity/v3/role_assignments?user.id=7977ff2f29eb435cbfeaa0b61d1d7a5d&scope.project.id=252d9673f08c479a98d04fee3f5856a4&role.id=6efe7f48768b4033a20e04bc93529e6d => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:25:07.690541 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79959f4d-d134-4eaa-bb39-9c337b05bde1 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:07.693521 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666dd108-696d-4c87-83ce-054ad6b708dc None None] PATH_INFO: `/` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:07.695477 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 452/901] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:25:07 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:25:07.702029 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601092bd-3c94-4ff3-9aac-e2c6fe2629c9 None None] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:07.702328 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601092bd-3c94-4ff3-9aac-e2c6fe2629c9 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:07.702617 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601092bd-3c94-4ff3-9aac-e2c6fe2629c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:07.720614 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.auth.core [None req-601092bd-3c94-4ff3-9aac-e2c6fe2629c9 None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143586) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:25:07.721876 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.fernet_utils [None req-601092bd-3c94-4ff3-9aac-e2c6fe2629c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143586) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:25:07.806621 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 453/902] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:25:07 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:25:07.830058 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601092bd-3c94-4ff3-9aac-e2c6fe2629c9 None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:07.846250 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79959f4d-d134-4eaa-bb39-9c337b05bde1 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:07.884348 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:07.899137 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2d9b48-1598-4157-a09b-262df02c2843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:07.900399 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2d9b48-1598-4157-a09b-262df02c2843 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:07.902004 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2d9b48-1598-4157-a09b-262df02c2843 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:07.902503 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2d9b48-1598-4157-a09b-262df02c2843 None admin] PATH_INFO: `/v3/endpoints` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:07.903504 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2d9b48-1598-4157-a09b-262df02c2843 None admin] RBAC: Adding query filter params (interface=public, service_id=6d9f5327a71c4441a7bc5ac213b1b285, region_id=RegionOne) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:25:07.904011 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2d9b48-1598-4157-a09b-262df02c2843 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:07.906475 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2d9b48-1598-4157-a09b-262df02c2843 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:07.919908 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 450/903] 10.222.0.57 () {56 vars in 1251 bytes} [Thu May 19 14:25:07 2022] GET /identity/v3/endpoints?service_id=6d9f5327a71c4441a7bc5ac213b1b285&interface=public®ion_id=RegionOne => generated 194 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:25:07.921101 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2d9b48-1598-4157-a09b-262df02c2843 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:07.966983 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601092bd-3c94-4ff3-9aac-e2c6fe2629c9 None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:07.991672 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:08.014007 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9b657d-6724-46e9-afa2-9afd57f27a5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:08.015171 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9b657d-6724-46e9-afa2-9afd57f27a5c None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:08.015645 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9b657d-6724-46e9-afa2-9afd57f27a5c None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:08.016112 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9b657d-6724-46e9-afa2-9afd57f27a5c None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:08.016956 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9b657d-6724-46e9-afa2-9afd57f27a5c None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:08.018976 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9b657d-6724-46e9-afa2-9afd57f27a5c None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:08.035792 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-fa9b657d-6724-46e9-afa2-9afd57f27a5c None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 14:25:08.035792 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:25:08.035792 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:25:08.035792 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:25:08.035792 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:25:08.035792 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:25:08.035792 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:25:08.035792 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:25:08.035792 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 14:25:08.035792 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:25:08.035792 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:25:08.035792 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:25:08.035792 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:25:08.035792 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:25:08.035792 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:25:08.035792 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:25:08.036635 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:25:08.036635 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:25:08.036635 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:25:08.036635 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:25:08.036635 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:25:08.036635 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:25:08.036635 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:25:08.036635 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:25:08.036635 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:25:08.036635 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:25:08.036635 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:25:08.036635 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:25:08.036635 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:25:08.036635 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:25:08.036635 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:25:08.036635 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:25:08.037372 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:25:08.037372 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:25:08.037372 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:25:08.037372 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:25:08.037372 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:25:08.037372 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:25:08.037372 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:25:08.037372 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:25:08.037372 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:25:08.037372 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:25:08.037372 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:25:08.037372 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:25:08.037372 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:25:08.037372 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 14:25:08.037372 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:25:08.040291 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 454/904] 10.222.0.57 () {56 vars in 1132 bytes} [Thu May 19 14:25:07 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:25:08.049490 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9b657d-6724-46e9-afa2-9afd57f27a5c None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:08.049645 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2d9b48-1598-4157-a09b-262df02c2843 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:08.094206 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda86db6-4fee-4dc7-9efa-2228247159d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:08.094825 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda86db6-4fee-4dc7-9efa-2228247159d0 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:08.095107 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda86db6-4fee-4dc7-9efa-2228247159d0 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:08.095384 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda86db6-4fee-4dc7-9efa-2228247159d0 None admin] PATH_INFO: `/v3/services` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:08.096065 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda86db6-4fee-4dc7-9efa-2228247159d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:08.097444 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda86db6-4fee-4dc7-9efa-2228247159d0 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:08.105320 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 451/905] 10.222.0.57 () {56 vars in 1087 bytes} [Thu May 19 14:25:07 2022] GET /identity/v3/services => generated 1268 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 14:25:08.167877 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9b657d-6724-46e9-afa2-9afd57f27a5c None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:08.169410 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:08.208131 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46ef058-a96c-43df-aef0-1e8a77aa2390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:08.208857 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46ef058-a96c-43df-aef0-1e8a77aa2390 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:08.209144 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46ef058-a96c-43df-aef0-1e8a77aa2390 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:08.209419 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46ef058-a96c-43df-aef0-1e8a77aa2390 None admin] PATH_INFO: `/v3/services` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:08.210171 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46ef058-a96c-43df-aef0-1e8a77aa2390 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:25:08.210478 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46ef058-a96c-43df-aef0-1e8a77aa2390 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:08.211872 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46ef058-a96c-43df-aef0-1e8a77aa2390 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:08.218778 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 455/906] 10.222.0.57 () {56 vars in 1126 bytes} [Thu May 19 14:25:08 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:25:08.224590 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda86db6-4fee-4dc7-9efa-2228247159d0 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:08.337287 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda86db6-4fee-4dc7-9efa-2228247159d0 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:08.357782 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:08.377555 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba2c785-b22c-4e7f-8f2b-e5b4779d1ac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:08.378583 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba2c785-b22c-4e7f-8f2b-e5b4779d1ac8 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:08.379153 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba2c785-b22c-4e7f-8f2b-e5b4779d1ac8 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:08.379714 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba2c785-b22c-4e7f-8f2b-e5b4779d1ac8 None admin] PATH_INFO: `/v3/services` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:08.380663 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba2c785-b22c-4e7f-8f2b-e5b4779d1ac8 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:25:08.381160 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba2c785-b22c-4e7f-8f2b-e5b4779d1ac8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:08.382801 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba2c785-b22c-4e7f-8f2b-e5b4779d1ac8 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:08.389117 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 452/907] 10.222.0.57 () {56 vars in 1126 bytes} [Thu May 19 14:25:08 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:25:08.394656 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46ef058-a96c-43df-aef0-1e8a77aa2390 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:08.502164 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46ef058-a96c-43df-aef0-1e8a77aa2390 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:08.538664 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0ca847-11ef-4c93-821c-b40d665fe1af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:08.539334 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0ca847-11ef-4c93-821c-b40d665fe1af None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:08.539622 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0ca847-11ef-4c93-821c-b40d665fe1af None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:08.539906 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0ca847-11ef-4c93-821c-b40d665fe1af None admin] PATH_INFO: `/v3/endpoints` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:08.540707 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0ca847-11ef-4c93-821c-b40d665fe1af None admin] RBAC: Adding query filter params (interface=public, service_id=2f34d185697147acb1fe7b5ce2a702c7, region_id=RegionOne) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:25:08.541021 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0ca847-11ef-4c93-821c-b40d665fe1af None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:08.543317 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0ca847-11ef-4c93-821c-b40d665fe1af None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:08.549691 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 456/908] 10.222.0.57 () {56 vars in 1251 bytes} [Thu May 19 14:25:08 2022] GET /identity/v3/endpoints?service_id=2f34d185697147acb1fe7b5ce2a702c7&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:25:08.555362 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba2c785-b22c-4e7f-8f2b-e5b4779d1ac8 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:08.674206 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba2c785-b22c-4e7f-8f2b-e5b4779d1ac8 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:08.717983 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec70faad-7f8c-46e9-b48d-61a5d64c765b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:08.718651 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec70faad-7f8c-46e9-b48d-61a5d64c765b None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:08.718930 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec70faad-7f8c-46e9-b48d-61a5d64c765b None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:08.719203 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec70faad-7f8c-46e9-b48d-61a5d64c765b None admin] PATH_INFO: `/v3/services` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:08.719888 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec70faad-7f8c-46e9-b48d-61a5d64c765b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:08.721257 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec70faad-7f8c-46e9-b48d-61a5d64c765b None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:08.727690 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 453/909] 10.222.0.57 () {56 vars in 1087 bytes} [Thu May 19 14:25:08 2022] GET /identity/v3/services => generated 1268 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 14:25:08.856911 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1675eab-2c9c-4346-82c9-3dce51e4105c None None] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:08.857217 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1675eab-2c9c-4346-82c9-3dce51e4105c None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:08.857730 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1675eab-2c9c-4346-82c9-3dce51e4105c None None] PATH_INFO: `/` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:08.859731 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 457/910] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:25:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:25:08.865879 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3f0378-f615-4118-a1f0-74a7a131b6e1 None None] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:08.866189 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3f0378-f615-4118-a1f0-74a7a131b6e1 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:08.866430 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3f0378-f615-4118-a1f0-74a7a131b6e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:08.884471 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.auth.core [None req-9a3f0378-f615-4118-a1f0-74a7a131b6e1 None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143587) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:25:08.886920 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-9a3f0378-f615-4118-a1f0-74a7a131b6e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:25:08.931497 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:08.970315 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 454/911] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:25:08 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:25:08.994714 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1675eab-2c9c-4346-82c9-3dce51e4105c None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:09.027415 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:09.122780 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1675eab-2c9c-4346-82c9-3dce51e4105c None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:09.159704 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca242ad-fa55-48a6-a983-15a145813fdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:09.160383 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca242ad-fa55-48a6-a983-15a145813fdb None admin] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:09.160669 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca242ad-fa55-48a6-a983-15a145813fdb None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:09.160950 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca242ad-fa55-48a6-a983-15a145813fdb None admin] PATH_INFO: `/v3/services` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:09.161691 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca242ad-fa55-48a6-a983-15a145813fdb None admin] RBAC: Authorizing `identity:create_service()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:09.163095 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca242ad-fa55-48a6-a983-15a145813fdb None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:09.180954 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 458/912] 10.222.0.57 () {60 vars in 1141 bytes} [Thu May 19 14:25:08 2022] POST /identity/v3/services => generated 241 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:25:10.098153 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998cf10f-89c1-4179-aea3-d85365ba805c None None] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:10.099115 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998cf10f-89c1-4179-aea3-d85365ba805c None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:10.099593 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998cf10f-89c1-4179-aea3-d85365ba805c None None] PATH_INFO: `/` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:10.101446 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 455/913] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:25:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:25:10.107665 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1d1d3d-147f-49fe-85d1-d15c36e02d6d None None] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:10.108500 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1d1d3d-147f-49fe-85d1-d15c36e02d6d None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:10.108992 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1d1d3d-147f-49fe-85d1-d15c36e02d6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:10.126330 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.auth.core [None req-7a1d1d3d-147f-49fe-85d1-d15c36e02d6d None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143586) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:25:10.127834 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.fernet_utils [None req-7a1d1d3d-147f-49fe-85d1-d15c36e02d6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143586) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:25:10.207544 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 459/914] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:25:10 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:25:10.231125 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998cf10f-89c1-4179-aea3-d85365ba805c None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:10.351481 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998cf10f-89c1-4179-aea3-d85365ba805c None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:10.388900 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75990ef1-3dfb-42dc-a79c-f76a36996c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:10.389719 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75990ef1-3dfb-42dc-a79c-f76a36996c95 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:10.390035 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75990ef1-3dfb-42dc-a79c-f76a36996c95 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:10.390313 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75990ef1-3dfb-42dc-a79c-f76a36996c95 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:10.402169 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75990ef1-3dfb-42dc-a79c-f76a36996c95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:25:10.410535 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-75990ef1-3dfb-42dc-a79c-f76a36996c95 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 14:25:10.410535 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:25:10.410535 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:25:10.410535 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:25:10.410535 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:25:10.410535 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:25:10.410535 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:25:10.410535 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:25:10.410535 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:25:10.410535 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:25:10.410535 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:25:10.410535 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:25:10.410535 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:25:10.410535 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:25:10.410535 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:25:10.410535 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:25:10.411335 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:25:10.411335 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:25:10.411335 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:25:10.411335 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:25:10.411335 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:25:10.411335 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:25:10.411335 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:25:10.411335 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:25:10.411335 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:25:10.411335 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:25:10.411335 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:25:10.411335 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:25:10.411335 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:25:10.411335 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:25:10.411335 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:25:10.411335 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:25:10.412043 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:25:10.412043 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:25:10.412043 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:25:10.412043 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:25:10.412043 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:25:10.412043 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:25:10.412043 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:25:10.412043 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:25:10.412043 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:25:10.412043 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:25:10.412043 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 14:25:10.412043 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:25:10.412592 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 456/915] 10.222.0.57 () {56 vars in 1120 bytes} [Thu May 19 14:25:10 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:25:10.419242 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1d1d3d-147f-49fe-85d1-d15c36e02d6d None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:10.438882 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:10.509179 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a8c0a9-8b62-48ef-be90-0a1f56abbd56 None None] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:10.511308 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a8c0a9-8b62-48ef-be90-0a1f56abbd56 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:10.511598 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a8c0a9-8b62-48ef-be90-0a1f56abbd56 None None] PATH_INFO: `/` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:10.513492 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 457/916] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:25:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:25:10.518707 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb70a53-fd94-4fe9-a6f4-2a01607b834c None None] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:10.519196 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb70a53-fd94-4fe9-a6f4-2a01607b834c None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:10.519671 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb70a53-fd94-4fe9-a6f4-2a01607b834c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:10.527942 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1d1d3d-147f-49fe-85d1-d15c36e02d6d None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:10.541407 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.auth.core [None req-9bb70a53-fd94-4fe9-a6f4-2a01607b834c None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143587) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:25:10.542808 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-9bb70a53-fd94-4fe9-a6f4-2a01607b834c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:25:10.578583 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd60a42-a79a-4ea8-b6e9-f4f2aaa57db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:10.579651 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:10.581371 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd60a42-a79a-4ea8-b6e9-f4f2aaa57db0 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:10.582249 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd60a42-a79a-4ea8-b6e9-f4f2aaa57db0 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:10.582552 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd60a42-a79a-4ea8-b6e9-f4f2aaa57db0 None admin] PATH_INFO: `/v3/roles` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:10.584150 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd60a42-a79a-4ea8-b6e9-f4f2aaa57db0 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:25:10.584682 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd60a42-a79a-4ea8-b6e9-f4f2aaa57db0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:10.586301 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd60a42-a79a-4ea8-b6e9-f4f2aaa57db0 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:10.596985 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 460/917] 10.222.0.57 () {56 vars in 1115 bytes} [Thu May 19 14:25:10 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:25:10.603390 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd60a42-a79a-4ea8-b6e9-f4f2aaa57db0 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:10.637346 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 458/918] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:25:10 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:25:10.656249 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb70a53-fd94-4fe9-a6f4-2a01607b834c None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:10.711557 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:10.734695 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd60a42-a79a-4ea8-b6e9-f4f2aaa57db0 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:10.774462 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f856c35-895f-450e-a8de-917863ed2358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:10.775261 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f856c35-895f-450e-a8de-917863ed2358 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:10.775593 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f856c35-895f-450e-a8de-917863ed2358 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:10.775936 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f856c35-895f-450e-a8de-917863ed2358 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:10.776382 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb70a53-fd94-4fe9-a6f4-2a01607b834c None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:10.787531 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f856c35-895f-450e-a8de-917863ed2358 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:10.788985 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f856c35-895f-450e-a8de-917863ed2358 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:10.808187 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-7f856c35-895f-450e-a8de-917863ed2358 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:25:10.808187 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:25:10.808187 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:25:10.808187 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:25:10.808187 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:25:10.808187 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:25:10.808187 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:25:10.808187 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:25:10.808187 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:25:10.808187 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:25:10.808187 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:25:10.808187 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:25:10.808187 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:25:10.808187 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:25:10.808187 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:25:10.809103 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:25:10.809103 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:25:10.809103 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:25:10.809103 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:25:10.809103 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:25:10.809103 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:25:10.809103 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:25:10.809103 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:25:10.809103 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:25:10.809103 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:25:10.809103 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:25:10.809103 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:25:10.809103 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:25:10.809103 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:25:10.809103 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:25:10.809103 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:25:10.809103 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:25:10.810315 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:25:10.810315 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:25:10.810315 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:25:10.810315 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:25:10.810315 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 461/919] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:25:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:25:10.815630 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f856c35-895f-450e-a8de-917863ed2358 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:10.819179 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72875479-4ba8-42c6-8412-4a2e61c7a749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:10.820289 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72875479-4ba8-42c6-8412-4a2e61c7a749 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:10.820803 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72875479-4ba8-42c6-8412-4a2e61c7a749 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:10.821274 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72875479-4ba8-42c6-8412-4a2e61c7a749 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:10.822159 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72875479-4ba8-42c6-8412-4a2e61c7a749 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:10.823824 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72875479-4ba8-42c6-8412-4a2e61c7a749 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:10.836881 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-72875479-4ba8-42c6-8412-4a2e61c7a749 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 14:25:10.836881 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:25:10.836881 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:25:10.836881 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:25:10.836881 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:25:10.836881 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:25:10.836881 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:25:10.836881 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:25:10.836881 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 14:25:10.836881 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:25:10.836881 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:25:10.836881 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:25:10.836881 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:25:10.836881 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:25:10.836881 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:25:10.836881 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:25:10.836881 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:25:10.837810 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:25:10.837810 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:25:10.837810 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:25:10.837810 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:25:10.837810 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:25:10.837810 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:25:10.837810 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:25:10.837810 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:25:10.837810 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:25:10.837810 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:25:10.837810 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:25:10.837810 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:25:10.837810 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:25:10.837810 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:25:10.837810 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:25:10.837810 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:25:10.838678 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:25:10.838678 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:25:10.838678 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:25:10.838678 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:25:10.838678 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:25:10.838678 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:25:10.838678 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:25:10.838678 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:25:10.838678 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:25:10.838678 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:25:10.838678 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:25:10.838678 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:25:10.838678 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 14:25:10.838678 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:25:10.846746 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 459/920] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 19 14:25:10 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:25:10.848269 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72875479-4ba8-42c6-8412-4a2e61c7a749 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:10.881275 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:10.926116 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f856c35-895f-450e-a8de-917863ed2358 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:10.958020 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:10.962915 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72875479-4ba8-42c6-8412-4a2e61c7a749 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:10.965639 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c4d363-53c6-4ea8-8dc2-410f2fc26756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:10.966477 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c4d363-53c6-4ea8-8dc2-410f2fc26756 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:10.966766 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c4d363-53c6-4ea8-8dc2-410f2fc26756 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:10.967046 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c4d363-53c6-4ea8-8dc2-410f2fc26756 None admin] PATH_INFO: `/v3/domains` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:10.967795 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c4d363-53c6-4ea8-8dc2-410f2fc26756 None admin] RBAC: Adding query filter params (name=Default) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:25:10.968984 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c4d363-53c6-4ea8-8dc2-410f2fc26756 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:10.969626 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c4d363-53c6-4ea8-8dc2-410f2fc26756 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:10.990223 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 462/921] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:25:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:25:10.991729 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c4d363-53c6-4ea8-8dc2-410f2fc26756 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:11.008820 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb61084-3f79-46fc-ad96-56650796ef83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:11.009953 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb61084-3f79-46fc-ad96-56650796ef83 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:11.010463 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb61084-3f79-46fc-ad96-56650796ef83 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:11.011016 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb61084-3f79-46fc-ad96-56650796ef83 None admin] PATH_INFO: `/v3/services` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:11.012038 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb61084-3f79-46fc-ad96-56650796ef83 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:25:11.012558 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb61084-3f79-46fc-ad96-56650796ef83 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:11.014254 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb61084-3f79-46fc-ad96-56650796ef83 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:11.025053 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 460/922] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 19 14:25:10 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:25:11.026774 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb61084-3f79-46fc-ad96-56650796ef83 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:11.105197 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c4d363-53c6-4ea8-8dc2-410f2fc26756 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:11.129169 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:11.139505 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb61084-3f79-46fc-ad96-56650796ef83 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:11.146546 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bfb50c-07aa-4901-bfef-2ec9c7a0f108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:11.147407 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bfb50c-07aa-4901-bfef-2ec9c7a0f108 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:11.147689 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bfb50c-07aa-4901-bfef-2ec9c7a0f108 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:11.147960 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bfb50c-07aa-4901-bfef-2ec9c7a0f108 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:11.156888 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bfb50c-07aa-4901-bfef-2ec9c7a0f108 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:11.158352 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bfb50c-07aa-4901-bfef-2ec9c7a0f108 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:11.175511 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-d2bfb50c-07aa-4901-bfef-2ec9c7a0f108 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 14:25:11.175511 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:25:11.175511 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:25:11.175511 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:25:11.175511 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:25:11.175511 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:25:11.175511 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:25:11.175511 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:25:11.175511 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:25:11.175511 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:25:11.175511 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:25:11.175511 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:25:11.175511 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:25:11.175511 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:25:11.175511 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:25:11.175511 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:25:11.176435 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:25:11.176435 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:25:11.176435 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:25:11.176435 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:25:11.176435 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:25:11.176435 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:25:11.176435 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:25:11.176435 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:25:11.176435 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:25:11.176435 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:25:11.176435 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:25:11.176435 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:25:11.176435 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:25:11.176435 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:25:11.176435 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:25:11.176435 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:25:11.176435 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:25:11.176435 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:25:11.177301 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:25:11.177301 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:25:11.177301 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:25:11.177301 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:25:11.177301 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:25:11.177301 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:25:11.177301 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:25:11.177301 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:25:11.177301 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 14:25:11.177301 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:25:11.177301 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 463/923] 10.222.0.57 () {56 vars in 1117 bytes} [Thu May 19 14:25:10 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:25:11.178440 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bea158-8335-443f-9f04-4d1e53d78b5a None None] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:11.178440 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bea158-8335-443f-9f04-4d1e53d78b5a None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:11.178587 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bea158-8335-443f-9f04-4d1e53d78b5a None None] PATH_INFO: `/` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:11.179911 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 464/924] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:25:11 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:25:11.184010 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bea158-8335-443f-9f04-4d1e53d78b5a None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:11.186415 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9366bea-5c1d-4c61-b985-6a831d16667b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:11.187083 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9366bea-5c1d-4c61-b985-6a831d16667b None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:11.187363 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9366bea-5c1d-4c61-b985-6a831d16667b None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:11.187637 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9366bea-5c1d-4c61-b985-6a831d16667b None admin] PATH_INFO: `/v3/services` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:11.188348 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9366bea-5c1d-4c61-b985-6a831d16667b None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:25:11.188655 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9366bea-5c1d-4c61-b985-6a831d16667b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:11.190056 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9366bea-5c1d-4c61-b985-6a831d16667b None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:11.195706 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 461/925] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 19 14:25:11 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:25:11.196957 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea50bd3-8e52-42ef-a685-ba7ca892e061 None None] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:11.197344 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea50bd3-8e52-42ef-a685-ba7ca892e061 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:11.198301 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea50bd3-8e52-42ef-a685-ba7ca892e061 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:11.216800 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.auth.core [None req-8ea50bd3-8e52-42ef-a685-ba7ca892e061 None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143587) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:25:11.218006 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-8ea50bd3-8e52-42ef-a685-ba7ca892e061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:25:11.239374 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:11.290579 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 462/926] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:25:11 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:25:11.292364 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea50bd3-8e52-42ef-a685-ba7ca892e061 None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:11.302855 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bea158-8335-443f-9f04-4d1e53d78b5a None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:11.350817 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b73fc4c-f8f3-4c7a-a2d8-3fb6909a6343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:11.351481 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b73fc4c-f8f3-4c7a-a2d8-3fb6909a6343 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:11.351815 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b73fc4c-f8f3-4c7a-a2d8-3fb6909a6343 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:11.352139 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b73fc4c-f8f3-4c7a-a2d8-3fb6909a6343 None admin] PATH_INFO: `/v3/users` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:11.352982 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b73fc4c-f8f3-4c7a-a2d8-3fb6909a6343 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:25:11.353288 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b73fc4c-f8f3-4c7a-a2d8-3fb6909a6343 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:11.355565 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b73fc4c-f8f3-4c7a-a2d8-3fb6909a6343 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:11.377674 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 465/927] 10.222.0.57 () {56 vars in 1149 bytes} [Thu May 19 14:25:11 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:25:11.377674 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b73fc4c-f8f3-4c7a-a2d8-3fb6909a6343 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:11.390678 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:11.407661 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea50bd3-8e52-42ef-a685-ba7ca892e061 None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:11.445343 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed616915-5777-4c74-b5b1-e35f9229c9e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:11.446071 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed616915-5777-4c74-b5b1-e35f9229c9e4 None admin] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:11.446322 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed616915-5777-4c74-b5b1-e35f9229c9e4 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:11.446601 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed616915-5777-4c74-b5b1-e35f9229c9e4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:11.447342 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed616915-5777-4c74-b5b1-e35f9229c9e4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:11.449281 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed616915-5777-4c74-b5b1-e35f9229c9e4 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:11.471843 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 463/928] 10.222.0.57 () {60 vars in 1144 bytes} [Thu May 19 14:25:11 2022] POST /identity/v3/endpoints => generated 346 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:25:11.473316 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed616915-5777-4c74-b5b1-e35f9229c9e4 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:11.497446 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b73fc4c-f8f3-4c7a-a2d8-3fb6909a6343 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:11.508183 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:11.546635 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14feba0-4fe5-40b7-abb7-b65edf12e632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:11.547537 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14feba0-4fe5-40b7-abb7-b65edf12e632 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:11.547757 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14feba0-4fe5-40b7-abb7-b65edf12e632 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:11.548030 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14feba0-4fe5-40b7-abb7-b65edf12e632 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:11.548705 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14feba0-4fe5-40b7-abb7-b65edf12e632 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:11.550141 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14feba0-4fe5-40b7-abb7-b65edf12e632 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:11.563848 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-d14feba0-4fe5-40b7-abb7-b65edf12e632 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 14:25:11.563848 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:25:11.563848 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:25:11.563848 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:25:11.563848 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:25:11.563848 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:25:11.563848 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:25:11.563848 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:25:11.563848 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 14:25:11.563848 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:25:11.563848 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:25:11.563848 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:25:11.563848 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:25:11.563848 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:25:11.563848 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:25:11.563848 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:25:11.564761 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:25:11.564761 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:25:11.564761 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:25:11.564761 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:25:11.564761 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:25:11.564761 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:25:11.564761 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:25:11.564761 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:25:11.564761 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:25:11.564761 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:25:11.564761 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:25:11.564761 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:25:11.564761 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:25:11.564761 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:25:11.564761 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:25:11.564761 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:25:11.565513 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:25:11.565513 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:25:11.565513 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:25:11.565513 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:25:11.565513 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:25:11.565513 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:25:11.565513 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:25:11.565513 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:25:11.565513 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:25:11.565513 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:25:11.565513 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:25:11.565513 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:25:11.565513 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:25:11.565513 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 14:25:11.565513 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:25:11.570421 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 466/929] 10.222.0.57 () {56 vars in 1132 bytes} [Thu May 19 14:25:11 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:25:11.571773 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6ca128-94a7-41f5-a686-29464cca2e11 None None] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:11.572060 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6ca128-94a7-41f5-a686-29464cca2e11 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:11.572337 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6ca128-94a7-41f5-a686-29464cca2e11 None None] PATH_INFO: `/` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:11.578018 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 467/930] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:25:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:25:11.578018 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6ca128-94a7-41f5-a686-29464cca2e11 None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:11.602466 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed616915-5777-4c74-b5b1-e35f9229c9e4 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:11.641162 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73db0aed-56d9-464b-9676-6e752980b6a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:11.641918 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73db0aed-56d9-464b-9676-6e752980b6a0 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:11.642204 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73db0aed-56d9-464b-9676-6e752980b6a0 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:11.642482 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73db0aed-56d9-464b-9676-6e752980b6a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:11.663296 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73db0aed-56d9-464b-9676-6e752980b6a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:11.664779 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73db0aed-56d9-464b-9676-6e752980b6a0 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:11.667867 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:11.686029 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-73db0aed-56d9-464b-9676-6e752980b6a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:25:11.686029 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:25:11.686029 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:25:11.686029 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:25:11.686029 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:25:11.686029 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:25:11.686029 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:25:11.686029 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:25:11.686029 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:25:11.686029 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:25:11.686029 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:25:11.686029 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:25:11.686029 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:25:11.686029 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:25:11.686029 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:25:11.687089 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:25:11.687089 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:25:11.687089 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:25:11.687089 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:25:11.687089 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:25:11.687089 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:25:11.687089 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:25:11.687089 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:25:11.687089 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:25:11.687089 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:25:11.687089 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:25:11.687089 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:25:11.687089 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:25:11.687089 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:25:11.687089 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:25:11.687089 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:25:11.687089 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:25:11.687979 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:25:11.687979 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:25:11.687979 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:25:11.687979 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:25:11.689546 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 464/931] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:25:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:25:11.691025 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c285bc6b-9324-463d-a8bd-4433c5ca17ae None None] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:11.691465 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c285bc6b-9324-463d-a8bd-4433c5ca17ae None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:11.691753 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c285bc6b-9324-463d-a8bd-4433c5ca17ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:11.700888 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6ca128-94a7-41f5-a686-29464cca2e11 None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:11.709588 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.auth.core [None req-c285bc6b-9324-463d-a8bd-4433c5ca17ae None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143587) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:25:11.711298 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-c285bc6b-9324-463d-a8bd-4433c5ca17ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:25:11.738505 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabf8bdf-9219-4df2-9f72-7c30885fefe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:11.739178 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabf8bdf-9219-4df2-9f72-7c30885fefe0 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:11.739463 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabf8bdf-9219-4df2-9f72-7c30885fefe0 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:11.739740 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabf8bdf-9219-4df2-9f72-7c30885fefe0 None admin] PATH_INFO: `/v3/services` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:11.740453 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabf8bdf-9219-4df2-9f72-7c30885fefe0 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:25:11.740762 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabf8bdf-9219-4df2-9f72-7c30885fefe0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:11.742150 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabf8bdf-9219-4df2-9f72-7c30885fefe0 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:11.751623 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 468/932] 10.222.0.57 () {56 vars in 1126 bytes} [Thu May 19 14:25:11 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:25:11.751623 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabf8bdf-9219-4df2-9f72-7c30885fefe0 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:11.785960 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:11.793986 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 465/933] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:25:11 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:25:11.795270 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c285bc6b-9324-463d-a8bd-4433c5ca17ae None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:11.863306 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabf8bdf-9219-4df2-9f72-7c30885fefe0 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:11.899611 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f426859-df3c-48f7-be24-5b2015e02d6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:11.900202 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f426859-df3c-48f7-be24-5b2015e02d6a None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:11.901010 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f426859-df3c-48f7-be24-5b2015e02d6a None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:11.901300 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f426859-df3c-48f7-be24-5b2015e02d6a None admin] PATH_INFO: `/v3/domains` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:11.902063 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f426859-df3c-48f7-be24-5b2015e02d6a None admin] RBAC: Adding query filter params (name=Default) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:25:11.902478 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f426859-df3c-48f7-be24-5b2015e02d6a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:11.903842 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f426859-df3c-48f7-be24-5b2015e02d6a None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:11.910174 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c285bc6b-9324-463d-a8bd-4433c5ca17ae None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:11.916830 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 469/934] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:25:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:25:11.917776 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f426859-df3c-48f7-be24-5b2015e02d6a None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:11.939036 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:11.952927 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff47a73-dc25-42a1-9482-b857b8fa5b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:11.953604 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff47a73-dc25-42a1-9482-b857b8fa5b1b None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:11.953916 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff47a73-dc25-42a1-9482-b857b8fa5b1b None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:11.954200 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff47a73-dc25-42a1-9482-b857b8fa5b1b None admin] PATH_INFO: `/v3/services` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:11.954915 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff47a73-dc25-42a1-9482-b857b8fa5b1b None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:25:11.955229 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff47a73-dc25-42a1-9482-b857b8fa5b1b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:11.956616 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff47a73-dc25-42a1-9482-b857b8fa5b1b None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:11.961939 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 466/935] 10.222.0.57 () {56 vars in 1126 bytes} [Thu May 19 14:25:11 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:25:11.962805 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff47a73-dc25-42a1-9482-b857b8fa5b1b None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:12.020888 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:12.033883 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f426859-df3c-48f7-be24-5b2015e02d6a None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:12.069764 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff47a73-dc25-42a1-9482-b857b8fa5b1b None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:12.070224 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c08376-49c0-40d1-9630-3ae87a827efa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:12.071019 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c08376-49c0-40d1-9630-3ae87a827efa None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:12.071309 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c08376-49c0-40d1-9630-3ae87a827efa None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:12.071585 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c08376-49c0-40d1-9630-3ae87a827efa None admin] PATH_INFO: `/v3/services/object-store` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:12.072242 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c08376-49c0-40d1-9630-3ae87a827efa None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:12.073661 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c08376-49c0-40d1-9630-3ae87a827efa None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:12.085944 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-d5c08376-49c0-40d1-9630-3ae87a827efa None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 14:25:12.085944 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:25:12.085944 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:25:12.085944 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:25:12.085944 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:25:12.085944 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:25:12.085944 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:25:12.085944 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:25:12.085944 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 14:25:12.085944 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:25:12.085944 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:25:12.085944 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:25:12.085944 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:25:12.085944 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:25:12.085944 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:25:12.085944 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:25:12.085944 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:25:12.086883 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:25:12.086883 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:25:12.086883 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:25:12.086883 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:25:12.086883 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:25:12.086883 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:25:12.086883 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:25:12.086883 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:25:12.086883 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:25:12.086883 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:25:12.086883 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:25:12.086883 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:25:12.086883 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:25:12.086883 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:25:12.086883 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:25:12.086883 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:25:12.087641 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:25:12.087641 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:25:12.087641 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:25:12.087641 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:25:12.087641 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:25:12.087641 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:25:12.087641 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:25:12.087641 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:25:12.087641 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:25:12.087641 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:25:12.087641 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:25:12.087641 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:25:12.087641 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 14:25:12.087641 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:25:12.087641 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 470/936] 10.222.0.57 () {56 vars in 1126 bytes} [Thu May 19 14:25:11 2022] GET /identity/v3/services/object-store => generated 93 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:25:12.088839 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c08376-49c0-40d1-9630-3ae87a827efa None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:12.112706 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc870279-e391-4407-be0a-eed2fd5bad9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:12.113502 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc870279-e391-4407-be0a-eed2fd5bad9c None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:12.113786 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc870279-e391-4407-be0a-eed2fd5bad9c None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:12.114089 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc870279-e391-4407-be0a-eed2fd5bad9c None admin] PATH_INFO: `/v3/projects/service` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:12.121321 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc870279-e391-4407-be0a-eed2fd5bad9c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:12.122816 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc870279-e391-4407-be0a-eed2fd5bad9c None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:12.137888 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-fc870279-e391-4407-be0a-eed2fd5bad9c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:25:12.137888 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:25:12.137888 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:25:12.137888 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:25:12.137888 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:25:12.137888 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:25:12.137888 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:25:12.137888 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:25:12.137888 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:25:12.137888 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:25:12.137888 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:25:12.137888 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:25:12.137888 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:25:12.137888 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:25:12.137888 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:25:12.139290 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:25:12.139290 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:25:12.139290 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:25:12.139290 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:25:12.139290 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:25:12.139290 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:25:12.139290 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:25:12.139290 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:25:12.139290 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:25:12.139290 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:25:12.139290 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:25:12.139290 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:25:12.139290 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:25:12.139290 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:25:12.139290 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:25:12.139290 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:25:12.139290 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:25:12.140261 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:25:12.140261 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:25:12.140261 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:25:12.140261 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:25:12.140261 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:25:12.140261 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:25:12.140261 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:25:12.140261 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:25:12.140261 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 467/937] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:25:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:25:12.140261 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc870279-e391-4407-be0a-eed2fd5bad9c None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:12.192891 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:12.208342 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c08376-49c0-40d1-9630-3ae87a827efa None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:12.245265 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc870279-e391-4407-be0a-eed2fd5bad9c None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:12.246443 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a69b89-c58b-4c92-96a1-eb27149c7090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:12.247221 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a69b89-c58b-4c92-96a1-eb27149c7090 None admin] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:12.247628 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a69b89-c58b-4c92-96a1-eb27149c7090 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:12.247909 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a69b89-c58b-4c92-96a1-eb27149c7090 None admin] PATH_INFO: `/v3/endpoints` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:12.248652 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a69b89-c58b-4c92-96a1-eb27149c7090 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:12.250054 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a69b89-c58b-4c92-96a1-eb27149c7090 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:12.274108 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 471/938] 10.222.0.57 () {60 vars in 1144 bytes} [Thu May 19 14:25:12 2022] POST /identity/v3/endpoints => generated 347 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:25:12.274959 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a69b89-c58b-4c92-96a1-eb27149c7090 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:12.303465 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:12.305639 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33817c4b-1ee0-414d-b67d-8533c9759493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:12.307639 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33817c4b-1ee0-414d-b67d-8533c9759493 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:12.308307 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33817c4b-1ee0-414d-b67d-8533c9759493 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:12.308657 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33817c4b-1ee0-414d-b67d-8533c9759493 None admin] PATH_INFO: `/v3/services` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:12.309695 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33817c4b-1ee0-414d-b67d-8533c9759493 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:25:12.310040 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33817c4b-1ee0-414d-b67d-8533c9759493 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:12.311707 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33817c4b-1ee0-414d-b67d-8533c9759493 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:12.322041 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 468/939] 10.222.0.57 () {56 vars in 1122 bytes} [Thu May 19 14:25:12 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:25:12.325704 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33817c4b-1ee0-414d-b67d-8533c9759493 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:12.437391 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a69b89-c58b-4c92-96a1-eb27149c7090 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:12.464753 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33817c4b-1ee0-414d-b67d-8533c9759493 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:12.478111 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:12.483162 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b955322e-b74b-45aa-aded-7512b9988118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:12.483844 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b955322e-b74b-45aa-aded-7512b9988118 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:12.484132 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b955322e-b74b-45aa-aded-7512b9988118 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:12.484411 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b955322e-b74b-45aa-aded-7512b9988118 None admin] PATH_INFO: `/v3/projects` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:12.485328 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b955322e-b74b-45aa-aded-7512b9988118 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:25:12.485532 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b955322e-b74b-45aa-aded-7512b9988118 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:12.487024 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b955322e-b74b-45aa-aded-7512b9988118 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:12.501628 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 472/940] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:25:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:25:12.508463 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b955322e-b74b-45aa-aded-7512b9988118 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:12.513081 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e41865-ec46-4373-a3c5-12f075b84c99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:12.513757 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e41865-ec46-4373-a3c5-12f075b84c99 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:12.514070 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e41865-ec46-4373-a3c5-12f075b84c99 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:12.514348 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e41865-ec46-4373-a3c5-12f075b84c99 None admin] PATH_INFO: `/v3/services` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:12.515120 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e41865-ec46-4373-a3c5-12f075b84c99 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:25:12.515437 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e41865-ec46-4373-a3c5-12f075b84c99 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:12.516960 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e41865-ec46-4373-a3c5-12f075b84c99 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:12.522984 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 469/941] 10.222.0.57 () {56 vars in 1122 bytes} [Thu May 19 14:25:12 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:25:12.530799 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e41865-ec46-4373-a3c5-12f075b84c99 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:12.587092 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:12.626942 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b955322e-b74b-45aa-aded-7512b9988118 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:12.643374 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e41865-ec46-4373-a3c5-12f075b84c99 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:12.683748 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1e114e-dc98-4c98-a148-d002fb28941b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:12.685235 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1e114e-dc98-4c98-a148-d002fb28941b None admin] REQUEST_METHOD: `PUT` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:12.685528 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1e114e-dc98-4c98-a148-d002fb28941b None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:12.685804 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1e114e-dc98-4c98-a148-d002fb28941b None admin] PATH_INFO: `/v3/projects/252d9673f08c479a98d04fee3f5856a4/users/7977ff2f29eb435cbfeaa0b61d1d7a5d/roles/6efe7f48768b4033a20e04bc93529e6d` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:12.689099 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1e114e-dc98-4c98-a148-d002fb28941b None admin] RBAC: Authorizing `identity:create_grant(project_id=252d9673f08c479a98d04fee3f5856a4, user_id=7977ff2f29eb435cbfeaa0b61d1d7a5d, role_id=6efe7f48768b4033a20e04bc93529e6d)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:12.690815 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1e114e-dc98-4c98-a148-d002fb28941b None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:12.697759 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d8e380-e18c-4de9-a3f4-48b73e7b0cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:12.698823 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d8e380-e18c-4de9-a3f4-48b73e7b0cb1 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:12.699326 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d8e380-e18c-4de9-a3f4-48b73e7b0cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:12.699800 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d8e380-e18c-4de9-a3f4-48b73e7b0cb1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:12.700773 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d8e380-e18c-4de9-a3f4-48b73e7b0cb1 None admin] RBAC: Adding query filter params (interface=public, service_id=5c28edec2a3f43cb86d8ef21f7f492d1, region_id=RegionOne) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:25:12.701384 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d8e380-e18c-4de9-a3f4-48b73e7b0cb1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:12.703053 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d8e380-e18c-4de9-a3f4-48b73e7b0cb1 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:12.709917 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 473/942] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 14:25:12 2022] PUT /identity/v3/projects/252d9673f08c479a98d04fee3f5856a4/users/7977ff2f29eb435cbfeaa0b61d1d7a5d/roles/6efe7f48768b4033a20e04bc93529e6d => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 14:25:12.716752 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 470/943] 10.222.0.57 () {56 vars in 1251 bytes} [Thu May 19 14:25:12 2022] GET /identity/v3/endpoints?service_id=5c28edec2a3f43cb86d8ef21f7f492d1&interface=public®ion_id=RegionOne => generated 194 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:25:12.723106 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1e114e-dc98-4c98-a148-d002fb28941b None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:12.843102 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1e114e-dc98-4c98-a148-d002fb28941b None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:12.877056 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:12.888421 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9827bf2-7917-49f2-b881-cb934d5c4acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:12.889192 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9827bf2-7917-49f2-b881-cb934d5c4acf None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:12.889523 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9827bf2-7917-49f2-b881-cb934d5c4acf None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:12.890228 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9827bf2-7917-49f2-b881-cb934d5c4acf None admin] PATH_INFO: `/v3/services` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:12.891202 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9827bf2-7917-49f2-b881-cb934d5c4acf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:12.892943 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9827bf2-7917-49f2-b881-cb934d5c4acf None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:12.900006 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 474/944] 10.222.0.57 () {56 vars in 1087 bytes} [Thu May 19 14:25:12 2022] GET /identity/v3/services => generated 1497 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 14:25:13.797735 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2368e808-a73f-40b2-9f8a-5c812f4abbb5 None None] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:13.798261 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2368e808-a73f-40b2-9f8a-5c812f4abbb5 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:13.798366 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2368e808-a73f-40b2-9f8a-5c812f4abbb5 None None] PATH_INFO: `/` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:13.799923 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 471/945] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:25:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:25:13.805275 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8bd630-58ca-4bd1-8efd-29af434624ce None None] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:13.805657 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8bd630-58ca-4bd1-8efd-29af434624ce None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:13.805832 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8bd630-58ca-4bd1-8efd-29af434624ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:13.823269 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.auth.core [None req-6d8bd630-58ca-4bd1-8efd-29af434624ce None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143586) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:25:13.824450 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.fernet_utils [None req-6d8bd630-58ca-4bd1-8efd-29af434624ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143586) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:25:13.892911 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 475/946] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:25:13 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:25:13.917739 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2368e808-a73f-40b2-9f8a-5c812f4abbb5 None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:13.959812 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:14.030375 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2368e808-a73f-40b2-9f8a-5c812f4abbb5 None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:14.065527 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cd5096-4f92-4e55-9bde-ec5964d9aee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:14.066219 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cd5096-4f92-4e55-9bde-ec5964d9aee8 None admin] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:14.066499 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cd5096-4f92-4e55-9bde-ec5964d9aee8 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:14.066772 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cd5096-4f92-4e55-9bde-ec5964d9aee8 None admin] PATH_INFO: `/v3/projects` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:14.067487 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cd5096-4f92-4e55-9bde-ec5964d9aee8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:14.068918 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cd5096-4f92-4e55-9bde-ec5964d9aee8 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:14.070308 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: WARNING keystone.server.flask.common [None req-31cd5096-4f92-4e55-9bde-ec5964d9aee8 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 19 14:25:14.091267 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.sql.core [None req-31cd5096-4f92-4e55-9bde-ec5964d9aee8 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 19 14:25:14.091267 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 19 14:25:14.091267 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [parameters: {'id': '6dabf262491347878cb0f3be4785822c', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 19 14:25:14.091267 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=143587) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 19 14:25:14.100074 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-31cd5096-4f92-4e55-9bde-ec5964d9aee8 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 6dabf262491347878cb0f3be4785822c.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 6dabf262491347878cb0f3be4785822c. May 19 14:25:14.100074 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:25:14.100074 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 14:25:14.100074 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 14:25:14.100074 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 14:25:14.100074 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 14:25:14.100074 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 14:25:14.100074 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application result = self._query(query) May 19 14:25:14.100074 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 14:25:14.100074 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application conn.query(q) May 19 14:25:14.100074 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 14:25:14.100074 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 14:25:14.100074 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 14:25:14.100864 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application result.read() May 19 14:25:14.100864 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 14:25:14.100864 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 14:25:14.100864 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 14:25:14.100864 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 14:25:14.100864 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 14:25:14.100864 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 14:25:14.100864 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 14:25:14.100864 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 14:25:14.100864 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 19 14:25:14.100864 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:25:14.100864 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 19 14:25:14.100864 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:25:14.100864 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:25:14.100864 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 19 14:25:14.100864 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 19 14:25:14.100864 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project May 19 14:25:14.100864 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return project_ref.to_dict() May 19 14:25:14.101727 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 14:25:14.101727 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application next(self.gen) May 19 14:25:14.101727 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 19 14:25:14.101727 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application yield resource May 19 14:25:14.101727 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 14:25:14.101727 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application next(self.gen) May 19 14:25:14.101727 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 19 14:25:14.101727 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application self.session.rollback() May 19 14:25:14.101727 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 19 14:25:14.101727 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application self.force_reraise() May 19 14:25:14.101727 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 19 14:25:14.101727 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise self.value May 19 14:25:14.101727 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 19 14:25:14.101727 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 19 14:25:14.101727 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 19 14:25:14.101727 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application session.commit() May 19 14:25:14.101727 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 19 14:25:14.102570 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 19 14:25:14.102570 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 19 14:25:14.102570 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application self._prepare_impl() May 19 14:25:14.102570 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 19 14:25:14.102570 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application self.session.flush() May 19 14:25:14.102570 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 19 14:25:14.102570 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application self._flush(objects) May 19 14:25:14.102570 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 19 14:25:14.102570 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 19 14:25:14.102570 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 19 14:25:14.102570 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application compat.raise_( May 19 14:25:14.102570 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 14:25:14.102570 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception May 19 14:25:14.102570 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 19 14:25:14.102570 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application flush_context.execute() May 19 14:25:14.102570 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 19 14:25:14.102570 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rec.execute(self) May 19 14:25:14.103380 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 19 14:25:14.103380 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 19 14:25:14.103380 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 19 14:25:14.103380 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application _emit_insert_statements( May 19 14:25:14.103380 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 19 14:25:14.103380 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application c = connection._execute_20( May 19 14:25:14.103380 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 19 14:25:14.103380 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 19 14:25:14.103380 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 19 14:25:14.103380 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 19 14:25:14.103380 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 19 14:25:14.103380 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application ret = self._execute_context( May 19 14:25:14.103380 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 19 14:25:14.103380 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 19 14:25:14.103380 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 19 14:25:14.105790 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 19 14:25:14.105790 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 14:25:14.105790 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception May 19 14:25:14.105790 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 14:25:14.105790 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 14:25:14.105790 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 14:25:14.105790 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 14:25:14.105790 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 14:25:14.105790 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application result = self._query(query) May 19 14:25:14.105790 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 14:25:14.105790 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application conn.query(q) May 19 14:25:14.105790 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 14:25:14.105790 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 14:25:14.105790 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 14:25:14.105790 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application result.read() May 19 14:25:14.105790 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 14:25:14.105790 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 14:25:14.106880 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 14:25:14.106880 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 14:25:14.106880 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 14:25:14.106880 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 14:25:14.106880 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 14:25:14.106880 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 14:25:14.106880 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 19 14:25:14.106880 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 19 14:25:14.106880 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [parameters: {'id': '6dabf262491347878cb0f3be4785822c', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 19 14:25:14.106880 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 19 14:25:14.106880 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:25:14.106880 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:25:14.106880 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:25:14.106880 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:25:14.106880 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project May 19 14:25:14.108615 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) May 19 14:25:14.108615 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 19 14:25:14.108615 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 19 14:25:14.108615 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. May 19 14:25:14.108615 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:25:14.108615 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:25:14.108615 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:25:14.108615 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:25:14.108615 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:25:14.108615 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:25:14.108615 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:25:14.108615 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:25:14.108615 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:25:14.108615 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:25:14.108615 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:25:14.108615 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:25:14.108615 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:25:14.110842 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:25:14.110842 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post May 19 14:25:14.110842 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( May 19 14:25:14.110842 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:25:14.110842 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:25:14.110842 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project May 19 14:25:14.110842 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.Conflict( May 19 14:25:14.110842 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 6dabf262491347878cb0f3be4785822c. May 19 14:25:14.110842 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:25:14.110842 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 472/947] 10.222.0.57 () {60 vars in 1140 bytes} [Thu May 19 14:25:13 2022] POST /identity/v3/projects => generated 262 bytes in 185 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 19 14:25:14.119215 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8bd630-58ca-4bd1-8efd-29af434624ce None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:14.215753 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8bd630-58ca-4bd1-8efd-29af434624ce None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:14.250257 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:14.255948 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c626dc4b-e99b-480e-a5d0-822ba07cd6b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:14.256847 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c626dc4b-e99b-480e-a5d0-822ba07cd6b3 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:14.257133 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c626dc4b-e99b-480e-a5d0-822ba07cd6b3 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:14.257408 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c626dc4b-e99b-480e-a5d0-822ba07cd6b3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:14.265307 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c626dc4b-e99b-480e-a5d0-822ba07cd6b3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:14.267034 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c626dc4b-e99b-480e-a5d0-822ba07cd6b3 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:14.283476 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-c626dc4b-e99b-480e-a5d0-822ba07cd6b3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:25:14.283476 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:25:14.283476 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:25:14.283476 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:25:14.283476 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:25:14.283476 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:25:14.283476 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:25:14.283476 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:25:14.283476 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:25:14.283476 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:25:14.283476 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:25:14.283476 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:25:14.283476 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:25:14.283476 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:25:14.283476 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:25:14.284307 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:25:14.284307 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:25:14.284307 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:25:14.284307 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:25:14.284307 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:25:14.284307 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:25:14.284307 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:25:14.284307 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:25:14.284307 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:25:14.284307 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:25:14.284307 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:25:14.284307 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:25:14.284307 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:25:14.284307 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:25:14.284307 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:25:14.284307 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:25:14.284307 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:25:14.285181 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:25:14.285181 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:25:14.285181 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:25:14.285181 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:25:14.285181 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:25:14.285181 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:25:14.285181 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:25:14.285181 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:25:14.285181 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 476/948] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:25:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:25:14.290632 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cd5096-4f92-4e55-9bde-ec5964d9aee8 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:14.392823 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:14.402773 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cd5096-4f92-4e55-9bde-ec5964d9aee8 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:14.438929 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242f5209-43cf-4499-a156-8a244f288dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:14.439595 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242f5209-43cf-4499-a156-8a244f288dd6 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:14.439874 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242f5209-43cf-4499-a156-8a244f288dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:14.440148 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242f5209-43cf-4499-a156-8a244f288dd6 None admin] PATH_INFO: `/v3/projects` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:14.440902 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242f5209-43cf-4499-a156-8a244f288dd6 None admin] RBAC: Adding query filter params (name=service) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:25:14.441209 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242f5209-43cf-4499-a156-8a244f288dd6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:14.442867 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242f5209-43cf-4499-a156-8a244f288dd6 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:14.464121 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 473/949] 10.222.0.57 () {56 vars in 1112 bytes} [Thu May 19 14:25:14 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:25:14.704529 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10c4afc-029e-4538-891f-38dd457a073d None None] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:14.705210 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10c4afc-029e-4538-891f-38dd457a073d None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:14.705699 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10c4afc-029e-4538-891f-38dd457a073d None None] PATH_INFO: `/` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:14.707604 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 477/950] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:25:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:25:14.713965 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f539dee6-0e6b-4850-8129-0724ce4480dd None None] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:14.714510 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f539dee6-0e6b-4850-8129-0724ce4480dd None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:14.714989 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f539dee6-0e6b-4850-8129-0724ce4480dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:14.734894 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.auth.core [None req-f539dee6-0e6b-4850-8129-0724ce4480dd None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143587) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:25:14.736432 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-f539dee6-0e6b-4850-8129-0724ce4480dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:25:14.814156 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 474/951] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:25:14 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:25:14.833812 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10c4afc-029e-4538-891f-38dd457a073d None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:14.953984 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10c4afc-029e-4538-891f-38dd457a073d None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:14.990687 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b2089e-9736-424a-b0ee-f8bbcf084450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:14.991483 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b2089e-9736-424a-b0ee-f8bbcf084450 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:14.991765 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b2089e-9736-424a-b0ee-f8bbcf084450 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:14.992043 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b2089e-9736-424a-b0ee-f8bbcf084450 None admin] PATH_INFO: `/v3/services/compute` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:14.992747 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b2089e-9736-424a-b0ee-f8bbcf084450 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:14.994371 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b2089e-9736-424a-b0ee-f8bbcf084450 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:15.006705 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-05b2089e-9736-424a-b0ee-f8bbcf084450 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 19 14:25:15.006705 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:25:15.006705 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:25:15.006705 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:25:15.006705 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:25:15.006705 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:25:15.006705 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:25:15.006705 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:25:15.006705 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 14:25:15.006705 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:25:15.006705 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:25:15.006705 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:25:15.006705 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:25:15.006705 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:25:15.006705 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:25:15.006705 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:25:15.006705 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:25:15.007611 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:25:15.007611 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:25:15.007611 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:25:15.007611 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:25:15.007611 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:25:15.007611 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:25:15.007611 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:25:15.007611 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:25:15.007611 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:25:15.007611 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:25:15.007611 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:25:15.007611 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:25:15.007611 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:25:15.007611 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:25:15.007611 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:25:15.007611 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:25:15.008587 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:25:15.008587 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:25:15.008587 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:25:15.008587 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:25:15.008587 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:25:15.008587 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:25:15.008587 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:25:15.008587 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:25:15.008587 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:25:15.008587 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:25:15.008587 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:25:15.008587 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:25:15.008587 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 14:25:15.008587 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:25:15.008587 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 478/952] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:25:14 2022] GET /identity/v3/services/compute => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:25:15.015835 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f539dee6-0e6b-4850-8129-0724ce4480dd None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:15.109115 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ab4676-2ab2-4d2d-b708-3ae4b40cf303 None None] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:15.109422 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ab4676-2ab2-4d2d-b708-3ae4b40cf303 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:15.109674 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ab4676-2ab2-4d2d-b708-3ae4b40cf303 None None] PATH_INFO: `/` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:15.111222 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 479/953] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:25:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:25:15.112663 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:15.116313 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d6f21f-b6b6-4965-89e9-e442ad011138 None None] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:15.116816 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d6f21f-b6b6-4965-89e9-e442ad011138 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:15.117254 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d6f21f-b6b6-4965-89e9-e442ad011138 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:15.128611 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f539dee6-0e6b-4850-8129-0724ce4480dd None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:15.138245 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.auth.core [None req-33d6f21f-b6b6-4965-89e9-e442ad011138 None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143586) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:25:15.140058 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.fernet_utils [None req-33d6f21f-b6b6-4965-89e9-e442ad011138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143586) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:25:15.178222 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe64d76-1b9a-46ab-b2c7-cdbe49d14fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:15.179197 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe64d76-1b9a-46ab-b2c7-cdbe49d14fce None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:15.179484 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe64d76-1b9a-46ab-b2c7-cdbe49d14fce None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:15.179763 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe64d76-1b9a-46ab-b2c7-cdbe49d14fce None admin] PATH_INFO: `/v3/services` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:15.180760 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe64d76-1b9a-46ab-b2c7-cdbe49d14fce None admin] RBAC: Adding query filter params (name=compute) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:25:15.181071 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe64d76-1b9a-46ab-b2c7-cdbe49d14fce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:15.183176 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe64d76-1b9a-46ab-b2c7-cdbe49d14fce None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:15.189885 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 475/954] 10.222.0.57 () {56 vars in 1112 bytes} [Thu May 19 14:25:15 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:25:15.196155 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe64d76-1b9a-46ab-b2c7-cdbe49d14fce None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:15.204809 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:15.228289 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 480/955] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:25:15 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:25:15.252484 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d6f21f-b6b6-4965-89e9-e442ad011138 None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:15.320472 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe64d76-1b9a-46ab-b2c7-cdbe49d14fce None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:15.366035 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2ed471-dac4-4ac3-a331-9f386adc7f76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:15.367009 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2ed471-dac4-4ac3-a331-9f386adc7f76 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:15.367291 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2ed471-dac4-4ac3-a331-9f386adc7f76 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:15.367721 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2ed471-dac4-4ac3-a331-9f386adc7f76 None admin] PATH_INFO: `/v3/services` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:15.368747 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2ed471-dac4-4ac3-a331-9f386adc7f76 None admin] RBAC: Adding query filter params (type=compute) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:25:15.369733 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2ed471-dac4-4ac3-a331-9f386adc7f76 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:15.371390 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2ed471-dac4-4ac3-a331-9f386adc7f76 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:15.371871 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d6f21f-b6b6-4965-89e9-e442ad011138 None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:15.381875 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 476/956] 10.222.0.57 () {56 vars in 1112 bytes} [Thu May 19 14:25:15 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:25:15.381875 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49fdf95-5282-4a25-a5ca-3a5ba281676d None None] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:15.381875 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49fdf95-5282-4a25-a5ca-3a5ba281676d None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:15.381875 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49fdf95-5282-4a25-a5ca-3a5ba281676d None None] PATH_INFO: `/` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:15.381875 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 477/957] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:25:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:25:15.388779 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13de435-6a04-4876-8f7c-16aaa083a9ab None None] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:15.389160 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13de435-6a04-4876-8f7c-16aaa083a9ab None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:15.389514 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13de435-6a04-4876-8f7c-16aaa083a9ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:15.407787 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.auth.core [None req-a13de435-6a04-4876-8f7c-16aaa083a9ab None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143587) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:25:15.409163 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-a13de435-6a04-4876-8f7c-16aaa083a9ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:25:15.416814 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c8b11a-a42d-49d3-8c31-86db22404d12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:15.417677 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c8b11a-a42d-49d3-8c31-86db22404d12 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:15.417996 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c8b11a-a42d-49d3-8c31-86db22404d12 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:15.418923 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c8b11a-a42d-49d3-8c31-86db22404d12 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:15.426233 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:15.435512 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c8b11a-a42d-49d3-8c31-86db22404d12 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:25:15.446953 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-d0c8b11a-a42d-49d3-8c31-86db22404d12 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 14:25:15.446953 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:25:15.446953 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:25:15.446953 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:25:15.446953 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:25:15.446953 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:25:15.446953 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:25:15.446953 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:25:15.446953 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:25:15.446953 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:25:15.446953 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:25:15.446953 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:25:15.446953 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:25:15.446953 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:25:15.446953 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:25:15.446953 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:25:15.448540 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:25:15.448540 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:25:15.448540 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:25:15.448540 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:25:15.448540 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:25:15.448540 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:25:15.448540 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:25:15.448540 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:25:15.448540 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:25:15.448540 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:25:15.448540 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:25:15.448540 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:25:15.448540 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:25:15.448540 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:25:15.448540 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:25:15.448540 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:25:15.449452 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:25:15.449452 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:25:15.449452 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:25:15.449452 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:25:15.449452 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:25:15.449452 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:25:15.449452 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:25:15.449452 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:25:15.449452 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:25:15.449452 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:25:15.449452 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 14:25:15.449452 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:25:15.449452 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 481/958] 10.222.0.57 () {56 vars in 1120 bytes} [Thu May 19 14:25:15 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:25:15.456584 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c8b11a-a42d-49d3-8c31-86db22404d12 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:15.501994 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 478/959] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:25:15 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:25:15.514255 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:15.523740 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13de435-6a04-4876-8f7c-16aaa083a9ab None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:15.595736 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c8b11a-a42d-49d3-8c31-86db22404d12 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:15.633329 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53450071-5bd2-404c-a190-dab8eb7a7db6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:15.634055 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53450071-5bd2-404c-a190-dab8eb7a7db6 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:15.634338 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53450071-5bd2-404c-a190-dab8eb7a7db6 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:15.634613 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53450071-5bd2-404c-a190-dab8eb7a7db6 None admin] PATH_INFO: `/v3/roles` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:15.635655 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53450071-5bd2-404c-a190-dab8eb7a7db6 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:25:15.636134 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53450071-5bd2-404c-a190-dab8eb7a7db6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:15.637629 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53450071-5bd2-404c-a190-dab8eb7a7db6 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:15.649886 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 482/960] 10.222.0.57 () {56 vars in 1115 bytes} [Thu May 19 14:25:15 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:25:15.652869 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53450071-5bd2-404c-a190-dab8eb7a7db6 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:15.664048 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13de435-6a04-4876-8f7c-16aaa083a9ab None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:15.704178 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d597fb5f-01e3-4de7-92c9-50511ea8ef97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:15.705225 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d597fb5f-01e3-4de7-92c9-50511ea8ef97 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:15.705512 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d597fb5f-01e3-4de7-92c9-50511ea8ef97 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:15.705788 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d597fb5f-01e3-4de7-92c9-50511ea8ef97 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:15.706518 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d597fb5f-01e3-4de7-92c9-50511ea8ef97 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:15.708023 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d597fb5f-01e3-4de7-92c9-50511ea8ef97 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:15.722259 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-d597fb5f-01e3-4de7-92c9-50511ea8ef97 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 14:25:15.722259 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:25:15.722259 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:25:15.722259 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:25:15.722259 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:25:15.722259 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:25:15.722259 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:25:15.722259 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:25:15.722259 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 14:25:15.722259 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:25:15.722259 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:25:15.722259 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:25:15.722259 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:25:15.722259 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:25:15.722259 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:25:15.722259 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:25:15.722259 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:25:15.723480 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:25:15.723480 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:25:15.723480 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:25:15.723480 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:25:15.723480 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:25:15.723480 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:25:15.723480 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:25:15.723480 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:25:15.723480 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:25:15.723480 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:25:15.723480 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:25:15.723480 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:25:15.723480 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:25:15.723480 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:25:15.723480 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:25:15.723480 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:25:15.724225 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:25:15.724225 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:25:15.724225 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:25:15.724225 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:25:15.724225 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:25:15.724225 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:25:15.724225 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:25:15.724225 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:25:15.724225 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:25:15.724225 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:25:15.724225 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:25:15.724225 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:25:15.724225 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 14:25:15.724225 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:25:15.729421 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 479/961] 10.222.0.57 () {56 vars in 1126 bytes} [Thu May 19 14:25:15 2022] GET /identity/v3/services/object-store => generated 93 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:25:15.730658 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d597fb5f-01e3-4de7-92c9-50511ea8ef97 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:15.734670 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:15.770037 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53450071-5bd2-404c-a190-dab8eb7a7db6 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:15.772741 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:15.818454 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c70a6f-f9eb-434d-8181-d663bd20bf59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:15.819570 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c70a6f-f9eb-434d-8181-d663bd20bf59 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:15.819932 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c70a6f-f9eb-434d-8181-d663bd20bf59 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:15.820515 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c70a6f-f9eb-434d-8181-d663bd20bf59 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:15.834175 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c70a6f-f9eb-434d-8181-d663bd20bf59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:15.836161 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c70a6f-f9eb-434d-8181-d663bd20bf59 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:15.845312 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d597fb5f-01e3-4de7-92c9-50511ea8ef97 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:15.858216 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-b7c70a6f-f9eb-434d-8181-d663bd20bf59 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:25:15.858216 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:25:15.858216 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:25:15.858216 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:25:15.858216 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:25:15.858216 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:25:15.858216 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:25:15.858216 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:25:15.858216 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:25:15.858216 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:25:15.858216 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:25:15.858216 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:25:15.858216 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:25:15.858216 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:25:15.858216 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:25:15.859221 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:25:15.859221 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:25:15.859221 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:25:15.859221 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:25:15.859221 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:25:15.859221 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:25:15.859221 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:25:15.859221 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:25:15.859221 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:25:15.859221 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:25:15.859221 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:25:15.859221 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:25:15.859221 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:25:15.859221 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:25:15.859221 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:25:15.859221 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:25:15.859221 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:25:15.860155 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:25:15.860155 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:25:15.860155 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:25:15.860155 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:25:15.861917 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 483/962] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:25:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:25:15.865097 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c70a6f-f9eb-434d-8181-d663bd20bf59 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:15.901338 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c0c44b-69bd-41f9-a50d-3a6e0c7ee4dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:15.902410 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c0c44b-69bd-41f9-a50d-3a6e0c7ee4dd None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:15.902694 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c0c44b-69bd-41f9-a50d-3a6e0c7ee4dd None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:15.902967 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c0c44b-69bd-41f9-a50d-3a6e0c7ee4dd None admin] PATH_INFO: `/v3/services` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:15.904081 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c0c44b-69bd-41f9-a50d-3a6e0c7ee4dd None admin] RBAC: Adding query filter params (name=object-store) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:25:15.904394 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c0c44b-69bd-41f9-a50d-3a6e0c7ee4dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:15.906121 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c0c44b-69bd-41f9-a50d-3a6e0c7ee4dd None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:15.913153 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 480/963] 10.222.0.57 () {56 vars in 1122 bytes} [Thu May 19 14:25:15 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:25:15.929781 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c0c44b-69bd-41f9-a50d-3a6e0c7ee4dd None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:15.987977 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c70a6f-f9eb-434d-8181-d663bd20bf59 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:16.023452 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c340beae-41e9-4fd8-9bb9-2e113728e85a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:16.024043 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:16.024159 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c340beae-41e9-4fd8-9bb9-2e113728e85a None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:16.024404 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c340beae-41e9-4fd8-9bb9-2e113728e85a None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:16.025177 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c340beae-41e9-4fd8-9bb9-2e113728e85a None admin] PATH_INFO: `/v3/domains` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:16.026400 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c340beae-41e9-4fd8-9bb9-2e113728e85a None admin] RBAC: Adding query filter params (name=Default) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:25:16.026752 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c340beae-41e9-4fd8-9bb9-2e113728e85a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:16.028139 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c340beae-41e9-4fd8-9bb9-2e113728e85a None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:16.046612 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 484/964] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:25:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:25:16.047930 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c340beae-41e9-4fd8-9bb9-2e113728e85a None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:16.064725 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:16.067945 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c0c44b-69bd-41f9-a50d-3a6e0c7ee4dd None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:16.107090 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13881bf6-890b-43d0-9111-57d174f20905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:16.108181 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13881bf6-890b-43d0-9111-57d174f20905 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:16.108742 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13881bf6-890b-43d0-9111-57d174f20905 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:16.109333 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13881bf6-890b-43d0-9111-57d174f20905 None admin] PATH_INFO: `/v3/services` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:16.110554 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13881bf6-890b-43d0-9111-57d174f20905 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:25:16.111050 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13881bf6-890b-43d0-9111-57d174f20905 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:16.113187 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13881bf6-890b-43d0-9111-57d174f20905 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:16.119638 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 481/965] 10.222.0.57 () {56 vars in 1122 bytes} [Thu May 19 14:25:15 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:25:16.125052 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13881bf6-890b-43d0-9111-57d174f20905 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:16.175498 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c340beae-41e9-4fd8-9bb9-2e113728e85a None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:16.215389 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcdf079-c1e9-4742-9994-f23b6f1d832d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:16.216214 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcdf079-c1e9-4742-9994-f23b6f1d832d None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:16.216500 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcdf079-c1e9-4742-9994-f23b6f1d832d None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:16.216787 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcdf079-c1e9-4742-9994-f23b6f1d832d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:16.224903 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcdf079-c1e9-4742-9994-f23b6f1d832d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:16.227093 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcdf079-c1e9-4742-9994-f23b6f1d832d None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:16.245489 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-cfcdf079-c1e9-4742-9994-f23b6f1d832d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 14:25:16.245489 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:25:16.245489 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:25:16.245489 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:25:16.245489 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:25:16.245489 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:25:16.245489 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:25:16.245489 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:25:16.245489 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:25:16.245489 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:25:16.245489 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:25:16.245489 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:25:16.245489 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:25:16.245489 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:25:16.245489 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:25:16.245489 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:25:16.246579 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:25:16.246579 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:25:16.246579 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:25:16.246579 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:25:16.246579 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:25:16.246579 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:25:16.246579 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:25:16.246579 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:25:16.246579 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:25:16.246579 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:25:16.246579 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:25:16.246579 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:25:16.246579 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:25:16.246579 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:25:16.246579 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:25:16.246579 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:25:16.246579 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:25:16.246579 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:25:16.247621 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:25:16.247621 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:25:16.247621 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:25:16.247621 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:25:16.247621 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:25:16.247621 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:25:16.247621 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:25:16.247621 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:25:16.247621 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 14:25:16.247621 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:25:16.254522 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 485/966] 10.222.0.57 () {56 vars in 1117 bytes} [Thu May 19 14:25:16 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:25:16.255855 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcdf079-c1e9-4742-9994-f23b6f1d832d None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:16.268356 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13881bf6-890b-43d0-9111-57d174f20905 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:16.312314 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:16.318844 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29304ab4-bb86-418d-9185-6e2d4cc345f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:16.320055 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29304ab4-bb86-418d-9185-6e2d4cc345f5 None admin] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:16.320778 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29304ab4-bb86-418d-9185-6e2d4cc345f5 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:16.321383 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29304ab4-bb86-418d-9185-6e2d4cc345f5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:16.322521 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29304ab4-bb86-418d-9185-6e2d4cc345f5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:16.323910 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29304ab4-bb86-418d-9185-6e2d4cc345f5 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:16.350012 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 482/967] 10.222.0.57 () {60 vars in 1144 bytes} [Thu May 19 14:25:16 2022] POST /identity/v3/endpoints => generated 349 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 14:25:16.365769 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:16.387567 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcdf079-c1e9-4742-9994-f23b6f1d832d None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:16.435684 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26d5da3-0543-4013-9ea0-5594c8fe8661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:16.436713 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26d5da3-0543-4013-9ea0-5594c8fe8661 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:16.437186 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26d5da3-0543-4013-9ea0-5594c8fe8661 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:16.437605 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26d5da3-0543-4013-9ea0-5594c8fe8661 None admin] PATH_INFO: `/v3/users` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:16.438690 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26d5da3-0543-4013-9ea0-5594c8fe8661 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:25:16.439147 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26d5da3-0543-4013-9ea0-5594c8fe8661 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:16.440776 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26d5da3-0543-4013-9ea0-5594c8fe8661 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:16.462566 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 486/968] 10.222.0.57 () {56 vars in 1149 bytes} [Thu May 19 14:25:16 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:25:16.471049 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29304ab4-bb86-418d-9185-6e2d4cc345f5 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:16.588126 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29304ab4-bb86-418d-9185-6e2d4cc345f5 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:16.624370 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84608082-6436-43ae-aac8-597c0a3102f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:16.625032 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84608082-6436-43ae-aac8-597c0a3102f9 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:16.625315 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84608082-6436-43ae-aac8-597c0a3102f9 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:16.625599 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84608082-6436-43ae-aac8-597c0a3102f9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:16.696711 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:16.713066 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84608082-6436-43ae-aac8-597c0a3102f9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:16.715043 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84608082-6436-43ae-aac8-597c0a3102f9 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:16.824212 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 483/969] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:25:16 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:25:16.829752 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26d5da3-0543-4013-9ea0-5594c8fe8661 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:16.879607 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bd7f85-823b-4a87-97ad-a3e649da6f75 None None] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:16.880096 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bd7f85-823b-4a87-97ad-a3e649da6f75 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:16.880194 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bd7f85-823b-4a87-97ad-a3e649da6f75 None None] PATH_INFO: `/` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:16.885780 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 484/970] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:25:16 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:25:16.887003 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dd4d44-4693-4038-be8e-90844287a146 None None] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:16.887288 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dd4d44-4693-4038-be8e-90844287a146 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:16.887563 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dd4d44-4693-4038-be8e-90844287a146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:16.906038 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.auth.core [None req-63dd4d44-4693-4038-be8e-90844287a146 None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143587) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:25:16.908081 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-63dd4d44-4693-4038-be8e-90844287a146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:25:16.941869 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26d5da3-0543-4013-9ea0-5594c8fe8661 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:16.970366 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:16.979955 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0be04e-f262-4408-b32a-976a0adf941c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:16.980832 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0be04e-f262-4408-b32a-976a0adf941c None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:16.981187 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0be04e-f262-4408-b32a-976a0adf941c None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:16.981524 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0be04e-f262-4408-b32a-976a0adf941c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:16.983175 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 485/971] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:25:16 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:25:16.999774 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0be04e-f262-4408-b32a-976a0adf941c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:17.001347 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0be04e-f262-4408-b32a-976a0adf941c None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:17.011875 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dd4d44-4693-4038-be8e-90844287a146 None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:17.021874 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-9b0be04e-f262-4408-b32a-976a0adf941c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:25:17.021874 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:25:17.021874 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:25:17.021874 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:25:17.021874 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:25:17.021874 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:25:17.021874 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:25:17.021874 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:25:17.021874 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:25:17.021874 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:25:17.021874 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:25:17.021874 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:25:17.021874 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:25:17.021874 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:25:17.021874 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:25:17.022679 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:25:17.022679 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:25:17.022679 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:25:17.022679 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:25:17.022679 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:25:17.022679 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:25:17.022679 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:25:17.022679 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:25:17.022679 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:25:17.022679 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:25:17.022679 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:25:17.022679 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:25:17.022679 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:25:17.022679 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:25:17.022679 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:25:17.022679 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:25:17.022679 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:25:17.023479 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:25:17.023479 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:25:17.023479 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:25:17.023479 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:25:17.023479 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 487/972] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:25:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:25:17.029752 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0be04e-f262-4408-b32a-976a0adf941c None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:17.074225 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:17.149317 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0be04e-f262-4408-b32a-976a0adf941c None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:17.154745 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dd4d44-4693-4038-be8e-90844287a146 None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:17.195245 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a57c9c-3819-4d85-bfbe-fd9dfeaeb68c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:17.195912 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a57c9c-3819-4d85-bfbe-fd9dfeaeb68c None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:17.196249 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a57c9c-3819-4d85-bfbe-fd9dfeaeb68c None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:17.196531 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a57c9c-3819-4d85-bfbe-fd9dfeaeb68c None admin] PATH_INFO: `/v3/domains` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:17.197433 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a57c9c-3819-4d85-bfbe-fd9dfeaeb68c None admin] RBAC: Adding query filter params (name=Default) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:25:17.198049 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a57c9c-3819-4d85-bfbe-fd9dfeaeb68c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:17.199138 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a57c9c-3819-4d85-bfbe-fd9dfeaeb68c None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:17.212006 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13736326-510d-4c1a-ac6c-def85a4ae0bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:17.212685 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13736326-510d-4c1a-ac6c-def85a4ae0bd None admin] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:17.213778 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13736326-510d-4c1a-ac6c-def85a4ae0bd None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:17.213778 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13736326-510d-4c1a-ac6c-def85a4ae0bd None admin] PATH_INFO: `/v3/users` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:17.214510 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13736326-510d-4c1a-ac6c-def85a4ae0bd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:17.215952 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13736326-510d-4c1a-ac6c-def85a4ae0bd None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:17.227906 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 488/973] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:25:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:25:17.229974 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a57c9c-3819-4d85-bfbe-fd9dfeaeb68c None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:17.232300 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.sql.core [None req-13736326-510d-4c1a-ac6c-def85a4ae0bd None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 19 14:25:17.232300 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 19 14:25:17.232300 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [parameters: {'user_id': 'ac5d258a4b734d689c7175dfc10da4af', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 19 14:25:17.232300 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=143587) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 19 14:25:17.242553 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-13736326-510d-4c1a-ac6c-def85a4ae0bd None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 19 14:25:17.242553 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:25:17.242553 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 14:25:17.242553 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 14:25:17.242553 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 14:25:17.242553 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 14:25:17.242553 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 14:25:17.242553 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application result = self._query(query) May 19 14:25:17.242553 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 14:25:17.242553 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application conn.query(q) May 19 14:25:17.242553 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 14:25:17.242553 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 14:25:17.242553 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 14:25:17.242553 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application result.read() May 19 14:25:17.242553 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 14:25:17.243488 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 14:25:17.243488 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 14:25:17.243488 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 14:25:17.243488 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 14:25:17.243488 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 14:25:17.243488 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 14:25:17.243488 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 14:25:17.243488 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 19 14:25:17.243488 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:25:17.243488 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 19 14:25:17.243488 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:25:17.243488 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:25:17.243488 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 19 14:25:17.243488 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 19 14:25:17.243488 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user May 19 14:25:17.243488 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) May 19 14:25:17.243488 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 14:25:17.243488 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application next(self.gen) May 19 14:25:17.244689 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 19 14:25:17.244689 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application yield resource May 19 14:25:17.244689 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 14:25:17.244689 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application next(self.gen) May 19 14:25:17.244689 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 19 14:25:17.244689 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application self.session.rollback() May 19 14:25:17.244689 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 19 14:25:17.244689 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application self.force_reraise() May 19 14:25:17.244689 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 19 14:25:17.244689 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise self.value May 19 14:25:17.244689 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 19 14:25:17.244689 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 19 14:25:17.244689 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 19 14:25:17.244689 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application session.commit() May 19 14:25:17.244689 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 19 14:25:17.244689 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 19 14:25:17.244689 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 19 14:25:17.245658 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application self._prepare_impl() May 19 14:25:17.245658 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 19 14:25:17.245658 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application self.session.flush() May 19 14:25:17.245658 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 19 14:25:17.245658 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application self._flush(objects) May 19 14:25:17.245658 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 19 14:25:17.245658 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 19 14:25:17.245658 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 19 14:25:17.245658 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application compat.raise_( May 19 14:25:17.245658 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 14:25:17.245658 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception May 19 14:25:17.245658 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 19 14:25:17.245658 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application flush_context.execute() May 19 14:25:17.245658 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 19 14:25:17.245658 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rec.execute(self) May 19 14:25:17.245658 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 19 14:25:17.245658 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 19 14:25:17.246728 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 19 14:25:17.246728 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application _emit_insert_statements( May 19 14:25:17.246728 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements May 19 14:25:17.246728 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application result = connection._execute_20( May 19 14:25:17.246728 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 19 14:25:17.246728 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 19 14:25:17.246728 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 19 14:25:17.246728 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 19 14:25:17.246728 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 19 14:25:17.246728 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application ret = self._execute_context( May 19 14:25:17.246728 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 19 14:25:17.246728 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 19 14:25:17.246728 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 19 14:25:17.246728 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 19 14:25:17.246728 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 14:25:17.246728 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception May 19 14:25:17.248021 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 14:25:17.248021 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 14:25:17.248021 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 14:25:17.248021 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 14:25:17.248021 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 14:25:17.248021 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application result = self._query(query) May 19 14:25:17.248021 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 14:25:17.248021 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application conn.query(q) May 19 14:25:17.248021 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 14:25:17.248021 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 14:25:17.248021 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 14:25:17.248021 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application result.read() May 19 14:25:17.248021 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 14:25:17.248021 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 14:25:17.248021 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 14:25:17.248021 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 14:25:17.248961 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 14:25:17.248961 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 14:25:17.248961 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 14:25:17.248961 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 14:25:17.248961 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 19 14:25:17.248961 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 19 14:25:17.248961 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [parameters: {'user_id': 'ac5d258a4b734d689c7175dfc10da4af', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 19 14:25:17.248961 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 19 14:25:17.248961 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:25:17.248961 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:25:17.248961 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:25:17.248961 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:25:17.248961 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:25:17.248961 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:25:17.248961 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:25:17.249750 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:25:17.249750 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:25:17.249750 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:25:17.249750 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:25:17.249750 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:25:17.249750 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:25:17.249750 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:25:17.249750 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post May 19 14:25:17.249750 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( May 19 14:25:17.249750 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:25:17.249750 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:25:17.249750 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:25:17.249750 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:25:17.249750 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:25:17.249750 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:25:17.249750 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user May 19 14:25:17.249750 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) May 19 14:25:17.250690 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects May 19 14:25:17.250690 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) May 19 14:25:17.250690 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 19 14:25:17.250690 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 19 14:25:17.250690 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 19 14:25:17.250690 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:25:17.258934 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 486/974] 10.222.0.57 () {60 vars in 1131 bytes} [Thu May 19 14:25:17 2022] POST /identity/v3/users => generated 160 bytes in 243 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) May 19 14:25:17.260312 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13736326-510d-4c1a-ac6c-def85a4ae0bd None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:17.297405 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:17.374968 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:17.388579 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a57c9c-3819-4d85-bfbe-fd9dfeaeb68c None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:17.401975 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13736326-510d-4c1a-ac6c-def85a4ae0bd None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:17.438447 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031785c9-d27d-40a0-b972-a163be956f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:17.440148 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031785c9-d27d-40a0-b972-a163be956f47 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:17.440578 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031785c9-d27d-40a0-b972-a163be956f47 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:17.441247 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031785c9-d27d-40a0-b972-a163be956f47 None admin] PATH_INFO: `/v3/projects/service` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:17.445595 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9019e17c-cd4e-4c28-90ef-587d15c561d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:17.446557 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9019e17c-cd4e-4c28-90ef-587d15c561d0 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:17.446943 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9019e17c-cd4e-4c28-90ef-587d15c561d0 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:17.447352 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9019e17c-cd4e-4c28-90ef-587d15c561d0 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:17.451468 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031785c9-d27d-40a0-b972-a163be956f47 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:17.453068 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031785c9-d27d-40a0-b972-a163be956f47 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:17.457486 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9019e17c-cd4e-4c28-90ef-587d15c561d0 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:17.459346 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9019e17c-cd4e-4c28-90ef-587d15c561d0 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:17.470495 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-031785c9-d27d-40a0-b972-a163be956f47 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:25:17.470495 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:25:17.470495 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:25:17.470495 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:25:17.470495 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:25:17.470495 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:25:17.470495 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:25:17.470495 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:25:17.470495 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:25:17.470495 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:25:17.470495 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:25:17.470495 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:25:17.470495 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:25:17.470495 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:25:17.470495 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:25:17.471373 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:25:17.471373 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:25:17.471373 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:25:17.471373 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:25:17.471373 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:25:17.471373 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:25:17.471373 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:25:17.471373 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:25:17.471373 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:25:17.471373 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:25:17.471373 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:25:17.471373 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:25:17.471373 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:25:17.471373 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:25:17.471373 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:25:17.471373 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:25:17.471373 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:25:17.472287 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:25:17.472287 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:25:17.472287 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:25:17.472287 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:25:17.472287 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:25:17.472287 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:25:17.472287 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:25:17.472287 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:25:17.474604 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 489/975] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:25:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:25:17.483220 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-9019e17c-cd4e-4c28-90ef-587d15c561d0 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 14:25:17.483220 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:25:17.483220 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:25:17.483220 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:25:17.483220 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:25:17.483220 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:25:17.483220 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:25:17.483220 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:25:17.483220 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:25:17.483220 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:25:17.483220 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:25:17.483220 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:25:17.483220 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:25:17.483220 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:25:17.483220 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:25:17.483220 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:25:17.484043 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:25:17.484043 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:25:17.484043 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:25:17.484043 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:25:17.484043 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:25:17.484043 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:25:17.484043 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:25:17.484043 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:25:17.484043 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:25:17.484043 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:25:17.484043 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:25:17.484043 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:25:17.484043 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:25:17.484043 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:25:17.484043 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:25:17.484043 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:25:17.484043 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:25:17.484043 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:25:17.484863 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:25:17.484863 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:25:17.484863 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:25:17.484863 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:25:17.484863 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:25:17.484863 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:25:17.484863 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:25:17.484863 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:25:17.484863 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 14:25:17.484863 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:25:17.484863 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 487/976] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:25:17 2022] GET /identity/v3/users/cinder => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:25:17.488141 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031785c9-d27d-40a0-b972-a163be956f47 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:17.491118 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9019e17c-cd4e-4c28-90ef-587d15c561d0 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:17.613726 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:17.616540 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031785c9-d27d-40a0-b972-a163be956f47 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:17.620292 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9019e17c-cd4e-4c28-90ef-587d15c561d0 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:17.664659 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4524092-9556-42b5-9d43-a0931c04a8e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:17.664659 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4524092-9556-42b5-9d43-a0931c04a8e7 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:17.664659 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4524092-9556-42b5-9d43-a0931c04a8e7 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:17.664659 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4524092-9556-42b5-9d43-a0931c04a8e7 None admin] PATH_INFO: `/v3/projects` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:17.667222 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4524092-9556-42b5-9d43-a0931c04a8e7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:25:17.667711 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4524092-9556-42b5-9d43-a0931c04a8e7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:17.669228 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4524092-9556-42b5-9d43-a0931c04a8e7 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:17.671613 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506debf8-b6a9-4f2d-8eb3-082c14ce0510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:17.672609 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506debf8-b6a9-4f2d-8eb3-082c14ce0510 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:17.673076 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506debf8-b6a9-4f2d-8eb3-082c14ce0510 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:17.673523 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506debf8-b6a9-4f2d-8eb3-082c14ce0510 None admin] PATH_INFO: `/v3/users` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:17.674523 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506debf8-b6a9-4f2d-8eb3-082c14ce0510 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:25:17.675006 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506debf8-b6a9-4f2d-8eb3-082c14ce0510 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:17.676675 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506debf8-b6a9-4f2d-8eb3-082c14ce0510 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:17.687443 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 490/977] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:25:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:25:17.690916 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4524092-9556-42b5-9d43-a0931c04a8e7 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:17.709675 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:17.733874 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 488/978] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:25:17 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:25:17.733874 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cece1a-ad3e-483b-8162-a63e2684c1f1 None None] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:17.733874 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cece1a-ad3e-483b-8162-a63e2684c1f1 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:17.733874 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cece1a-ad3e-483b-8162-a63e2684c1f1 None None] PATH_INFO: `/` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:17.734589 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 489/979] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:25:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:25:17.740487 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19918031-dad6-45f7-a496-207fa639c66f None None] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:17.740775 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19918031-dad6-45f7-a496-207fa639c66f None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:17.741051 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19918031-dad6-45f7-a496-207fa639c66f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:17.764273 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.auth.core [None req-19918031-dad6-45f7-a496-207fa639c66f None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143587) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:25:17.766675 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-19918031-dad6-45f7-a496-207fa639c66f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:25:17.818191 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4524092-9556-42b5-9d43-a0931c04a8e7 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:17.859601 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c285339b-af2a-49e8-a185-ccc3d570965d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:17.861225 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c285339b-af2a-49e8-a185-ccc3d570965d None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:17.861750 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c285339b-af2a-49e8-a185-ccc3d570965d None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:17.862254 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c285339b-af2a-49e8-a185-ccc3d570965d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:17.863375 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c285339b-af2a-49e8-a185-ccc3d570965d None admin] RBAC: Adding query filter params (role.id=6efe7f48768b4033a20e04bc93529e6d, scope.project.id=252d9673f08c479a98d04fee3f5856a4, user.id=7977ff2f29eb435cbfeaa0b61d1d7a5d) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:25:17.863881 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c285339b-af2a-49e8-a185-ccc3d570965d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:17.873870 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 490/980] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:25:17 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:25:17.874089 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c285339b-af2a-49e8-a185-ccc3d570965d None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:17.874472 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 491/981] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:25:17 2022] GET /identity/v3/role_assignments?user.id=7977ff2f29eb435cbfeaa0b61d1d7a5d&scope.project.id=252d9673f08c479a98d04fee3f5856a4&role.id=6efe7f48768b4033a20e04bc93529e6d => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:25:17.890058 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19918031-dad6-45f7-a496-207fa639c66f None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:18.013639 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19918031-dad6-45f7-a496-207fa639c66f None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:18.046468 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:18.054894 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3545f7f5-84be-4121-8515-ccc7afea9914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:18.056157 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3545f7f5-84be-4121-8515-ccc7afea9914 None admin] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:18.056833 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3545f7f5-84be-4121-8515-ccc7afea9914 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:18.057093 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3545f7f5-84be-4121-8515-ccc7afea9914 None admin] PATH_INFO: `/v3/services` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:18.057802 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3545f7f5-84be-4121-8515-ccc7afea9914 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:18.059371 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3545f7f5-84be-4121-8515-ccc7afea9914 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:18.076472 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 491/982] 10.222.0.57 () {60 vars in 1141 bytes} [Thu May 19 14:25:17 2022] POST /identity/v3/services => generated 242 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:25:18.781641 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dface41-f8e8-456f-aa06-fd16f57e2147 None None] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:18.782184 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dface41-f8e8-456f-aa06-fd16f57e2147 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:18.782362 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dface41-f8e8-456f-aa06-fd16f57e2147 None None] PATH_INFO: `/` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:18.783784 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 492/983] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:25:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:25:18.788901 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c6ce68-b8e2-407e-bf42-1dee77686974 None None] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:18.789207 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c6ce68-b8e2-407e-bf42-1dee77686974 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:18.789496 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c6ce68-b8e2-407e-bf42-1dee77686974 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:18.806912 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.auth.core [None req-c4c6ce68-b8e2-407e-bf42-1dee77686974 None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143587) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:25:18.808254 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-c4c6ce68-b8e2-407e-bf42-1dee77686974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:25:18.888154 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 492/984] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:25:18 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:25:18.910747 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dface41-f8e8-456f-aa06-fd16f57e2147 None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:18.937508 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:19.017646 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dface41-f8e8-456f-aa06-fd16f57e2147 None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:19.051824 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be5ae12-09cc-452c-bd90-32c792c1c0c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:19.052633 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be5ae12-09cc-452c-bd90-32c792c1c0c0 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:19.052918 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be5ae12-09cc-452c-bd90-32c792c1c0c0 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:19.053197 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be5ae12-09cc-452c-bd90-32c792c1c0c0 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:19.053871 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be5ae12-09cc-452c-bd90-32c792c1c0c0 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:19.055249 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be5ae12-09cc-452c-bd90-32c792c1c0c0 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:19.068039 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-0be5ae12-09cc-452c-bd90-32c792c1c0c0 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 14:25:19.068039 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:25:19.068039 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:25:19.068039 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:25:19.068039 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:25:19.068039 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:25:19.068039 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:25:19.068039 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:25:19.068039 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 14:25:19.068039 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:25:19.068039 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:25:19.068039 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:25:19.068039 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:25:19.068039 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:25:19.068039 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:25:19.068039 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:25:19.068039 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:25:19.068932 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:25:19.068932 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:25:19.068932 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:25:19.068932 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:25:19.068932 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:25:19.068932 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:25:19.068932 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:25:19.068932 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:25:19.068932 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:25:19.068932 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:25:19.068932 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:25:19.068932 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:25:19.068932 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:25:19.068932 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:25:19.068932 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:25:19.068932 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:25:19.069709 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:25:19.069709 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:25:19.069709 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:25:19.069709 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:25:19.069709 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:25:19.069709 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:25:19.069709 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:25:19.069709 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:25:19.069709 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:25:19.069709 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:25:19.069709 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:25:19.069709 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:25:19.069709 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 14:25:19.069709 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:25:19.074392 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 493/985] 10.222.0.57 () {56 vars in 1126 bytes} [Thu May 19 14:25:18 2022] GET /identity/v3/services/object-store => generated 93 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:25:19.075719 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c6ce68-b8e2-407e-bf42-1dee77686974 None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:19.179444 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c6ce68-b8e2-407e-bf42-1dee77686974 None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:19.196614 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:19.219043 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a72a4be-7917-4644-a0d7-73b4c53825f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:19.219739 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a72a4be-7917-4644-a0d7-73b4c53825f5 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:19.220022 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a72a4be-7917-4644-a0d7-73b4c53825f5 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:19.220298 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a72a4be-7917-4644-a0d7-73b4c53825f5 None admin] PATH_INFO: `/v3/services` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:19.221033 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a72a4be-7917-4644-a0d7-73b4c53825f5 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:25:19.221351 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a72a4be-7917-4644-a0d7-73b4c53825f5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:19.223075 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a72a4be-7917-4644-a0d7-73b4c53825f5 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:19.229836 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 493/986] 10.222.0.57 () {56 vars in 1122 bytes} [Thu May 19 14:25:19 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:25:19.235384 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be5ae12-09cc-452c-bd90-32c792c1c0c0 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:19.329971 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:19.344007 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be5ae12-09cc-452c-bd90-32c792c1c0c0 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:19.380499 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e6bc1d-8aaf-4bb0-b476-33b232fb8e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:19.381166 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e6bc1d-8aaf-4bb0-b476-33b232fb8e05 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:19.381452 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e6bc1d-8aaf-4bb0-b476-33b232fb8e05 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:19.381729 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e6bc1d-8aaf-4bb0-b476-33b232fb8e05 None admin] PATH_INFO: `/v3/services` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:19.382467 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e6bc1d-8aaf-4bb0-b476-33b232fb8e05 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:25:19.382777 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e6bc1d-8aaf-4bb0-b476-33b232fb8e05 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:19.384139 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e6bc1d-8aaf-4bb0-b476-33b232fb8e05 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:19.394109 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 494/987] 10.222.0.57 () {56 vars in 1122 bytes} [Thu May 19 14:25:19 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:25:19.395112 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a72a4be-7917-4644-a0d7-73b4c53825f5 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:19.497300 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a72a4be-7917-4644-a0d7-73b4c53825f5 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:19.531262 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18343f27-5701-4b3b-bb67-ad41d3d26336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:19.531921 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18343f27-5701-4b3b-bb67-ad41d3d26336 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:19.532207 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18343f27-5701-4b3b-bb67-ad41d3d26336 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:19.532484 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18343f27-5701-4b3b-bb67-ad41d3d26336 None admin] PATH_INFO: `/v3/endpoints` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:19.533275 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18343f27-5701-4b3b-bb67-ad41d3d26336 None admin] RBAC: Adding query filter params (interface=admin, service_id=5c28edec2a3f43cb86d8ef21f7f492d1, region_id=RegionOne) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:25:19.533586 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18343f27-5701-4b3b-bb67-ad41d3d26336 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:19.534989 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18343f27-5701-4b3b-bb67-ad41d3d26336 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:19.540350 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 494/988] 10.222.0.57 () {56 vars in 1249 bytes} [Thu May 19 14:25:19 2022] GET /identity/v3/endpoints?service_id=5c28edec2a3f43cb86d8ef21f7f492d1&interface=admin®ion_id=RegionOne => generated 193 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:25:19.546423 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e6bc1d-8aaf-4bb0-b476-33b232fb8e05 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:19.651379 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e6bc1d-8aaf-4bb0-b476-33b232fb8e05 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:19.689797 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcdafd7-16f2-446f-8324-28620aed77c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:19.690842 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcdafd7-16f2-446f-8324-28620aed77c5 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:19.691305 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcdafd7-16f2-446f-8324-28620aed77c5 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:19.691760 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcdafd7-16f2-446f-8324-28620aed77c5 None admin] PATH_INFO: `/v3/services` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:19.692618 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcdafd7-16f2-446f-8324-28620aed77c5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:19.694218 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcdafd7-16f2-446f-8324-28620aed77c5 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:19.701291 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 495/989] 10.222.0.57 () {56 vars in 1087 bytes} [Thu May 19 14:25:19 2022] GET /identity/v3/services => generated 1727 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 14:25:20.221636 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbbba5c-f0fb-49b5-b215-54f9d188d7e5 None None] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:20.222181 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbbba5c-f0fb-49b5-b215-54f9d188d7e5 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:20.222507 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbbba5c-f0fb-49b5-b215-54f9d188d7e5 None None] PATH_INFO: `/` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:20.224001 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 495/990] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:25:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:25:20.229908 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91e3050-1264-4139-9caf-ca4322c65842 None None] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:20.230340 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91e3050-1264-4139-9caf-ca4322c65842 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:20.230518 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91e3050-1264-4139-9caf-ca4322c65842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:20.247570 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.auth.core [None req-a91e3050-1264-4139-9caf-ca4322c65842 None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143586) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:25:20.248765 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.fernet_utils [None req-a91e3050-1264-4139-9caf-ca4322c65842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143586) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:25:20.260488 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:20.321091 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 496/991] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:25:20 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:25:20.344177 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbbba5c-f0fb-49b5-b215-54f9d188d7e5 None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:20.374088 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c96378-8038-4d3a-8a49-61b57d507b52 None None] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:20.374400 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c96378-8038-4d3a-8a49-61b57d507b52 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:20.374671 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c96378-8038-4d3a-8a49-61b57d507b52 None None] PATH_INFO: `/` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:20.376185 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 497/992] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:25:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:25:20.381629 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ded57a-9dbe-4ccb-95cc-6a651bd5026b None None] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:20.382012 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ded57a-9dbe-4ccb-95cc-6a651bd5026b None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:20.382290 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ded57a-9dbe-4ccb-95cc-6a651bd5026b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:20.394759 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:20.402096 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.auth.core [None req-88ded57a-9dbe-4ccb-95cc-6a651bd5026b None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143586) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:25:20.403302 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.fernet_utils [None req-88ded57a-9dbe-4ccb-95cc-6a651bd5026b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143586) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:25:20.460774 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbbba5c-f0fb-49b5-b215-54f9d188d7e5 None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:20.481993 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 498/993] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:25:20 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:25:20.501178 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ff55cf-df39-4edf-a6a1-15627e6596d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:20.501852 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ff55cf-df39-4edf-a6a1-15627e6596d1 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:20.502161 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ff55cf-df39-4edf-a6a1-15627e6596d1 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:20.502441 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ff55cf-df39-4edf-a6a1-15627e6596d1 None admin] PATH_INFO: `/v3/services/image` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:20.502948 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ded57a-9dbe-4ccb-95cc-6a651bd5026b None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:20.503104 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ff55cf-df39-4edf-a6a1-15627e6596d1 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:20.504492 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ff55cf-df39-4edf-a6a1-15627e6596d1 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:20.516888 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-62ff55cf-df39-4edf-a6a1-15627e6596d1 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 19 14:25:20.516888 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:25:20.516888 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:25:20.516888 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:25:20.516888 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:25:20.516888 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:25:20.516888 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:25:20.516888 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:25:20.516888 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 14:25:20.516888 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:25:20.516888 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:25:20.516888 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:25:20.516888 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:25:20.516888 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:25:20.516888 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:25:20.516888 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:25:20.516888 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:25:20.518036 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:25:20.518036 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:25:20.518036 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:25:20.518036 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:25:20.518036 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:25:20.518036 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:25:20.518036 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:25:20.518036 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:25:20.518036 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:25:20.518036 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:25:20.518036 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:25:20.518036 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:25:20.518036 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:25:20.518036 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:25:20.518036 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:25:20.518036 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:25:20.519015 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:25:20.519015 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:25:20.519015 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:25:20.519015 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:25:20.519015 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:25:20.519015 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:25:20.519015 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:25:20.519015 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:25:20.519015 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:25:20.519015 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:25:20.519015 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:25:20.519015 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:25:20.519015 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 14:25:20.519015 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:25:20.519015 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 496/994] 10.222.0.57 () {56 vars in 1105 bytes} [Thu May 19 14:25:20 2022] GET /identity/v3/services/image => generated 86 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:25:20.523976 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ff55cf-df39-4edf-a6a1-15627e6596d1 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:20.591758 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:20.621214 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ded57a-9dbe-4ccb-95cc-6a651bd5026b None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:20.631706 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ff55cf-df39-4edf-a6a1-15627e6596d1 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:20.640164 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:20.662741 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e9680b-6c23-4331-91af-82b222de5829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:20.663822 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e9680b-6c23-4331-91af-82b222de5829 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:20.664332 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e9680b-6c23-4331-91af-82b222de5829 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:20.664609 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e9680b-6c23-4331-91af-82b222de5829 None admin] PATH_INFO: `/v3/services/compute` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:20.665425 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e9680b-6c23-4331-91af-82b222de5829 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:20.667591 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e9680b-6c23-4331-91af-82b222de5829 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:20.671718 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09e3d3f-7018-47e9-8c58-4e10c2995754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:20.672389 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09e3d3f-7018-47e9-8c58-4e10c2995754 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:20.672677 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09e3d3f-7018-47e9-8c58-4e10c2995754 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:20.672956 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09e3d3f-7018-47e9-8c58-4e10c2995754 None admin] PATH_INFO: `/v3/services` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:20.673671 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09e3d3f-7018-47e9-8c58-4e10c2995754 None admin] RBAC: Adding query filter params (name=image) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:25:20.674021 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09e3d3f-7018-47e9-8c58-4e10c2995754 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:20.675389 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09e3d3f-7018-47e9-8c58-4e10c2995754 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:20.680065 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-20e9680b-6c23-4331-91af-82b222de5829 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 19 14:25:20.680065 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:25:20.680065 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:25:20.680065 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:25:20.680065 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:25:20.680065 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:25:20.680065 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:25:20.680065 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:25:20.680065 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 14:25:20.680065 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:25:20.680065 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:25:20.680065 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:25:20.680065 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:25:20.680065 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:25:20.680065 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:25:20.680065 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:25:20.680065 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:25:20.680990 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:25:20.680990 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:25:20.680990 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:25:20.680990 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:25:20.680990 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:25:20.680990 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:25:20.680990 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:25:20.680990 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:25:20.680990 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:25:20.680990 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:25:20.680990 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:25:20.680990 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:25:20.680990 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:25:20.680990 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:25:20.680990 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:25:20.680990 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:25:20.687464 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:25:20.687464 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:25:20.687464 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:25:20.687464 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:25:20.687464 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:25:20.687464 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:25:20.687464 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:25:20.687464 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:25:20.687464 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:25:20.687464 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:25:20.687464 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:25:20.687464 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:25:20.687464 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 14:25:20.687464 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:25:20.687464 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 499/995] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:25:20 2022] GET /identity/v3/services/compute => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:25:20.688165 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 497/996] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:25:20 2022] GET /identity/v3/services?name=image => generated 122 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:25:20.688268 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e9680b-6c23-4331-91af-82b222de5829 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:20.688383 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09e3d3f-7018-47e9-8c58-4e10c2995754 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:20.796048 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09e3d3f-7018-47e9-8c58-4e10c2995754 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:20.797328 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e9680b-6c23-4331-91af-82b222de5829 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:20.833607 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17aa4fae-e2f9-408d-ade6-87fc909cae79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:20.834299 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aa4fae-e2f9-408d-ade6-87fc909cae79 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:20.834585 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aa4fae-e2f9-408d-ade6-87fc909cae79 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:20.834680 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f508d2-14b3-43c1-98cd-24c8b93eacf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:20.834865 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aa4fae-e2f9-408d-ade6-87fc909cae79 None admin] PATH_INFO: `/v3/services` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:20.835286 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f508d2-14b3-43c1-98cd-24c8b93eacf1 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:20.835566 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17aa4fae-e2f9-408d-ade6-87fc909cae79 None admin] RBAC: Adding query filter params (name=compute) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:25:20.835656 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f508d2-14b3-43c1-98cd-24c8b93eacf1 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:20.835846 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f508d2-14b3-43c1-98cd-24c8b93eacf1 None admin] PATH_INFO: `/v3/services` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:20.835943 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17aa4fae-e2f9-408d-ade6-87fc909cae79 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:20.836556 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f508d2-14b3-43c1-98cd-24c8b93eacf1 None admin] RBAC: Adding query filter params (type=image) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:25:20.836869 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f508d2-14b3-43c1-98cd-24c8b93eacf1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:20.837242 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17aa4fae-e2f9-408d-ade6-87fc909cae79 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:20.838316 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f508d2-14b3-43c1-98cd-24c8b93eacf1 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:20.842964 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 498/997] 10.222.0.57 () {56 vars in 1112 bytes} [Thu May 19 14:25:20 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:25:20.848841 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17aa4fae-e2f9-408d-ade6-87fc909cae79 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:20.850233 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 500/998] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:25:20 2022] GET /identity/v3/services?type=image => generated 122 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:25:20.894010 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:20.970150 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17aa4fae-e2f9-408d-ade6-87fc909cae79 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:21.007544 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85229b04-9025-4231-bc64-4aa94a3901a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:21.008226 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85229b04-9025-4231-bc64-4aa94a3901a2 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:21.008507 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85229b04-9025-4231-bc64-4aa94a3901a2 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:21.008785 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85229b04-9025-4231-bc64-4aa94a3901a2 None admin] PATH_INFO: `/v3/services` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:21.010022 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85229b04-9025-4231-bc64-4aa94a3901a2 None admin] RBAC: Adding query filter params (type=compute) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:25:21.010348 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85229b04-9025-4231-bc64-4aa94a3901a2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:21.011785 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85229b04-9025-4231-bc64-4aa94a3901a2 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:21.026094 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 499/999] 10.222.0.57 () {56 vars in 1112 bytes} [Thu May 19 14:25:20 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:25:21.030065 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f508d2-14b3-43c1-98cd-24c8b93eacf1 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:21.034126 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:21.142558 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f508d2-14b3-43c1-98cd-24c8b93eacf1 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:21.177480 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1657d1-859a-40c4-878d-865d71ce15e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:21.178160 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1657d1-859a-40c4-878d-865d71ce15e1 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:21.178441 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1657d1-859a-40c4-878d-865d71ce15e1 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:21.178713 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1657d1-859a-40c4-878d-865d71ce15e1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:21.179495 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1657d1-859a-40c4-878d-865d71ce15e1 None admin] RBAC: Adding query filter params (interface=public, service_id=60a62cb33bfc42f299f0096331459387, region_id=RegionOne) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:25:21.179805 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1657d1-859a-40c4-878d-865d71ce15e1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:21.181194 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1657d1-859a-40c4-878d-865d71ce15e1 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:21.186994 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 501/1000] 10.222.0.57 () {56 vars in 1251 bytes} [Thu May 19 14:25:21 2022] GET /identity/v3/endpoints?service_id=60a62cb33bfc42f299f0096331459387&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:25:21.192256 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85229b04-9025-4231-bc64-4aa94a3901a2 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:21.294403 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:21.302193 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85229b04-9025-4231-bc64-4aa94a3901a2 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:21.342783 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99085e2d-1e1c-4109-a8e0-8ba75e4fd8e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:21.343505 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99085e2d-1e1c-4109-a8e0-8ba75e4fd8e6 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:21.343817 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99085e2d-1e1c-4109-a8e0-8ba75e4fd8e6 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:21.344117 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99085e2d-1e1c-4109-a8e0-8ba75e4fd8e6 None admin] PATH_INFO: `/v3/services` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:21.344827 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99085e2d-1e1c-4109-a8e0-8ba75e4fd8e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:21.346271 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99085e2d-1e1c-4109-a8e0-8ba75e4fd8e6 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:21.353263 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 500/1001] 10.222.0.57 () {56 vars in 1087 bytes} [Thu May 19 14:25:21 2022] GET /identity/v3/services => generated 1727 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 14:25:22.004009 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895fc36f-30f8-4f18-919d-882f467aba89 None None] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:22.004501 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895fc36f-30f8-4f18-919d-882f467aba89 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:22.004501 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895fc36f-30f8-4f18-919d-882f467aba89 None None] PATH_INFO: `/` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:22.006036 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 502/1002] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:25:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:25:22.011551 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337ffe47-f87e-4af8-ad8d-7ae5d4ff8aa6 None None] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:22.011846 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337ffe47-f87e-4af8-ad8d-7ae5d4ff8aa6 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:22.012133 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337ffe47-f87e-4af8-ad8d-7ae5d4ff8aa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:22.029516 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.auth.core [None req-337ffe47-f87e-4af8-ad8d-7ae5d4ff8aa6 None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143587) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:25:22.030742 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-337ffe47-f87e-4af8-ad8d-7ae5d4ff8aa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:25:22.101885 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 501/1003] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:25:22 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:25:22.126212 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895fc36f-30f8-4f18-919d-882f467aba89 None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:22.196196 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:22.233406 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895fc36f-30f8-4f18-919d-882f467aba89 None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:22.267216 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd4dde5-e0b5-4573-a42f-f2c0a68a9cde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:22.267952 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd4dde5-e0b5-4573-a42f-f2c0a68a9cde None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:22.268239 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd4dde5-e0b5-4573-a42f-f2c0a68a9cde None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:22.268517 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd4dde5-e0b5-4573-a42f-f2c0a68a9cde None admin] PATH_INFO: `/v3/services/object-store` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:22.269173 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd4dde5-e0b5-4573-a42f-f2c0a68a9cde None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:22.270575 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd4dde5-e0b5-4573-a42f-f2c0a68a9cde None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:22.282592 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-4cd4dde5-e0b5-4573-a42f-f2c0a68a9cde None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 14:25:22.282592 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:25:22.282592 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:25:22.282592 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:25:22.282592 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:25:22.282592 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:25:22.282592 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:25:22.282592 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:25:22.282592 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 14:25:22.282592 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:25:22.282592 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:25:22.282592 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:25:22.282592 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:25:22.282592 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:25:22.282592 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:25:22.282592 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:25:22.282592 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:25:22.283449 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:25:22.283449 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:25:22.283449 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:25:22.283449 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:25:22.283449 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:25:22.283449 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:25:22.283449 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:25:22.283449 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:25:22.283449 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:25:22.283449 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:25:22.283449 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:25:22.283449 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:25:22.283449 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:25:22.283449 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:25:22.283449 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:25:22.283449 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:25:22.284230 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:25:22.284230 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:25:22.284230 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:25:22.284230 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:25:22.284230 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:25:22.284230 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:25:22.284230 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:25:22.284230 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:25:22.284230 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:25:22.284230 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:25:22.284230 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:25:22.284230 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:25:22.284230 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 14:25:22.284230 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:25:22.285920 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 503/1004] 10.222.0.57 () {56 vars in 1126 bytes} [Thu May 19 14:25:22 2022] GET /identity/v3/services/object-store => generated 93 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:25:22.290217 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337ffe47-f87e-4af8-ad8d-7ae5d4ff8aa6 None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:22.389386 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:22.399729 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337ffe47-f87e-4af8-ad8d-7ae5d4ff8aa6 None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:22.435850 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe809927-1474-41ba-a356-727d990ec231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:22.436523 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe809927-1474-41ba-a356-727d990ec231 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:22.436807 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe809927-1474-41ba-a356-727d990ec231 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:22.437086 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe809927-1474-41ba-a356-727d990ec231 None admin] PATH_INFO: `/v3/services` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:22.437804 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe809927-1474-41ba-a356-727d990ec231 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:25:22.438145 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe809927-1474-41ba-a356-727d990ec231 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:22.439521 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe809927-1474-41ba-a356-727d990ec231 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:22.450157 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 502/1005] 10.222.0.57 () {56 vars in 1122 bytes} [Thu May 19 14:25:22 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:25:22.451083 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd4dde5-e0b5-4573-a42f-f2c0a68a9cde None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:22.558930 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd4dde5-e0b5-4573-a42f-f2c0a68a9cde None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:22.591426 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:22.597944 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876a3032-a434-4a3e-b29c-66b2ec759ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:22.598871 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876a3032-a434-4a3e-b29c-66b2ec759ad8 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:22.599362 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876a3032-a434-4a3e-b29c-66b2ec759ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:22.599849 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876a3032-a434-4a3e-b29c-66b2ec759ad8 None admin] PATH_INFO: `/v3/services` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:22.600750 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876a3032-a434-4a3e-b29c-66b2ec759ad8 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:25:22.601247 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876a3032-a434-4a3e-b29c-66b2ec759ad8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:22.602829 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876a3032-a434-4a3e-b29c-66b2ec759ad8 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:22.615632 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 504/1006] 10.222.0.57 () {56 vars in 1122 bytes} [Thu May 19 14:25:22 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:25:22.617153 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe809927-1474-41ba-a356-727d990ec231 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:22.732872 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe809927-1474-41ba-a356-727d990ec231 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:22.768529 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6757ce-b83e-4bab-a56a-7ff0856997d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:22.769192 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6757ce-b83e-4bab-a56a-7ff0856997d7 None admin] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:22.769480 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6757ce-b83e-4bab-a56a-7ff0856997d7 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:22.769762 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6757ce-b83e-4bab-a56a-7ff0856997d7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:22.770594 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6757ce-b83e-4bab-a56a-7ff0856997d7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:22.771969 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6757ce-b83e-4bab-a56a-7ff0856997d7 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:22.793012 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 503/1007] 10.222.0.57 () {60 vars in 1144 bytes} [Thu May 19 14:25:22 2022] POST /identity/v3/endpoints => generated 325 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:25:23.131518 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7752352c-3617-4bd6-a9e9-8721a39934ff None None] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:23.132035 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7752352c-3617-4bd6-a9e9-8721a39934ff None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:23.132117 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7752352c-3617-4bd6-a9e9-8721a39934ff None None] PATH_INFO: `/` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:23.133452 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 505/1008] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:25:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:25:23.139115 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db22900-60f6-4bce-92f9-13f3efa8f43e None None] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:23.139408 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db22900-60f6-4bce-92f9-13f3efa8f43e None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:23.139688 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db22900-60f6-4bce-92f9-13f3efa8f43e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:23.157533 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.auth.core [None req-7db22900-60f6-4bce-92f9-13f3efa8f43e None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143587) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:25:23.158750 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-7db22900-60f6-4bce-92f9-13f3efa8f43e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:25:23.171106 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:23.238605 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 504/1009] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:25:23 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:25:23.261430 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7752352c-3617-4bd6-a9e9-8721a39934ff None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:23.368374 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7752352c-3617-4bd6-a9e9-8721a39934ff None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:23.401992 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7a5b5a-928b-48f4-af09-5f198f9e21b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:23.402591 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7a5b5a-928b-48f4-af09-5f198f9e21b9 None admin] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:23.402876 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7a5b5a-928b-48f4-af09-5f198f9e21b9 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:23.403150 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7a5b5a-928b-48f4-af09-5f198f9e21b9 None admin] PATH_INFO: `/v3/services` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:23.403889 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7a5b5a-928b-48f4-af09-5f198f9e21b9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:23.405259 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7a5b5a-928b-48f4-af09-5f198f9e21b9 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:23.422954 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 506/1010] 10.222.0.57 () {60 vars in 1141 bytes} [Thu May 19 14:25:23 2022] POST /identity/v3/services => generated 242 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:25:23.627303 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca8495a-7289-4852-ac3e-0e87d075a0ca None None] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:23.627828 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca8495a-7289-4852-ac3e-0e87d075a0ca None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:23.627828 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca8495a-7289-4852-ac3e-0e87d075a0ca None None] PATH_INFO: `/` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:23.629225 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 505/1011] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:25:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:25:23.635178 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc614e44-48da-4d86-86ac-027fb68c09b9 None None] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:23.635558 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc614e44-48da-4d86-86ac-027fb68c09b9 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:23.635851 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc614e44-48da-4d86-86ac-027fb68c09b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:23.660570 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.auth.core [None req-bc614e44-48da-4d86-86ac-027fb68c09b9 None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143586) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:25:23.661771 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.fernet_utils [None req-bc614e44-48da-4d86-86ac-027fb68c09b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143586) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:25:23.715174 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:23.740353 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 507/1012] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:25:23 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:25:23.763621 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca8495a-7289-4852-ac3e-0e87d075a0ca None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:23.866281 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca8495a-7289-4852-ac3e-0e87d075a0ca None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:23.901365 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaec5f2-f237-4856-9447-d6ce8172a40d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:23.902298 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaec5f2-f237-4856-9447-d6ce8172a40d None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:23.902579 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaec5f2-f237-4856-9447-d6ce8172a40d None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:23.902850 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaec5f2-f237-4856-9447-d6ce8172a40d None admin] PATH_INFO: `/v3/services/compute` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:23.903498 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddaec5f2-f237-4856-9447-d6ce8172a40d None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:23.904873 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddaec5f2-f237-4856-9447-d6ce8172a40d None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:23.916905 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-ddaec5f2-f237-4856-9447-d6ce8172a40d None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 19 14:25:23.916905 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:25:23.916905 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:25:23.916905 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:25:23.916905 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:25:23.916905 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:25:23.916905 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:25:23.916905 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:25:23.916905 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 14:25:23.916905 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:25:23.916905 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:25:23.916905 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:25:23.916905 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:25:23.916905 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:25:23.916905 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:25:23.916905 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:25:23.916905 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:25:23.917778 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:25:23.917778 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:25:23.917778 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:25:23.917778 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:25:23.917778 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:25:23.917778 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:25:23.917778 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:25:23.917778 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:25:23.917778 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:25:23.917778 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:25:23.917778 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:25:23.917778 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:25:23.917778 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:25:23.917778 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:25:23.917778 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:25:23.917778 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:25:23.919096 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:25:23.919096 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:25:23.919096 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:25:23.919096 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:25:23.919096 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:25:23.919096 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:25:23.919096 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:25:23.919096 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:25:23.919096 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:25:23.919096 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:25:23.919096 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:25:23.919096 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:25:23.919096 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 14:25:23.919096 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:25:23.919096 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 506/1013] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:25:23 2022] GET /identity/v3/services/compute => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:25:23.924237 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc614e44-48da-4d86-86ac-027fb68c09b9 None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:24.031913 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc614e44-48da-4d86-86ac-027fb68c09b9 None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:24.067970 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dad1201-5111-4f64-a21f-2abee55e0b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:24.068641 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dad1201-5111-4f64-a21f-2abee55e0b78 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:24.068928 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dad1201-5111-4f64-a21f-2abee55e0b78 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:24.069210 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dad1201-5111-4f64-a21f-2abee55e0b78 None admin] PATH_INFO: `/v3/services` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:24.069950 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dad1201-5111-4f64-a21f-2abee55e0b78 None admin] RBAC: Adding query filter params (name=compute) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:25:24.070266 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dad1201-5111-4f64-a21f-2abee55e0b78 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:24.071641 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dad1201-5111-4f64-a21f-2abee55e0b78 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:24.082153 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 508/1014] 10.222.0.57 () {56 vars in 1112 bytes} [Thu May 19 14:25:23 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:25:24.083293 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaec5f2-f237-4856-9447-d6ce8172a40d None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:24.092815 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:24.189336 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaec5f2-f237-4856-9447-d6ce8172a40d None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:24.224211 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8f968b-9890-4008-b1e3-72d2fd7607a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:24.224876 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8f968b-9890-4008-b1e3-72d2fd7607a8 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:24.225160 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8f968b-9890-4008-b1e3-72d2fd7607a8 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:24.225558 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8f968b-9890-4008-b1e3-72d2fd7607a8 None admin] PATH_INFO: `/v3/services` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:24.226305 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8f968b-9890-4008-b1e3-72d2fd7607a8 None admin] RBAC: Adding query filter params (type=compute) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:25:24.226614 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8f968b-9890-4008-b1e3-72d2fd7607a8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:24.227988 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8f968b-9890-4008-b1e3-72d2fd7607a8 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:24.238304 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 507/1015] 10.222.0.57 () {56 vars in 1112 bytes} [Thu May 19 14:25:24 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:25:24.239547 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dad1201-5111-4f64-a21f-2abee55e0b78 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:24.289506 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:24.348141 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dad1201-5111-4f64-a21f-2abee55e0b78 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:24.383570 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960e9d89-bada-4ed8-abd2-9852de707e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:24.384480 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960e9d89-bada-4ed8-abd2-9852de707e97 None admin] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:24.384767 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960e9d89-bada-4ed8-abd2-9852de707e97 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:24.385065 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960e9d89-bada-4ed8-abd2-9852de707e97 None admin] PATH_INFO: `/v3/endpoints` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:24.385796 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960e9d89-bada-4ed8-abd2-9852de707e97 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:24.387207 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960e9d89-bada-4ed8-abd2-9852de707e97 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:24.408336 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 509/1016] 10.222.0.57 () {60 vars in 1144 bytes} [Thu May 19 14:25:24 2022] POST /identity/v3/endpoints => generated 334 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:25:25.108410 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cf84bb-0b9e-4bd9-a391-b4a7ea21f4eb None None] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:25.108914 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cf84bb-0b9e-4bd9-a391-b4a7ea21f4eb None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:25.108914 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cf84bb-0b9e-4bd9-a391-b4a7ea21f4eb None None] PATH_INFO: `/` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:25.110291 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 508/1017] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:25:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:25:25.115521 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088a76ee-1e0b-494d-a6aa-e215bcb24457 None None] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:25.115815 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088a76ee-1e0b-494d-a6aa-e215bcb24457 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:25.116100 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088a76ee-1e0b-494d-a6aa-e215bcb24457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:25.133482 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.auth.core [None req-088a76ee-1e0b-494d-a6aa-e215bcb24457 None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143586) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:25:25.135445 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.fernet_utils [None req-088a76ee-1e0b-494d-a6aa-e215bcb24457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143586) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:25:25.210695 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 510/1018] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:25:25 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:25:25.235011 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cf84bb-0b9e-4bd9-a391-b4a7ea21f4eb None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:25.310401 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:25.345792 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cf84bb-0b9e-4bd9-a391-b4a7ea21f4eb None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:25.383065 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e6693a-4e72-4007-8fd1-21a81b911432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:25.383858 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e6693a-4e72-4007-8fd1-21a81b911432 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:25.384142 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e6693a-4e72-4007-8fd1-21a81b911432 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:25.384449 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e6693a-4e72-4007-8fd1-21a81b911432 None admin] PATH_INFO: `/v3/domains/default` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:25.385651 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e6693a-4e72-4007-8fd1-21a81b911432 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:25.387148 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e6693a-4e72-4007-8fd1-21a81b911432 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:25.388970 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 509/1019] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:25:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:25:25.394661 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088a76ee-1e0b-494d-a6aa-e215bcb24457 None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:25.423839 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:25.499155 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088a76ee-1e0b-494d-a6aa-e215bcb24457 None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:25.534097 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597cf582-2e83-4960-8671-c654fdb66b06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:25.534801 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597cf582-2e83-4960-8671-c654fdb66b06 None admin] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:25.535088 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597cf582-2e83-4960-8671-c654fdb66b06 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:25.535368 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597cf582-2e83-4960-8671-c654fdb66b06 None admin] PATH_INFO: `/v3/projects` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:25.536124 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597cf582-2e83-4960-8671-c654fdb66b06 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:25.537614 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597cf582-2e83-4960-8671-c654fdb66b06 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:25.569877 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 511/1020] 10.222.0.57 () {60 vars in 1141 bytes} [Thu May 19 14:25:25 2022] POST /identity/v3/projects => generated 311 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:25:25.731262 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c516aef-d704-426d-9e58-6f8ec562865b None None] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:25.731605 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c516aef-d704-426d-9e58-6f8ec562865b None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:25.731831 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c516aef-d704-426d-9e58-6f8ec562865b None None] PATH_INFO: `/` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:25.733419 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 510/1021] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:25:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:25:25.739056 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a5a5b5-97f5-4720-89af-619e29ba56e3 None None] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:25.739351 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a5a5b5-97f5-4720-89af-619e29ba56e3 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:25.739636 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a5a5b5-97f5-4720-89af-619e29ba56e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:25.757473 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.auth.core [None req-76a5a5b5-97f5-4720-89af-619e29ba56e3 None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143586) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:25:25.758692 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.fernet_utils [None req-76a5a5b5-97f5-4720-89af-619e29ba56e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143586) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:25:25.833673 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 512/1022] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:25:25 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:25:25.857741 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c516aef-d704-426d-9e58-6f8ec562865b None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:25.962347 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c516aef-d704-426d-9e58-6f8ec562865b None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:25.996565 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b1675b-c3fd-4907-b564-f9da3697ee5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:25.997352 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b1675b-c3fd-4907-b564-f9da3697ee5d None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:25.997629 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b1675b-c3fd-4907-b564-f9da3697ee5d None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:25.997925 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b1675b-c3fd-4907-b564-f9da3697ee5d None admin] PATH_INFO: `/v3/services/image` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:25.998576 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b1675b-c3fd-4907-b564-f9da3697ee5d None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:25.999948 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b1675b-c3fd-4907-b564-f9da3697ee5d None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:26.012582 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-75b1675b-c3fd-4907-b564-f9da3697ee5d None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 19 14:25:26.012582 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:25:26.012582 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:25:26.012582 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:25:26.012582 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:25:26.012582 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:25:26.012582 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:25:26.012582 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:25:26.012582 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 14:25:26.012582 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:25:26.012582 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:25:26.012582 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:25:26.012582 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:25:26.012582 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:25:26.012582 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:25:26.012582 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:25:26.012582 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:25:26.013481 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:25:26.013481 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:25:26.013481 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:25:26.013481 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:25:26.013481 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:25:26.013481 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:25:26.013481 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:25:26.013481 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:25:26.013481 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:25:26.013481 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:25:26.013481 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:25:26.013481 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:25:26.013481 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:25:26.013481 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:25:26.013481 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:25:26.013481 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:25:26.014663 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:25:26.014663 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:25:26.014663 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:25:26.014663 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:25:26.014663 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:25:26.014663 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:25:26.014663 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:25:26.014663 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:25:26.014663 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:25:26.014663 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:25:26.014663 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:25:26.014663 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:25:26.014663 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 14:25:26.014663 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:25:26.015711 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 511/1023] 10.222.0.57 () {56 vars in 1105 bytes} [Thu May 19 14:25:25 2022] GET /identity/v3/services/image => generated 86 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:25:26.020093 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a5a5b5-97f5-4720-89af-619e29ba56e3 None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:26.033094 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:26.128135 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a5a5b5-97f5-4720-89af-619e29ba56e3 None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:26.165753 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32125e9c-e7d9-41c1-bd28-634f4c679ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:26.166536 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32125e9c-e7d9-41c1-bd28-634f4c679ddf None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:26.166857 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32125e9c-e7d9-41c1-bd28-634f4c679ddf None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:26.167140 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32125e9c-e7d9-41c1-bd28-634f4c679ddf None admin] PATH_INFO: `/v3/services` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:26.167866 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32125e9c-e7d9-41c1-bd28-634f4c679ddf None admin] RBAC: Adding query filter params (name=image) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:25:26.168178 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32125e9c-e7d9-41c1-bd28-634f4c679ddf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:26.169561 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32125e9c-e7d9-41c1-bd28-634f4c679ddf None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:26.179403 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 513/1024] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:25:26 2022] GET /identity/v3/services?name=image => generated 122 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:25:26.180597 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b1675b-c3fd-4907-b564-f9da3697ee5d None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:26.184296 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:26.291115 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b1675b-c3fd-4907-b564-f9da3697ee5d None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:26.328793 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a630536f-519e-4101-ac41-79a0c97400ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:26.329464 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a630536f-519e-4101-ac41-79a0c97400ed None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:26.329749 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a630536f-519e-4101-ac41-79a0c97400ed None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:26.330059 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a630536f-519e-4101-ac41-79a0c97400ed None admin] PATH_INFO: `/v3/services` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:26.330768 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a630536f-519e-4101-ac41-79a0c97400ed None admin] RBAC: Adding query filter params (type=image) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:25:26.331078 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a630536f-519e-4101-ac41-79a0c97400ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:26.332456 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a630536f-519e-4101-ac41-79a0c97400ed None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:26.338010 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 512/1025] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:25:26 2022] GET /identity/v3/services?type=image => generated 350 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:25:26.343735 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32125e9c-e7d9-41c1-bd28-634f4c679ddf None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:26.425493 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:26.454615 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32125e9c-e7d9-41c1-bd28-634f4c679ddf None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:26.488853 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c19b941-dfac-448d-afc1-c298f345335e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:26.489520 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c19b941-dfac-448d-afc1-c298f345335e None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:26.489806 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c19b941-dfac-448d-afc1-c298f345335e None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:26.490112 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c19b941-dfac-448d-afc1-c298f345335e None admin] PATH_INFO: `/v3/endpoints` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:26.490916 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c19b941-dfac-448d-afc1-c298f345335e None admin] RBAC: Adding query filter params (interface=public, service_id=6ff89d53e5d14f9985fa4c8189e94f07, region_id=RegionOne) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:25:26.491231 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c19b941-dfac-448d-afc1-c298f345335e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:26.492606 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c19b941-dfac-448d-afc1-c298f345335e None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:26.498767 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 514/1026] 10.222.0.57 () {56 vars in 1251 bytes} [Thu May 19 14:25:26 2022] GET /identity/v3/endpoints?service_id=6ff89d53e5d14f9985fa4c8189e94f07&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:25:26.504120 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a630536f-519e-4101-ac41-79a0c97400ed None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:26.571317 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:26.612734 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a630536f-519e-4101-ac41-79a0c97400ed None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:26.648509 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa74851-cb64-4dcf-81fe-43557697a551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:26.649174 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa74851-cb64-4dcf-81fe-43557697a551 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:26.649457 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa74851-cb64-4dcf-81fe-43557697a551 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:26.649734 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa74851-cb64-4dcf-81fe-43557697a551 None admin] PATH_INFO: `/v3/services` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:26.650495 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa74851-cb64-4dcf-81fe-43557697a551 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:26.654759 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa74851-cb64-4dcf-81fe-43557697a551 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:26.663125 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 513/1027] 10.222.0.57 () {56 vars in 1087 bytes} [Thu May 19 14:25:26 2022] GET /identity/v3/services => generated 1957 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 14:25:26.782644 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971fb12c-7ba4-4264-ba2d-dc5709018e3d None None] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:26.782997 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971fb12c-7ba4-4264-ba2d-dc5709018e3d None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:26.783208 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971fb12c-7ba4-4264-ba2d-dc5709018e3d None None] PATH_INFO: `/` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:26.784867 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 515/1028] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:25:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:25:26.791881 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa93686-6c8a-460b-af84-3fea5a241ca8 None None] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:26.792365 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa93686-6c8a-460b-af84-3fea5a241ca8 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:26.792460 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa93686-6c8a-460b-af84-3fea5a241ca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:26.811543 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.auth.core [None req-efa93686-6c8a-460b-af84-3fea5a241ca8 None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143587) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:25:26.812759 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-efa93686-6c8a-460b-af84-3fea5a241ca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:25:26.889949 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 514/1029] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:25:26 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:25:26.914765 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971fb12c-7ba4-4264-ba2d-dc5709018e3d None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:27.018889 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971fb12c-7ba4-4264-ba2d-dc5709018e3d None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:27.054456 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9790ff7e-0b3a-4483-ba8d-26692b7322de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:27.055267 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9790ff7e-0b3a-4483-ba8d-26692b7322de None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:27.055547 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9790ff7e-0b3a-4483-ba8d-26692b7322de None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:27.055822 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9790ff7e-0b3a-4483-ba8d-26692b7322de None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:27.062859 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:27.067225 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9790ff7e-0b3a-4483-ba8d-26692b7322de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:25:27.075863 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-9790ff7e-0b3a-4483-ba8d-26692b7322de None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 14:25:27.075863 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:25:27.075863 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:25:27.075863 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:25:27.075863 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:25:27.075863 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:25:27.075863 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:25:27.075863 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:25:27.075863 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:25:27.075863 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:25:27.075863 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:25:27.075863 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:25:27.075863 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:25:27.075863 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:25:27.075863 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:25:27.075863 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:25:27.076966 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:25:27.076966 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:25:27.076966 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:25:27.076966 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:25:27.076966 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:25:27.076966 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:25:27.076966 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:25:27.076966 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:25:27.076966 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:25:27.076966 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:25:27.076966 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:25:27.076966 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:25:27.076966 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:25:27.076966 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:25:27.076966 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:25:27.076966 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:25:27.077948 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:25:27.077948 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:25:27.077948 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:25:27.077948 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:25:27.077948 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:25:27.077948 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:25:27.077948 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:25:27.077948 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:25:27.077948 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:25:27.077948 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:25:27.077948 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 14:25:27.077948 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:25:27.082026 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 516/1030] 10.222.0.57 () {56 vars in 1120 bytes} [Thu May 19 14:25:26 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:25:27.086670 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa93686-6c8a-460b-af84-3fea5a241ca8 None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:27.148738 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:27.192679 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa93686-6c8a-460b-af84-3fea5a241ca8 None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:27.229452 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707210cc-1718-492f-b171-f5d5a06d5f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:27.230149 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707210cc-1718-492f-b171-f5d5a06d5f2e None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:27.230426 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707210cc-1718-492f-b171-f5d5a06d5f2e None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:27.230696 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707210cc-1718-492f-b171-f5d5a06d5f2e None admin] PATH_INFO: `/v3/roles` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:27.231443 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707210cc-1718-492f-b171-f5d5a06d5f2e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:25:27.231750 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707210cc-1718-492f-b171-f5d5a06d5f2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:27.233139 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707210cc-1718-492f-b171-f5d5a06d5f2e None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:27.241697 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 515/1031] 10.222.0.57 () {56 vars in 1115 bytes} [Thu May 19 14:25:27 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:25:27.247521 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9790ff7e-0b3a-4483-ba8d-26692b7322de None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:27.350305 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9790ff7e-0b3a-4483-ba8d-26692b7322de None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:27.384910 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76a17ce-c121-4d7e-bfe6-ec9546a9231f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:27.385783 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76a17ce-c121-4d7e-bfe6-ec9546a9231f None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:27.386116 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76a17ce-c121-4d7e-bfe6-ec9546a9231f None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:27.386398 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76a17ce-c121-4d7e-bfe6-ec9546a9231f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:27.397486 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76a17ce-c121-4d7e-bfe6-ec9546a9231f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:27.399017 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76a17ce-c121-4d7e-bfe6-ec9546a9231f None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:27.417023 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-f76a17ce-c121-4d7e-bfe6-ec9546a9231f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:25:27.417023 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:25:27.417023 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:25:27.417023 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:25:27.417023 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:25:27.417023 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:25:27.417023 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:25:27.417023 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:25:27.417023 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:25:27.417023 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:25:27.417023 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:25:27.417023 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:25:27.417023 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:25:27.417023 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:25:27.417023 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:25:27.418268 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:25:27.418268 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:25:27.418268 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:25:27.418268 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:25:27.418268 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:25:27.418268 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:25:27.418268 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:25:27.418268 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:25:27.418268 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:25:27.418268 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:25:27.418268 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:25:27.418268 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:25:27.418268 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:25:27.418268 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:25:27.418268 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:25:27.418268 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:25:27.418268 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:25:27.419097 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:25:27.419097 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:25:27.419097 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:25:27.419097 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:25:27.423042 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 517/1032] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:25:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:25:27.424473 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707210cc-1718-492f-b171-f5d5a06d5f2e None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:27.529076 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707210cc-1718-492f-b171-f5d5a06d5f2e None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:27.551775 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:27.564342 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df816f5b-935d-4cb6-8857-3da046dfe9b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:27.565012 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df816f5b-935d-4cb6-8857-3da046dfe9b8 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:27.565296 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df816f5b-935d-4cb6-8857-3da046dfe9b8 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:27.565576 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df816f5b-935d-4cb6-8857-3da046dfe9b8 None admin] PATH_INFO: `/v3/domains` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:27.566332 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df816f5b-935d-4cb6-8857-3da046dfe9b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:25:27.566643 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df816f5b-935d-4cb6-8857-3da046dfe9b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:27.568021 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df816f5b-935d-4cb6-8857-3da046dfe9b8 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:27.585141 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 516/1033] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:25:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:25:27.586589 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76a17ce-c121-4d7e-bfe6-ec9546a9231f None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:27.638754 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:27.704949 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76a17ce-c121-4d7e-bfe6-ec9546a9231f None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:27.739044 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f291cb87-447a-418a-973e-f5d37605a4e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:27.739841 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f291cb87-447a-418a-973e-f5d37605a4e0 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:27.740127 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f291cb87-447a-418a-973e-f5d37605a4e0 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:27.740406 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f291cb87-447a-418a-973e-f5d37605a4e0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:27.750356 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f291cb87-447a-418a-973e-f5d37605a4e0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:27.751855 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f291cb87-447a-418a-973e-f5d37605a4e0 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:27.768149 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-f291cb87-447a-418a-973e-f5d37605a4e0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 14:25:27.768149 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:25:27.768149 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:25:27.768149 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:25:27.768149 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:25:27.768149 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:25:27.768149 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:25:27.768149 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:25:27.768149 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:25:27.768149 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:25:27.768149 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:25:27.768149 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:25:27.768149 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:25:27.768149 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:25:27.768149 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:25:27.768149 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:25:27.769205 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:25:27.769205 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:25:27.769205 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:25:27.769205 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:25:27.769205 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:25:27.769205 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:25:27.769205 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:25:27.769205 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:25:27.769205 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:25:27.769205 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:25:27.769205 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:25:27.769205 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:25:27.769205 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:25:27.769205 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:25:27.769205 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:25:27.769205 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:25:27.769205 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:25:27.769205 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:25:27.770118 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:25:27.770118 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:25:27.770118 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:25:27.770118 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:25:27.770118 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:25:27.770118 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:25:27.770118 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:25:27.770118 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:25:27.770118 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 14:25:27.770118 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:25:27.770118 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 518/1034] 10.222.0.57 () {56 vars in 1093 bytes} [Thu May 19 14:25:27 2022] GET /identity/v3/users/nova => generated 82 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:25:27.777269 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df816f5b-935d-4cb6-8857-3da046dfe9b8 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:27.882507 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df816f5b-935d-4cb6-8857-3da046dfe9b8 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:27.918904 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bd239a-1b60-444e-99b2-386b67db4c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:27.919503 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bd239a-1b60-444e-99b2-386b67db4c23 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:27.919787 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bd239a-1b60-444e-99b2-386b67db4c23 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:27.920063 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bd239a-1b60-444e-99b2-386b67db4c23 None admin] PATH_INFO: `/v3/users` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:27.920904 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bd239a-1b60-444e-99b2-386b67db4c23 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:25:27.921216 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bd239a-1b60-444e-99b2-386b67db4c23 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:27.922686 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bd239a-1b60-444e-99b2-386b67db4c23 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:27.944743 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 517/1035] 10.222.0.57 () {56 vars in 1133 bytes} [Thu May 19 14:25:27 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:25:27.946437 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f291cb87-447a-418a-973e-f5d37605a4e0 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:27.954261 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad27a9d8-99cc-4db9-8df0-df200d07a76a None None] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:27.954562 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad27a9d8-99cc-4db9-8df0-df200d07a76a None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:27.954843 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad27a9d8-99cc-4db9-8df0-df200d07a76a None None] PATH_INFO: `/` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:27.956344 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 518/1036] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:25:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:25:27.961375 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d03bc0d-16b3-425d-ae65-c194894a39c8 None None] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:27.961671 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d03bc0d-16b3-425d-ae65-c194894a39c8 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:27.961988 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d03bc0d-16b3-425d-ae65-c194894a39c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:27.980072 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.auth.core [None req-6d03bc0d-16b3-425d-ae65-c194894a39c8 None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143587) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:25:27.981270 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-6d03bc0d-16b3-425d-ae65-c194894a39c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:25:28.020875 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:28.058474 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f291cb87-447a-418a-973e-f5d37605a4e0 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:28.066171 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 519/1037] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:25:27 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:25:28.068949 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:28.084023 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d03bc0d-16b3-425d-ae65-c194894a39c8 None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:28.096784 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8999112c-a5a9-4bda-925c-37a823714d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:28.097692 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8999112c-a5a9-4bda-925c-37a823714d6e None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:28.098005 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8999112c-a5a9-4bda-925c-37a823714d6e None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:28.098285 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8999112c-a5a9-4bda-925c-37a823714d6e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:28.174646 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8999112c-a5a9-4bda-925c-37a823714d6e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:28.176061 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8999112c-a5a9-4bda-925c-37a823714d6e None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:28.189342 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d03bc0d-16b3-425d-ae65-c194894a39c8 None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:28.231643 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455ccde5-8d20-49d6-b562-a724ba80582a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:28.233300 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455ccde5-8d20-49d6-b562-a724ba80582a None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:28.233585 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455ccde5-8d20-49d6-b562-a724ba80582a None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:28.233882 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455ccde5-8d20-49d6-b562-a724ba80582a None admin] PATH_INFO: `/v3/domains/default` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:28.235081 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455ccde5-8d20-49d6-b562-a724ba80582a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:28.236665 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455ccde5-8d20-49d6-b562-a724ba80582a None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:28.242008 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 520/1038] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:25:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:25:28.244921 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455ccde5-8d20-49d6-b562-a724ba80582a None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:28.286070 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 519/1039] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:25:27 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 19 14:25:28.286936 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:28.292443 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8999112c-a5a9-4bda-925c-37a823714d6e None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:28.303553 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:28.357228 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455ccde5-8d20-49d6-b562-a724ba80582a None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:28.396287 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae7e232-4aa6-4271-89c5-2126e55ceeff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:28.396961 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae7e232-4aa6-4271-89c5-2126e55ceeff None admin] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:28.397248 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae7e232-4aa6-4271-89c5-2126e55ceeff None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:28.397526 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae7e232-4aa6-4271-89c5-2126e55ceeff None admin] PATH_INFO: `/v3/users` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:28.398268 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae7e232-4aa6-4271-89c5-2126e55ceeff None admin] RBAC: Authorizing `identity:create_user()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:28.400238 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae7e232-4aa6-4271-89c5-2126e55ceeff None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:28.401300 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8999112c-a5a9-4bda-925c-37a823714d6e None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:28.421353 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 521/1040] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 19 14:25:28 2022] POST /identity/v3/users => generated 285 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 14:25:28.453039 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0badb03d-6b1c-4430-8857-04fb7d7041a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:28.454018 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0badb03d-6b1c-4430-8857-04fb7d7041a5 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:28.454369 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0badb03d-6b1c-4430-8857-04fb7d7041a5 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:28.454718 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0badb03d-6b1c-4430-8857-04fb7d7041a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:28.467297 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0badb03d-6b1c-4430-8857-04fb7d7041a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:28.468808 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0badb03d-6b1c-4430-8857-04fb7d7041a5 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:28.489229 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-0badb03d-6b1c-4430-8857-04fb7d7041a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:25:28.489229 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:25:28.489229 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:25:28.489229 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:25:28.489229 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:25:28.489229 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:25:28.489229 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:25:28.489229 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:25:28.489229 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:25:28.489229 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:25:28.489229 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:25:28.489229 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:25:28.489229 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:25:28.489229 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:25:28.489229 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:25:28.490478 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:25:28.490478 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:25:28.490478 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:25:28.490478 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:25:28.490478 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:25:28.490478 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:25:28.490478 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:25:28.490478 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:25:28.490478 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:25:28.490478 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:25:28.490478 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:25:28.490478 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:25:28.490478 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:25:28.490478 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:25:28.490478 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:25:28.490478 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:25:28.490478 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:25:28.491324 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:25:28.491324 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:25:28.491324 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:25:28.491324 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:25:28.491324 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 520/1041] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:25:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:25:28.497364 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae7e232-4aa6-4271-89c5-2126e55ceeff None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:28.601249 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:28.610693 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae7e232-4aa6-4271-89c5-2126e55ceeff None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:28.653135 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e58ab1-e022-4b68-a96e-3d1129d564eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:28.654208 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e58ab1-e022-4b68-a96e-3d1129d564eb None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:28.654694 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e58ab1-e022-4b68-a96e-3d1129d564eb None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:28.655167 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e58ab1-e022-4b68-a96e-3d1129d564eb None admin] PATH_INFO: `/v3/domains` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:28.656120 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e58ab1-e022-4b68-a96e-3d1129d564eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:25:28.657115 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e58ab1-e022-4b68-a96e-3d1129d564eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:28.660656 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e58ab1-e022-4b68-a96e-3d1129d564eb None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:28.672203 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 522/1042] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:25:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:25:28.677964 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0badb03d-6b1c-4430-8857-04fb7d7041a5 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:28.750223 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:28.795207 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0badb03d-6b1c-4430-8857-04fb7d7041a5 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:28.831198 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75177953-6d1d-48a3-b0da-9cc0dce3ebb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:28.832001 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75177953-6d1d-48a3-b0da-9cc0dce3ebb0 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:28.832289 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75177953-6d1d-48a3-b0da-9cc0dce3ebb0 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:28.832570 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75177953-6d1d-48a3-b0da-9cc0dce3ebb0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:28.839676 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75177953-6d1d-48a3-b0da-9cc0dce3ebb0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:28.841126 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75177953-6d1d-48a3-b0da-9cc0dce3ebb0 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:28.856058 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-75177953-6d1d-48a3-b0da-9cc0dce3ebb0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:25:28.856058 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:25:28.856058 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:25:28.856058 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:25:28.856058 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:25:28.856058 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:25:28.856058 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:25:28.856058 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:25:28.856058 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:25:28.856058 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:25:28.856058 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:25:28.856058 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:25:28.856058 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:25:28.856058 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:25:28.856058 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:25:28.856954 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:25:28.856954 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:25:28.856954 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:25:28.856954 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:25:28.856954 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:25:28.856954 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:25:28.856954 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:25:28.856954 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:25:28.856954 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:25:28.856954 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:25:28.856954 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:25:28.856954 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:25:28.856954 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:25:28.856954 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:25:28.856954 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:25:28.856954 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:25:28.856954 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:25:28.857967 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:25:28.857967 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:25:28.857967 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:25:28.857967 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:25:28.857967 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:25:28.857967 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:25:28.857967 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:25:28.857967 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:25:28.857967 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 521/1043] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:25:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:25:28.863246 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e58ab1-e022-4b68-a96e-3d1129d564eb None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:28.937113 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9995455b-cfcb-46bd-b927-479bbd32a2a2 None None] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:28.937425 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9995455b-cfcb-46bd-b927-479bbd32a2a2 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:28.937674 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9995455b-cfcb-46bd-b927-479bbd32a2a2 None None] PATH_INFO: `/` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:28.939214 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 522/1044] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:25:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:25:28.944061 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9748bc47-7e59-4d80-aadc-683ea055e826 None None] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:28.944362 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9748bc47-7e59-4d80-aadc-683ea055e826 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:28.944651 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9748bc47-7e59-4d80-aadc-683ea055e826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:28.962032 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.auth.core [None req-9748bc47-7e59-4d80-aadc-683ea055e826 None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143586) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:25:28.963346 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.fernet_utils [None req-9748bc47-7e59-4d80-aadc-683ea055e826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143586) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:25:28.971589 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e58ab1-e022-4b68-a96e-3d1129d564eb None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:29.011733 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d531e4f-2e09-4905-bdb0-080c856f12a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:29.012503 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d531e4f-2e09-4905-bdb0-080c856f12a5 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:29.012982 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d531e4f-2e09-4905-bdb0-080c856f12a5 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:29.013473 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d531e4f-2e09-4905-bdb0-080c856f12a5 None admin] PATH_INFO: `/v3/projects` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:29.014469 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d531e4f-2e09-4905-bdb0-080c856f12a5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:25:29.015003 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d531e4f-2e09-4905-bdb0-080c856f12a5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:29.016966 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d531e4f-2e09-4905-bdb0-080c856f12a5 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:29.037083 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 523/1045] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:25:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:25:29.038876 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d531e4f-2e09-4905-bdb0-080c856f12a5 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:29.046878 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 523/1046] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:25:28 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:25:29.056081 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:29.070424 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9748bc47-7e59-4d80-aadc-683ea055e826 None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:29.123995 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:29.154120 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d531e4f-2e09-4905-bdb0-080c856f12a5 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:29.176328 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9748bc47-7e59-4d80-aadc-683ea055e826 None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:29.189309 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c65bcf-0e3d-42c1-b1a1-8e219e1d632c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:29.189928 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c65bcf-0e3d-42c1-b1a1-8e219e1d632c None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:29.190218 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c65bcf-0e3d-42c1-b1a1-8e219e1d632c None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:29.190499 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c65bcf-0e3d-42c1-b1a1-8e219e1d632c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:29.191334 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c65bcf-0e3d-42c1-b1a1-8e219e1d632c None admin] RBAC: Adding query filter params (role.id=6efe7f48768b4033a20e04bc93529e6d, scope.project.id=252d9673f08c479a98d04fee3f5856a4, user.id=5f28de22c2ee4c53b6d14d5b0ddf7fe9) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:25:29.191653 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c65bcf-0e3d-42c1-b1a1-8e219e1d632c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:29.193104 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c65bcf-0e3d-42c1-b1a1-8e219e1d632c None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:29.199740 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 524/1047] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:25:29 2022] GET /identity/v3/role_assignments?user.id=5f28de22c2ee4c53b6d14d5b0ddf7fe9&scope.project.id=252d9673f08c479a98d04fee3f5856a4&role.id=6efe7f48768b4033a20e04bc93529e6d => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:25:29.217136 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab498e5-ac3a-4694-9005-6237cecb01bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:29.218101 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab498e5-ac3a-4694-9005-6237cecb01bc None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:29.218457 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab498e5-ac3a-4694-9005-6237cecb01bc None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:29.218803 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab498e5-ac3a-4694-9005-6237cecb01bc None admin] PATH_INFO: `/v3/services/image` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:29.219576 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab498e5-ac3a-4694-9005-6237cecb01bc None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:29.221007 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab498e5-ac3a-4694-9005-6237cecb01bc None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:29.234801 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-1ab498e5-ac3a-4694-9005-6237cecb01bc None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 19 14:25:29.234801 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:25:29.234801 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:25:29.234801 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:25:29.234801 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:25:29.234801 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:25:29.234801 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:25:29.234801 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:25:29.234801 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 14:25:29.234801 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:25:29.234801 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:25:29.234801 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:25:29.234801 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:25:29.234801 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:25:29.234801 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:25:29.234801 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:25:29.234801 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:25:29.235792 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:25:29.235792 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:25:29.235792 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:25:29.235792 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:25:29.235792 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:25:29.235792 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:25:29.235792 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:25:29.235792 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:25:29.235792 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:25:29.235792 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:25:29.235792 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:25:29.235792 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:25:29.235792 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:25:29.235792 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:25:29.235792 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:25:29.235792 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:25:29.242257 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:25:29.242257 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:25:29.242257 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:25:29.242257 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:25:29.242257 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:25:29.242257 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:25:29.242257 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:25:29.242257 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:25:29.242257 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:25:29.242257 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:25:29.242257 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:25:29.242257 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:25:29.242257 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 14:25:29.242257 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:25:29.242257 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 524/1048] 10.222.0.57 () {56 vars in 1105 bytes} [Thu May 19 14:25:29 2022] GET /identity/v3/services/image => generated 86 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:25:29.244365 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c65bcf-0e3d-42c1-b1a1-8e219e1d632c None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:29.332603 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:29.361160 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c65bcf-0e3d-42c1-b1a1-8e219e1d632c None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:29.399447 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d5bc77-6073-4e40-bcd3-e2c186db2d6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:29.400624 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d5bc77-6073-4e40-bcd3-e2c186db2d6a None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:29.400904 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d5bc77-6073-4e40-bcd3-e2c186db2d6a None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:29.401172 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d5bc77-6073-4e40-bcd3-e2c186db2d6a None admin] PATH_INFO: `/v3/services` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:29.401893 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d5bc77-6073-4e40-bcd3-e2c186db2d6a None admin] RBAC: Adding query filter params (name=image) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:25:29.402201 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d5bc77-6073-4e40-bcd3-e2c186db2d6a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:29.403570 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d5bc77-6073-4e40-bcd3-e2c186db2d6a None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:29.408957 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 525/1049] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:25:29 2022] GET /identity/v3/services?name=image => generated 122 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:25:29.414426 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab498e5-ac3a-4694-9005-6237cecb01bc None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:29.520704 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab498e5-ac3a-4694-9005-6237cecb01bc None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:29.537529 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:29.557275 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1ebf84-f10c-4ba2-9045-23c01cc3949b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:29.557965 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1ebf84-f10c-4ba2-9045-23c01cc3949b None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:29.558249 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1ebf84-f10c-4ba2-9045-23c01cc3949b None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:29.558526 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1ebf84-f10c-4ba2-9045-23c01cc3949b None admin] PATH_INFO: `/v3/services` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:29.559224 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1ebf84-f10c-4ba2-9045-23c01cc3949b None admin] RBAC: Adding query filter params (type=image) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:25:29.559528 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1ebf84-f10c-4ba2-9045-23c01cc3949b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:29.560881 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1ebf84-f10c-4ba2-9045-23c01cc3949b None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:29.566161 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 525/1050] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:25:29 2022] GET /identity/v3/services?type=image => generated 350 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:25:29.572121 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d5bc77-6073-4e40-bcd3-e2c186db2d6a None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:29.680496 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d5bc77-6073-4e40-bcd3-e2c186db2d6a None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:29.715808 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185fd9ba-3913-477d-80e0-369085f66919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:29.716465 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185fd9ba-3913-477d-80e0-369085f66919 None admin] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:29.716742 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185fd9ba-3913-477d-80e0-369085f66919 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:29.717017 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185fd9ba-3913-477d-80e0-369085f66919 None admin] PATH_INFO: `/v3/endpoints` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:29.717796 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185fd9ba-3913-477d-80e0-369085f66919 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:29.719206 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185fd9ba-3913-477d-80e0-369085f66919 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:29.734940 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:29.743042 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 526/1051] 10.222.0.57 () {60 vars in 1144 bytes} [Thu May 19 14:25:29 2022] POST /identity/v3/endpoints => generated 327 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:25:30.806471 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6351bba4-5bf4-4370-b42e-1a9d018e0883 None None] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:30.807055 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6351bba4-5bf4-4370-b42e-1a9d018e0883 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:30.807055 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6351bba4-5bf4-4370-b42e-1a9d018e0883 None None] PATH_INFO: `/` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:30.808381 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 526/1052] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:25:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:25:30.813972 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a425c480-5c28-4288-80b9-6304da5d1c2a None None] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:30.814713 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a425c480-5c28-4288-80b9-6304da5d1c2a None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:30.815002 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a425c480-5c28-4288-80b9-6304da5d1c2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:30.832621 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.auth.core [None req-a425c480-5c28-4288-80b9-6304da5d1c2a None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143587) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:25:30.833822 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-a425c480-5c28-4288-80b9-6304da5d1c2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:25:30.912262 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 527/1053] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:25:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:25:30.936937 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6351bba4-5bf4-4370-b42e-1a9d018e0883 None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:31.046071 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6351bba4-5bf4-4370-b42e-1a9d018e0883 None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:31.080690 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3b579c-e150-4aa0-84cc-bc3617588c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:31.081485 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3b579c-e150-4aa0-84cc-bc3617588c43 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:31.081766 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3b579c-e150-4aa0-84cc-bc3617588c43 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:31.082073 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3b579c-e150-4aa0-84cc-bc3617588c43 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:31.093954 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3b579c-e150-4aa0-84cc-bc3617588c43 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:25:31.101778 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-cf3b579c-e150-4aa0-84cc-bc3617588c43 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:25:31.101778 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:25:31.101778 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:25:31.101778 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:25:31.101778 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:25:31.101778 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:25:31.101778 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:25:31.101778 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:25:31.101778 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:25:31.101778 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:25:31.101778 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:25:31.101778 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:25:31.101778 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:25:31.101778 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:25:31.101778 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:25:31.101778 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:25:31.103022 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:25:31.103022 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:25:31.103022 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:25:31.103022 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:25:31.103022 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:25:31.103022 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:25:31.103022 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:25:31.103022 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:25:31.103022 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:25:31.103022 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:25:31.103022 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:25:31.103022 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:25:31.103022 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:25:31.103022 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:25:31.103022 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:25:31.103022 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:25:31.104108 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:25:31.104108 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:25:31.104108 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:25:31.104108 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:25:31.104108 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:25:31.104108 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:25:31.104108 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:25:31.104108 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:25:31.104108 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:25:31.104108 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:25:31.104108 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:25:31.104108 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:25:31.108137 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 527/1054] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:25:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:25:31.109509 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a425c480-5c28-4288-80b9-6304da5d1c2a None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:31.215281 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a425c480-5c28-4288-80b9-6304da5d1c2a None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:31.239083 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:31.251159 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0ec621-c659-4766-8f4d-4ac4824b3040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:31.251998 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0ec621-c659-4766-8f4d-4ac4824b3040 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:31.252419 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0ec621-c659-4766-8f4d-4ac4824b3040 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:31.252833 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0ec621-c659-4766-8f4d-4ac4824b3040 None admin] PATH_INFO: `/v3/roles` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:31.253739 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0ec621-c659-4766-8f4d-4ac4824b3040 None admin] RBAC: Adding query filter params (name=admin) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:25:31.254210 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0ec621-c659-4766-8f4d-4ac4824b3040 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:31.255727 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0ec621-c659-4766-8f4d-4ac4824b3040 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:31.264844 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 528/1055] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:25:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:25:31.270320 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3b579c-e150-4aa0-84cc-bc3617588c43 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:31.317505 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:31.382974 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3b579c-e150-4aa0-84cc-bc3617588c43 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:31.418991 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a899e5a3-9518-4f8f-b66f-5d3e24620c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:31.419859 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a899e5a3-9518-4f8f-b66f-5d3e24620c11 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:31.420148 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a899e5a3-9518-4f8f-b66f-5d3e24620c11 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:31.420427 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a899e5a3-9518-4f8f-b66f-5d3e24620c11 None admin] PATH_INFO: `/v3/users/9d6861ec911f47beb2e2d3c426655104` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:31.434861 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a899e5a3-9518-4f8f-b66f-5d3e24620c11 None admin] RBAC: Authorizing `identity:get_user(user_id=9d6861ec911f47beb2e2d3c426655104)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:31.436312 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a899e5a3-9518-4f8f-b66f-5d3e24620c11 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:31.441941 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 528/1056] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:25:31 2022] GET /identity/v3/users/9d6861ec911f47beb2e2d3c426655104 => generated 285 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:25:31.444082 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0ec621-c659-4766-8f4d-4ac4824b3040 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:31.500018 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc66196-451d-4f6f-865a-55cdae346550 None None] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:31.500281 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc66196-451d-4f6f-865a-55cdae346550 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:31.500503 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc66196-451d-4f6f-865a-55cdae346550 None None] PATH_INFO: `/` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:31.501897 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 529/1057] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:25:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:25:31.506934 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac931ed5-c9d6-4165-9bbf-0a33d69a5a95 None None] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:31.507226 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac931ed5-c9d6-4165-9bbf-0a33d69a5a95 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:31.507546 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac931ed5-c9d6-4165-9bbf-0a33d69a5a95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:31.524742 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.auth.core [None req-ac931ed5-c9d6-4165-9bbf-0a33d69a5a95 None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143586) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:25:31.525962 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.fernet_utils [None req-ac931ed5-c9d6-4165-9bbf-0a33d69a5a95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143586) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:25:31.564695 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0ec621-c659-4766-8f4d-4ac4824b3040 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:31.600789 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36bd785-c83e-4fa9-b914-e53663444790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:31.601442 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36bd785-c83e-4fa9-b914-e53663444790 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:31.601730 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36bd785-c83e-4fa9-b914-e53663444790 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:31.602035 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36bd785-c83e-4fa9-b914-e53663444790 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:31.609276 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 530/1058] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:25:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:25:31.632770 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac931ed5-c9d6-4165-9bbf-0a33d69a5a95 None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:31.661602 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:31.669348 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:31.689476 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36bd785-c83e-4fa9-b914-e53663444790 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:31.690980 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36bd785-c83e-4fa9-b914-e53663444790 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:31.740417 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac931ed5-c9d6-4165-9bbf-0a33d69a5a95 None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:31.777814 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a03c2d-f35c-4631-a75e-c67bc376feb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:31.778730 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a03c2d-f35c-4631-a75e-c67bc376feb0 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:31.779010 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a03c2d-f35c-4631-a75e-c67bc376feb0 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:31.779285 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a03c2d-f35c-4631-a75e-c67bc376feb0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:31.789680 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a03c2d-f35c-4631-a75e-c67bc376feb0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:25:31.798932 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-27a03c2d-f35c-4631-a75e-c67bc376feb0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 14:25:31.798932 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:25:31.798932 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:25:31.798932 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:25:31.798932 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:25:31.798932 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:25:31.798932 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:25:31.798932 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:25:31.798932 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:25:31.798932 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:25:31.798932 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:25:31.798932 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:25:31.798932 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:25:31.798932 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:25:31.798932 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:25:31.798932 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:25:31.799813 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:25:31.799813 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:25:31.799813 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:25:31.799813 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:25:31.799813 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:25:31.799813 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:25:31.799813 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:25:31.799813 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:25:31.799813 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:25:31.799813 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:25:31.799813 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:25:31.799813 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:25:31.799813 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:25:31.799813 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:25:31.799813 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:25:31.799813 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:25:31.800602 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:25:31.800602 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:25:31.800602 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:25:31.800602 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:25:31.800602 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:25:31.800602 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:25:31.800602 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:25:31.800602 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:25:31.800602 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:25:31.800602 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:25:31.800602 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 14:25:31.800602 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:25:31.800602 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 531/1059] 10.222.0.57 () {56 vars in 1120 bytes} [Thu May 19 14:25:31 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:25:31.810057 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 529/1060] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:25:31 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:25:31.810224 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a03c2d-f35c-4631-a75e-c67bc376feb0 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:31.811008 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36bd785-c83e-4fa9-b914-e53663444790 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:31.905670 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:31.909306 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:31.920125 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a03c2d-f35c-4631-a75e-c67bc376feb0 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:31.924577 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36bd785-c83e-4fa9-b914-e53663444790 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:31.958526 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eb0005-9151-4734-9af2-65f4f7fe9560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:31.959201 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eb0005-9151-4734-9af2-65f4f7fe9560 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:31.959488 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eb0005-9151-4734-9af2-65f4f7fe9560 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:31.959768 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eb0005-9151-4734-9af2-65f4f7fe9560 None admin] PATH_INFO: `/v3/roles` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:31.960520 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99eb0005-9151-4734-9af2-65f4f7fe9560 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:25:31.960895 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99eb0005-9151-4734-9af2-65f4f7fe9560 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:31.961496 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e514760-41da-4225-ad98-8303eb91ba4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:31.963397 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99eb0005-9151-4734-9af2-65f4f7fe9560 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:31.964096 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e514760-41da-4225-ad98-8303eb91ba4d None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:31.964096 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e514760-41da-4225-ad98-8303eb91ba4d None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:31.964096 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e514760-41da-4225-ad98-8303eb91ba4d None admin] PATH_INFO: `/v3/projects/b6532fb566c74a37ab1d52330b32f145` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:31.965675 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e514760-41da-4225-ad98-8303eb91ba4d None admin] RBAC: Authorizing `identity:get_project(project_id=b6532fb566c74a37ab1d52330b32f145)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:31.967155 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e514760-41da-4225-ad98-8303eb91ba4d None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:31.974905 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 530/1061] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:25:31 2022] GET /identity/v3/projects/b6532fb566c74a37ab1d52330b32f145 => generated 311 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:25:31.975058 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 532/1062] 10.222.0.57 () {56 vars in 1115 bytes} [Thu May 19 14:25:31 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:25:31.975613 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e514760-41da-4225-ad98-8303eb91ba4d None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:31.976762 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eb0005-9151-4734-9af2-65f4f7fe9560 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:32.084810 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eb0005-9151-4734-9af2-65f4f7fe9560 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:32.086853 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e514760-41da-4225-ad98-8303eb91ba4d None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:32.119190 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baced762-31af-452c-b170-3c66db1257ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:32.119980 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baced762-31af-452c-b170-3c66db1257ef None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:32.120267 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baced762-31af-452c-b170-3c66db1257ef None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:32.120546 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baced762-31af-452c-b170-3c66db1257ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:32.122847 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3ae5fa-e4eb-4f31-b117-30a9f4df67cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:32.123485 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3ae5fa-e4eb-4f31-b117-30a9f4df67cf None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:32.123760 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3ae5fa-e4eb-4f31-b117-30a9f4df67cf None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:32.124036 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3ae5fa-e4eb-4f31-b117-30a9f4df67cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:32.124853 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3ae5fa-e4eb-4f31-b117-30a9f4df67cf None admin] RBAC: Adding query filter params (role.id=a3b793754e7b40b0bb21afe74eec3c18, scope.project.id=b6532fb566c74a37ab1d52330b32f145, user.id=9d6861ec911f47beb2e2d3c426655104) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:25:32.125163 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3ae5fa-e4eb-4f31-b117-30a9f4df67cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:32.126619 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3ae5fa-e4eb-4f31-b117-30a9f4df67cf None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:32.131157 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baced762-31af-452c-b170-3c66db1257ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:32.132090 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 531/1063] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:25:31 2022] GET /identity/v3/role_assignments?user.id=9d6861ec911f47beb2e2d3c426655104&scope.project.id=b6532fb566c74a37ab1d52330b32f145&role.id=a3b793754e7b40b0bb21afe74eec3c18 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:25:32.132568 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baced762-31af-452c-b170-3c66db1257ef None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:32.133395 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a403d5-836d-4a21-984e-286c2bee4464 None None] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:32.133690 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a403d5-836d-4a21-984e-286c2bee4464 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:32.134550 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a403d5-836d-4a21-984e-286c2bee4464 None None] PATH_INFO: `/` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:32.136092 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 532/1064] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:25:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:25:32.142591 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53014e00-429f-4900-84f2-15af0edb42cf None None] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:32.142839 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53014e00-429f-4900-84f2-15af0edb42cf None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:32.143129 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53014e00-429f-4900-84f2-15af0edb42cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:32.153635 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-baced762-31af-452c-b170-3c66db1257ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:25:32.153635 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:25:32.153635 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:25:32.153635 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:25:32.153635 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:25:32.153635 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:25:32.153635 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:25:32.153635 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:25:32.153635 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:25:32.153635 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:25:32.153635 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:25:32.153635 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:25:32.153635 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:25:32.153635 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:25:32.153635 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:25:32.154710 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:25:32.154710 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:25:32.154710 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:25:32.154710 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:25:32.154710 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:25:32.154710 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:25:32.154710 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:25:32.154710 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:25:32.154710 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:25:32.154710 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:25:32.154710 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:25:32.154710 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:25:32.154710 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:25:32.154710 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:25:32.154710 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:25:32.154710 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:25:32.154710 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:25:32.155727 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:25:32.155727 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:25:32.155727 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:25:32.155727 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:25:32.157544 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 533/1065] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:25:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:25:32.163454 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baced762-31af-452c-b170-3c66db1257ef None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:32.166642 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.auth.core [None req-53014e00-429f-4900-84f2-15af0edb42cf None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143587) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:25:32.167417 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:32.167927 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-53014e00-429f-4900-84f2-15af0edb42cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:25:32.183851 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:32.250067 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 533/1066] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:25:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:25:32.276705 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53014e00-429f-4900-84f2-15af0edb42cf None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:32.288764 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baced762-31af-452c-b170-3c66db1257ef None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:32.330674 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716ce105-f1b7-4e65-99c8-ceae8b651655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:32.331748 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716ce105-f1b7-4e65-99c8-ceae8b651655 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:32.332206 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716ce105-f1b7-4e65-99c8-ceae8b651655 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:32.332468 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716ce105-f1b7-4e65-99c8-ceae8b651655 None admin] PATH_INFO: `/v3/domains` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:32.333216 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716ce105-f1b7-4e65-99c8-ceae8b651655 None admin] RBAC: Adding query filter params (name=Default) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:25:32.333612 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716ce105-f1b7-4e65-99c8-ceae8b651655 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:32.335011 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716ce105-f1b7-4e65-99c8-ceae8b651655 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:32.351460 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 534/1067] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:25:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:25:32.358352 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716ce105-f1b7-4e65-99c8-ceae8b651655 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:32.391605 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53014e00-429f-4900-84f2-15af0edb42cf None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:32.423959 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:32.429112 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b23677f-fd93-49e2-99f8-ed403a6b804c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:32.429770 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b23677f-fd93-49e2-99f8-ed403a6b804c None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:32.430087 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b23677f-fd93-49e2-99f8-ed403a6b804c None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:32.430371 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b23677f-fd93-49e2-99f8-ed403a6b804c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:32.452083 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:32.471857 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716ce105-f1b7-4e65-99c8-ceae8b651655 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:32.507467 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b23677f-fd93-49e2-99f8-ed403a6b804c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:32.509176 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b23677f-fd93-49e2-99f8-ed403a6b804c None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:32.512540 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a72e6c4-6f98-4410-be02-eae999ed341d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:32.513568 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a72e6c4-6f98-4410-be02-eae999ed341d None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:32.513680 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a72e6c4-6f98-4410-be02-eae999ed341d None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:32.513919 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a72e6c4-6f98-4410-be02-eae999ed341d None admin] PATH_INFO: `/v3/users/nova` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:32.522740 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a72e6c4-6f98-4410-be02-eae999ed341d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:32.524172 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a72e6c4-6f98-4410-be02-eae999ed341d None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:32.539584 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-9a72e6c4-6f98-4410-be02-eae999ed341d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 14:25:32.539584 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:25:32.539584 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:25:32.539584 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:25:32.539584 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:25:32.539584 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:25:32.539584 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:25:32.539584 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:25:32.539584 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:25:32.539584 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:25:32.539584 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:25:32.539584 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:25:32.539584 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:25:32.539584 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:25:32.539584 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:25:32.539584 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:25:32.541437 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:25:32.541437 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:25:32.541437 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:25:32.541437 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:25:32.541437 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:25:32.541437 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:25:32.541437 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:25:32.541437 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:25:32.541437 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:25:32.541437 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:25:32.541437 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:25:32.541437 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:25:32.541437 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:25:32.541437 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:25:32.541437 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:25:32.541437 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:25:32.541437 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:25:32.541437 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:25:32.542564 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:25:32.542564 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:25:32.542564 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:25:32.542564 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:25:32.542564 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:25:32.542564 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:25:32.542564 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:25:32.542564 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:25:32.542564 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 14:25:32.542564 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:25:32.542564 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 535/1068] 10.222.0.57 () {56 vars in 1093 bytes} [Thu May 19 14:25:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:25:32.547403 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a72e6c4-6f98-4410-be02-eae999ed341d None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:32.626978 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 534/1069] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:25:32 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 19 14:25:32.632496 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b23677f-fd93-49e2-99f8-ed403a6b804c None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:32.669347 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a72e6c4-6f98-4410-be02-eae999ed341d None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:32.695766 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:32.703637 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:32.709504 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23635775-b761-4915-bcdc-7a4448f806d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:32.710337 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23635775-b761-4915-bcdc-7a4448f806d3 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:32.710622 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23635775-b761-4915-bcdc-7a4448f806d3 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:32.710997 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23635775-b761-4915-bcdc-7a4448f806d3 None admin] PATH_INFO: `/v3/users` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:32.711854 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23635775-b761-4915-bcdc-7a4448f806d3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:25:32.712210 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23635775-b761-4915-bcdc-7a4448f806d3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:32.713645 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23635775-b761-4915-bcdc-7a4448f806d3 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:32.733256 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 536/1070] 10.222.0.57 () {56 vars in 1133 bytes} [Thu May 19 14:25:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:25:32.738615 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23635775-b761-4915-bcdc-7a4448f806d3 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:32.764225 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b23677f-fd93-49e2-99f8-ed403a6b804c None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:32.804195 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d7d49a-aa26-47d8-8b4e-c10d4549cf0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:32.804997 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d7d49a-aa26-47d8-8b4e-c10d4549cf0d None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:32.805282 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d7d49a-aa26-47d8-8b4e-c10d4549cf0d None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:32.805788 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d7d49a-aa26-47d8-8b4e-c10d4549cf0d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:32.818653 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d7d49a-aa26-47d8-8b4e-c10d4549cf0d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:32.820116 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d7d49a-aa26-47d8-8b4e-c10d4549cf0d None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:32.839022 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-27d7d49a-aa26-47d8-8b4e-c10d4549cf0d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:25:32.839022 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:25:32.839022 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:25:32.839022 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:25:32.839022 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:25:32.839022 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:25:32.839022 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:25:32.839022 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:25:32.839022 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:25:32.839022 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:25:32.839022 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:25:32.839022 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:25:32.839022 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:25:32.839022 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:25:32.839022 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:25:32.839911 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:25:32.839911 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:25:32.839911 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:25:32.839911 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:25:32.839911 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:25:32.839911 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:25:32.839911 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:25:32.839911 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:25:32.839911 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:25:32.839911 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:25:32.839911 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:25:32.839911 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:25:32.839911 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:25:32.839911 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:25:32.839911 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:25:32.839911 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:25:32.839911 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:25:32.840750 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:25:32.840750 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:25:32.840750 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:25:32.840750 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:25:32.840750 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 535/1071] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:25:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:25:32.846025 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d7d49a-aa26-47d8-8b4e-c10d4549cf0d None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:32.846573 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23635775-b761-4915-bcdc-7a4448f806d3 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:32.893611 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6106f7-e4c7-4d2a-9677-52242dc56de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:32.894363 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6106f7-e4c7-4d2a-9677-52242dc56de8 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:32.894650 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6106f7-e4c7-4d2a-9677-52242dc56de8 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:32.894932 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6106f7-e4c7-4d2a-9677-52242dc56de8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:32.910676 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6106f7-e4c7-4d2a-9677-52242dc56de8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:32.913246 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6106f7-e4c7-4d2a-9677-52242dc56de8 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:32.933388 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-4e6106f7-e4c7-4d2a-9677-52242dc56de8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:25:32.933388 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:25:32.933388 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:25:32.933388 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:25:32.933388 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:25:32.933388 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:25:32.933388 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:25:32.933388 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:25:32.933388 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:25:32.933388 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:25:32.933388 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:25:32.933388 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:25:32.933388 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:25:32.933388 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:25:32.933388 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:25:32.934309 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:25:32.934309 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:25:32.934309 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:25:32.934309 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:25:32.934309 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:25:32.934309 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:25:32.934309 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:25:32.934309 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:25:32.934309 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:25:32.934309 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:25:32.934309 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:25:32.934309 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:25:32.934309 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:25:32.934309 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:25:32.934309 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:25:32.934309 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:25:32.934309 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:25:32.935139 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:25:32.935139 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:25:32.935139 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:25:32.935139 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:25:32.935139 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 537/1072] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:25:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:25:32.940103 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6106f7-e4c7-4d2a-9677-52242dc56de8 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:32.955897 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:32.959768 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d7d49a-aa26-47d8-8b4e-c10d4549cf0d None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:32.972457 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:33.004481 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f005e561-48ab-4645-8bfb-bb2bbd0b088c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:33.005317 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f005e561-48ab-4645-8bfb-bb2bbd0b088c None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:33.005963 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f005e561-48ab-4645-8bfb-bb2bbd0b088c None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:33.006254 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f005e561-48ab-4645-8bfb-bb2bbd0b088c None admin] PATH_INFO: `/v3/domains` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:33.007167 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f005e561-48ab-4645-8bfb-bb2bbd0b088c None admin] RBAC: Adding query filter params (name=Default) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:25:33.007480 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f005e561-48ab-4645-8bfb-bb2bbd0b088c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:33.008912 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f005e561-48ab-4645-8bfb-bb2bbd0b088c None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:33.020903 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 536/1073] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:25:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:25:33.051220 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6106f7-e4c7-4d2a-9677-52242dc56de8 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:33.089241 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a15bdc-ba0c-4e50-95e0-c3b2cba19311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:33.089962 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a15bdc-ba0c-4e50-95e0-c3b2cba19311 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:33.090245 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a15bdc-ba0c-4e50-95e0-c3b2cba19311 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:33.090519 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a15bdc-ba0c-4e50-95e0-c3b2cba19311 None admin] PATH_INFO: `/v3/domains` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:33.091254 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a15bdc-ba0c-4e50-95e0-c3b2cba19311 None admin] RBAC: Adding query filter params (name=Default) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:25:33.091560 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a15bdc-ba0c-4e50-95e0-c3b2cba19311 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:33.092960 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a15bdc-ba0c-4e50-95e0-c3b2cba19311 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:33.107156 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 538/1074] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:25:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:25:33.112764 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f005e561-48ab-4645-8bfb-bb2bbd0b088c None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:33.224798 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f005e561-48ab-4645-8bfb-bb2bbd0b088c None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:33.263208 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282d1d21-8d3c-487b-849e-062ceec265d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:33.264038 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282d1d21-8d3c-487b-849e-062ceec265d1 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:33.264917 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282d1d21-8d3c-487b-849e-062ceec265d1 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:33.264917 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282d1d21-8d3c-487b-849e-062ceec265d1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:33.272440 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282d1d21-8d3c-487b-849e-062ceec265d1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:33.273972 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282d1d21-8d3c-487b-849e-062ceec265d1 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:33.289997 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-282d1d21-8d3c-487b-849e-062ceec265d1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:25:33.289997 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:25:33.289997 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:25:33.289997 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:25:33.289997 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:25:33.289997 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:25:33.289997 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:25:33.289997 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:25:33.289997 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:25:33.289997 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:25:33.289997 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:25:33.289997 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:25:33.289997 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:25:33.289997 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:25:33.289997 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:25:33.291438 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:25:33.291438 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:25:33.291438 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:25:33.291438 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:25:33.291438 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:25:33.291438 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:25:33.291438 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:25:33.291438 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:25:33.291438 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:25:33.291438 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:25:33.291438 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:25:33.291438 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:25:33.291438 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:25:33.291438 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:25:33.291438 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:25:33.291438 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:25:33.291438 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:25:33.292292 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:25:33.292292 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:25:33.292292 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:25:33.292292 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:25:33.292292 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:25:33.292292 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:25:33.292292 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:25:33.292292 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:25:33.292292 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 537/1075] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:25:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:25:33.297686 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a15bdc-ba0c-4e50-95e0-c3b2cba19311 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:33.400148 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:33.410508 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a15bdc-ba0c-4e50-95e0-c3b2cba19311 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:33.443308 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d42ca1-7753-44ed-ab61-35efdeef5804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:33.443959 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d42ca1-7753-44ed-ab61-35efdeef5804 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:33.444240 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d42ca1-7753-44ed-ab61-35efdeef5804 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:33.444512 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d42ca1-7753-44ed-ab61-35efdeef5804 None admin] PATH_INFO: `/v3/projects` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:33.445290 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d42ca1-7753-44ed-ab61-35efdeef5804 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:25:33.445601 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d42ca1-7753-44ed-ab61-35efdeef5804 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:33.447601 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d42ca1-7753-44ed-ab61-35efdeef5804 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:33.460168 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 539/1076] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:25:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:25:33.466714 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282d1d21-8d3c-487b-849e-062ceec265d1 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:33.488182 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:33.564402 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282d1d21-8d3c-487b-849e-062ceec265d1 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:33.598060 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7ea142-5a8d-4402-99f6-db8761007863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:33.598928 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7ea142-5a8d-4402-99f6-db8761007863 None admin] REQUEST_METHOD: `PUT` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:33.599207 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7ea142-5a8d-4402-99f6-db8761007863 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:33.599480 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7ea142-5a8d-4402-99f6-db8761007863 None admin] PATH_INFO: `/v3/projects/252d9673f08c479a98d04fee3f5856a4/users/5f28de22c2ee4c53b6d14d5b0ddf7fe9/roles/6efe7f48768b4033a20e04bc93529e6d` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:33.601638 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7ea142-5a8d-4402-99f6-db8761007863 None admin] RBAC: Authorizing `identity:create_grant(project_id=252d9673f08c479a98d04fee3f5856a4, user_id=5f28de22c2ee4c53b6d14d5b0ddf7fe9, role_id=6efe7f48768b4033a20e04bc93529e6d)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:33.603136 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7ea142-5a8d-4402-99f6-db8761007863 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:33.616135 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 538/1077] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 14:25:33 2022] PUT /identity/v3/projects/252d9673f08c479a98d04fee3f5856a4/users/5f28de22c2ee4c53b6d14d5b0ddf7fe9/roles/6efe7f48768b4033a20e04bc93529e6d => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:25:34.400035 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0c8c11-2b01-4f6d-81ca-938ac04c14d7 None None] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:34.400722 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0c8c11-2b01-4f6d-81ca-938ac04c14d7 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:34.400722 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0c8c11-2b01-4f6d-81ca-938ac04c14d7 None None] PATH_INFO: `/` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:34.401974 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 540/1078] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:25:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:25:34.407420 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272516b7-273e-45f9-878a-7841e801a17f None None] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:34.407870 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272516b7-273e-45f9-878a-7841e801a17f None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:34.407987 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272516b7-273e-45f9-878a-7841e801a17f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:34.425577 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.auth.core [None req-272516b7-273e-45f9-878a-7841e801a17f None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143587) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:25:34.426803 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-272516b7-273e-45f9-878a-7841e801a17f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:25:34.494323 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 539/1079] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:25:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:25:34.516861 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0c8c11-2b01-4f6d-81ca-938ac04c14d7 None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:34.615933 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0c8c11-2b01-4f6d-81ca-938ac04c14d7 None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:34.650753 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf6b036-34c1-4b0c-ac5a-1ef6f2d4b8f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:34.651471 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf6b036-34c1-4b0c-ac5a-1ef6f2d4b8f8 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:34.651755 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf6b036-34c1-4b0c-ac5a-1ef6f2d4b8f8 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:34.654829 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf6b036-34c1-4b0c-ac5a-1ef6f2d4b8f8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:34.664183 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf6b036-34c1-4b0c-ac5a-1ef6f2d4b8f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:25:34.671779 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-acf6b036-34c1-4b0c-ac5a-1ef6f2d4b8f8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:25:34.671779 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:25:34.671779 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:25:34.671779 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:25:34.671779 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:25:34.671779 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:25:34.671779 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:25:34.671779 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:25:34.671779 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:25:34.671779 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:25:34.671779 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:25:34.671779 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:25:34.671779 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:25:34.671779 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:25:34.671779 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:25:34.671779 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:25:34.672819 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:25:34.672819 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:25:34.672819 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:25:34.672819 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:25:34.672819 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:25:34.672819 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:25:34.672819 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:25:34.672819 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:25:34.672819 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:25:34.672819 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:25:34.672819 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:25:34.672819 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:25:34.672819 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:25:34.672819 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:25:34.672819 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:25:34.672819 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:25:34.673657 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:25:34.673657 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:25:34.673657 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:25:34.673657 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:25:34.673657 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:25:34.673657 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:25:34.673657 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:25:34.673657 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:25:34.673657 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:25:34.673657 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:25:34.673657 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:25:34.673657 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:25:34.673657 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 541/1080] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:25:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:25:34.679130 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272516b7-273e-45f9-878a-7841e801a17f None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:34.693096 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:34.784735 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272516b7-273e-45f9-878a-7841e801a17f None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:34.820144 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a790ffb6-c9aa-4bff-8ca7-961cc9d6fc6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:34.820812 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a790ffb6-c9aa-4bff-8ca7-961cc9d6fc6f None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:34.821095 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a790ffb6-c9aa-4bff-8ca7-961cc9d6fc6f None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:34.821404 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a790ffb6-c9aa-4bff-8ca7-961cc9d6fc6f None admin] PATH_INFO: `/v3/roles` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:34.822274 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a790ffb6-c9aa-4bff-8ca7-961cc9d6fc6f None admin] RBAC: Adding query filter params (name=admin) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:25:34.822590 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a790ffb6-c9aa-4bff-8ca7-961cc9d6fc6f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:34.823971 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a790ffb6-c9aa-4bff-8ca7-961cc9d6fc6f None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:34.832273 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 540/1081] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:25:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:25:34.838153 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf6b036-34c1-4b0c-ac5a-1ef6f2d4b8f8 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:34.854448 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:34.939195 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf6b036-34c1-4b0c-ac5a-1ef6f2d4b8f8 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:34.972317 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc397c49-86f6-45a4-b8e0-c6bd0ebe3ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:34.973183 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc397c49-86f6-45a4-b8e0-c6bd0ebe3ad6 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:34.973467 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc397c49-86f6-45a4-b8e0-c6bd0ebe3ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:34.973745 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc397c49-86f6-45a4-b8e0-c6bd0ebe3ad6 None admin] PATH_INFO: `/v3/users/9d6861ec911f47beb2e2d3c426655104` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:34.974978 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc397c49-86f6-45a4-b8e0-c6bd0ebe3ad6 None admin] RBAC: Authorizing `identity:get_user(user_id=9d6861ec911f47beb2e2d3c426655104)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:34.976410 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc397c49-86f6-45a4-b8e0-c6bd0ebe3ad6 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:34.978734 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 542/1082] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:25:34 2022] GET /identity/v3/users/9d6861ec911f47beb2e2d3c426655104 => generated 285 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:25:34.984489 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a790ffb6-c9aa-4bff-8ca7-961cc9d6fc6f None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:35.053096 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:35.083992 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a790ffb6-c9aa-4bff-8ca7-961cc9d6fc6f None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:35.118168 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d869591-fbd1-4535-8873-039d4bcac0cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:35.119708 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d869591-fbd1-4535-8873-039d4bcac0cb None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:35.120006 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d869591-fbd1-4535-8873-039d4bcac0cb None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:35.120279 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d869591-fbd1-4535-8873-039d4bcac0cb None admin] PATH_INFO: `/v3/projects/b6532fb566c74a37ab1d52330b32f145` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:35.121458 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d869591-fbd1-4535-8873-039d4bcac0cb None admin] RBAC: Authorizing `identity:get_project(project_id=b6532fb566c74a37ab1d52330b32f145)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:35.122926 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d869591-fbd1-4535-8873-039d4bcac0cb None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:35.124653 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 541/1083] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:25:34 2022] GET /identity/v3/projects/b6532fb566c74a37ab1d52330b32f145 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:25:35.130529 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc397c49-86f6-45a4-b8e0-c6bd0ebe3ad6 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:35.214591 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:35.234133 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc397c49-86f6-45a4-b8e0-c6bd0ebe3ad6 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:35.266522 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c757c188-5242-4bb2-a6a5-56823421ca5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:35.267404 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c757c188-5242-4bb2-a6a5-56823421ca5f None admin] REQUEST_METHOD: `PUT` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:35.267685 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c757c188-5242-4bb2-a6a5-56823421ca5f None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:35.267957 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c757c188-5242-4bb2-a6a5-56823421ca5f None admin] PATH_INFO: `/v3/projects/b6532fb566c74a37ab1d52330b32f145/users/9d6861ec911f47beb2e2d3c426655104/roles/a3b793754e7b40b0bb21afe74eec3c18` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:35.269918 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c757c188-5242-4bb2-a6a5-56823421ca5f None admin] RBAC: Authorizing `identity:create_grant(project_id=b6532fb566c74a37ab1d52330b32f145, user_id=9d6861ec911f47beb2e2d3c426655104, role_id=a3b793754e7b40b0bb21afe74eec3c18)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:35.271345 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c757c188-5242-4bb2-a6a5-56823421ca5f None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:35.285152 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 543/1084] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 14:25:35 2022] PUT /identity/v3/projects/b6532fb566c74a37ab1d52330b32f145/users/9d6861ec911f47beb2e2d3c426655104/roles/a3b793754e7b40b0bb21afe74eec3c18 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 14:25:35.871335 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69992da7-5508-461b-86f5-34254c9db730 None None] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:35.871851 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69992da7-5508-461b-86f5-34254c9db730 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:35.871935 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69992da7-5508-461b-86f5-34254c9db730 None None] PATH_INFO: `/` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:35.873273 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 542/1085] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:25:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:25:35.878840 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff7cfc0-323f-4bb9-903a-ea979b5eb541 None None] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:35.879264 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff7cfc0-323f-4bb9-903a-ea979b5eb541 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:35.879544 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff7cfc0-323f-4bb9-903a-ea979b5eb541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:35.896270 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.auth.core [None req-eff7cfc0-323f-4bb9-903a-ea979b5eb541 None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143586) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:25:35.897457 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.fernet_utils [None req-eff7cfc0-323f-4bb9-903a-ea979b5eb541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143586) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:25:35.964813 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 544/1086] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:25:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:25:35.988816 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69992da7-5508-461b-86f5-34254c9db730 None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:36.092476 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69992da7-5508-461b-86f5-34254c9db730 None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:36.126443 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da7e6f5-f79e-4ff3-8be1-c29e4ebf33b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:36.127262 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da7e6f5-f79e-4ff3-8be1-c29e4ebf33b2 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:36.127545 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da7e6f5-f79e-4ff3-8be1-c29e4ebf33b2 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:36.127823 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da7e6f5-f79e-4ff3-8be1-c29e4ebf33b2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:36.135606 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:36.139315 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da7e6f5-f79e-4ff3-8be1-c29e4ebf33b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:25:36.147638 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-6da7e6f5-f79e-4ff3-8be1-c29e4ebf33b2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 14:25:36.147638 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:25:36.147638 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:25:36.147638 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:25:36.147638 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:25:36.147638 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:25:36.147638 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:25:36.147638 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:25:36.147638 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:25:36.147638 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:25:36.147638 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:25:36.147638 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:25:36.147638 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:25:36.147638 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:25:36.147638 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:25:36.147638 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:25:36.148501 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:25:36.148501 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:25:36.148501 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:25:36.148501 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:25:36.148501 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:25:36.148501 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:25:36.148501 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:25:36.148501 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:25:36.148501 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:25:36.148501 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:25:36.148501 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:25:36.148501 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:25:36.148501 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:25:36.148501 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:25:36.148501 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:25:36.148501 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:25:36.158143 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:25:36.158143 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:25:36.158143 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:25:36.158143 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:25:36.158143 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:25:36.158143 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:25:36.158143 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:25:36.158143 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:25:36.158143 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:25:36.158143 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:25:36.158143 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 14:25:36.158143 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:25:36.158143 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 543/1087] 10.222.0.57 () {56 vars in 1120 bytes} [Thu May 19 14:25:35 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:25:36.158985 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff7cfc0-323f-4bb9-903a-ea979b5eb541 None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:36.225731 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:36.256863 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff7cfc0-323f-4bb9-903a-ea979b5eb541 None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:36.292608 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebec965-183b-4a85-a640-93dd5105c178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:36.293274 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebec965-183b-4a85-a640-93dd5105c178 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:36.293561 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebec965-183b-4a85-a640-93dd5105c178 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:36.293858 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebec965-183b-4a85-a640-93dd5105c178 None admin] PATH_INFO: `/v3/roles` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:36.294604 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebec965-183b-4a85-a640-93dd5105c178 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:25:36.294913 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebec965-183b-4a85-a640-93dd5105c178 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:36.296278 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebec965-183b-4a85-a640-93dd5105c178 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:36.304302 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 545/1088] 10.222.0.57 () {56 vars in 1115 bytes} [Thu May 19 14:25:36 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:25:36.310115 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da7e6f5-f79e-4ff3-8be1-c29e4ebf33b2 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:36.411005 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da7e6f5-f79e-4ff3-8be1-c29e4ebf33b2 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:36.445738 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095f6785-7f79-4a27-be50-e1e7d000e71f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:36.447251 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095f6785-7f79-4a27-be50-e1e7d000e71f None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:36.447533 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095f6785-7f79-4a27-be50-e1e7d000e71f None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:36.447796 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095f6785-7f79-4a27-be50-e1e7d000e71f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:36.458049 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095f6785-7f79-4a27-be50-e1e7d000e71f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:36.459511 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095f6785-7f79-4a27-be50-e1e7d000e71f None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:36.478275 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-095f6785-7f79-4a27-be50-e1e7d000e71f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:25:36.478275 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:25:36.478275 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:25:36.478275 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:25:36.478275 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:25:36.478275 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:25:36.478275 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:25:36.478275 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:25:36.478275 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:25:36.478275 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:25:36.478275 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:25:36.478275 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:25:36.478275 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:25:36.478275 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:25:36.478275 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:25:36.479137 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:25:36.479137 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:25:36.479137 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:25:36.479137 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:25:36.479137 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:25:36.479137 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:25:36.479137 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:25:36.479137 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:25:36.479137 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:25:36.479137 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:25:36.479137 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:25:36.479137 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:25:36.479137 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:25:36.479137 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:25:36.479137 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:25:36.479137 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:25:36.479137 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:25:36.480292 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:25:36.480292 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:25:36.480292 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:25:36.480292 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:25:36.480292 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 544/1089] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:25:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:25:36.485178 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebec965-183b-4a85-a640-93dd5105c178 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:36.585233 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebec965-183b-4a85-a640-93dd5105c178 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:36.614009 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:36.620107 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bd1593-b65c-4c64-b197-a1b6b38d1f81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:36.620762 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bd1593-b65c-4c64-b197-a1b6b38d1f81 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:36.621049 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bd1593-b65c-4c64-b197-a1b6b38d1f81 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:36.621330 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bd1593-b65c-4c64-b197-a1b6b38d1f81 None admin] PATH_INFO: `/v3/domains` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:36.622083 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bd1593-b65c-4c64-b197-a1b6b38d1f81 None admin] RBAC: Adding query filter params (name=Default) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:25:36.622397 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bd1593-b65c-4c64-b197-a1b6b38d1f81 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:36.623765 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bd1593-b65c-4c64-b197-a1b6b38d1f81 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:36.636445 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 546/1090] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:25:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:25:36.642300 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095f6785-7f79-4a27-be50-e1e7d000e71f None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:36.698298 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:36.746716 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095f6785-7f79-4a27-be50-e1e7d000e71f None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:36.779605 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80b08b8-0c16-41b3-ac90-8804cec2f38b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:36.780389 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80b08b8-0c16-41b3-ac90-8804cec2f38b None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:36.780669 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80b08b8-0c16-41b3-ac90-8804cec2f38b None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:36.780938 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80b08b8-0c16-41b3-ac90-8804cec2f38b None admin] PATH_INFO: `/v3/users/nova` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:36.788108 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80b08b8-0c16-41b3-ac90-8804cec2f38b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:36.789529 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80b08b8-0c16-41b3-ac90-8804cec2f38b None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:36.804899 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-d80b08b8-0c16-41b3-ac90-8804cec2f38b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 14:25:36.804899 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:25:36.804899 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:25:36.804899 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:25:36.804899 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:25:36.804899 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:25:36.804899 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:25:36.804899 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:25:36.804899 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:25:36.804899 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:25:36.804899 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:25:36.804899 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:25:36.804899 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:25:36.804899 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:25:36.804899 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:25:36.804899 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:25:36.806560 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:25:36.806560 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:25:36.806560 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:25:36.806560 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:25:36.806560 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:25:36.806560 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:25:36.806560 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:25:36.806560 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:25:36.806560 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:25:36.806560 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:25:36.806560 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:25:36.806560 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:25:36.806560 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:25:36.806560 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:25:36.806560 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:25:36.806560 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:25:36.806560 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:25:36.806560 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:25:36.807447 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:25:36.807447 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:25:36.807447 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:25:36.807447 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:25:36.807447 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:25:36.807447 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:25:36.807447 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:25:36.807447 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:25:36.807447 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 14:25:36.807447 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:25:36.807447 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 545/1091] 10.222.0.57 () {56 vars in 1093 bytes} [Thu May 19 14:25:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:25:36.812583 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bd1593-b65c-4c64-b197-a1b6b38d1f81 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:36.911167 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bd1593-b65c-4c64-b197-a1b6b38d1f81 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:36.945229 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01885ca0-4d78-4005-a7ef-e628def82a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:36.945915 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01885ca0-4d78-4005-a7ef-e628def82a2d None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:36.946282 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01885ca0-4d78-4005-a7ef-e628def82a2d None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:36.947370 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01885ca0-4d78-4005-a7ef-e628def82a2d None admin] PATH_INFO: `/v3/users` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:36.947965 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01885ca0-4d78-4005-a7ef-e628def82a2d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:25:36.948273 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01885ca0-4d78-4005-a7ef-e628def82a2d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:36.949683 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01885ca0-4d78-4005-a7ef-e628def82a2d None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:36.966808 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 547/1092] 10.222.0.57 () {56 vars in 1133 bytes} [Thu May 19 14:25:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:25:36.972853 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80b08b8-0c16-41b3-ac90-8804cec2f38b None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:37.075868 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80b08b8-0c16-41b3-ac90-8804cec2f38b None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:37.089301 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:37.111117 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f306b446-f722-49c3-a2f1-15c9995296d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:37.111765 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f306b446-f722-49c3-a2f1-15c9995296d5 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:37.112053 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f306b446-f722-49c3-a2f1-15c9995296d5 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:37.112333 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f306b446-f722-49c3-a2f1-15c9995296d5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:37.186634 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f306b446-f722-49c3-a2f1-15c9995296d5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:37.188653 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f306b446-f722-49c3-a2f1-15c9995296d5 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:37.287381 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 546/1093] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:25:36 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:25:37.293225 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01885ca0-4d78-4005-a7ef-e628def82a2d None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:37.345316 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:37.398496 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01885ca0-4d78-4005-a7ef-e628def82a2d None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:37.433569 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6e937a-059b-4e3f-bcc1-9d00952acfde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:37.434388 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6e937a-059b-4e3f-bcc1-9d00952acfde None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:37.434677 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6e937a-059b-4e3f-bcc1-9d00952acfde None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:37.434957 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6e937a-059b-4e3f-bcc1-9d00952acfde None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:37.445318 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6e937a-059b-4e3f-bcc1-9d00952acfde None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:37.447394 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6e937a-059b-4e3f-bcc1-9d00952acfde None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:37.465746 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-bd6e937a-059b-4e3f-bcc1-9d00952acfde None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:25:37.465746 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:25:37.465746 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:25:37.465746 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:25:37.465746 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:25:37.465746 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:25:37.465746 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:25:37.465746 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:25:37.465746 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:25:37.465746 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:25:37.465746 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:25:37.465746 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:25:37.465746 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:25:37.465746 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:25:37.465746 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:25:37.466638 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:25:37.466638 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:25:37.466638 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:25:37.466638 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:25:37.466638 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:25:37.466638 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:25:37.466638 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:25:37.466638 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:25:37.466638 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:25:37.466638 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:25:37.466638 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:25:37.466638 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:25:37.466638 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:25:37.466638 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:25:37.466638 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:25:37.466638 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:25:37.466638 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:25:37.467485 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:25:37.467485 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:25:37.467485 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:25:37.467485 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:25:37.467485 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 548/1094] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:25:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:25:37.472898 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f306b446-f722-49c3-a2f1-15c9995296d5 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:37.487117 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:37.531966 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcc0cd6-da17-485d-b283-dae50ad9b53f None None] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:37.532254 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcc0cd6-da17-485d-b283-dae50ad9b53f None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:37.532539 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcc0cd6-da17-485d-b283-dae50ad9b53f None None] PATH_INFO: `/` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:37.533945 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 549/1095] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:25:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:25:37.538857 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de85a499-87b6-499c-beb1-cbf617246a84 None None] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:37.539146 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de85a499-87b6-499c-beb1-cbf617246a84 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:37.539428 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de85a499-87b6-499c-beb1-cbf617246a84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:37.556152 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.auth.core [None req-de85a499-87b6-499c-beb1-cbf617246a84 None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143586) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:25:37.557512 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.fernet_utils [None req-de85a499-87b6-499c-beb1-cbf617246a84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143586) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:25:37.575592 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f306b446-f722-49c3-a2f1-15c9995296d5 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:37.609758 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb4ff26-c738-4227-a959-ac02f73fab02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:37.610380 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb4ff26-c738-4227-a959-ac02f73fab02 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:37.610690 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb4ff26-c738-4227-a959-ac02f73fab02 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:37.611369 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb4ff26-c738-4227-a959-ac02f73fab02 None admin] PATH_INFO: `/v3/domains` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:37.612111 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb4ff26-c738-4227-a959-ac02f73fab02 None admin] RBAC: Adding query filter params (name=Default) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:25:37.612673 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb4ff26-c738-4227-a959-ac02f73fab02 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:37.614068 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb4ff26-c738-4227-a959-ac02f73fab02 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:37.625377 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 550/1096] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:25:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:25:37.626171 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 547/1097] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:25:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:25:37.631544 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de85a499-87b6-499c-beb1-cbf617246a84 None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:37.650158 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb4ff26-c738-4227-a959-ac02f73fab02 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:37.665236 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:37.723645 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:37.738150 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de85a499-87b6-499c-beb1-cbf617246a84 None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:37.753686 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb4ff26-c738-4227-a959-ac02f73fab02 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:37.772267 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e37657-f0af-48d9-955b-c0d8428c6a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:37.773063 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e37657-f0af-48d9-955b-c0d8428c6a2e None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:37.773348 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e37657-f0af-48d9-955b-c0d8428c6a2e None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:37.773623 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e37657-f0af-48d9-955b-c0d8428c6a2e None admin] PATH_INFO: `/v3/projects/service` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:37.781039 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e37657-f0af-48d9-955b-c0d8428c6a2e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:37.782601 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e37657-f0af-48d9-955b-c0d8428c6a2e None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:37.786691 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14ac6bf-3efe-41fd-8e6e-a206608439e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:37.787474 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14ac6bf-3efe-41fd-8e6e-a206608439e9 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:37.787754 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14ac6bf-3efe-41fd-8e6e-a206608439e9 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:37.788034 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14ac6bf-3efe-41fd-8e6e-a206608439e9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:37.797212 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-02e37657-f0af-48d9-955b-c0d8428c6a2e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:25:37.797212 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:25:37.797212 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:25:37.797212 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:25:37.797212 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:25:37.797212 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:25:37.797212 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:25:37.797212 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:25:37.797212 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:25:37.797212 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:25:37.797212 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:25:37.797212 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:25:37.797212 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:25:37.797212 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:25:37.797212 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:25:37.798294 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:25:37.798294 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:25:37.798294 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:25:37.798294 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:25:37.798294 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:25:37.798294 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:25:37.798294 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:25:37.798294 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:25:37.798294 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:25:37.798294 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:25:37.798294 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:25:37.798294 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:25:37.798294 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:25:37.798294 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:25:37.798294 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:25:37.798294 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:25:37.798294 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:25:37.799487 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:25:37.799487 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:25:37.799487 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:25:37.799487 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:25:37.799487 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:25:37.799487 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:25:37.799487 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:25:37.799487 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:25:37.799487 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 551/1098] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:25:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:25:37.800985 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14ac6bf-3efe-41fd-8e6e-a206608439e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:25:37.804206 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e37657-f0af-48d9-955b-c0d8428c6a2e None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:37.808731 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-e14ac6bf-3efe-41fd-8e6e-a206608439e9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:25:37.808731 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:25:37.808731 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:25:37.808731 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:25:37.808731 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:25:37.808731 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:25:37.808731 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:25:37.808731 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:25:37.808731 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:25:37.808731 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:25:37.808731 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:25:37.808731 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:25:37.808731 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:25:37.808731 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:25:37.808731 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:25:37.808731 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:25:37.809614 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:25:37.809614 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:25:37.809614 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:25:37.809614 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:25:37.809614 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:25:37.809614 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:25:37.809614 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:25:37.809614 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:25:37.809614 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:25:37.809614 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:25:37.809614 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:25:37.809614 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:25:37.809614 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:25:37.809614 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:25:37.809614 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:25:37.809614 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:25:37.810448 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:25:37.810448 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:25:37.810448 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:25:37.810448 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:25:37.810448 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:25:37.810448 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:25:37.810448 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:25:37.810448 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:25:37.810448 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:25:37.810448 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:25:37.810448 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:25:37.810448 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:25:37.810448 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 548/1099] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:25:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:25:37.815891 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14ac6bf-3efe-41fd-8e6e-a206608439e9 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:37.898448 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:37.908431 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e37657-f0af-48d9-955b-c0d8428c6a2e None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:37.915043 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14ac6bf-3efe-41fd-8e6e-a206608439e9 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:37.941975 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6663a1-b6b9-4a62-9e9a-01eb0f9ba1b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:37.942749 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6663a1-b6b9-4a62-9e9a-01eb0f9ba1b7 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:37.943031 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6663a1-b6b9-4a62-9e9a-01eb0f9ba1b7 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:37.943315 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6663a1-b6b9-4a62-9e9a-01eb0f9ba1b7 None admin] PATH_INFO: `/v3/projects` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:37.943615 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:37.944106 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6663a1-b6b9-4a62-9e9a-01eb0f9ba1b7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:25:37.944422 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6663a1-b6b9-4a62-9e9a-01eb0f9ba1b7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:37.945929 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6663a1-b6b9-4a62-9e9a-01eb0f9ba1b7 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:37.950111 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12028be-0873-4b81-b749-179ef88702ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:37.950774 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12028be-0873-4b81-b749-179ef88702ee None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:37.951054 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12028be-0873-4b81-b749-179ef88702ee None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:37.951327 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12028be-0873-4b81-b749-179ef88702ee None admin] PATH_INFO: `/v3/roles` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:37.952068 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12028be-0873-4b81-b749-179ef88702ee None admin] RBAC: Adding query filter params (name=admin) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:25:37.952374 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12028be-0873-4b81-b749-179ef88702ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:37.953732 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12028be-0873-4b81-b749-179ef88702ee None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:37.958792 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 552/1100] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:25:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:25:37.963542 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 549/1101] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:25:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:25:37.969069 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12028be-0873-4b81-b749-179ef88702ee None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:37.969192 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6663a1-b6b9-4a62-9e9a-01eb0f9ba1b7 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:38.067989 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12028be-0873-4b81-b749-179ef88702ee None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:38.068552 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6663a1-b6b9-4a62-9e9a-01eb0f9ba1b7 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:38.102948 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a2ed58-cffc-4ddf-9406-840bf518c2c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:38.103284 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972f5612-53f8-4994-82fa-ca2a7d960215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:38.103784 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a2ed58-cffc-4ddf-9406-840bf518c2c9 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:38.104070 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a2ed58-cffc-4ddf-9406-840bf518c2c9 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:38.104169 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972f5612-53f8-4994-82fa-ca2a7d960215 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:38.104348 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a2ed58-cffc-4ddf-9406-840bf518c2c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:38.104436 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972f5612-53f8-4994-82fa-ca2a7d960215 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:38.104689 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972f5612-53f8-4994-82fa-ca2a7d960215 None admin] PATH_INFO: `/v3/users/9d6861ec911f47beb2e2d3c426655104` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:38.105187 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a2ed58-cffc-4ddf-9406-840bf518c2c9 None admin] RBAC: Adding query filter params (role.id=6efe7f48768b4033a20e04bc93529e6d, scope.project.id=252d9673f08c479a98d04fee3f5856a4, user.id=5f28de22c2ee4c53b6d14d5b0ddf7fe9) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:25:38.105502 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a2ed58-cffc-4ddf-9406-840bf518c2c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:38.105981 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972f5612-53f8-4994-82fa-ca2a7d960215 None admin] RBAC: Authorizing `identity:get_user(user_id=9d6861ec911f47beb2e2d3c426655104)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:38.106977 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a2ed58-cffc-4ddf-9406-840bf518c2c9 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:38.107407 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972f5612-53f8-4994-82fa-ca2a7d960215 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:38.109058 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 550/1102] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:25:37 2022] GET /identity/v3/users/9d6861ec911f47beb2e2d3c426655104 => generated 285 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:25:38.112954 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 553/1103] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:25:37 2022] GET /identity/v3/role_assignments?user.id=5f28de22c2ee4c53b6d14d5b0ddf7fe9&scope.project.id=252d9673f08c479a98d04fee3f5856a4&role.id=6efe7f48768b4033a20e04bc93529e6d => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:25:38.119117 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972f5612-53f8-4994-82fa-ca2a7d960215 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:38.186157 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:38.227452 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972f5612-53f8-4994-82fa-ca2a7d960215 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:38.264931 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36350f2d-6ddd-4f22-8000-3fb70eef8a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:38.265780 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36350f2d-6ddd-4f22-8000-3fb70eef8a91 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:38.266099 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36350f2d-6ddd-4f22-8000-3fb70eef8a91 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:38.266376 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36350f2d-6ddd-4f22-8000-3fb70eef8a91 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:38.338907 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36350f2d-6ddd-4f22-8000-3fb70eef8a91 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:38.340379 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36350f2d-6ddd-4f22-8000-3fb70eef8a91 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:38.402208 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:38.441963 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 551/1104] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:25:38 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:25:38.449658 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a2ed58-cffc-4ddf-9406-840bf518c2c9 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:38.469950 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:38.559807 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a2ed58-cffc-4ddf-9406-840bf518c2c9 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:38.594634 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762ae748-3094-4ac6-9c7f-020ff13e4213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:38.595508 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762ae748-3094-4ac6-9c7f-020ff13e4213 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:38.595795 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762ae748-3094-4ac6-9c7f-020ff13e4213 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:38.596077 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762ae748-3094-4ac6-9c7f-020ff13e4213 None admin] PATH_INFO: `/v3/projects/b6532fb566c74a37ab1d52330b32f145` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:38.597546 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762ae748-3094-4ac6-9c7f-020ff13e4213 None admin] RBAC: Authorizing `identity:get_project(project_id=b6532fb566c74a37ab1d52330b32f145)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:38.599012 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762ae748-3094-4ac6-9c7f-020ff13e4213 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:38.600720 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 554/1105] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:25:38 2022] GET /identity/v3/projects/b6532fb566c74a37ab1d52330b32f145 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:25:38.607876 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36350f2d-6ddd-4f22-8000-3fb70eef8a91 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:38.717288 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36350f2d-6ddd-4f22-8000-3fb70eef8a91 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:38.754536 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10997a5e-8c13-4cae-99ed-2cc80e439e74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:38.755198 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10997a5e-8c13-4cae-99ed-2cc80e439e74 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:38.755482 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10997a5e-8c13-4cae-99ed-2cc80e439e74 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:38.755761 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10997a5e-8c13-4cae-99ed-2cc80e439e74 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:38.756603 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10997a5e-8c13-4cae-99ed-2cc80e439e74 None admin] RBAC: Adding query filter params (role.id=a3b793754e7b40b0bb21afe74eec3c18, scope.project.id=b6532fb566c74a37ab1d52330b32f145, user.id=9d6861ec911f47beb2e2d3c426655104) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:25:38.756918 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10997a5e-8c13-4cae-99ed-2cc80e439e74 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:38.758388 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10997a5e-8c13-4cae-99ed-2cc80e439e74 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:38.764810 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 552/1106] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:25:38 2022] GET /identity/v3/role_assignments?user.id=9d6861ec911f47beb2e2d3c426655104&scope.project.id=b6532fb566c74a37ab1d52330b32f145&role.id=a3b793754e7b40b0bb21afe74eec3c18 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:25:41.015750 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14edaf3-81e9-4fe0-87fa-dc17a9d43d5c None None] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:41.016248 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14edaf3-81e9-4fe0-87fa-dc17a9d43d5c None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:41.016248 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14edaf3-81e9-4fe0-87fa-dc17a9d43d5c None None] PATH_INFO: `/` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:41.017569 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 555/1107] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:25:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:25:41.022877 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab218032-de52-407f-8c77-da6aeb8c446e None None] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:41.023313 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab218032-de52-407f-8c77-da6aeb8c446e None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:41.023446 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab218032-de52-407f-8c77-da6aeb8c446e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:41.040956 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.auth.core [None req-ab218032-de52-407f-8c77-da6aeb8c446e None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143587) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:25:41.042194 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-ab218032-de52-407f-8c77-da6aeb8c446e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:25:41.062148 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:41.109075 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 553/1108] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:25:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:25:41.134228 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14edaf3-81e9-4fe0-87fa-dc17a9d43d5c None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:41.218105 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:41.235248 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14edaf3-81e9-4fe0-87fa-dc17a9d43d5c None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:41.269098 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93723ee-140f-4748-8bbf-262bb1e35e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:41.269825 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93723ee-140f-4748-8bbf-262bb1e35e3e None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:41.270137 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93723ee-140f-4748-8bbf-262bb1e35e3e None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:41.270418 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93723ee-140f-4748-8bbf-262bb1e35e3e None admin] PATH_INFO: `/v3/domains/default` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:41.271606 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93723ee-140f-4748-8bbf-262bb1e35e3e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:41.273040 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93723ee-140f-4748-8bbf-262bb1e35e3e None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:41.274880 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 556/1109] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:25:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:25:41.280546 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab218032-de52-407f-8c77-da6aeb8c446e None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:41.377226 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab218032-de52-407f-8c77-da6aeb8c446e None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:41.410296 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480fd3c2-307f-40a2-88b4-ea43aa52161b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:41.410951 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480fd3c2-307f-40a2-88b4-ea43aa52161b None admin] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:41.411229 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480fd3c2-307f-40a2-88b4-ea43aa52161b None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:41.411500 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480fd3c2-307f-40a2-88b4-ea43aa52161b None admin] PATH_INFO: `/v3/users` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:41.412208 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480fd3c2-307f-40a2-88b4-ea43aa52161b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:41.413638 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480fd3c2-307f-40a2-88b4-ea43aa52161b None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:41.434103 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 554/1110] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 19 14:25:41 2022] POST /identity/v3/users => generated 286 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:25:43.686061 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bff0a6-69fd-477c-bd74-be8cf22676ba None None] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:43.686523 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bff0a6-69fd-477c-bd74-be8cf22676ba None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:43.686635 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bff0a6-69fd-477c-bd74-be8cf22676ba None None] PATH_INFO: `/` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:43.687989 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 557/1111] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:25:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:25:43.693300 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8dbe22-616e-4d04-8f1c-cc6e0517746a None None] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:43.693679 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8dbe22-616e-4d04-8f1c-cc6e0517746a None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:43.693882 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8dbe22-616e-4d04-8f1c-cc6e0517746a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:43.710530 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.auth.core [None req-3f8dbe22-616e-4d04-8f1c-cc6e0517746a None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143587) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:25:43.712284 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-3f8dbe22-616e-4d04-8f1c-cc6e0517746a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:25:43.719419 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:43.782039 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 555/1112] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:25:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:25:43.806616 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bff0a6-69fd-477c-bd74-be8cf22676ba None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:43.905697 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bff0a6-69fd-477c-bd74-be8cf22676ba None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:43.937707 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25baf82-fe5f-4e93-8e8a-0e6f65c0f8a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:43.938504 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25baf82-fe5f-4e93-8e8a-0e6f65c0f8a4 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:43.938783 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25baf82-fe5f-4e93-8e8a-0e6f65c0f8a4 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:43.939051 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25baf82-fe5f-4e93-8e8a-0e6f65c0f8a4 None admin] PATH_INFO: `/v3/roles/b954b2c7ce764aa0afac78e075a011a6` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:43.940505 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25baf82-fe5f-4e93-8e8a-0e6f65c0f8a4 None admin] RBAC: Authorizing `identity:get_role(role_id=b954b2c7ce764aa0afac78e075a011a6)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:43.941866 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25baf82-fe5f-4e93-8e8a-0e6f65c0f8a4 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:43.943214 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 558/1113] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:25:43 2022] GET /identity/v3/roles/b954b2c7ce764aa0afac78e075a011a6 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:25:43.949700 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8dbe22-616e-4d04-8f1c-cc6e0517746a None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:44.047329 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8dbe22-616e-4d04-8f1c-cc6e0517746a None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:44.080063 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317d32a9-54e2-47e9-b5d1-8cd18abf54a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:44.080932 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317d32a9-54e2-47e9-b5d1-8cd18abf54a6 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:44.081210 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317d32a9-54e2-47e9-b5d1-8cd18abf54a6 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:44.081483 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317d32a9-54e2-47e9-b5d1-8cd18abf54a6 None admin] PATH_INFO: `/v3/users/d931209f11e143119db59340be3ecefe` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:44.092780 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:44.095196 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317d32a9-54e2-47e9-b5d1-8cd18abf54a6 None admin] RBAC: Authorizing `identity:get_user(user_id=d931209f11e143119db59340be3ecefe)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:44.096919 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317d32a9-54e2-47e9-b5d1-8cd18abf54a6 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:44.098834 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 556/1114] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:25:43 2022] GET /identity/v3/users/d931209f11e143119db59340be3ecefe => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:25:44.104616 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25baf82-fe5f-4e93-8e8a-0e6f65c0f8a4 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:44.126042 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:44.203507 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25baf82-fe5f-4e93-8e8a-0e6f65c0f8a4 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:44.237569 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1e091b-7c49-44b9-a73b-d1df7a40432c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:44.238240 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1e091b-7c49-44b9-a73b-d1df7a40432c None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:44.238526 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1e091b-7c49-44b9-a73b-d1df7a40432c None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:44.238804 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1e091b-7c49-44b9-a73b-d1df7a40432c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:44.304281 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1e091b-7c49-44b9-a73b-d1df7a40432c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:44.306508 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1e091b-7c49-44b9-a73b-d1df7a40432c None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:44.333815 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:44.412633 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 559/1115] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:25:44 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:25:44.419103 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317d32a9-54e2-47e9-b5d1-8cd18abf54a6 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:44.522361 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317d32a9-54e2-47e9-b5d1-8cd18abf54a6 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:44.558308 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c3e9b5-21d4-4cb7-b652-b8100d72a29a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:44.559177 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c3e9b5-21d4-4cb7-b652-b8100d72a29a None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:44.559463 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c3e9b5-21d4-4cb7-b652-b8100d72a29a None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:44.559747 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c3e9b5-21d4-4cb7-b652-b8100d72a29a None admin] PATH_INFO: `/v3/projects/b6532fb566c74a37ab1d52330b32f145` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:44.560958 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c3e9b5-21d4-4cb7-b652-b8100d72a29a None admin] RBAC: Authorizing `identity:get_project(project_id=b6532fb566c74a37ab1d52330b32f145)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:44.562448 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c3e9b5-21d4-4cb7-b652-b8100d72a29a None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:44.564171 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 557/1116] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:25:44 2022] GET /identity/v3/projects/b6532fb566c74a37ab1d52330b32f145 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:25:44.570434 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1e091b-7c49-44b9-a73b-d1df7a40432c None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:44.672450 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1e091b-7c49-44b9-a73b-d1df7a40432c None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:44.692838 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:44.707048 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f65360-4840-4c70-b74d-09d3eec3668c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:44.707702 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f65360-4840-4c70-b74d-09d3eec3668c None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:44.707985 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f65360-4840-4c70-b74d-09d3eec3668c None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:44.708260 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f65360-4840-4c70-b74d-09d3eec3668c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:44.709117 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f65360-4840-4c70-b74d-09d3eec3668c None admin] RBAC: Adding query filter params (role.id=b954b2c7ce764aa0afac78e075a011a6, scope.project.id=b6532fb566c74a37ab1d52330b32f145, user.id=d931209f11e143119db59340be3ecefe) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:25:44.709432 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f65360-4840-4c70-b74d-09d3eec3668c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:44.710917 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f65360-4840-4c70-b74d-09d3eec3668c None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:44.716478 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 560/1117] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:25:44 2022] GET /identity/v3/role_assignments?user.id=d931209f11e143119db59340be3ecefe&scope.project.id=b6532fb566c74a37ab1d52330b32f145&role.id=b954b2c7ce764aa0afac78e075a011a6 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:25:46.911143 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660def4a-bdde-436d-8727-ce74e679e5bd None None] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:46.911639 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660def4a-bdde-436d-8727-ce74e679e5bd None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:46.911639 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660def4a-bdde-436d-8727-ce74e679e5bd None None] PATH_INFO: `/` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:46.912980 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 558/1118] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:25:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:25:46.918294 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790b19b4-bf33-41dd-b0d4-e9163c4b186e None None] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:46.918693 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790b19b4-bf33-41dd-b0d4-e9163c4b186e None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:46.918854 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790b19b4-bf33-41dd-b0d4-e9163c4b186e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:46.935575 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.auth.core [None req-790b19b4-bf33-41dd-b0d4-e9163c4b186e None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143586) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:25:46.936754 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.fernet_utils [None req-790b19b4-bf33-41dd-b0d4-e9163c4b186e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143586) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:25:47.002490 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 561/1119] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:25:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:25:47.026295 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660def4a-bdde-436d-8727-ce74e679e5bd None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:47.092723 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:47.125299 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660def4a-bdde-436d-8727-ce74e679e5bd None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:47.158547 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fffcc5-a13a-4cf1-bd7c-a449d812a7da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:47.159389 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fffcc5-a13a-4cf1-bd7c-a449d812a7da None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:47.159666 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fffcc5-a13a-4cf1-bd7c-a449d812a7da None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:47.159934 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fffcc5-a13a-4cf1-bd7c-a449d812a7da None admin] PATH_INFO: `/v3/roles/b954b2c7ce764aa0afac78e075a011a6` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:47.161437 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fffcc5-a13a-4cf1-bd7c-a449d812a7da None admin] RBAC: Authorizing `identity:get_role(role_id=b954b2c7ce764aa0afac78e075a011a6)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:47.162775 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fffcc5-a13a-4cf1-bd7c-a449d812a7da None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:47.164141 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 559/1120] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:25:47 2022] GET /identity/v3/roles/b954b2c7ce764aa0afac78e075a011a6 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:25:47.170045 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790b19b4-bf33-41dd-b0d4-e9163c4b186e None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:47.267339 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790b19b4-bf33-41dd-b0d4-e9163c4b186e None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:47.280924 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:47.300712 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f974f480-2de8-4cbe-b631-f678490af28e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:47.301578 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f974f480-2de8-4cbe-b631-f678490af28e None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:47.301889 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f974f480-2de8-4cbe-b631-f678490af28e None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:47.302173 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f974f480-2de8-4cbe-b631-f678490af28e None admin] PATH_INFO: `/v3/users/d931209f11e143119db59340be3ecefe` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:47.303355 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f974f480-2de8-4cbe-b631-f678490af28e None admin] RBAC: Authorizing `identity:get_user(user_id=d931209f11e143119db59340be3ecefe)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:47.304783 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f974f480-2de8-4cbe-b631-f678490af28e None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:47.306958 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 562/1121] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:25:47 2022] GET /identity/v3/users/d931209f11e143119db59340be3ecefe => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:25:47.312684 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fffcc5-a13a-4cf1-bd7c-a449d812a7da None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:47.411373 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fffcc5-a13a-4cf1-bd7c-a449d812a7da None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:47.444582 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2170e9d-725a-4feb-8c1d-442922bdd390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:47.445370 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2170e9d-725a-4feb-8c1d-442922bdd390 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:47.445659 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2170e9d-725a-4feb-8c1d-442922bdd390 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:47.446092 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2170e9d-725a-4feb-8c1d-442922bdd390 None admin] PATH_INFO: `/v3/projects/b6532fb566c74a37ab1d52330b32f145` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:47.447368 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2170e9d-725a-4feb-8c1d-442922bdd390 None admin] RBAC: Authorizing `identity:get_project(project_id=b6532fb566c74a37ab1d52330b32f145)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:47.448836 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2170e9d-725a-4feb-8c1d-442922bdd390 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:47.450564 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 560/1122] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:25:47 2022] GET /identity/v3/projects/b6532fb566c74a37ab1d52330b32f145 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:25:47.456485 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f974f480-2de8-4cbe-b631-f678490af28e None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:47.554033 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f974f480-2de8-4cbe-b631-f678490af28e None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:47.587343 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c9211e-6af6-45d6-ab5b-443ee11a824d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:47.588202 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c9211e-6af6-45d6-ab5b-443ee11a824d None admin] REQUEST_METHOD: `PUT` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:47.588481 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c9211e-6af6-45d6-ab5b-443ee11a824d None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:47.588754 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c9211e-6af6-45d6-ab5b-443ee11a824d None admin] PATH_INFO: `/v3/projects/b6532fb566c74a37ab1d52330b32f145/users/d931209f11e143119db59340be3ecefe/roles/b954b2c7ce764aa0afac78e075a011a6` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:47.590964 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c9211e-6af6-45d6-ab5b-443ee11a824d None admin] RBAC: Authorizing `identity:create_grant(project_id=b6532fb566c74a37ab1d52330b32f145, user_id=d931209f11e143119db59340be3ecefe, role_id=b954b2c7ce764aa0afac78e075a011a6)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:47.592368 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c9211e-6af6-45d6-ab5b-443ee11a824d None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:47.604823 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 563/1123] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 14:25:47 2022] PUT /identity/v3/projects/b6532fb566c74a37ab1d52330b32f145/users/d931209f11e143119db59340be3ecefe/roles/b954b2c7ce764aa0afac78e075a011a6 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:25:49.802249 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa330ee-edd5-4fd3-a319-42e087a7e458 None None] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:49.802776 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa330ee-edd5-4fd3-a319-42e087a7e458 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:49.802873 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa330ee-edd5-4fd3-a319-42e087a7e458 None None] PATH_INFO: `/` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:49.804185 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 561/1124] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:25:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:25:49.809630 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c453f0-5369-49d1-aa0e-98daeeeb39b7 None None] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:49.810090 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c453f0-5369-49d1-aa0e-98daeeeb39b7 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:49.810216 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c453f0-5369-49d1-aa0e-98daeeeb39b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:49.828098 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.auth.core [None req-b1c453f0-5369-49d1-aa0e-98daeeeb39b7 None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143586) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:25:49.829269 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.fernet_utils [None req-b1c453f0-5369-49d1-aa0e-98daeeeb39b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143586) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:25:49.863448 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:49.895508 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 564/1125] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:25:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:25:49.919867 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa330ee-edd5-4fd3-a319-42e087a7e458 None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:49.923661 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:50.020197 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa330ee-edd5-4fd3-a319-42e087a7e458 None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:50.053586 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d095acd-c2ce-4807-a103-559363d91047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:50.054454 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d095acd-c2ce-4807-a103-559363d91047 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:50.054741 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d095acd-c2ce-4807-a103-559363d91047 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:50.055102 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d095acd-c2ce-4807-a103-559363d91047 None admin] PATH_INFO: `/v3/roles/b954b2c7ce764aa0afac78e075a011a6` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:50.056755 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d095acd-c2ce-4807-a103-559363d91047 None admin] RBAC: Authorizing `identity:get_role(role_id=b954b2c7ce764aa0afac78e075a011a6)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:50.058148 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d095acd-c2ce-4807-a103-559363d91047 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:50.059533 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 562/1126] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:25:49 2022] GET /identity/v3/roles/b954b2c7ce764aa0afac78e075a011a6 => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:25:50.065400 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c453f0-5369-49d1-aa0e-98daeeeb39b7 None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:50.162591 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c453f0-5369-49d1-aa0e-98daeeeb39b7 None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:50.195185 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8a61d0-868c-4c2d-807d-06b43d9d0cb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:50.197206 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8a61d0-868c-4c2d-807d-06b43d9d0cb9 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:50.197488 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8a61d0-868c-4c2d-807d-06b43d9d0cb9 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:50.197765 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8a61d0-868c-4c2d-807d-06b43d9d0cb9 None admin] PATH_INFO: `/v3/users/d931209f11e143119db59340be3ecefe` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:50.198988 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8a61d0-868c-4c2d-807d-06b43d9d0cb9 None admin] RBAC: Authorizing `identity:get_user(user_id=d931209f11e143119db59340be3ecefe)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:50.200420 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8a61d0-868c-4c2d-807d-06b43d9d0cb9 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:50.202058 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 565/1127] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:25:50 2022] GET /identity/v3/users/d931209f11e143119db59340be3ecefe => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:25:50.207733 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d095acd-c2ce-4807-a103-559363d91047 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:50.276637 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:50.306513 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d095acd-c2ce-4807-a103-559363d91047 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:50.349229 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059caeb5-09ea-478c-a8e0-39ec369669d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:50.349895 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059caeb5-09ea-478c-a8e0-39ec369669d9 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:50.350184 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059caeb5-09ea-478c-a8e0-39ec369669d9 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:50.350466 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059caeb5-09ea-478c-a8e0-39ec369669d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:50.423903 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059caeb5-09ea-478c-a8e0-39ec369669d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:50.425367 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059caeb5-09ea-478c-a8e0-39ec369669d9 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:50.491029 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:50.521630 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 563/1128] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:25:50 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:25:50.527741 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8a61d0-868c-4c2d-807d-06b43d9d0cb9 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:50.569083 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:50.630970 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8a61d0-868c-4c2d-807d-06b43d9d0cb9 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:50.663397 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20832b38-9ef1-4603-b437-2554e054263b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:50.664245 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20832b38-9ef1-4603-b437-2554e054263b None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:50.665164 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20832b38-9ef1-4603-b437-2554e054263b None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:50.665434 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20832b38-9ef1-4603-b437-2554e054263b None admin] PATH_INFO: `/v3/projects/b6532fb566c74a37ab1d52330b32f145` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:50.666647 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20832b38-9ef1-4603-b437-2554e054263b None admin] RBAC: Authorizing `identity:get_project(project_id=b6532fb566c74a37ab1d52330b32f145)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:50.668074 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20832b38-9ef1-4603-b437-2554e054263b None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:50.669749 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 566/1129] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:25:50 2022] GET /identity/v3/projects/b6532fb566c74a37ab1d52330b32f145 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:25:50.675889 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059caeb5-09ea-478c-a8e0-39ec369669d9 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:50.772281 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059caeb5-09ea-478c-a8e0-39ec369669d9 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:50.805855 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb2551c-0d1d-432e-b047-8821cb6ae2dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:50.806513 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb2551c-0d1d-432e-b047-8821cb6ae2dd None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:50.806787 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb2551c-0d1d-432e-b047-8821cb6ae2dd None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:50.807057 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb2551c-0d1d-432e-b047-8821cb6ae2dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:50.807881 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb2551c-0d1d-432e-b047-8821cb6ae2dd None admin] RBAC: Adding query filter params (role.id=b954b2c7ce764aa0afac78e075a011a6, scope.project.id=b6532fb566c74a37ab1d52330b32f145, user.id=d931209f11e143119db59340be3ecefe) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:25:50.808192 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb2551c-0d1d-432e-b047-8821cb6ae2dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:50.809622 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb2551c-0d1d-432e-b047-8821cb6ae2dd None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:50.815398 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 564/1130] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:25:50 2022] GET /identity/v3/role_assignments?user.id=d931209f11e143119db59340be3ecefe&scope.project.id=b6532fb566c74a37ab1d52330b32f145&role.id=b954b2c7ce764aa0afac78e075a011a6 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:25:53.053579 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f9e394-a140-473a-99ca-fb23877b31ad None None] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:53.054125 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f9e394-a140-473a-99ca-fb23877b31ad None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:53.054125 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f9e394-a140-473a-99ca-fb23877b31ad None None] PATH_INFO: `/` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:53.055660 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 567/1131] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:25:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:25:53.060870 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033e4892-92c9-449e-86df-1010c78cd77c None None] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:53.061281 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033e4892-92c9-449e-86df-1010c78cd77c None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:53.061423 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033e4892-92c9-449e-86df-1010c78cd77c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:53.078306 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.auth.core [None req-033e4892-92c9-449e-86df-1010c78cd77c None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143587) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:25:53.079489 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-033e4892-92c9-449e-86df-1010c78cd77c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:25:53.108122 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:53.147301 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 565/1132] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:25:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:25:53.171007 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f9e394-a140-473a-99ca-fb23877b31ad None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:53.270290 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f9e394-a140-473a-99ca-fb23877b31ad None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:53.279340 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:53.303214 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede91320-3830-43d5-b504-e5ffc92a1859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:53.304039 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede91320-3830-43d5-b504-e5ffc92a1859 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:53.304276 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede91320-3830-43d5-b504-e5ffc92a1859 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:53.304549 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede91320-3830-43d5-b504-e5ffc92a1859 None admin] PATH_INFO: `/v3/domains/default` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:53.306362 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede91320-3830-43d5-b504-e5ffc92a1859 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:53.307784 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede91320-3830-43d5-b504-e5ffc92a1859 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:53.309375 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 568/1133] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:25:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:25:53.315215 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033e4892-92c9-449e-86df-1010c78cd77c None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:53.413435 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033e4892-92c9-449e-86df-1010c78cd77c None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:53.447735 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431d79b2-1a64-4379-873f-49d0c34e969e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:53.448382 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431d79b2-1a64-4379-873f-49d0c34e969e None admin] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:53.448661 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431d79b2-1a64-4379-873f-49d0c34e969e None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:53.448929 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431d79b2-1a64-4379-873f-49d0c34e969e None admin] PATH_INFO: `/v3/projects` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:53.449631 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431d79b2-1a64-4379-873f-49d0c34e969e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:53.451090 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431d79b2-1a64-4379-873f-49d0c34e969e None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:53.475985 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 566/1134] 10.222.0.57 () {60 vars in 1141 bytes} [Thu May 19 14:25:53 2022] POST /identity/v3/projects => generated 311 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:25:55.733300 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b37b71-eb6c-4b93-ae87-14db9fba689c None None] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:55.733833 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b37b71-eb6c-4b93-ae87-14db9fba689c None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:55.733975 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b37b71-eb6c-4b93-ae87-14db9fba689c None None] PATH_INFO: `/` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:55.735276 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 569/1135] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:25:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:25:55.740584 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c195a1e-2ef6-41af-bc57-9592716756dc None None] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:55.740960 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c195a1e-2ef6-41af-bc57-9592716756dc None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:55.741164 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c195a1e-2ef6-41af-bc57-9592716756dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:55.758645 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.auth.core [None req-3c195a1e-2ef6-41af-bc57-9592716756dc None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143587) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:25:55.759826 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-3c195a1e-2ef6-41af-bc57-9592716756dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:25:55.772545 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:55.827444 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 567/1136] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:25:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:25:55.853990 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b37b71-eb6c-4b93-ae87-14db9fba689c None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:55.954782 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b37b71-eb6c-4b93-ae87-14db9fba689c None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:55.988768 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504be1dc-b962-42c3-8264-ec7aaece3796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:55.989551 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504be1dc-b962-42c3-8264-ec7aaece3796 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:55.989836 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504be1dc-b962-42c3-8264-ec7aaece3796 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:55.990138 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504be1dc-b962-42c3-8264-ec7aaece3796 None admin] PATH_INFO: `/v3/domains/default` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:55.991306 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504be1dc-b962-42c3-8264-ec7aaece3796 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:55.993390 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504be1dc-b962-42c3-8264-ec7aaece3796 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:55.995050 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 570/1137] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:25:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:25:56.000850 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c195a1e-2ef6-41af-bc57-9592716756dc None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:56.098910 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c195a1e-2ef6-41af-bc57-9592716756dc None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:56.132444 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04429bbc-d9dd-4940-b010-271426db50d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:56.133189 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04429bbc-d9dd-4940-b010-271426db50d5 None admin] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:56.133636 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04429bbc-d9dd-4940-b010-271426db50d5 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:56.133941 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04429bbc-d9dd-4940-b010-271426db50d5 None admin] PATH_INFO: `/v3/users` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:56.134639 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04429bbc-d9dd-4940-b010-271426db50d5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:56.136070 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04429bbc-d9dd-4940-b010-271426db50d5 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:56.156795 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 568/1138] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 19 14:25:55 2022] POST /identity/v3/users => generated 286 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:25:58.424769 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9681c398-b200-4e89-b468-f5d92d29e931 None None] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:58.425331 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9681c398-b200-4e89-b468-f5d92d29e931 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:58.425331 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9681c398-b200-4e89-b468-f5d92d29e931 None None] PATH_INFO: `/` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:58.426703 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 571/1139] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:25:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:25:58.432128 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381cc70d-9f00-4a69-bd82-8d5889cbf26b None None] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:58.432537 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381cc70d-9f00-4a69-bd82-8d5889cbf26b None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:58.432688 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381cc70d-9f00-4a69-bd82-8d5889cbf26b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:58.438661 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:58.450692 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.auth.core [None req-381cc70d-9f00-4a69-bd82-8d5889cbf26b None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143587) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:25:58.452015 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-381cc70d-9f00-4a69-bd82-8d5889cbf26b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:25:58.517701 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 569/1140] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:25:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:25:58.542049 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9681c398-b200-4e89-b468-f5d92d29e931 None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:58.588352 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:58.647681 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9681c398-b200-4e89-b468-f5d92d29e931 None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:58.685191 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce9e690-a5d6-4272-979c-0b3f345ce262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:58.685936 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce9e690-a5d6-4272-979c-0b3f345ce262 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:58.686220 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce9e690-a5d6-4272-979c-0b3f345ce262 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:58.686497 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce9e690-a5d6-4272-979c-0b3f345ce262 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:58.697230 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce9e690-a5d6-4272-979c-0b3f345ce262 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:25:58.704892 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-dce9e690-a5d6-4272-979c-0b3f345ce262 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:25:58.704892 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:25:58.704892 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:25:58.704892 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:25:58.704892 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:25:58.704892 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:25:58.704892 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:25:58.704892 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:25:58.704892 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:25:58.704892 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:25:58.704892 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:25:58.704892 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:25:58.704892 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:25:58.704892 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:25:58.704892 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:25:58.704892 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:25:58.706041 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:25:58.706041 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:25:58.706041 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:25:58.706041 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:25:58.706041 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:25:58.706041 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:25:58.706041 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:25:58.706041 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:25:58.706041 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:25:58.706041 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:25:58.706041 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:25:58.706041 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:25:58.706041 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:25:58.706041 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:25:58.706041 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:25:58.706041 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:25:58.707164 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:25:58.707164 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:25:58.707164 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:25:58.707164 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:25:58.707164 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:25:58.707164 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:25:58.707164 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:25:58.707164 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:25:58.707164 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:25:58.707164 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:25:58.707164 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:25:58.707164 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:25:58.707164 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 572/1141] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:25:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:25:58.712881 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381cc70d-9f00-4a69-bd82-8d5889cbf26b None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:58.821100 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381cc70d-9f00-4a69-bd82-8d5889cbf26b None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:58.855577 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2bfc42-2412-4840-a8b7-5d824aa8db44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:58.856375 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2bfc42-2412-4840-a8b7-5d824aa8db44 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:58.856648 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2bfc42-2412-4840-a8b7-5d824aa8db44 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:58.856925 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2bfc42-2412-4840-a8b7-5d824aa8db44 None admin] PATH_INFO: `/v3/roles` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:58.857675 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2bfc42-2412-4840-a8b7-5d824aa8db44 None admin] RBAC: Adding query filter params (name=admin) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:25:58.858016 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2bfc42-2412-4840-a8b7-5d824aa8db44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:58.859403 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2bfc42-2412-4840-a8b7-5d824aa8db44 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:58.860825 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:58.868558 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 570/1142] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:25:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:25:58.874445 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce9e690-a5d6-4272-979c-0b3f345ce262 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:58.975219 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce9e690-a5d6-4272-979c-0b3f345ce262 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:58.978271 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:59.009723 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a30487c-2f60-48bd-85db-c3ab05cbc93d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:59.010756 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a30487c-2f60-48bd-85db-c3ab05cbc93d None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:59.011042 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a30487c-2f60-48bd-85db-c3ab05cbc93d None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:59.011319 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a30487c-2f60-48bd-85db-c3ab05cbc93d None admin] PATH_INFO: `/v3/users/80968c17108047d4b8a74bfc1e773204` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:59.024863 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a30487c-2f60-48bd-85db-c3ab05cbc93d None admin] RBAC: Authorizing `identity:get_user(user_id=80968c17108047d4b8a74bfc1e773204)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:59.026347 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a30487c-2f60-48bd-85db-c3ab05cbc93d None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:59.028005 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 573/1143] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:25:58 2022] GET /identity/v3/users/80968c17108047d4b8a74bfc1e773204 => generated 286 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:25:59.033813 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2bfc42-2412-4840-a8b7-5d824aa8db44 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:59.138450 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2bfc42-2412-4840-a8b7-5d824aa8db44 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:59.173545 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f75220-fa86-48b5-8668-24b5f7b5aed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:59.174223 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f75220-fa86-48b5-8668-24b5f7b5aed3 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:59.174508 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f75220-fa86-48b5-8668-24b5f7b5aed3 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:59.174785 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f75220-fa86-48b5-8668-24b5f7b5aed3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:59.238464 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:59.246483 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f75220-fa86-48b5-8668-24b5f7b5aed3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:59.247986 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f75220-fa86-48b5-8668-24b5f7b5aed3 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:59.344660 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 571/1144] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:25:59 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:25:59.350930 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a30487c-2f60-48bd-85db-c3ab05cbc93d None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:59.449303 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a30487c-2f60-48bd-85db-c3ab05cbc93d None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:59.482468 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439d224f-5e71-442e-b71a-ba1600835def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:59.483316 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439d224f-5e71-442e-b71a-ba1600835def None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:59.483600 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439d224f-5e71-442e-b71a-ba1600835def None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:59.483875 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439d224f-5e71-442e-b71a-ba1600835def None admin] PATH_INFO: `/v3/projects/a476639a3ca94e1ab4846fd1530f75c9` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:59.485054 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439d224f-5e71-442e-b71a-ba1600835def None admin] RBAC: Authorizing `identity:get_project(project_id=a476639a3ca94e1ab4846fd1530f75c9)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:59.486514 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439d224f-5e71-442e-b71a-ba1600835def None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:59.488203 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 574/1145] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:25:59 2022] GET /identity/v3/projects/a476639a3ca94e1ab4846fd1530f75c9 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:25:59.495089 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f75220-fa86-48b5-8668-24b5f7b5aed3 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:59.587635 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:59.593725 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f75220-fa86-48b5-8668-24b5f7b5aed3 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:59.628896 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075bd6c3-2317-4372-b35a-f23819628de1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:59.629527 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075bd6c3-2317-4372-b35a-f23819628de1 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:59.629804 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075bd6c3-2317-4372-b35a-f23819628de1 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:59.630131 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075bd6c3-2317-4372-b35a-f23819628de1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:59.630914 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075bd6c3-2317-4372-b35a-f23819628de1 None admin] RBAC: Adding query filter params (role.id=a3b793754e7b40b0bb21afe74eec3c18, scope.project.id=a476639a3ca94e1ab4846fd1530f75c9, user.id=80968c17108047d4b8a74bfc1e773204) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:25:59.631220 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075bd6c3-2317-4372-b35a-f23819628de1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:59.632678 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075bd6c3-2317-4372-b35a-f23819628de1 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:59.638546 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 572/1146] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:25:59 2022] GET /identity/v3/role_assignments?user.id=80968c17108047d4b8a74bfc1e773204&scope.project.id=a476639a3ca94e1ab4846fd1530f75c9&role.id=a3b793754e7b40b0bb21afe74eec3c18 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:26:01.827966 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f525446-7950-40e5-a2aa-574fad44e8aa None None] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:26:01.828479 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f525446-7950-40e5-a2aa-574fad44e8aa None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:26:01.828585 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f525446-7950-40e5-a2aa-574fad44e8aa None None] PATH_INFO: `/` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:26:01.829888 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 575/1147] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:26:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:26:01.835021 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9963181e-0436-4fd4-940f-5b38c76a1fa8 None None] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:26:01.835448 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9963181e-0436-4fd4-940f-5b38c76a1fa8 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:26:01.835577 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9963181e-0436-4fd4-940f-5b38c76a1fa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:26:01.852974 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.auth.core [None req-9963181e-0436-4fd4-940f-5b38c76a1fa8 None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143587) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:26:01.854202 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-9963181e-0436-4fd4-940f-5b38c76a1fa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:26:01.918974 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 573/1148] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:26:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:26:01.941745 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f525446-7950-40e5-a2aa-574fad44e8aa None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:26:01.973227 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:26:02.044332 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f525446-7950-40e5-a2aa-574fad44e8aa None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:26:02.077156 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05516ae-1153-4fc9-af0c-46a7f7bc7722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:26:02.077967 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05516ae-1153-4fc9-af0c-46a7f7bc7722 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:26:02.078253 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05516ae-1153-4fc9-af0c-46a7f7bc7722 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:26:02.078530 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05516ae-1153-4fc9-af0c-46a7f7bc7722 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:26:02.089124 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05516ae-1153-4fc9-af0c-46a7f7bc7722 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:26:02.096912 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-d05516ae-1153-4fc9-af0c-46a7f7bc7722 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:26:02.096912 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:26:02.096912 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:26:02.096912 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:26:02.096912 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:26:02.096912 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:26:02.096912 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:26:02.096912 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:26:02.096912 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:26:02.096912 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:26:02.096912 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:26:02.096912 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:26:02.096912 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:26:02.096912 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:26:02.096912 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:26:02.096912 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:26:02.097958 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:26:02.097958 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:26:02.097958 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:26:02.097958 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:26:02.097958 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:26:02.097958 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:26:02.097958 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:26:02.097958 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:26:02.097958 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:26:02.097958 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:26:02.097958 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:26:02.097958 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:26:02.097958 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:26:02.097958 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:26:02.097958 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:26:02.097958 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:26:02.098779 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:26:02.098779 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:26:02.098779 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:26:02.098779 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:26:02.098779 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:26:02.098779 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:26:02.098779 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:26:02.098779 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:26:02.098779 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:26:02.098779 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:26:02.098779 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:26:02.098779 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:26:02.098779 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 576/1149] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:26:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:26:02.104924 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9963181e-0436-4fd4-940f-5b38c76a1fa8 None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:26:02.193409 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:26:02.203428 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9963181e-0436-4fd4-940f-5b38c76a1fa8 None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:26:02.237418 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a0401c-6b2a-41d4-ab20-9abfd5efabad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:26:02.238102 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a0401c-6b2a-41d4-ab20-9abfd5efabad None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:26:02.238385 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a0401c-6b2a-41d4-ab20-9abfd5efabad None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:26:02.238658 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a0401c-6b2a-41d4-ab20-9abfd5efabad None admin] PATH_INFO: `/v3/roles` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:26:02.239394 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a0401c-6b2a-41d4-ab20-9abfd5efabad None admin] RBAC: Adding query filter params (name=admin) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:26:02.239696 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a0401c-6b2a-41d4-ab20-9abfd5efabad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:26:02.241061 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a0401c-6b2a-41d4-ab20-9abfd5efabad None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:26:02.249003 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 574/1150] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:26:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:26:02.254929 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05516ae-1153-4fc9-af0c-46a7f7bc7722 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:26:02.344389 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:26:02.355668 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05516ae-1153-4fc9-af0c-46a7f7bc7722 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:26:02.390169 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee80064-644f-4f31-991d-78ebd18eff59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:26:02.391126 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee80064-644f-4f31-991d-78ebd18eff59 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:26:02.391414 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee80064-644f-4f31-991d-78ebd18eff59 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:26:02.391691 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee80064-644f-4f31-991d-78ebd18eff59 None admin] PATH_INFO: `/v3/users/80968c17108047d4b8a74bfc1e773204` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:26:02.392896 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee80064-644f-4f31-991d-78ebd18eff59 None admin] RBAC: Authorizing `identity:get_user(user_id=80968c17108047d4b8a74bfc1e773204)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:26:02.394373 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee80064-644f-4f31-991d-78ebd18eff59 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:26:02.396526 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 577/1151] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:26:02 2022] GET /identity/v3/users/80968c17108047d4b8a74bfc1e773204 => generated 286 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:26:02.402832 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a0401c-6b2a-41d4-ab20-9abfd5efabad None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:26:02.502870 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a0401c-6b2a-41d4-ab20-9abfd5efabad None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:26:02.535684 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd00dea-5f2c-4ad4-a44f-deafac38b37a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:26:02.536522 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd00dea-5f2c-4ad4-a44f-deafac38b37a None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:26:02.536803 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd00dea-5f2c-4ad4-a44f-deafac38b37a None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:26:02.537076 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd00dea-5f2c-4ad4-a44f-deafac38b37a None admin] PATH_INFO: `/v3/projects/a476639a3ca94e1ab4846fd1530f75c9` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:26:02.538265 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd00dea-5f2c-4ad4-a44f-deafac38b37a None admin] RBAC: Authorizing `identity:get_project(project_id=a476639a3ca94e1ab4846fd1530f75c9)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:26:02.540523 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd00dea-5f2c-4ad4-a44f-deafac38b37a None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:26:02.542218 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 575/1152] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:26:02 2022] GET /identity/v3/projects/a476639a3ca94e1ab4846fd1530f75c9 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:26:02.548033 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee80064-644f-4f31-991d-78ebd18eff59 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:26:02.644765 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee80064-644f-4f31-991d-78ebd18eff59 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:26:02.677779 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc79321-badb-4c49-9589-947e345b2ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:26:02.678678 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc79321-badb-4c49-9589-947e345b2ed3 None admin] REQUEST_METHOD: `PUT` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:26:02.678954 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc79321-badb-4c49-9589-947e345b2ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:26:02.679224 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc79321-badb-4c49-9589-947e345b2ed3 None admin] PATH_INFO: `/v3/projects/a476639a3ca94e1ab4846fd1530f75c9/users/80968c17108047d4b8a74bfc1e773204/roles/a3b793754e7b40b0bb21afe74eec3c18` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:26:02.682533 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc79321-badb-4c49-9589-947e345b2ed3 None admin] RBAC: Authorizing `identity:create_grant(project_id=a476639a3ca94e1ab4846fd1530f75c9, user_id=80968c17108047d4b8a74bfc1e773204, role_id=a3b793754e7b40b0bb21afe74eec3c18)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:26:02.683950 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc79321-badb-4c49-9589-947e345b2ed3 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:26:02.696873 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 578/1153] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 14:26:02 2022] PUT /identity/v3/projects/a476639a3ca94e1ab4846fd1530f75c9/users/80968c17108047d4b8a74bfc1e773204/roles/a3b793754e7b40b0bb21afe74eec3c18 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 14:26:04.893528 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a38ff7-1c1f-405b-9c0f-58fe024639cb None None] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:26:04.894138 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a38ff7-1c1f-405b-9c0f-58fe024639cb None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:26:04.894244 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a38ff7-1c1f-405b-9c0f-58fe024639cb None None] PATH_INFO: `/` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:26:04.895619 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 576/1154] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:26:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:26:04.901012 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ec4f17-d9da-44a7-bf1d-d7d8e29d7008 None None] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:26:04.901448 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ec4f17-d9da-44a7-bf1d-d7d8e29d7008 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:26:04.901584 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ec4f17-d9da-44a7-bf1d-d7d8e29d7008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:26:04.918924 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.auth.core [None req-a7ec4f17-d9da-44a7-bf1d-d7d8e29d7008 None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143586) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:26:04.920047 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.fernet_utils [None req-a7ec4f17-d9da-44a7-bf1d-d7d8e29d7008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143586) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:26:04.934329 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:26:04.989704 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 579/1155] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:26:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:26:05.014510 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a38ff7-1c1f-405b-9c0f-58fe024639cb None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:26:05.036239 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:26:05.116374 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a38ff7-1c1f-405b-9c0f-58fe024639cb None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:26:05.149696 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026e673c-4aa1-4823-948d-b488bc013956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:26:05.150499 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026e673c-4aa1-4823-948d-b488bc013956 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:26:05.150783 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026e673c-4aa1-4823-948d-b488bc013956 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:26:05.151058 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026e673c-4aa1-4823-948d-b488bc013956 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:26:05.161337 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026e673c-4aa1-4823-948d-b488bc013956 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:26:05.169603 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-026e673c-4aa1-4823-948d-b488bc013956 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:26:05.169603 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:26:05.169603 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:26:05.169603 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:26:05.169603 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:26:05.169603 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:26:05.169603 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:26:05.169603 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:26:05.169603 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:26:05.169603 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:26:05.169603 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:26:05.169603 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:26:05.169603 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:26:05.169603 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:26:05.169603 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:26:05.169603 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:26:05.170672 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:26:05.170672 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:26:05.170672 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:26:05.170672 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:26:05.170672 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:26:05.170672 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:26:05.170672 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:26:05.170672 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:26:05.170672 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:26:05.170672 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:26:05.170672 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:26:05.170672 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:26:05.170672 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:26:05.170672 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:26:05.170672 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:26:05.170672 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:26:05.171605 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:26:05.171605 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:26:05.171605 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:26:05.171605 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:26:05.171605 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:26:05.171605 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:26:05.171605 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:26:05.171605 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:26:05.171605 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:26:05.171605 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:26:05.171605 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:26:05.171605 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:26:05.171605 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 577/1156] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:26:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:26:05.176763 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ec4f17-d9da-44a7-bf1d-d7d8e29d7008 None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:26:05.272717 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ec4f17-d9da-44a7-bf1d-d7d8e29d7008 None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:26:05.306691 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a713271f-e409-4c3d-b1b5-7b1cbbeaa7f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:26:05.307348 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a713271f-e409-4c3d-b1b5-7b1cbbeaa7f1 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:26:05.307630 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a713271f-e409-4c3d-b1b5-7b1cbbeaa7f1 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:26:05.307904 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a713271f-e409-4c3d-b1b5-7b1cbbeaa7f1 None admin] PATH_INFO: `/v3/roles` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:26:05.308649 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a713271f-e409-4c3d-b1b5-7b1cbbeaa7f1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:26:05.308956 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a713271f-e409-4c3d-b1b5-7b1cbbeaa7f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:26:05.310340 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a713271f-e409-4c3d-b1b5-7b1cbbeaa7f1 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:26:05.318229 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 580/1157] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:26:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:26:05.324175 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026e673c-4aa1-4823-948d-b488bc013956 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:26:05.406348 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:26:05.426339 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026e673c-4aa1-4823-948d-b488bc013956 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:26:05.459964 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9d4b88-ee1a-4e30-9f15-bffedd6ea551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:26:05.460864 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9d4b88-ee1a-4e30-9f15-bffedd6ea551 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:26:05.461142 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9d4b88-ee1a-4e30-9f15-bffedd6ea551 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:26:05.461417 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9d4b88-ee1a-4e30-9f15-bffedd6ea551 None admin] PATH_INFO: `/v3/users/80968c17108047d4b8a74bfc1e773204` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:26:05.462628 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9d4b88-ee1a-4e30-9f15-bffedd6ea551 None admin] RBAC: Authorizing `identity:get_user(user_id=80968c17108047d4b8a74bfc1e773204)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:26:05.464070 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9d4b88-ee1a-4e30-9f15-bffedd6ea551 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:26:05.465697 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 578/1158] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:26:05 2022] GET /identity/v3/users/80968c17108047d4b8a74bfc1e773204 => generated 286 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:26:05.471724 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a713271f-e409-4c3d-b1b5-7b1cbbeaa7f1 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:26:05.484319 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:26:05.570669 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a713271f-e409-4c3d-b1b5-7b1cbbeaa7f1 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:26:05.604090 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bfbbd3-fe90-495f-b7e2-e8834fe11bd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:26:05.604753 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bfbbd3-fe90-495f-b7e2-e8834fe11bd3 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:26:05.605039 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bfbbd3-fe90-495f-b7e2-e8834fe11bd3 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:26:05.605318 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bfbbd3-fe90-495f-b7e2-e8834fe11bd3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:26:05.673142 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bfbbd3-fe90-495f-b7e2-e8834fe11bd3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:26:05.674628 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bfbbd3-fe90-495f-b7e2-e8834fe11bd3 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:26:05.686264 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:26:05.771987 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 581/1159] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:26:05 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:26:05.778216 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9d4b88-ee1a-4e30-9f15-bffedd6ea551 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:26:05.876156 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9d4b88-ee1a-4e30-9f15-bffedd6ea551 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:26:05.909016 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941287cd-24a0-4048-992f-68e09bf3ed51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:26:05.909882 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941287cd-24a0-4048-992f-68e09bf3ed51 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:26:05.910166 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941287cd-24a0-4048-992f-68e09bf3ed51 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:26:05.910444 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941287cd-24a0-4048-992f-68e09bf3ed51 None admin] PATH_INFO: `/v3/projects/a476639a3ca94e1ab4846fd1530f75c9` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:26:05.911616 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941287cd-24a0-4048-992f-68e09bf3ed51 None admin] RBAC: Authorizing `identity:get_project(project_id=a476639a3ca94e1ab4846fd1530f75c9)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:26:05.913057 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941287cd-24a0-4048-992f-68e09bf3ed51 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:26:05.915576 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 579/1160] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:26:05 2022] GET /identity/v3/projects/a476639a3ca94e1ab4846fd1530f75c9 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:26:05.921594 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bfbbd3-fe90-495f-b7e2-e8834fe11bd3 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:26:06.031380 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bfbbd3-fe90-495f-b7e2-e8834fe11bd3 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:26:06.043018 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:26:06.066472 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc2a693-2e5e-4c89-8bc6-4cd6c2c2dab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:26:06.067125 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc2a693-2e5e-4c89-8bc6-4cd6c2c2dab8 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:26:06.067412 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc2a693-2e5e-4c89-8bc6-4cd6c2c2dab8 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:26:06.067692 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc2a693-2e5e-4c89-8bc6-4cd6c2c2dab8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:26:06.068523 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc2a693-2e5e-4c89-8bc6-4cd6c2c2dab8 None admin] RBAC: Adding query filter params (role.id=a3b793754e7b40b0bb21afe74eec3c18, scope.project.id=a476639a3ca94e1ab4846fd1530f75c9, user.id=80968c17108047d4b8a74bfc1e773204) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:26:06.068840 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc2a693-2e5e-4c89-8bc6-4cd6c2c2dab8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:26:06.070995 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc2a693-2e5e-4c89-8bc6-4cd6c2c2dab8 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:26:06.076657 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 582/1161] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:26:05 2022] GET /identity/v3/role_assignments?user.id=80968c17108047d4b8a74bfc1e773204&scope.project.id=a476639a3ca94e1ab4846fd1530f75c9&role.id=a3b793754e7b40b0bb21afe74eec3c18 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:26:08.442761 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceadad9-0f4c-45ea-8542-6d78acc4cd88 None None] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:26:08.443290 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceadad9-0f4c-45ea-8542-6d78acc4cd88 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:26:08.443387 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceadad9-0f4c-45ea-8542-6d78acc4cd88 None None] PATH_INFO: `/` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:26:08.444722 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 580/1162] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:26:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:26:08.450223 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62335296-fd4e-41c7-bb58-29f1126f095b None None] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:26:08.450640 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62335296-fd4e-41c7-bb58-29f1126f095b None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:26:08.450890 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62335296-fd4e-41c7-bb58-29f1126f095b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:26:08.468218 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.auth.core [None req-62335296-fd4e-41c7-bb58-29f1126f095b None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143586) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:26:08.469406 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.fernet_utils [None req-62335296-fd4e-41c7-bb58-29f1126f095b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143586) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:26:08.535129 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 583/1163] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:26:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:26:08.558021 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceadad9-0f4c-45ea-8542-6d78acc4cd88 None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:26:08.576415 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:26:08.658794 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceadad9-0f4c-45ea-8542-6d78acc4cd88 None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:26:08.692250 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65acea7e-d58b-47b6-8cb0-af92c517a7d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:26:08.692938 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65acea7e-d58b-47b6-8cb0-af92c517a7d4 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:26:08.693169 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65acea7e-d58b-47b6-8cb0-af92c517a7d4 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:26:08.693443 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65acea7e-d58b-47b6-8cb0-af92c517a7d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:26:08.759653 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65acea7e-d58b-47b6-8cb0-af92c517a7d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:26:08.761091 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65acea7e-d58b-47b6-8cb0-af92c517a7d4 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:26:08.777656 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:26:08.857405 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 581/1164] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:26:08 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:26:08.863488 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62335296-fd4e-41c7-bb58-29f1126f095b None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:26:08.960134 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62335296-fd4e-41c7-bb58-29f1126f095b None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:26:08.962570 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:26:08.996472 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10ffc74-0f06-4ef6-a2c8-d763e9f87fc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:26:08.997267 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10ffc74-0f06-4ef6-a2c8-d763e9f87fc7 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:26:08.997548 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10ffc74-0f06-4ef6-a2c8-d763e9f87fc7 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:26:08.997821 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10ffc74-0f06-4ef6-a2c8-d763e9f87fc7 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:26:09.004381 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10ffc74-0f06-4ef6-a2c8-d763e9f87fc7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:26:09.005817 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10ffc74-0f06-4ef6-a2c8-d763e9f87fc7 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:26:09.020720 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-f10ffc74-0f06-4ef6-a2c8-d763e9f87fc7 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. May 19 14:26:09.020720 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:26:09.020720 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 19 14:26:09.020720 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 19 14:26:09.020720 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:26:09.020720 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:26:09.020720 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:26:09.020720 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:26:09.020720 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. May 19 14:26:09.020720 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:26:09.020720 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:26:09.020720 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:26:09.020720 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:26:09.020720 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:26:09.020720 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:26:09.020720 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:26:09.020720 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:26:09.021936 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:26:09.021936 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:26:09.021936 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:26:09.021936 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:26:09.021936 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:26:09.021936 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:26:09.021936 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:26:09.021936 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:26:09.021936 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:26:09.021936 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:26:09.021936 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:26:09.021936 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:26:09.021936 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:26:09.021936 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:26:09.021936 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:26:09.021936 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:26:09.022994 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:26:09.022994 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:26:09.022994 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:26:09.022994 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:26:09.022994 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:26:09.022994 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:26:09.022994 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:26:09.022994 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:26:09.022994 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:26:09.022994 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:26:09.022994 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 19 14:26:09.022994 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:26:09.022994 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. May 19 14:26:09.022994 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:26:09.022994 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 584/1165] 10.222.0.57 () {56 vars in 1117 bytes} [Thu May 19 14:26:08 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:26:09.028132 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65acea7e-d58b-47b6-8cb0-af92c517a7d4 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:26:09.126833 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65acea7e-d58b-47b6-8cb0-af92c517a7d4 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:26:09.143394 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:26:09.161356 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfee631-14a5-4ed2-8132-126537c6b098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:26:09.162048 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfee631-14a5-4ed2-8132-126537c6b098 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:26:09.162342 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfee631-14a5-4ed2-8132-126537c6b098 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:26:09.162622 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfee631-14a5-4ed2-8132-126537c6b098 None admin] PATH_INFO: `/v3/domains` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:26:09.163338 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfee631-14a5-4ed2-8132-126537c6b098 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:26:09.163734 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfee631-14a5-4ed2-8132-126537c6b098 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:26:09.165121 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfee631-14a5-4ed2-8132-126537c6b098 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:26:09.172198 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 582/1166] 10.222.0.57 () {56 vars in 1115 bytes} [Thu May 19 14:26:09 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:26:11.366940 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581a2164-81c5-4bc6-bdfd-95455bb87ad5 None None] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:26:11.367496 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581a2164-81c5-4bc6-bdfd-95455bb87ad5 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:26:11.367601 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581a2164-81c5-4bc6-bdfd-95455bb87ad5 None None] PATH_INFO: `/` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:26:11.368918 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 585/1167] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:26:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:26:11.374269 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3823ee75-1c33-4bb9-9f4c-4f1745386aae None None] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:26:11.374689 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3823ee75-1c33-4bb9-9f4c-4f1745386aae None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:26:11.374828 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3823ee75-1c33-4bb9-9f4c-4f1745386aae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:26:11.391681 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.auth.core [None req-3823ee75-1c33-4bb9-9f4c-4f1745386aae None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143587) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:26:11.392870 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-3823ee75-1c33-4bb9-9f4c-4f1745386aae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:26:11.458252 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 583/1168] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:26:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:26:11.482237 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581a2164-81c5-4bc6-bdfd-95455bb87ad5 None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:26:11.581980 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581a2164-81c5-4bc6-bdfd-95455bb87ad5 None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:26:11.614647 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6470c485-878b-4b98-896c-a78094b4e5d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:26:11.615297 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6470c485-878b-4b98-896c-a78094b4e5d3 None admin] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:26:11.615579 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6470c485-878b-4b98-896c-a78094b4e5d3 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:26:11.615855 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6470c485-878b-4b98-896c-a78094b4e5d3 None admin] PATH_INFO: `/v3/domains` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:26:11.617126 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6470c485-878b-4b98-896c-a78094b4e5d3 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:26:11.618084 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6470c485-878b-4b98-896c-a78094b4e5d3 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:26:11.632717 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 586/1169] 10.222.0.57 () {60 vars in 1138 bytes} [Thu May 19 14:26:11 2022] POST /identity/v3/domains => generated 267 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:26:13.877606 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc71e77-4fee-40ca-b76d-9764b207ed68 None None] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:26:13.878197 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc71e77-4fee-40ca-b76d-9764b207ed68 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:26:13.878197 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc71e77-4fee-40ca-b76d-9764b207ed68 None None] PATH_INFO: `/` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:26:13.879584 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 584/1170] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:26:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:26:13.885041 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4a4661-5db8-47fa-90b0-bb6ad502c28d None None] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:26:13.885448 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4a4661-5db8-47fa-90b0-bb6ad502c28d None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:26:13.885601 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4a4661-5db8-47fa-90b0-bb6ad502c28d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:26:13.901997 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.auth.core [None req-ca4a4661-5db8-47fa-90b0-bb6ad502c28d None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143586) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:26:13.903160 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.fernet_utils [None req-ca4a4661-5db8-47fa-90b0-bb6ad502c28d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143586) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:26:13.920252 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:26:13.967620 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 587/1171] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:26:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:26:13.992119 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc71e77-4fee-40ca-b76d-9764b207ed68 None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:26:14.090540 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc71e77-4fee-40ca-b76d-9764b207ed68 None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:26:14.102554 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:26:14.126777 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25f56ba-6cde-4946-9170-5dbd36b2e188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:26:14.127633 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25f56ba-6cde-4946-9170-5dbd36b2e188 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:26:14.127916 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25f56ba-6cde-4946-9170-5dbd36b2e188 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:26:14.128189 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25f56ba-6cde-4946-9170-5dbd36b2e188 None admin] PATH_INFO: `/v3/domains/be48aee21e0e44ccab60b90780a8c5f0` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:26:14.138683 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25f56ba-6cde-4946-9170-5dbd36b2e188 None admin] RBAC: Authorizing `identity:get_domain(domain_id=be48aee21e0e44ccab60b90780a8c5f0)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:26:14.140138 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25f56ba-6cde-4946-9170-5dbd36b2e188 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:26:14.141776 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 585/1172] 10.222.0.57 () {56 vars in 1183 bytes} [Thu May 19 14:26:13 2022] GET /identity/v3/domains/be48aee21e0e44ccab60b90780a8c5f0 => generated 267 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:26:14.148091 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4a4661-5db8-47fa-90b0-bb6ad502c28d None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:26:14.244282 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4a4661-5db8-47fa-90b0-bb6ad502c28d None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:26:14.276941 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81711247-1666-4308-a823-95dc4d98872c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:26:14.277587 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81711247-1666-4308-a823-95dc4d98872c None admin] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:26:14.277888 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81711247-1666-4308-a823-95dc4d98872c None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:26:14.278164 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81711247-1666-4308-a823-95dc4d98872c None admin] PATH_INFO: `/v3/projects` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:26:14.278873 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81711247-1666-4308-a823-95dc4d98872c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:26:14.280276 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81711247-1666-4308-a823-95dc4d98872c None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:26:14.304939 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 588/1173] 10.222.0.57 () {60 vars in 1141 bytes} [Thu May 19 14:26:14 2022] POST /identity/v3/projects => generated 361 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:26:16.559999 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b614e3-c320-496f-a0fa-3f4ce0fa4859 None None] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:26:16.560549 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b614e3-c320-496f-a0fa-3f4ce0fa4859 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:26:16.560549 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b614e3-c320-496f-a0fa-3f4ce0fa4859 None None] PATH_INFO: `/` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:26:16.562071 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 586/1174] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:26:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:26:16.567303 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce2fbe9-0fd9-4780-9c1b-ffe259053881 None None] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:26:16.567739 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce2fbe9-0fd9-4780-9c1b-ffe259053881 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:26:16.567864 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce2fbe9-0fd9-4780-9c1b-ffe259053881 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:26:16.580072 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:26:16.585708 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.auth.core [None req-3ce2fbe9-0fd9-4780-9c1b-ffe259053881 None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143586) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:26:16.587390 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.fernet_utils [None req-3ce2fbe9-0fd9-4780-9c1b-ffe259053881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143586) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:26:16.653998 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 589/1175] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:26:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:26:16.680353 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b614e3-c320-496f-a0fa-3f4ce0fa4859 None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:26:16.781042 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b614e3-c320-496f-a0fa-3f4ce0fa4859 None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:26:16.814228 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a17c2f7-bd30-4f96-be2e-fe3b3cce47df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:26:16.815014 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a17c2f7-bd30-4f96-be2e-fe3b3cce47df None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:26:16.815297 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a17c2f7-bd30-4f96-be2e-fe3b3cce47df None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:26:16.815573 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a17c2f7-bd30-4f96-be2e-fe3b3cce47df None admin] PATH_INFO: `/v3/domains/be48aee21e0e44ccab60b90780a8c5f0` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:26:16.816757 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a17c2f7-bd30-4f96-be2e-fe3b3cce47df None admin] RBAC: Authorizing `identity:get_domain(domain_id=be48aee21e0e44ccab60b90780a8c5f0)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:26:16.818289 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a17c2f7-bd30-4f96-be2e-fe3b3cce47df None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:26:16.820537 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 587/1176] 10.222.0.57 () {56 vars in 1183 bytes} [Thu May 19 14:26:16 2022] GET /identity/v3/domains/be48aee21e0e44ccab60b90780a8c5f0 => generated 267 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:26:16.826594 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce2fbe9-0fd9-4780-9c1b-ffe259053881 None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:26:16.924029 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce2fbe9-0fd9-4780-9c1b-ffe259053881 None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:26:16.956848 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8aafeb-c93c-4f63-8f6d-86dba78aa9b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:26:16.957508 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8aafeb-c93c-4f63-8f6d-86dba78aa9b8 None admin] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:26:16.957791 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8aafeb-c93c-4f63-8f6d-86dba78aa9b8 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:26:16.958097 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8aafeb-c93c-4f63-8f6d-86dba78aa9b8 None admin] PATH_INFO: `/v3/users` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:26:16.958812 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8aafeb-c93c-4f63-8f6d-86dba78aa9b8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:26:16.960343 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8aafeb-c93c-4f63-8f6d-86dba78aa9b8 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:26:16.967798 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:26:16.981548 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 590/1177] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 19 14:26:16 2022] POST /identity/v3/users => generated 311 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:26:19.240414 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df93a24-8890-4344-96d6-838ed69b178c None None] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:26:19.240964 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df93a24-8890-4344-96d6-838ed69b178c None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:26:19.240964 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df93a24-8890-4344-96d6-838ed69b178c None None] PATH_INFO: `/` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:26:19.243024 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 588/1178] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:26:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:26:19.248475 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda0f223-f9aa-4ec3-81cb-02ac07e3306f None None] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:26:19.248880 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda0f223-f9aa-4ec3-81cb-02ac07e3306f None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:26:19.249040 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda0f223-f9aa-4ec3-81cb-02ac07e3306f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:26:19.266320 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.auth.core [None req-dda0f223-f9aa-4ec3-81cb-02ac07e3306f None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143586) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:26:19.267497 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.fernet_utils [None req-dda0f223-f9aa-4ec3-81cb-02ac07e3306f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143586) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:26:19.333120 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 591/1179] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:26:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:26:19.356980 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df93a24-8890-4344-96d6-838ed69b178c None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:26:19.398411 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:26:19.457321 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df93a24-8890-4344-96d6-838ed69b178c None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:26:19.491126 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b863f00-6a6a-409a-980a-d6f0cb3f699e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:26:19.491877 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b863f00-6a6a-409a-980a-d6f0cb3f699e None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:26:19.492160 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b863f00-6a6a-409a-980a-d6f0cb3f699e None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:26:19.492437 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b863f00-6a6a-409a-980a-d6f0cb3f699e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:26:19.503247 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b863f00-6a6a-409a-980a-d6f0cb3f699e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:26:19.510885 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-5b863f00-6a6a-409a-980a-d6f0cb3f699e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:26:19.510885 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:26:19.510885 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:26:19.510885 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:26:19.510885 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:26:19.510885 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:26:19.510885 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:26:19.510885 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:26:19.510885 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:26:19.510885 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:26:19.510885 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:26:19.510885 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:26:19.510885 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:26:19.510885 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:26:19.510885 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:26:19.510885 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:26:19.511923 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:26:19.511923 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:26:19.511923 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:26:19.511923 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:26:19.511923 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:26:19.511923 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:26:19.511923 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:26:19.511923 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:26:19.511923 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:26:19.511923 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:26:19.511923 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:26:19.511923 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:26:19.511923 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:26:19.511923 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:26:19.511923 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:26:19.511923 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:26:19.517869 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:26:19.517869 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:26:19.517869 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:26:19.517869 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:26:19.517869 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:26:19.517869 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:26:19.517869 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:26:19.517869 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:26:19.517869 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:26:19.517869 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:26:19.517869 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:26:19.517869 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:26:19.517869 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 589/1180] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:26:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:26:19.518674 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda0f223-f9aa-4ec3-81cb-02ac07e3306f None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:26:19.614380 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda0f223-f9aa-4ec3-81cb-02ac07e3306f None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:26:19.635972 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:26:19.648378 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fc17d6-c460-417f-865b-cc77caddb797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:26:19.649028 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fc17d6-c460-417f-865b-cc77caddb797 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:26:19.649308 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fc17d6-c460-417f-865b-cc77caddb797 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:26:19.649579 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fc17d6-c460-417f-865b-cc77caddb797 None admin] PATH_INFO: `/v3/roles` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:26:19.650347 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fc17d6-c460-417f-865b-cc77caddb797 None admin] RBAC: Adding query filter params (name=admin) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:26:19.650653 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fc17d6-c460-417f-865b-cc77caddb797 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:26:19.652045 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fc17d6-c460-417f-865b-cc77caddb797 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:26:19.661546 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 592/1181] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:26:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:26:19.668112 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b863f00-6a6a-409a-980a-d6f0cb3f699e None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:26:19.761585 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:26:19.765720 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b863f00-6a6a-409a-980a-d6f0cb3f699e None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:26:19.799345 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387cf8f9-3ea8-4c8c-b89d-feee1df95b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:26:19.800199 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387cf8f9-3ea8-4c8c-b89d-feee1df95b2d None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:26:19.800480 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387cf8f9-3ea8-4c8c-b89d-feee1df95b2d None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:26:19.800755 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387cf8f9-3ea8-4c8c-b89d-feee1df95b2d None admin] PATH_INFO: `/v3/users/f8eb93fd2aed47f18a39abcedf54ec9d` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:26:19.813442 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387cf8f9-3ea8-4c8c-b89d-feee1df95b2d None admin] RBAC: Authorizing `identity:get_user(user_id=f8eb93fd2aed47f18a39abcedf54ec9d)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:26:19.814913 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387cf8f9-3ea8-4c8c-b89d-feee1df95b2d None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:26:19.816557 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 590/1182] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:26:19 2022] GET /identity/v3/users/f8eb93fd2aed47f18a39abcedf54ec9d => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:26:19.822892 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fc17d6-c460-417f-865b-cc77caddb797 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:26:19.921384 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fc17d6-c460-417f-865b-cc77caddb797 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:26:19.953816 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884de26f-0195-4658-8f2f-8f893c5d1bd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:26:19.954482 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884de26f-0195-4658-8f2f-8f893c5d1bd1 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:26:19.954765 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884de26f-0195-4658-8f2f-8f893c5d1bd1 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:26:19.955037 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884de26f-0195-4658-8f2f-8f893c5d1bd1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:26:20.005686 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:26:20.021563 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884de26f-0195-4658-8f2f-8f893c5d1bd1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:26:20.023182 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884de26f-0195-4658-8f2f-8f893c5d1bd1 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:26:20.119022 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 593/1183] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:26:19 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:26:20.124910 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387cf8f9-3ea8-4c8c-b89d-feee1df95b2d None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:26:20.221316 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387cf8f9-3ea8-4c8c-b89d-feee1df95b2d None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:26:20.254512 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba59812-e180-475b-892c-d87424601a74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:26:20.255361 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba59812-e180-475b-892c-d87424601a74 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:26:20.255644 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba59812-e180-475b-892c-d87424601a74 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:26:20.255921 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba59812-e180-475b-892c-d87424601a74 None admin] PATH_INFO: `/v3/projects/ea5fc23f66074ac0a43eecd7a2d44569` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:26:20.257208 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba59812-e180-475b-892c-d87424601a74 None admin] RBAC: Authorizing `identity:get_project(project_id=ea5fc23f66074ac0a43eecd7a2d44569)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:26:20.258665 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba59812-e180-475b-892c-d87424601a74 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:26:20.260371 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 591/1184] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:26:20 2022] GET /identity/v3/projects/ea5fc23f66074ac0a43eecd7a2d44569 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:26:20.266440 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884de26f-0195-4658-8f2f-8f893c5d1bd1 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:26:20.348685 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:26:20.364532 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884de26f-0195-4658-8f2f-8f893c5d1bd1 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:26:20.398036 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50447810-5c8f-47f7-9a7b-8001abe58cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:26:20.398669 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50447810-5c8f-47f7-9a7b-8001abe58cb5 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:26:20.398953 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50447810-5c8f-47f7-9a7b-8001abe58cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:26:20.399224 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50447810-5c8f-47f7-9a7b-8001abe58cb5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:26:20.400044 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50447810-5c8f-47f7-9a7b-8001abe58cb5 None admin] RBAC: Adding query filter params (role.id=a3b793754e7b40b0bb21afe74eec3c18, scope.project.id=ea5fc23f66074ac0a43eecd7a2d44569, user.id=f8eb93fd2aed47f18a39abcedf54ec9d) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:26:20.400352 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50447810-5c8f-47f7-9a7b-8001abe58cb5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:26:20.401774 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50447810-5c8f-47f7-9a7b-8001abe58cb5 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:26:20.407098 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 594/1185] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:26:20 2022] GET /identity/v3/role_assignments?user.id=f8eb93fd2aed47f18a39abcedf54ec9d&scope.project.id=ea5fc23f66074ac0a43eecd7a2d44569&role.id=a3b793754e7b40b0bb21afe74eec3c18 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:26:22.606217 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6303f887-a6f6-4221-b15e-fc7fd21f834b None None] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:26:22.606774 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6303f887-a6f6-4221-b15e-fc7fd21f834b None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:26:22.606774 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6303f887-a6f6-4221-b15e-fc7fd21f834b None None] PATH_INFO: `/` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:26:22.608154 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 592/1186] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:26:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:26:22.613516 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74496f4-55dc-4f74-b0b0-593f8044db81 None None] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:26:22.613962 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74496f4-55dc-4f74-b0b0-593f8044db81 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:26:22.614110 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74496f4-55dc-4f74-b0b0-593f8044db81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:26:22.630582 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.auth.core [None req-e74496f4-55dc-4f74-b0b0-593f8044db81 None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143586) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:26:22.631808 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.fernet_utils [None req-e74496f4-55dc-4f74-b0b0-593f8044db81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143586) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:26:22.700104 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 595/1187] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:26:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:26:22.722656 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6303f887-a6f6-4221-b15e-fc7fd21f834b None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:26:22.747907 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:26:22.822938 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6303f887-a6f6-4221-b15e-fc7fd21f834b None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:26:22.857133 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c870939f-4c46-4765-9b10-522db15540cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:26:22.857865 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c870939f-4c46-4765-9b10-522db15540cc None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:26:22.858147 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c870939f-4c46-4765-9b10-522db15540cc None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:26:22.858420 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c870939f-4c46-4765-9b10-522db15540cc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:26:22.869189 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c870939f-4c46-4765-9b10-522db15540cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:26:22.876660 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-c870939f-4c46-4765-9b10-522db15540cc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:26:22.876660 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:26:22.876660 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:26:22.876660 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:26:22.876660 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:26:22.876660 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:26:22.876660 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:26:22.876660 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:26:22.876660 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:26:22.876660 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:26:22.876660 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:26:22.876660 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:26:22.876660 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:26:22.876660 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:26:22.876660 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:26:22.876660 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:26:22.877700 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:26:22.877700 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:26:22.877700 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:26:22.877700 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:26:22.877700 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:26:22.877700 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:26:22.877700 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:26:22.877700 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:26:22.877700 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:26:22.877700 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:26:22.877700 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:26:22.877700 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:26:22.877700 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:26:22.877700 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:26:22.877700 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:26:22.877700 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:26:22.878574 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:26:22.878574 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:26:22.878574 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:26:22.878574 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:26:22.878574 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:26:22.878574 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:26:22.878574 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:26:22.878574 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:26:22.878574 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:26:22.878574 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:26:22.878574 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:26:22.878574 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:26:22.878574 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 593/1188] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:26:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:26:22.884610 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74496f4-55dc-4f74-b0b0-593f8044db81 None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:26:22.961927 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:26:22.983103 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74496f4-55dc-4f74-b0b0-593f8044db81 None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:26:23.016459 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4567964-699c-476d-917f-203e948b8268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:26:23.017123 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4567964-699c-476d-917f-203e948b8268 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:26:23.017404 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4567964-699c-476d-917f-203e948b8268 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:26:23.017679 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4567964-699c-476d-917f-203e948b8268 None admin] PATH_INFO: `/v3/roles` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:26:23.018467 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4567964-699c-476d-917f-203e948b8268 None admin] RBAC: Adding query filter params (name=admin) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:26:23.018775 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4567964-699c-476d-917f-203e948b8268 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:26:23.020146 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4567964-699c-476d-917f-203e948b8268 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:26:23.029074 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 596/1189] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:26:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:26:23.035091 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c870939f-4c46-4765-9b10-522db15540cc None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:26:23.120945 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:26:23.137434 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c870939f-4c46-4765-9b10-522db15540cc None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:26:23.170404 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31be3da-f5c2-4001-8c52-1305550a93dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:26:23.171269 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31be3da-f5c2-4001-8c52-1305550a93dd None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:26:23.171549 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31be3da-f5c2-4001-8c52-1305550a93dd None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:26:23.171821 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31be3da-f5c2-4001-8c52-1305550a93dd None admin] PATH_INFO: `/v3/users/f8eb93fd2aed47f18a39abcedf54ec9d` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:26:23.172999 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31be3da-f5c2-4001-8c52-1305550a93dd None admin] RBAC: Authorizing `identity:get_user(user_id=f8eb93fd2aed47f18a39abcedf54ec9d)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:26:23.174487 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31be3da-f5c2-4001-8c52-1305550a93dd None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:26:23.176111 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 594/1190] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:26:23 2022] GET /identity/v3/users/f8eb93fd2aed47f18a39abcedf54ec9d => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:26:23.182515 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4567964-699c-476d-917f-203e948b8268 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:26:23.280939 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4567964-699c-476d-917f-203e948b8268 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:26:23.314033 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a212ed-85be-4c6b-add4-0804ccc67bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:26:23.314886 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a212ed-85be-4c6b-add4-0804ccc67bbf None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:26:23.315171 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a212ed-85be-4c6b-add4-0804ccc67bbf None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:26:23.315449 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a212ed-85be-4c6b-add4-0804ccc67bbf None admin] PATH_INFO: `/v3/projects/ea5fc23f66074ac0a43eecd7a2d44569` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:26:23.316636 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a212ed-85be-4c6b-add4-0804ccc67bbf None admin] RBAC: Authorizing `identity:get_project(project_id=ea5fc23f66074ac0a43eecd7a2d44569)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:26:23.318090 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a212ed-85be-4c6b-add4-0804ccc67bbf None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:26:23.320511 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 597/1191] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:26:23 2022] GET /identity/v3/projects/ea5fc23f66074ac0a43eecd7a2d44569 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:26:23.326583 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31be3da-f5c2-4001-8c52-1305550a93dd None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:26:23.424699 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31be3da-f5c2-4001-8c52-1305550a93dd None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:26:23.458634 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572299b4-3ec3-4633-86e3-372e65aead43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:26:23.459511 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572299b4-3ec3-4633-86e3-372e65aead43 None admin] REQUEST_METHOD: `PUT` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:26:23.459834 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572299b4-3ec3-4633-86e3-372e65aead43 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:26:23.460107 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572299b4-3ec3-4633-86e3-372e65aead43 None admin] PATH_INFO: `/v3/projects/ea5fc23f66074ac0a43eecd7a2d44569/users/f8eb93fd2aed47f18a39abcedf54ec9d/roles/a3b793754e7b40b0bb21afe74eec3c18` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:26:23.462157 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572299b4-3ec3-4633-86e3-372e65aead43 None admin] RBAC: Authorizing `identity:create_grant(project_id=ea5fc23f66074ac0a43eecd7a2d44569, user_id=f8eb93fd2aed47f18a39abcedf54ec9d, role_id=a3b793754e7b40b0bb21afe74eec3c18)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:26:23.463578 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572299b4-3ec3-4633-86e3-372e65aead43 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:26:23.476446 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 595/1192] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 14:26:23 2022] PUT /identity/v3/projects/ea5fc23f66074ac0a43eecd7a2d44569/users/f8eb93fd2aed47f18a39abcedf54ec9d/roles/a3b793754e7b40b0bb21afe74eec3c18 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:26:25.668708 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d9f4d6-c025-402a-bae7-8e633c8a3012 None None] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:26:25.669132 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d9f4d6-c025-402a-bae7-8e633c8a3012 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:26:25.669270 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d9f4d6-c025-402a-bae7-8e633c8a3012 None None] PATH_INFO: `/` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:26:25.670668 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 598/1193] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:26:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:26:25.675787 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dbb0ba-9875-4ea2-9077-341e2de5ec08 None None] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:26:25.676189 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dbb0ba-9875-4ea2-9077-341e2de5ec08 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:26:25.676346 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dbb0ba-9875-4ea2-9077-341e2de5ec08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:26:25.693280 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.auth.core [None req-23dbb0ba-9875-4ea2-9077-341e2de5ec08 None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143587) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:26:25.694730 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-23dbb0ba-9875-4ea2-9077-341e2de5ec08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:26:25.701803 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:26:25.760901 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 596/1194] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:26:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:26:25.784540 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d9f4d6-c025-402a-bae7-8e633c8a3012 None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:26:25.796467 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:26:25.885835 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d9f4d6-c025-402a-bae7-8e633c8a3012 None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:26:25.918456 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e01fcf4-30e1-4f0f-ad80-61c38cd3a449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:26:25.919168 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e01fcf4-30e1-4f0f-ad80-61c38cd3a449 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:26:25.919450 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e01fcf4-30e1-4f0f-ad80-61c38cd3a449 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:26:25.919727 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e01fcf4-30e1-4f0f-ad80-61c38cd3a449 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:26:25.930272 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e01fcf4-30e1-4f0f-ad80-61c38cd3a449 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:26:25.937730 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-0e01fcf4-30e1-4f0f-ad80-61c38cd3a449 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:26:25.937730 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:26:25.937730 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:26:25.937730 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:26:25.937730 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:26:25.937730 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:26:25.937730 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:26:25.937730 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:26:25.937730 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:26:25.937730 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:26:25.937730 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:26:25.937730 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:26:25.937730 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:26:25.937730 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:26:25.937730 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:26:25.937730 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:26:25.938802 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:26:25.938802 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:26:25.938802 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:26:25.938802 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:26:25.938802 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:26:25.938802 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:26:25.938802 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:26:25.938802 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:26:25.938802 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:26:25.938802 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:26:25.938802 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:26:25.938802 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:26:25.938802 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:26:25.938802 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:26:25.938802 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:26:25.938802 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:26:25.939774 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:26:25.939774 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:26:25.939774 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:26:25.939774 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:26:25.939774 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:26:25.939774 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:26:25.939774 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:26:25.939774 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:26:25.939774 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:26:25.939774 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:26:25.939774 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:26:25.939774 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:26:25.939774 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 599/1195] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:26:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:26:25.945604 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dbb0ba-9875-4ea2-9077-341e2de5ec08 None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:26:26.043086 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dbb0ba-9875-4ea2-9077-341e2de5ec08 None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:26:26.075693 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090e970b-0c98-4ff8-9612-6a5e42f1924b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:26:26.076341 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090e970b-0c98-4ff8-9612-6a5e42f1924b None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:26:26.076622 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090e970b-0c98-4ff8-9612-6a5e42f1924b None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:26:26.076896 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090e970b-0c98-4ff8-9612-6a5e42f1924b None admin] PATH_INFO: `/v3/roles` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:26:26.077632 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090e970b-0c98-4ff8-9612-6a5e42f1924b None admin] RBAC: Adding query filter params (name=admin) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:26:26.077966 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090e970b-0c98-4ff8-9612-6a5e42f1924b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:26:26.079328 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090e970b-0c98-4ff8-9612-6a5e42f1924b None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:26:26.087818 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 597/1196] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:26:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:26:26.093558 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e01fcf4-30e1-4f0f-ad80-61c38cd3a449 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:26:26.159174 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:26:26.192574 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e01fcf4-30e1-4f0f-ad80-61c38cd3a449 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:26:26.226331 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847e910d-4785-49bb-bdf5-460938362c77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:26:26.227186 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847e910d-4785-49bb-bdf5-460938362c77 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:26:26.227467 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847e910d-4785-49bb-bdf5-460938362c77 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:26:26.227739 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847e910d-4785-49bb-bdf5-460938362c77 None admin] PATH_INFO: `/v3/users/f8eb93fd2aed47f18a39abcedf54ec9d` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:26:26.228931 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847e910d-4785-49bb-bdf5-460938362c77 None admin] RBAC: Authorizing `identity:get_user(user_id=f8eb93fd2aed47f18a39abcedf54ec9d)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:26:26.230396 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847e910d-4785-49bb-bdf5-460938362c77 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:26:26.232011 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 600/1197] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:26:26 2022] GET /identity/v3/users/f8eb93fd2aed47f18a39abcedf54ec9d => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:26:26.237992 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090e970b-0c98-4ff8-9612-6a5e42f1924b None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:26:26.242478 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:26:26.337103 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090e970b-0c98-4ff8-9612-6a5e42f1924b None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:26:26.370637 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3034bf27-6c1b-4ae2-91e9-7525d5e6f328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:26:26.371280 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3034bf27-6c1b-4ae2-91e9-7525d5e6f328 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:26:26.371566 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3034bf27-6c1b-4ae2-91e9-7525d5e6f328 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:26:26.371846 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3034bf27-6c1b-4ae2-91e9-7525d5e6f328 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:26:26.441073 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3034bf27-6c1b-4ae2-91e9-7525d5e6f328 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:26:26.442555 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3034bf27-6c1b-4ae2-91e9-7525d5e6f328 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:26:26.445812 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:26:26.539662 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 598/1198] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:26:26 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:26:26.545823 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847e910d-4785-49bb-bdf5-460938362c77 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:26:26.643149 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847e910d-4785-49bb-bdf5-460938362c77 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:26:26.677106 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dad13f3-3b38-420d-8ec0-d6f4430e9c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:26:26.677992 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dad13f3-3b38-420d-8ec0-d6f4430e9c2d None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:26:26.678318 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dad13f3-3b38-420d-8ec0-d6f4430e9c2d None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:26:26.679223 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dad13f3-3b38-420d-8ec0-d6f4430e9c2d None admin] PATH_INFO: `/v3/projects/ea5fc23f66074ac0a43eecd7a2d44569` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:26:26.680610 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dad13f3-3b38-420d-8ec0-d6f4430e9c2d None admin] RBAC: Authorizing `identity:get_project(project_id=ea5fc23f66074ac0a43eecd7a2d44569)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:26:26.682089 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dad13f3-3b38-420d-8ec0-d6f4430e9c2d None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:26:26.683772 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 601/1199] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:26:26 2022] GET /identity/v3/projects/ea5fc23f66074ac0a43eecd7a2d44569 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:26:26.689950 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3034bf27-6c1b-4ae2-91e9-7525d5e6f328 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:26:26.786118 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3034bf27-6c1b-4ae2-91e9-7525d5e6f328 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:26:26.788939 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:26:26.820786 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cff0a8-6449-4d64-9661-d7ba5d0affa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:26:26.821433 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cff0a8-6449-4d64-9661-d7ba5d0affa4 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:26:26.821718 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cff0a8-6449-4d64-9661-d7ba5d0affa4 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:26:26.822027 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cff0a8-6449-4d64-9661-d7ba5d0affa4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:26:26.822865 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cff0a8-6449-4d64-9661-d7ba5d0affa4 None admin] RBAC: Adding query filter params (role.id=a3b793754e7b40b0bb21afe74eec3c18, scope.project.id=ea5fc23f66074ac0a43eecd7a2d44569, user.id=f8eb93fd2aed47f18a39abcedf54ec9d) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:26:26.823181 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cff0a8-6449-4d64-9661-d7ba5d0affa4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:26:26.824629 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cff0a8-6449-4d64-9661-d7ba5d0affa4 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:26:26.830386 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 599/1200] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:26:26 2022] GET /identity/v3/role_assignments?user.id=f8eb93fd2aed47f18a39abcedf54ec9d&scope.project.id=ea5fc23f66074ac0a43eecd7a2d44569&role.id=a3b793754e7b40b0bb21afe74eec3c18 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:26:36.672874 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=98 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:26:36.827701 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=16 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:27:25.413256 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca403ee1-476f-4303-8d1c-31f4fdc2a39e None None] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:25.413882 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca403ee1-476f-4303-8d1c-31f4fdc2a39e None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:25.413882 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca403ee1-476f-4303-8d1c-31f4fdc2a39e None None] PATH_INFO: `/` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:25.415348 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 602/1201] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:27:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:27:25.421314 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de2a0de-2145-40d7-9a2f-00b21b2a2b03 None None] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:25.421915 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de2a0de-2145-40d7-9a2f-00b21b2a2b03 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:25.421915 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de2a0de-2145-40d7-9a2f-00b21b2a2b03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:25.442625 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.auth.core [None req-9de2a0de-2145-40d7-9a2f-00b21b2a2b03 None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143587) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:27:25.443833 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-9de2a0de-2145-40d7-9a2f-00b21b2a2b03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:27:25.539237 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 600/1202] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:27:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:27:25.564552 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca403ee1-476f-4303-8d1c-31f4fdc2a39e None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:27:25.681462 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca403ee1-476f-4303-8d1c-31f4fdc2a39e None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:27:25.721707 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97607cce-7e13-441d-8293-4aa51f78fef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:27:25.723280 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97607cce-7e13-441d-8293-4aa51f78fef6 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:25.723572 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97607cce-7e13-441d-8293-4aa51f78fef6 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:25.723849 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97607cce-7e13-441d-8293-4aa51f78fef6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:25.734944 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97607cce-7e13-441d-8293-4aa51f78fef6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:27:25.736401 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97607cce-7e13-441d-8293-4aa51f78fef6 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:27:25.756128 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-97607cce-7e13-441d-8293-4aa51f78fef6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:27:25.756128 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:27:25.756128 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:27:25.756128 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:27:25.756128 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:27:25.756128 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:27:25.756128 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:27:25.756128 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:27:25.756128 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:27:25.756128 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:27:25.756128 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:27:25.756128 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:27:25.756128 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:27:25.756128 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:27:25.756128 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:27:25.757037 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:27:25.757037 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:27:25.757037 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:27:25.757037 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:27:25.757037 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:27:25.757037 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:27:25.757037 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:27:25.757037 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:27:25.757037 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:27:25.757037 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:27:25.757037 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:27:25.757037 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:27:25.757037 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:27:25.757037 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:27:25.757037 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:27:25.757037 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:27:25.757037 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:27:25.758368 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:27:25.758368 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:27:25.758368 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:27:25.758368 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:27:25.758832 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 603/1203] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:27:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:27:25.763419 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de2a0de-2145-40d7-9a2f-00b21b2a2b03 None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:27:25.872050 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de2a0de-2145-40d7-9a2f-00b21b2a2b03 None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:27:25.906096 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:27:25.914926 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e5f9d9-78e0-485e-a10c-2a9a97f63d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:27:25.915594 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e5f9d9-78e0-485e-a10c-2a9a97f63d89 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:25.920025 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e5f9d9-78e0-485e-a10c-2a9a97f63d89 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:25.920025 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e5f9d9-78e0-485e-a10c-2a9a97f63d89 None admin] PATH_INFO: `/v3/domains` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:25.920025 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e5f9d9-78e0-485e-a10c-2a9a97f63d89 None admin] RBAC: Adding query filter params (name=Default) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:27:25.920025 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e5f9d9-78e0-485e-a10c-2a9a97f63d89 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:27:25.920025 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e5f9d9-78e0-485e-a10c-2a9a97f63d89 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:27:25.937982 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 601/1204] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:27:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:27:25.939190 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97607cce-7e13-441d-8293-4aa51f78fef6 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:27:26.013494 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:27:26.088291 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97607cce-7e13-441d-8293-4aa51f78fef6 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:27:26.136850 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abcc8ac-a7c0-44fb-875c-095ef2e29266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:27:26.137562 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abcc8ac-a7c0-44fb-875c-095ef2e29266 None admin] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:26.137861 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abcc8ac-a7c0-44fb-875c-095ef2e29266 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:26.138202 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abcc8ac-a7c0-44fb-875c-095ef2e29266 None admin] PATH_INFO: `/v3/users` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:26.138956 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abcc8ac-a7c0-44fb-875c-095ef2e29266 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:27:26.140510 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abcc8ac-a7c0-44fb-875c-095ef2e29266 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:27:26.162988 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 604/1205] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 19 14:27:25 2022] POST /identity/v3/users => generated 251 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:27:28.539904 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2aae54-5992-4b95-83f0-efcb26ca9c35 None None] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:28.540460 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2aae54-5992-4b95-83f0-efcb26ca9c35 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:28.540460 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2aae54-5992-4b95-83f0-efcb26ca9c35 None None] PATH_INFO: `/` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:28.542543 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 602/1206] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:27:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:27:28.548049 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dd3ca5-e537-4236-b292-a014d9c4cea9 None None] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:28.548497 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dd3ca5-e537-4236-b292-a014d9c4cea9 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:28.548598 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dd3ca5-e537-4236-b292-a014d9c4cea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:28.567254 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.auth.core [None req-e3dd3ca5-e537-4236-b292-a014d9c4cea9 None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143586) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:27:28.568447 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.fernet_utils [None req-e3dd3ca5-e537-4236-b292-a014d9c4cea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143586) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:27:28.646962 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 605/1207] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:27:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:27:28.673463 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2aae54-5992-4b95-83f0-efcb26ca9c35 None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:27:28.808151 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2aae54-5992-4b95-83f0-efcb26ca9c35 None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:27:28.856035 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a145e00-8525-468e-a749-3223ac96ab57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:27:28.856837 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a145e00-8525-468e-a749-3223ac96ab57 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:28.857116 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a145e00-8525-468e-a749-3223ac96ab57 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:28.857392 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a145e00-8525-468e-a749-3223ac96ab57 None admin] PATH_INFO: `/v3/roles/service` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:28.874229 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a145e00-8525-468e-a749-3223ac96ab57 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:27:28.892481 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-5a145e00-8525-468e-a749-3223ac96ab57 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:27:28.892481 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:27:28.892481 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:27:28.892481 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:27:28.892481 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:27:28.892481 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:27:28.892481 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:27:28.892481 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:27:28.892481 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:27:28.892481 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:27:28.892481 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:27:28.892481 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:27:28.892481 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:27:28.892481 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:27:28.892481 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:27:28.892481 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:27:28.893458 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:27:28.893458 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:27:28.893458 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:27:28.893458 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:27:28.893458 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:27:28.893458 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:27:28.893458 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:27:28.893458 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:27:28.893458 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:27:28.893458 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:27:28.893458 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:27:28.893458 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:27:28.893458 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:27:28.893458 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:27:28.893458 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:27:28.893458 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:27:28.895164 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:27:28.895164 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:27:28.895164 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:27:28.895164 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:27:28.895164 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:27:28.895164 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:27:28.895164 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:27:28.895164 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:27:28.895164 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:27:28.895164 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:27:28.895164 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:27:28.895164 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:27:28.896089 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 603/1208] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 14:27:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:27:28.901671 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dd3ca5-e537-4236-b292-a014d9c4cea9 None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:27:28.951345 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:27:29.029809 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dd3ca5-e537-4236-b292-a014d9c4cea9 None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:27:29.070195 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da8b2f0-3010-42af-86d2-81df7728e1cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:27:29.070796 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da8b2f0-3010-42af-86d2-81df7728e1cd None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:29.071083 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da8b2f0-3010-42af-86d2-81df7728e1cd None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:29.071359 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da8b2f0-3010-42af-86d2-81df7728e1cd None admin] PATH_INFO: `/v3/roles` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:29.072343 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da8b2f0-3010-42af-86d2-81df7728e1cd None admin] RBAC: Adding query filter params (name=service) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:27:29.072633 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da8b2f0-3010-42af-86d2-81df7728e1cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:27:29.074178 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da8b2f0-3010-42af-86d2-81df7728e1cd None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:27:29.084802 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 606/1209] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 19 14:27:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:27:29.090792 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a145e00-8525-468e-a749-3223ac96ab57 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:27:29.148634 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:27:29.207078 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a145e00-8525-468e-a749-3223ac96ab57 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:27:29.246006 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d691990a-562f-422a-898f-2a94d7a7b78e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:27:29.247107 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d691990a-562f-422a-898f-2a94d7a7b78e None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:29.247545 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d691990a-562f-422a-898f-2a94d7a7b78e None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:29.247892 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d691990a-562f-422a-898f-2a94d7a7b78e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:29.263019 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d691990a-562f-422a-898f-2a94d7a7b78e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:27:29.264783 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d691990a-562f-422a-898f-2a94d7a7b78e None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:27:29.286141 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-d691990a-562f-422a-898f-2a94d7a7b78e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:27:29.286141 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:27:29.286141 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:27:29.286141 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:27:29.286141 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:27:29.286141 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:27:29.286141 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:27:29.286141 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:27:29.286141 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:27:29.286141 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:27:29.286141 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:27:29.286141 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:27:29.286141 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:27:29.286141 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:27:29.286141 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:27:29.287309 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:27:29.287309 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:27:29.287309 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:27:29.287309 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:27:29.287309 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:27:29.287309 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:27:29.287309 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:27:29.287309 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:27:29.287309 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:27:29.287309 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:27:29.287309 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:27:29.287309 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:27:29.287309 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:27:29.287309 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:27:29.287309 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:27:29.287309 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:27:29.287309 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:27:29.288316 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:27:29.288316 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:27:29.288316 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:27:29.288316 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:27:29.289691 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 604/1210] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:27:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:27:29.295945 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da8b2f0-3010-42af-86d2-81df7728e1cd None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:27:29.413633 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:27:29.416822 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da8b2f0-3010-42af-86d2-81df7728e1cd None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:27:29.458453 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b2320d-ea5c-43e5-981c-64f79fe44758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:27:29.459144 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b2320d-ea5c-43e5-981c-64f79fe44758 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:29.459391 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b2320d-ea5c-43e5-981c-64f79fe44758 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:29.459660 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b2320d-ea5c-43e5-981c-64f79fe44758 None admin] PATH_INFO: `/v3/domains` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:29.460392 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b2320d-ea5c-43e5-981c-64f79fe44758 None admin] RBAC: Adding query filter params (name=Default) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:27:29.460697 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b2320d-ea5c-43e5-981c-64f79fe44758 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:27:29.462186 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b2320d-ea5c-43e5-981c-64f79fe44758 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:27:29.475493 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 607/1211] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:27:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:27:29.481413 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d691990a-562f-422a-898f-2a94d7a7b78e None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:27:29.597098 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d691990a-562f-422a-898f-2a94d7a7b78e None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:27:29.598745 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:27:29.642891 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0e855d-c0ab-4797-96f3-cc3cfc6104d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:27:29.643701 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0e855d-c0ab-4797-96f3-cc3cfc6104d7 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:29.644040 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0e855d-c0ab-4797-96f3-cc3cfc6104d7 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:29.644572 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0e855d-c0ab-4797-96f3-cc3cfc6104d7 None admin] PATH_INFO: `/v3/users/placement` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:29.652966 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0e855d-c0ab-4797-96f3-cc3cfc6104d7 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:27:29.654458 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0e855d-c0ab-4797-96f3-cc3cfc6104d7 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:27:29.671214 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-ea0e855d-c0ab-4797-96f3-cc3cfc6104d7 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 14:27:29.671214 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:27:29.671214 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:27:29.671214 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:27:29.671214 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:27:29.671214 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:27:29.671214 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:27:29.671214 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:27:29.671214 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:27:29.671214 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:27:29.671214 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:27:29.671214 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:27:29.671214 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:27:29.671214 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:27:29.671214 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:27:29.671214 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:27:29.672557 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:27:29.672557 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:27:29.672557 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:27:29.672557 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:27:29.672557 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:27:29.672557 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:27:29.672557 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:27:29.672557 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:27:29.672557 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:27:29.672557 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:27:29.672557 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:27:29.672557 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:27:29.672557 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:27:29.672557 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:27:29.672557 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:27:29.672557 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:27:29.672557 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:27:29.672557 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:27:29.673482 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:27:29.673482 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:27:29.673482 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:27:29.673482 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:27:29.673482 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:27:29.673482 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:27:29.673482 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:27:29.673482 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:27:29.673482 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 14:27:29.673482 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:27:29.673482 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 605/1212] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:27:29 2022] GET /identity/v3/users/placement => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:27:29.678825 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b2320d-ea5c-43e5-981c-64f79fe44758 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:27:29.803754 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b2320d-ea5c-43e5-981c-64f79fe44758 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:27:29.841029 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1373ed5-e772-41b1-a1d0-01881fd852a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:27:29.841727 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1373ed5-e772-41b1-a1d0-01881fd852a3 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:29.842041 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1373ed5-e772-41b1-a1d0-01881fd852a3 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:29.842324 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1373ed5-e772-41b1-a1d0-01881fd852a3 None admin] PATH_INFO: `/v3/users` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:29.843470 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1373ed5-e772-41b1-a1d0-01881fd852a3 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:27:29.843766 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1373ed5-e772-41b1-a1d0-01881fd852a3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:27:29.845237 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1373ed5-e772-41b1-a1d0-01881fd852a3 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:27:29.871902 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 608/1213] 10.222.0.57 () {56 vars in 1143 bytes} [Thu May 19 14:27:29 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:27:29.872724 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0e855d-c0ab-4797-96f3-cc3cfc6104d7 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:27:29.996784 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0e855d-c0ab-4797-96f3-cc3cfc6104d7 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:27:30.044267 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452b180e-b050-4a2a-a714-39bb0310d5d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:27:30.045025 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452b180e-b050-4a2a-a714-39bb0310d5d0 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:30.045312 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452b180e-b050-4a2a-a714-39bb0310d5d0 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:30.045628 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452b180e-b050-4a2a-a714-39bb0310d5d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:30.079038 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:27:30.133485 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452b180e-b050-4a2a-a714-39bb0310d5d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:27:30.135790 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452b180e-b050-4a2a-a714-39bb0310d5d0 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:27:30.258609 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 606/1214] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:27:29 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:27:30.263935 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1373ed5-e772-41b1-a1d0-01881fd852a3 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:27:30.286318 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:27:30.392660 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1373ed5-e772-41b1-a1d0-01881fd852a3 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:27:30.437788 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691c43a7-773d-44ea-b7e0-669729fd7b50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:27:30.438644 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691c43a7-773d-44ea-b7e0-669729fd7b50 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:30.438948 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691c43a7-773d-44ea-b7e0-669729fd7b50 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:30.439367 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691c43a7-773d-44ea-b7e0-669729fd7b50 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:30.452586 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691c43a7-773d-44ea-b7e0-669729fd7b50 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:27:30.454352 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691c43a7-773d-44ea-b7e0-669729fd7b50 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:27:30.476880 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-691c43a7-773d-44ea-b7e0-669729fd7b50 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:27:30.476880 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:27:30.476880 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:27:30.476880 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:27:30.476880 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:27:30.476880 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:27:30.476880 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:27:30.476880 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:27:30.476880 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:27:30.476880 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:27:30.476880 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:27:30.476880 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:27:30.476880 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:27:30.476880 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:27:30.476880 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:27:30.477983 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:27:30.477983 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:27:30.477983 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:27:30.477983 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:27:30.477983 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:27:30.477983 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:27:30.477983 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:27:30.477983 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:27:30.477983 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:27:30.477983 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:27:30.477983 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:27:30.477983 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:27:30.477983 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:27:30.477983 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:27:30.477983 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:27:30.477983 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:27:30.477983 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:27:30.478836 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:27:30.478836 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:27:30.478836 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:27:30.478836 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:27:30.481246 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 609/1215] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:27:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:27:30.486845 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452b180e-b050-4a2a-a714-39bb0310d5d0 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:27:30.560722 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:27:30.609194 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452b180e-b050-4a2a-a714-39bb0310d5d0 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:27:30.654447 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4493519-1db4-4046-95ed-39e4a91baab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:27:30.655093 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4493519-1db4-4046-95ed-39e4a91baab2 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:30.655367 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4493519-1db4-4046-95ed-39e4a91baab2 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:30.655643 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4493519-1db4-4046-95ed-39e4a91baab2 None admin] PATH_INFO: `/v3/domains` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:30.656367 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4493519-1db4-4046-95ed-39e4a91baab2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:27:30.656793 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4493519-1db4-4046-95ed-39e4a91baab2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:27:30.658122 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4493519-1db4-4046-95ed-39e4a91baab2 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:27:30.672081 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 607/1216] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:27:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:27:30.677477 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691c43a7-773d-44ea-b7e0-669729fd7b50 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:27:30.762540 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:27:30.806555 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691c43a7-773d-44ea-b7e0-669729fd7b50 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:27:30.845389 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259400aa-98f5-4d35-8399-dfc71a342f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:27:30.846160 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259400aa-98f5-4d35-8399-dfc71a342f35 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:30.846442 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259400aa-98f5-4d35-8399-dfc71a342f35 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:30.846719 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259400aa-98f5-4d35-8399-dfc71a342f35 None admin] PATH_INFO: `/v3/projects/service` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:30.855653 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259400aa-98f5-4d35-8399-dfc71a342f35 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:27:30.857335 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259400aa-98f5-4d35-8399-dfc71a342f35 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:27:30.873174 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-259400aa-98f5-4d35-8399-dfc71a342f35 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:27:30.873174 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:27:30.873174 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:27:30.873174 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:27:30.873174 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:27:30.873174 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:27:30.873174 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:27:30.873174 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:27:30.873174 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:27:30.873174 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:27:30.873174 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:27:30.873174 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:27:30.873174 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:27:30.873174 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:27:30.873174 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:27:30.874278 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:27:30.874278 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:27:30.874278 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:27:30.874278 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:27:30.874278 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:27:30.874278 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:27:30.874278 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:27:30.874278 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:27:30.874278 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:27:30.874278 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:27:30.874278 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:27:30.874278 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:27:30.874278 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:27:30.874278 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:27:30.874278 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:27:30.874278 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:27:30.874278 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:27:30.875416 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:27:30.875416 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:27:30.875416 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:27:30.875416 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:27:30.875416 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:27:30.875416 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:27:30.875416 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:27:30.875416 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:27:30.875416 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 610/1217] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:27:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:27:30.881048 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4493519-1db4-4046-95ed-39e4a91baab2 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:27:31.007051 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4493519-1db4-4046-95ed-39e4a91baab2 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:27:31.038210 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:27:31.052948 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82688437-d46d-475e-8808-adb8858e0ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:27:31.053704 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82688437-d46d-475e-8808-adb8858e0ba4 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:31.054050 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82688437-d46d-475e-8808-adb8858e0ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:31.054374 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82688437-d46d-475e-8808-adb8858e0ba4 None admin] PATH_INFO: `/v3/projects` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:31.055257 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82688437-d46d-475e-8808-adb8858e0ba4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:27:31.055604 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82688437-d46d-475e-8808-adb8858e0ba4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:27:31.057253 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82688437-d46d-475e-8808-adb8858e0ba4 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:27:31.072254 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 608/1218] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:27:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:27:31.078157 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259400aa-98f5-4d35-8399-dfc71a342f35 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:27:31.211245 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259400aa-98f5-4d35-8399-dfc71a342f35 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:27:31.252051 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:27:31.262461 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40cb180-d16a-4547-b024-b9cdb92897a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:27:31.263363 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40cb180-d16a-4547-b024-b9cdb92897a4 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:31.263619 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40cb180-d16a-4547-b024-b9cdb92897a4 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:31.263942 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40cb180-d16a-4547-b024-b9cdb92897a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:31.264849 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40cb180-d16a-4547-b024-b9cdb92897a4 None admin] RBAC: Adding query filter params (role.id=b0b74fefb27e4a0ebd895893aa9862d7, scope.project.id=252d9673f08c479a98d04fee3f5856a4, user.id=59ec84a77cb449a1a22c86ac9873b8af) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:27:31.265237 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40cb180-d16a-4547-b024-b9cdb92897a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:27:31.266688 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40cb180-d16a-4547-b024-b9cdb92897a4 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:27:31.274519 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 611/1219] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:27:31 2022] GET /identity/v3/role_assignments?user.id=59ec84a77cb449a1a22c86ac9873b8af&scope.project.id=252d9673f08c479a98d04fee3f5856a4&role.id=b0b74fefb27e4a0ebd895893aa9862d7 => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:27:33.611204 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6adf467-4a5e-4140-bb7b-933601036211 None None] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:33.611784 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6adf467-4a5e-4140-bb7b-933601036211 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:33.611784 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6adf467-4a5e-4140-bb7b-933601036211 None None] PATH_INFO: `/` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:33.613615 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 609/1220] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:27:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:27:33.619166 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f428909-c160-4712-b08e-bf9119ec6416 None None] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:33.619693 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f428909-c160-4712-b08e-bf9119ec6416 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:33.619784 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f428909-c160-4712-b08e-bf9119ec6416 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:33.638324 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.auth.core [None req-4f428909-c160-4712-b08e-bf9119ec6416 None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143586) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:27:33.639522 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.fernet_utils [None req-4f428909-c160-4712-b08e-bf9119ec6416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143586) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:27:33.715687 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 612/1221] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:27:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:27:33.741059 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6adf467-4a5e-4140-bb7b-933601036211 None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:27:33.855141 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6adf467-4a5e-4140-bb7b-933601036211 None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:27:33.893655 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613f602d-19cd-414b-a11b-61227274ff38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:27:33.894519 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613f602d-19cd-414b-a11b-61227274ff38 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:33.894822 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613f602d-19cd-414b-a11b-61227274ff38 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:33.895078 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613f602d-19cd-414b-a11b-61227274ff38 None admin] PATH_INFO: `/v3/roles/service` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:33.906270 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613f602d-19cd-414b-a11b-61227274ff38 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:27:33.914622 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-613f602d-19cd-414b-a11b-61227274ff38 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:27:33.914622 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:27:33.914622 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:27:33.914622 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:27:33.914622 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:27:33.914622 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:27:33.914622 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:27:33.914622 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:27:33.914622 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:27:33.914622 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:27:33.914622 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:27:33.914622 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:27:33.914622 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:27:33.914622 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:27:33.914622 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:27:33.914622 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:27:33.915840 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:27:33.915840 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:27:33.915840 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:27:33.915840 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:27:33.915840 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:27:33.915840 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:27:33.915840 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:27:33.915840 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:27:33.915840 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:27:33.915840 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:27:33.915840 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:27:33.915840 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:27:33.915840 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:27:33.915840 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:27:33.915840 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:27:33.915840 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:27:33.916651 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:27:33.916651 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:27:33.916651 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:27:33.916651 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:27:33.916651 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:27:33.916651 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:27:33.916651 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:27:33.916651 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:27:33.916651 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:27:33.916651 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:27:33.916651 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:27:33.916651 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:27:33.916651 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 610/1222] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 14:27:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:27:33.922153 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f428909-c160-4712-b08e-bf9119ec6416 None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:27:34.063186 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f428909-c160-4712-b08e-bf9119ec6416 None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:27:34.096504 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:27:34.112354 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c0d458-ca80-4930-b33d-eb8fcbfeee9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:27:34.113741 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c0d458-ca80-4930-b33d-eb8fcbfeee9b None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:34.114122 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c0d458-ca80-4930-b33d-eb8fcbfeee9b None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:34.114585 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c0d458-ca80-4930-b33d-eb8fcbfeee9b None admin] PATH_INFO: `/v3/roles` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:34.115270 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c0d458-ca80-4930-b33d-eb8fcbfeee9b None admin] RBAC: Adding query filter params (name=service) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:27:34.115611 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c0d458-ca80-4930-b33d-eb8fcbfeee9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:27:34.117069 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c0d458-ca80-4930-b33d-eb8fcbfeee9b None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:27:34.126271 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 613/1223] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 19 14:27:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:27:34.132385 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613f602d-19cd-414b-a11b-61227274ff38 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:27:34.184618 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:27:34.257340 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613f602d-19cd-414b-a11b-61227274ff38 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:27:34.294359 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d10fba-97d1-44d7-b956-ae7d830832e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:27:34.295161 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d10fba-97d1-44d7-b956-ae7d830832e0 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:34.295444 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d10fba-97d1-44d7-b956-ae7d830832e0 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:34.295723 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d10fba-97d1-44d7-b956-ae7d830832e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:34.307362 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d10fba-97d1-44d7-b956-ae7d830832e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:27:34.308808 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d10fba-97d1-44d7-b956-ae7d830832e0 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:27:34.325814 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-21d10fba-97d1-44d7-b956-ae7d830832e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:27:34.325814 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:27:34.325814 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:27:34.325814 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:27:34.325814 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:27:34.325814 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:27:34.325814 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:27:34.325814 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:27:34.325814 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:27:34.325814 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:27:34.325814 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:27:34.325814 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:27:34.325814 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:27:34.325814 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:27:34.325814 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:27:34.326802 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:27:34.326802 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:27:34.326802 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:27:34.326802 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:27:34.326802 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:27:34.326802 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:27:34.326802 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:27:34.326802 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:27:34.326802 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:27:34.326802 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:27:34.326802 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:27:34.326802 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:27:34.326802 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:27:34.326802 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:27:34.326802 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:27:34.326802 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:27:34.326802 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:27:34.327783 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:27:34.327783 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:27:34.327783 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:27:34.327783 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:27:34.329998 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 611/1224] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:27:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:27:34.332804 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c0d458-ca80-4930-b33d-eb8fcbfeee9b None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:27:34.458446 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c0d458-ca80-4930-b33d-eb8fcbfeee9b None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:27:34.497344 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48cea57-387f-4de1-a3e1-bb506d898ab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:27:34.498437 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48cea57-387f-4de1-a3e1-bb506d898ab4 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:34.498870 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48cea57-387f-4de1-a3e1-bb506d898ab4 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:34.499432 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48cea57-387f-4de1-a3e1-bb506d898ab4 None admin] PATH_INFO: `/v3/domains` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:34.500770 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48cea57-387f-4de1-a3e1-bb506d898ab4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:27:34.501382 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48cea57-387f-4de1-a3e1-bb506d898ab4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:27:34.504689 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48cea57-387f-4de1-a3e1-bb506d898ab4 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:27:34.520191 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 614/1225] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:27:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:27:34.527483 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d10fba-97d1-44d7-b956-ae7d830832e0 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:27:34.637484 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:27:34.645429 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d10fba-97d1-44d7-b956-ae7d830832e0 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:27:34.683878 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18cf407-3e5b-4fe4-a551-9f8d381cff10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:27:34.684746 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18cf407-3e5b-4fe4-a551-9f8d381cff10 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:34.685036 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18cf407-3e5b-4fe4-a551-9f8d381cff10 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:34.685358 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18cf407-3e5b-4fe4-a551-9f8d381cff10 None admin] PATH_INFO: `/v3/users/placement` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:34.694473 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18cf407-3e5b-4fe4-a551-9f8d381cff10 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:27:34.696140 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18cf407-3e5b-4fe4-a551-9f8d381cff10 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:27:34.712493 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-d18cf407-3e5b-4fe4-a551-9f8d381cff10 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 14:27:34.712493 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:27:34.712493 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:27:34.712493 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:27:34.712493 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:27:34.712493 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:27:34.712493 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:27:34.712493 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:27:34.712493 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:27:34.712493 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:27:34.712493 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:27:34.712493 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:27:34.712493 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:27:34.712493 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:27:34.712493 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:27:34.712493 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:27:34.713429 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:27:34.713429 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:27:34.713429 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:27:34.713429 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:27:34.713429 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:27:34.713429 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:27:34.713429 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:27:34.713429 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:27:34.713429 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:27:34.713429 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:27:34.713429 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:27:34.713429 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:27:34.713429 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:27:34.713429 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:27:34.713429 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:27:34.713429 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:27:34.713429 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:27:34.713429 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:27:34.714346 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:27:34.714346 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:27:34.714346 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:27:34.714346 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:27:34.714346 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:27:34.714346 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:27:34.714346 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:27:34.714346 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:27:34.714346 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 14:27:34.714346 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:27:34.714346 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 612/1226] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:27:34 2022] GET /identity/v3/users/placement => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:27:34.721223 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48cea57-387f-4de1-a3e1-bb506d898ab4 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:27:34.777535 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:27:34.836790 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48cea57-387f-4de1-a3e1-bb506d898ab4 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:27:34.877775 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2767061-3ab2-45bc-9c83-d2e418d56b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:27:34.878732 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2767061-3ab2-45bc-9c83-d2e418d56b54 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:34.879021 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2767061-3ab2-45bc-9c83-d2e418d56b54 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:34.879301 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2767061-3ab2-45bc-9c83-d2e418d56b54 None admin] PATH_INFO: `/v3/users` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:34.880188 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2767061-3ab2-45bc-9c83-d2e418d56b54 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:27:34.880539 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2767061-3ab2-45bc-9c83-d2e418d56b54 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:27:34.882259 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2767061-3ab2-45bc-9c83-d2e418d56b54 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:27:34.902277 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 615/1227] 10.222.0.57 () {56 vars in 1143 bytes} [Thu May 19 14:27:34 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:27:34.907810 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18cf407-3e5b-4fe4-a551-9f8d381cff10 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:27:35.015872 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18cf407-3e5b-4fe4-a551-9f8d381cff10 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:27:35.058591 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3037ca-ff29-4fde-b8d3-d72b6cd2fad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:27:35.059816 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3037ca-ff29-4fde-b8d3-d72b6cd2fad3 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:35.060131 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3037ca-ff29-4fde-b8d3-d72b6cd2fad3 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:35.060570 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3037ca-ff29-4fde-b8d3-d72b6cd2fad3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:35.074446 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3037ca-ff29-4fde-b8d3-d72b6cd2fad3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:27:35.076135 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3037ca-ff29-4fde-b8d3-d72b6cd2fad3 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:27:35.100362 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-ae3037ca-ff29-4fde-b8d3-d72b6cd2fad3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:27:35.100362 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:27:35.100362 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:27:35.100362 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:27:35.100362 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:27:35.100362 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:27:35.100362 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:27:35.100362 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:27:35.100362 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:27:35.100362 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:27:35.100362 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:27:35.100362 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:27:35.100362 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:27:35.100362 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:27:35.100362 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:27:35.101527 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:27:35.101527 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:27:35.101527 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:27:35.101527 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:27:35.101527 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:27:35.101527 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:27:35.101527 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:27:35.101527 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:27:35.101527 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:27:35.101527 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:27:35.101527 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:27:35.101527 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:27:35.101527 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:27:35.101527 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:27:35.101527 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:27:35.101527 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:27:35.101527 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:27:35.102591 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:27:35.102591 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:27:35.102591 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:27:35.102591 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:27:35.102591 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 613/1228] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:27:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:27:35.107778 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2767061-3ab2-45bc-9c83-d2e418d56b54 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:27:35.221252 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2767061-3ab2-45bc-9c83-d2e418d56b54 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:27:35.223696 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:27:35.259887 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ea9387-35c9-4dfe-b0ce-43ecc2adb434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:27:35.260550 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ea9387-35c9-4dfe-b0ce-43ecc2adb434 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:35.260831 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ea9387-35c9-4dfe-b0ce-43ecc2adb434 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:35.261108 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ea9387-35c9-4dfe-b0ce-43ecc2adb434 None admin] PATH_INFO: `/v3/domains` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:35.261826 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ea9387-35c9-4dfe-b0ce-43ecc2adb434 None admin] RBAC: Adding query filter params (name=Default) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:27:35.262161 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ea9387-35c9-4dfe-b0ce-43ecc2adb434 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:27:35.263515 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ea9387-35c9-4dfe-b0ce-43ecc2adb434 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:27:35.280352 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 616/1229] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:27:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:27:35.281625 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3037ca-ff29-4fde-b8d3-d72b6cd2fad3 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:27:35.291656 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:27:35.394934 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3037ca-ff29-4fde-b8d3-d72b6cd2fad3 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:27:35.433020 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988055c0-6b72-4506-b206-31dfb2ed8043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:27:35.433818 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988055c0-6b72-4506-b206-31dfb2ed8043 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:35.434133 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988055c0-6b72-4506-b206-31dfb2ed8043 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:35.434412 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988055c0-6b72-4506-b206-31dfb2ed8043 None admin] PATH_INFO: `/v3/projects/service` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:35.441985 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988055c0-6b72-4506-b206-31dfb2ed8043 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:27:35.443439 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988055c0-6b72-4506-b206-31dfb2ed8043 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:27:35.459672 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-988055c0-6b72-4506-b206-31dfb2ed8043 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:27:35.459672 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:27:35.459672 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:27:35.459672 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:27:35.459672 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:27:35.459672 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:27:35.459672 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:27:35.459672 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:27:35.459672 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:27:35.459672 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:27:35.459672 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:27:35.459672 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:27:35.459672 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:27:35.459672 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:27:35.459672 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:27:35.460849 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:27:35.460849 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:27:35.460849 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:27:35.460849 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:27:35.460849 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:27:35.460849 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:27:35.460849 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:27:35.460849 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:27:35.460849 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:27:35.460849 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:27:35.460849 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:27:35.460849 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:27:35.460849 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:27:35.460849 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:27:35.460849 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:27:35.460849 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:27:35.460849 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:27:35.462182 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:27:35.462182 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:27:35.462182 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:27:35.462182 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:27:35.462182 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:27:35.462182 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:27:35.462182 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:27:35.462182 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:27:35.462182 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 614/1230] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:27:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:27:35.467288 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ea9387-35c9-4dfe-b0ce-43ecc2adb434 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:27:35.584669 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ea9387-35c9-4dfe-b0ce-43ecc2adb434 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:27:35.629315 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365c00b9-3356-4f33-bad4-973243b5a511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:27:35.630005 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365c00b9-3356-4f33-bad4-973243b5a511 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:35.630288 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365c00b9-3356-4f33-bad4-973243b5a511 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:35.630562 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365c00b9-3356-4f33-bad4-973243b5a511 None admin] PATH_INFO: `/v3/projects` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:35.631585 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365c00b9-3356-4f33-bad4-973243b5a511 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:27:35.631904 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365c00b9-3356-4f33-bad4-973243b5a511 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:27:35.633675 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365c00b9-3356-4f33-bad4-973243b5a511 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:27:35.648911 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 617/1231] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:27:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:27:35.654770 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988055c0-6b72-4506-b206-31dfb2ed8043 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:27:35.723868 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:27:35.762786 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988055c0-6b72-4506-b206-31dfb2ed8043 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:27:35.801327 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae44d2c-a195-498e-835c-70b74a0e151c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:27:35.802485 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae44d2c-a195-498e-835c-70b74a0e151c None admin] REQUEST_METHOD: `PUT` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:35.802816 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae44d2c-a195-498e-835c-70b74a0e151c None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:35.803106 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae44d2c-a195-498e-835c-70b74a0e151c None admin] PATH_INFO: `/v3/projects/252d9673f08c479a98d04fee3f5856a4/users/59ec84a77cb449a1a22c86ac9873b8af/roles/b0b74fefb27e4a0ebd895893aa9862d7` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:35.820963 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae44d2c-a195-498e-835c-70b74a0e151c None admin] RBAC: Authorizing `identity:create_grant(project_id=252d9673f08c479a98d04fee3f5856a4, user_id=59ec84a77cb449a1a22c86ac9873b8af, role_id=b0b74fefb27e4a0ebd895893aa9862d7)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:27:35.822544 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae44d2c-a195-498e-835c-70b74a0e151c None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:27:35.846668 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 615/1232] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 14:27:35 2022] PUT /identity/v3/projects/252d9673f08c479a98d04fee3f5856a4/users/59ec84a77cb449a1a22c86ac9873b8af/roles/b0b74fefb27e4a0ebd895893aa9862d7 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:27:38.158724 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daf62fa-189b-480a-8d00-99a223aa0c18 None None] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:38.159357 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daf62fa-189b-480a-8d00-99a223aa0c18 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:38.159357 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daf62fa-189b-480a-8d00-99a223aa0c18 None None] PATH_INFO: `/` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:38.160648 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 618/1233] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:27:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:27:38.166266 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2010f0bf-badd-41fd-af00-00689f6c6d1d None None] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:38.166704 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2010f0bf-badd-41fd-af00-00689f6c6d1d None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:38.166831 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2010f0bf-badd-41fd-af00-00689f6c6d1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:38.184261 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.auth.core [None req-2010f0bf-badd-41fd-af00-00689f6c6d1d None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143587) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:27:38.185491 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-2010f0bf-badd-41fd-af00-00689f6c6d1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:27:38.257519 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 616/1234] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:27:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:27:38.281468 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daf62fa-189b-480a-8d00-99a223aa0c18 None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:27:38.310741 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:27:38.399158 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daf62fa-189b-480a-8d00-99a223aa0c18 None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:27:38.438513 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f8ca17-96eb-41bd-a92d-5a52259368c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:27:38.439178 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f8ca17-96eb-41bd-a92d-5a52259368c9 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:38.439434 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f8ca17-96eb-41bd-a92d-5a52259368c9 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:38.439707 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f8ca17-96eb-41bd-a92d-5a52259368c9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:38.452327 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f8ca17-96eb-41bd-a92d-5a52259368c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:27:38.463241 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-34f8ca17-96eb-41bd-a92d-5a52259368c9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:27:38.463241 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:27:38.463241 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:27:38.463241 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:27:38.463241 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:27:38.463241 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:27:38.463241 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:27:38.463241 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:27:38.463241 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:27:38.463241 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:27:38.463241 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:27:38.463241 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:27:38.463241 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:27:38.463241 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:27:38.463241 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:27:38.463241 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:27:38.464216 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:27:38.464216 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:27:38.464216 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:27:38.464216 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:27:38.464216 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:27:38.464216 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:27:38.464216 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:27:38.464216 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:27:38.464216 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:27:38.464216 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:27:38.464216 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:27:38.464216 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:27:38.464216 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:27:38.464216 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:27:38.464216 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:27:38.464216 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:27:38.465058 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:27:38.465058 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:27:38.465058 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:27:38.465058 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:27:38.465058 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:27:38.465058 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:27:38.465058 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:27:38.465058 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:27:38.465058 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:27:38.465058 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:27:38.465058 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:27:38.465058 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:27:38.465058 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 619/1235] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 14:27:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:27:38.471147 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2010f0bf-badd-41fd-af00-00689f6c6d1d None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:27:38.533384 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:27:38.605593 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2010f0bf-badd-41fd-af00-00689f6c6d1d None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:27:38.647625 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794520c3-b3df-46e4-afa8-c23478389c2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:27:38.648432 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794520c3-b3df-46e4-afa8-c23478389c2a None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:38.648930 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794520c3-b3df-46e4-afa8-c23478389c2a None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:38.649401 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794520c3-b3df-46e4-afa8-c23478389c2a None admin] PATH_INFO: `/v3/roles` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:38.650424 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794520c3-b3df-46e4-afa8-c23478389c2a None admin] RBAC: Adding query filter params (name=service) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:27:38.650759 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794520c3-b3df-46e4-afa8-c23478389c2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:27:38.652731 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794520c3-b3df-46e4-afa8-c23478389c2a None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:27:38.663839 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 617/1236] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 19 14:27:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:27:38.669991 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f8ca17-96eb-41bd-a92d-5a52259368c9 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:27:38.761160 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:27:38.781994 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f8ca17-96eb-41bd-a92d-5a52259368c9 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:27:38.821213 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31e65b6-a1e2-4a13-9852-540f89bd9c99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:27:38.822042 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31e65b6-a1e2-4a13-9852-540f89bd9c99 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:38.822369 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31e65b6-a1e2-4a13-9852-540f89bd9c99 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:38.822680 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31e65b6-a1e2-4a13-9852-540f89bd9c99 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:38.835032 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31e65b6-a1e2-4a13-9852-540f89bd9c99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:27:38.836516 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31e65b6-a1e2-4a13-9852-540f89bd9c99 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:27:38.856615 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-b31e65b6-a1e2-4a13-9852-540f89bd9c99 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:27:38.856615 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:27:38.856615 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:27:38.856615 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:27:38.856615 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:27:38.856615 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:27:38.856615 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:27:38.856615 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:27:38.856615 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:27:38.856615 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:27:38.856615 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:27:38.856615 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:27:38.856615 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:27:38.856615 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:27:38.856615 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:27:38.857771 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:27:38.857771 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:27:38.857771 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:27:38.857771 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:27:38.857771 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:27:38.857771 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:27:38.857771 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:27:38.857771 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:27:38.857771 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:27:38.857771 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:27:38.857771 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:27:38.857771 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:27:38.857771 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:27:38.857771 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:27:38.857771 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:27:38.857771 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:27:38.857771 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:27:38.858723 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:27:38.858723 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:27:38.858723 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:27:38.858723 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:27:38.858723 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 620/1237] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:27:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:27:38.864614 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794520c3-b3df-46e4-afa8-c23478389c2a None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:27:38.975677 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794520c3-b3df-46e4-afa8-c23478389c2a None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:27:38.985114 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:27:39.013624 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f499cc-fe64-4c4d-847a-7a8baa86763d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:27:39.014279 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f499cc-fe64-4c4d-847a-7a8baa86763d None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:39.014625 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f499cc-fe64-4c4d-847a-7a8baa86763d None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:39.014877 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f499cc-fe64-4c4d-847a-7a8baa86763d None admin] PATH_INFO: `/v3/domains` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:39.015675 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f499cc-fe64-4c4d-847a-7a8baa86763d None admin] RBAC: Adding query filter params (name=Default) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:27:39.016056 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f499cc-fe64-4c4d-847a-7a8baa86763d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:27:39.017482 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f499cc-fe64-4c4d-847a-7a8baa86763d None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:27:39.029830 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 618/1238] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:27:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:27:39.037003 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31e65b6-a1e2-4a13-9852-540f89bd9c99 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:27:39.144769 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31e65b6-a1e2-4a13-9852-540f89bd9c99 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:27:39.184461 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6446d3b2-12fa-4580-81f9-4a9769e4482b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:27:39.185266 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6446d3b2-12fa-4580-81f9-4a9769e4482b None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:39.185553 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6446d3b2-12fa-4580-81f9-4a9769e4482b None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:39.185833 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6446d3b2-12fa-4580-81f9-4a9769e4482b None admin] PATH_INFO: `/v3/users/placement` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:39.188573 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:27:39.194619 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6446d3b2-12fa-4580-81f9-4a9769e4482b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:27:39.196279 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6446d3b2-12fa-4580-81f9-4a9769e4482b None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:27:39.212425 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-6446d3b2-12fa-4580-81f9-4a9769e4482b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 14:27:39.212425 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:27:39.212425 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:27:39.212425 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:27:39.212425 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:27:39.212425 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:27:39.212425 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:27:39.212425 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:27:39.212425 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:27:39.212425 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:27:39.212425 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:27:39.212425 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:27:39.212425 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:27:39.212425 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:27:39.212425 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:27:39.212425 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:27:39.213454 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:27:39.213454 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:27:39.213454 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:27:39.213454 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:27:39.213454 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:27:39.213454 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:27:39.213454 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:27:39.213454 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:27:39.213454 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:27:39.213454 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:27:39.213454 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:27:39.213454 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:27:39.213454 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:27:39.213454 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:27:39.213454 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:27:39.213454 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:27:39.213454 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:27:39.213454 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:27:39.214404 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:27:39.214404 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:27:39.214404 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:27:39.214404 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:27:39.214404 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:27:39.214404 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:27:39.214404 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:27:39.214404 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:27:39.214404 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 14:27:39.214404 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:27:39.214404 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 621/1239] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:27:39 2022] GET /identity/v3/users/placement => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:27:39.220007 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f499cc-fe64-4c4d-847a-7a8baa86763d None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:27:39.339685 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f499cc-fe64-4c4d-847a-7a8baa86763d None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:27:39.382273 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf82872-a8ef-44e7-8ac1-624cf48deb34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:27:39.382934 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf82872-a8ef-44e7-8ac1-624cf48deb34 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:39.383298 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf82872-a8ef-44e7-8ac1-624cf48deb34 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:39.383701 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf82872-a8ef-44e7-8ac1-624cf48deb34 None admin] PATH_INFO: `/v3/users` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:39.384886 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf82872-a8ef-44e7-8ac1-624cf48deb34 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:27:39.385292 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf82872-a8ef-44e7-8ac1-624cf48deb34 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:27:39.387153 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf82872-a8ef-44e7-8ac1-624cf48deb34 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:27:39.412125 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 619/1240] 10.222.0.57 () {56 vars in 1143 bytes} [Thu May 19 14:27:39 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:27:39.418965 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6446d3b2-12fa-4580-81f9-4a9769e4482b None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:27:39.543494 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6446d3b2-12fa-4580-81f9-4a9769e4482b None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:27:39.584046 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6530e75-c60f-4db3-955e-15f176a81693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:27:39.584702 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6530e75-c60f-4db3-955e-15f176a81693 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:39.584983 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6530e75-c60f-4db3-955e-15f176a81693 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:39.585257 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6530e75-c60f-4db3-955e-15f176a81693 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:39.653300 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:27:39.662063 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6530e75-c60f-4db3-955e-15f176a81693 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:27:39.663576 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6530e75-c60f-4db3-955e-15f176a81693 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:27:39.787597 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 622/1241] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:27:39 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:27:39.793833 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf82872-a8ef-44e7-8ac1-624cf48deb34 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:27:39.825863 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:27:39.915103 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf82872-a8ef-44e7-8ac1-624cf48deb34 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:27:39.960649 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b03ffcd-95bc-4c9c-9eb7-17da9c16abff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:27:39.961445 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b03ffcd-95bc-4c9c-9eb7-17da9c16abff None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:39.961733 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b03ffcd-95bc-4c9c-9eb7-17da9c16abff None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:39.962044 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b03ffcd-95bc-4c9c-9eb7-17da9c16abff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:39.973994 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b03ffcd-95bc-4c9c-9eb7-17da9c16abff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:27:39.975459 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b03ffcd-95bc-4c9c-9eb7-17da9c16abff None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:27:39.993161 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-1b03ffcd-95bc-4c9c-9eb7-17da9c16abff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:27:39.993161 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:27:39.993161 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:27:39.993161 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:27:39.993161 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:27:39.993161 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:27:39.993161 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:27:39.993161 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:27:39.993161 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:27:39.993161 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:27:39.993161 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:27:39.993161 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:27:39.993161 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:27:39.993161 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:27:39.993161 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:27:39.994371 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:27:39.994371 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:27:39.994371 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:27:39.994371 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:27:39.994371 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:27:39.994371 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:27:39.994371 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:27:39.994371 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:27:39.994371 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:27:39.994371 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:27:39.994371 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:27:39.994371 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:27:39.994371 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:27:39.994371 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:27:39.994371 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:27:39.994371 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:27:39.994371 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:27:39.995382 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:27:39.995382 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:27:39.995382 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:27:39.995382 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:27:39.995382 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 620/1242] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:27:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:27:40.000399 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6530e75-c60f-4db3-955e-15f176a81693 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:27:40.109890 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:27:40.117296 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6530e75-c60f-4db3-955e-15f176a81693 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:27:40.155633 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fd58c2-b2a2-41db-bec7-d46bd7b7f19e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:27:40.156301 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fd58c2-b2a2-41db-bec7-d46bd7b7f19e None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:40.156590 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fd58c2-b2a2-41db-bec7-d46bd7b7f19e None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:40.156860 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fd58c2-b2a2-41db-bec7-d46bd7b7f19e None admin] PATH_INFO: `/v3/domains` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:40.157600 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fd58c2-b2a2-41db-bec7-d46bd7b7f19e None admin] RBAC: Adding query filter params (name=Default) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:27:40.157938 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fd58c2-b2a2-41db-bec7-d46bd7b7f19e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:27:40.159907 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fd58c2-b2a2-41db-bec7-d46bd7b7f19e None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:27:40.172014 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 623/1243] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:27:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:27:40.178333 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b03ffcd-95bc-4c9c-9eb7-17da9c16abff None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:27:40.270426 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:27:40.295644 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b03ffcd-95bc-4c9c-9eb7-17da9c16abff None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:27:40.341087 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c29066c-7264-426a-af93-5b7511aac6b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:27:40.342099 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c29066c-7264-426a-af93-5b7511aac6b9 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:40.342359 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c29066c-7264-426a-af93-5b7511aac6b9 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:40.342731 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c29066c-7264-426a-af93-5b7511aac6b9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:40.350590 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c29066c-7264-426a-af93-5b7511aac6b9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:27:40.352049 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c29066c-7264-426a-af93-5b7511aac6b9 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:27:40.369219 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-7c29066c-7264-426a-af93-5b7511aac6b9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:27:40.369219 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:27:40.369219 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:27:40.369219 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:27:40.369219 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:27:40.369219 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:27:40.369219 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:27:40.369219 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:27:40.369219 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:27:40.369219 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:27:40.369219 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:27:40.369219 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:27:40.369219 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:27:40.369219 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:27:40.369219 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:27:40.376390 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:27:40.376390 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:27:40.376390 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:27:40.376390 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:27:40.376390 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:27:40.376390 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:27:40.376390 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:27:40.376390 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:27:40.376390 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:27:40.376390 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:27:40.376390 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:27:40.376390 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:27:40.376390 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:27:40.376390 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:27:40.376390 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:27:40.376390 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:27:40.376390 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:27:40.377285 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:27:40.377285 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:27:40.377285 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:27:40.377285 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:27:40.377285 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:27:40.377285 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:27:40.377285 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:27:40.377285 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:27:40.377285 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 621/1244] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:27:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:27:40.378405 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fd58c2-b2a2-41db-bec7-d46bd7b7f19e None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:27:40.505641 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fd58c2-b2a2-41db-bec7-d46bd7b7f19e None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:27:40.543428 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8f593a-f322-4198-b854-f51f497e44f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:27:40.544282 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8f593a-f322-4198-b854-f51f497e44f3 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:40.544578 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8f593a-f322-4198-b854-f51f497e44f3 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:40.544865 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8f593a-f322-4198-b854-f51f497e44f3 None admin] PATH_INFO: `/v3/projects` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:40.545911 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8f593a-f322-4198-b854-f51f497e44f3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:27:40.546264 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8f593a-f322-4198-b854-f51f497e44f3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:27:40.547827 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8f593a-f322-4198-b854-f51f497e44f3 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:27:40.566704 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 624/1245] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:27:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:27:40.568123 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c29066c-7264-426a-af93-5b7511aac6b9 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:27:40.689119 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c29066c-7264-426a-af93-5b7511aac6b9 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:27:40.730629 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14a94a1-3e75-4c11-93ec-1d154f67915e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:27:40.731358 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14a94a1-3e75-4c11-93ec-1d154f67915e None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:40.731748 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14a94a1-3e75-4c11-93ec-1d154f67915e None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:40.732112 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14a94a1-3e75-4c11-93ec-1d154f67915e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:40.733147 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14a94a1-3e75-4c11-93ec-1d154f67915e None admin] RBAC: Adding query filter params (role.id=b0b74fefb27e4a0ebd895893aa9862d7, scope.project.id=252d9673f08c479a98d04fee3f5856a4, user.id=59ec84a77cb449a1a22c86ac9873b8af) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:27:40.733526 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14a94a1-3e75-4c11-93ec-1d154f67915e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:27:40.735312 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14a94a1-3e75-4c11-93ec-1d154f67915e None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:27:40.742851 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 622/1246] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:27:40 2022] GET /identity/v3/role_assignments?user.id=59ec84a77cb449a1a22c86ac9873b8af&scope.project.id=252d9673f08c479a98d04fee3f5856a4&role.id=b0b74fefb27e4a0ebd895893aa9862d7 => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:27:43.344983 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a544c0-6552-453d-9ae0-76043a05857a None None] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:43.345531 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a544c0-6552-453d-9ae0-76043a05857a None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:43.345531 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a544c0-6552-453d-9ae0-76043a05857a None None] PATH_INFO: `/` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:43.348092 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 625/1247] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:27:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:27:43.354461 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cfe02c-b8a6-4a37-be0e-d67e538e0bce None None] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:43.354892 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cfe02c-b8a6-4a37-be0e-d67e538e0bce None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:43.355109 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cfe02c-b8a6-4a37-be0e-d67e538e0bce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:43.360763 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:27:43.376428 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.auth.core [None req-31cfe02c-b8a6-4a37-be0e-d67e538e0bce None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143587) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:27:43.378490 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-31cfe02c-b8a6-4a37-be0e-d67e538e0bce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:27:43.458081 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 623/1248] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:27:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:27:43.484384 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a544c0-6552-453d-9ae0-76043a05857a None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:27:43.505352 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:27:43.599500 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a544c0-6552-453d-9ae0-76043a05857a None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:27:43.637687 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d8f544-cd69-405f-be0f-9a9c88af8957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:27:43.638513 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d8f544-cd69-405f-be0f-9a9c88af8957 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:43.638796 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d8f544-cd69-405f-be0f-9a9c88af8957 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:43.639073 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d8f544-cd69-405f-be0f-9a9c88af8957 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:43.652437 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d8f544-cd69-405f-be0f-9a9c88af8957 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:27:43.660625 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-c6d8f544-cd69-405f-be0f-9a9c88af8957 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:27:43.660625 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:27:43.660625 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:27:43.660625 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:27:43.660625 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:27:43.660625 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:27:43.660625 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:27:43.660625 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:27:43.660625 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:27:43.660625 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:27:43.660625 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:27:43.660625 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:27:43.660625 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:27:43.660625 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:27:43.660625 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:27:43.660625 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:27:43.661617 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:27:43.661617 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:27:43.661617 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:27:43.661617 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:27:43.661617 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:27:43.661617 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:27:43.661617 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:27:43.661617 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:27:43.661617 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:27:43.661617 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:27:43.661617 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:27:43.661617 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:27:43.661617 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:27:43.661617 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:27:43.661617 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:27:43.661617 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:27:43.662457 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:27:43.662457 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:27:43.662457 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:27:43.662457 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:27:43.662457 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:27:43.662457 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:27:43.662457 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:27:43.662457 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:27:43.662457 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:27:43.662457 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:27:43.662457 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:27:43.662457 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:27:43.662457 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 626/1249] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:27:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:27:43.668496 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cfe02c-b8a6-4a37-be0e-d67e538e0bce None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:27:43.787884 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cfe02c-b8a6-4a37-be0e-d67e538e0bce None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:27:43.820051 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:27:43.826662 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236990c4-21df-4bd3-afe7-ebcd89da8a83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:27:43.827479 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236990c4-21df-4bd3-afe7-ebcd89da8a83 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:43.827935 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236990c4-21df-4bd3-afe7-ebcd89da8a83 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:43.828327 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236990c4-21df-4bd3-afe7-ebcd89da8a83 None admin] PATH_INFO: `/v3/roles` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:43.829095 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236990c4-21df-4bd3-afe7-ebcd89da8a83 None admin] RBAC: Adding query filter params (name=admin) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:27:43.829510 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236990c4-21df-4bd3-afe7-ebcd89da8a83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:27:43.831301 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236990c4-21df-4bd3-afe7-ebcd89da8a83 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:27:43.841800 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 624/1250] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:27:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:27:43.849734 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d8f544-cd69-405f-be0f-9a9c88af8957 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:27:43.943108 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:27:43.973184 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d8f544-cd69-405f-be0f-9a9c88af8957 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:27:44.019698 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4c3090-efc0-48c1-bdb5-f7740cdf30ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:27:44.020495 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4c3090-efc0-48c1-bdb5-f7740cdf30ad None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:44.020776 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4c3090-efc0-48c1-bdb5-f7740cdf30ad None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:44.021049 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4c3090-efc0-48c1-bdb5-f7740cdf30ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:44.033812 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4c3090-efc0-48c1-bdb5-f7740cdf30ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:27:44.036355 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4c3090-efc0-48c1-bdb5-f7740cdf30ad None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:27:44.057043 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-9a4c3090-efc0-48c1-bdb5-f7740cdf30ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:27:44.057043 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:27:44.057043 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:27:44.057043 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:27:44.057043 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:27:44.057043 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:27:44.057043 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:27:44.057043 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:27:44.057043 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:27:44.057043 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:27:44.057043 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:27:44.057043 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:27:44.057043 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:27:44.057043 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:27:44.057043 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:27:44.058221 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:27:44.058221 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:27:44.058221 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:27:44.058221 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:27:44.058221 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:27:44.058221 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:27:44.058221 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:27:44.058221 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:27:44.058221 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:27:44.058221 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:27:44.058221 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:27:44.058221 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:27:44.058221 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:27:44.058221 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:27:44.058221 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:27:44.058221 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:27:44.058221 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:27:44.059285 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:27:44.059285 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:27:44.059285 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:27:44.059285 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:27:44.059285 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 627/1251] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:27:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:27:44.064265 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236990c4-21df-4bd3-afe7-ebcd89da8a83 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:27:44.181164 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236990c4-21df-4bd3-afe7-ebcd89da8a83 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:27:44.222330 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec00974-9d83-4984-be81-e947bef7ef96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:27:44.223356 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec00974-9d83-4984-be81-e947bef7ef96 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:44.223641 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec00974-9d83-4984-be81-e947bef7ef96 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:44.224099 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec00974-9d83-4984-be81-e947bef7ef96 None admin] PATH_INFO: `/v3/domains` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:44.224919 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec00974-9d83-4984-be81-e947bef7ef96 None admin] RBAC: Adding query filter params (name=Default) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:27:44.225233 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec00974-9d83-4984-be81-e947bef7ef96 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:27:44.226648 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec00974-9d83-4984-be81-e947bef7ef96 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:27:44.240319 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 625/1252] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:27:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:27:44.246232 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4c3090-efc0-48c1-bdb5-f7740cdf30ad None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:27:44.368462 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4c3090-efc0-48c1-bdb5-f7740cdf30ad None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:27:44.407606 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:27:44.418931 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a9f780-af54-492f-8f1e-88003c66d3da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:27:44.419836 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a9f780-af54-492f-8f1e-88003c66d3da None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:44.420107 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a9f780-af54-492f-8f1e-88003c66d3da None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:44.420395 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a9f780-af54-492f-8f1e-88003c66d3da None admin] PATH_INFO: `/v3/users/placement` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:44.432287 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a9f780-af54-492f-8f1e-88003c66d3da None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:27:44.433682 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a9f780-af54-492f-8f1e-88003c66d3da None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:27:44.450890 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-a0a9f780-af54-492f-8f1e-88003c66d3da None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 14:27:44.450890 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:27:44.450890 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:27:44.450890 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:27:44.450890 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:27:44.450890 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:27:44.450890 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:27:44.450890 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:27:44.450890 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:27:44.450890 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:27:44.450890 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:27:44.450890 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:27:44.450890 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:27:44.450890 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:27:44.450890 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:27:44.450890 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:27:44.452027 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:27:44.452027 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:27:44.452027 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:27:44.452027 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:27:44.452027 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:27:44.452027 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:27:44.452027 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:27:44.452027 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:27:44.452027 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:27:44.452027 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:27:44.452027 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:27:44.452027 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:27:44.452027 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:27:44.452027 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:27:44.452027 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:27:44.452027 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:27:44.452027 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:27:44.452027 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:27:44.458563 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:27:44.458563 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:27:44.458563 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:27:44.458563 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:27:44.458563 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:27:44.458563 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:27:44.458563 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:27:44.458563 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:27:44.458563 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 14:27:44.458563 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:27:44.458563 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 628/1253] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:27:44 2022] GET /identity/v3/users/placement => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:27:44.459359 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec00974-9d83-4984-be81-e947bef7ef96 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:27:44.530395 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:27:44.583538 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec00974-9d83-4984-be81-e947bef7ef96 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:27:44.620294 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9015fac4-146f-4783-8f35-05acbc1ea341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:27:44.620967 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9015fac4-146f-4783-8f35-05acbc1ea341 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:44.621246 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9015fac4-146f-4783-8f35-05acbc1ea341 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:44.621546 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9015fac4-146f-4783-8f35-05acbc1ea341 None admin] PATH_INFO: `/v3/users` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:44.622428 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9015fac4-146f-4783-8f35-05acbc1ea341 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:27:44.622737 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9015fac4-146f-4783-8f35-05acbc1ea341 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:27:44.624184 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9015fac4-146f-4783-8f35-05acbc1ea341 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:27:44.643087 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 626/1254] 10.222.0.57 () {56 vars in 1143 bytes} [Thu May 19 14:27:44 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:27:44.650086 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a9f780-af54-492f-8f1e-88003c66d3da None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:27:44.777885 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a9f780-af54-492f-8f1e-88003c66d3da None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:27:44.818491 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be690b75-6d6e-4d44-bd14-c686cc6b6093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:27:44.819213 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be690b75-6d6e-4d44-bd14-c686cc6b6093 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:44.819492 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be690b75-6d6e-4d44-bd14-c686cc6b6093 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:44.819958 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be690b75-6d6e-4d44-bd14-c686cc6b6093 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:44.890103 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:27:44.905442 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be690b75-6d6e-4d44-bd14-c686cc6b6093 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:27:44.907005 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be690b75-6d6e-4d44-bd14-c686cc6b6093 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:27:45.043389 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 629/1255] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:27:44 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:27:45.048903 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9015fac4-146f-4783-8f35-05acbc1ea341 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:27:45.182651 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9015fac4-146f-4783-8f35-05acbc1ea341 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:27:45.198179 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:27:45.224210 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33590047-4898-4118-bdf0-f3119c9151ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:27:45.225008 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33590047-4898-4118-bdf0-f3119c9151ee None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:45.225291 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33590047-4898-4118-bdf0-f3119c9151ee None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:45.225566 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33590047-4898-4118-bdf0-f3119c9151ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:45.236624 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33590047-4898-4118-bdf0-f3119c9151ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:27:45.238365 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33590047-4898-4118-bdf0-f3119c9151ee None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:27:45.257329 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-33590047-4898-4118-bdf0-f3119c9151ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:27:45.257329 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:27:45.257329 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:27:45.257329 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:27:45.257329 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:27:45.257329 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:27:45.257329 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:27:45.257329 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:27:45.257329 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:27:45.257329 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:27:45.257329 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:27:45.257329 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:27:45.257329 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:27:45.257329 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:27:45.257329 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:27:45.258473 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:27:45.258473 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:27:45.258473 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:27:45.258473 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:27:45.258473 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:27:45.258473 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:27:45.258473 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:27:45.258473 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:27:45.258473 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:27:45.258473 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:27:45.258473 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:27:45.258473 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:27:45.258473 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:27:45.258473 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:27:45.258473 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:27:45.258473 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:27:45.258473 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:27:45.259401 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:27:45.259401 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:27:45.259401 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:27:45.259401 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:27:45.266135 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 627/1256] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:27:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:27:45.272726 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be690b75-6d6e-4d44-bd14-c686cc6b6093 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:27:45.385007 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:27:45.401595 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be690b75-6d6e-4d44-bd14-c686cc6b6093 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:27:45.447900 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78beb3a1-6785-44bc-a8ba-6fc3256ff595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:27:45.448568 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78beb3a1-6785-44bc-a8ba-6fc3256ff595 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:45.448852 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78beb3a1-6785-44bc-a8ba-6fc3256ff595 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:45.449132 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78beb3a1-6785-44bc-a8ba-6fc3256ff595 None admin] PATH_INFO: `/v3/domains` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:45.449872 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78beb3a1-6785-44bc-a8ba-6fc3256ff595 None admin] RBAC: Adding query filter params (name=Default) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:27:45.450190 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78beb3a1-6785-44bc-a8ba-6fc3256ff595 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:27:45.451701 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78beb3a1-6785-44bc-a8ba-6fc3256ff595 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:27:45.464189 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 630/1257] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:27:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:27:45.470507 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33590047-4898-4118-bdf0-f3119c9151ee None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:27:45.582875 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33590047-4898-4118-bdf0-f3119c9151ee None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:27:45.624114 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190ed43b-83e1-41d1-bf5e-6ba8ce00cbdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:27:45.624910 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190ed43b-83e1-41d1-bf5e-6ba8ce00cbdc None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:45.625192 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190ed43b-83e1-41d1-bf5e-6ba8ce00cbdc None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:45.625465 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190ed43b-83e1-41d1-bf5e-6ba8ce00cbdc None admin] PATH_INFO: `/v3/projects/service` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:45.634035 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190ed43b-83e1-41d1-bf5e-6ba8ce00cbdc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:27:45.635590 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190ed43b-83e1-41d1-bf5e-6ba8ce00cbdc None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:27:45.650587 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-190ed43b-83e1-41d1-bf5e-6ba8ce00cbdc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:27:45.650587 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:27:45.650587 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:27:45.650587 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:27:45.650587 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:27:45.650587 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:27:45.650587 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:27:45.650587 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:27:45.650587 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:27:45.650587 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:27:45.650587 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:27:45.650587 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:27:45.650587 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:27:45.650587 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:27:45.650587 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:27:45.653266 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:27:45.653266 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:27:45.653266 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:27:45.653266 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:27:45.653266 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:27:45.653266 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:27:45.653266 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:27:45.653266 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:27:45.653266 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:27:45.653266 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:27:45.653266 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:27:45.653266 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:27:45.653266 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:27:45.653266 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:27:45.653266 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:27:45.653266 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:27:45.653266 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:27:45.654322 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:27:45.654322 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:27:45.654322 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:27:45.654322 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:27:45.654322 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:27:45.654322 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:27:45.654322 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:27:45.654322 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:27:45.654322 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 628/1258] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:27:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:27:45.657868 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78beb3a1-6785-44bc-a8ba-6fc3256ff595 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:27:45.766364 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78beb3a1-6785-44bc-a8ba-6fc3256ff595 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:27:45.803205 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc80f23b-405e-4d42-832f-2b731f98d9c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:27:45.804069 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc80f23b-405e-4d42-832f-2b731f98d9c4 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:45.804438 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc80f23b-405e-4d42-832f-2b731f98d9c4 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:45.804791 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc80f23b-405e-4d42-832f-2b731f98d9c4 None admin] PATH_INFO: `/v3/projects` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:45.805715 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc80f23b-405e-4d42-832f-2b731f98d9c4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:27:45.806139 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc80f23b-405e-4d42-832f-2b731f98d9c4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:27:45.807689 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc80f23b-405e-4d42-832f-2b731f98d9c4 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:27:45.821322 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 631/1259] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:27:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:27:45.827666 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190ed43b-83e1-41d1-bf5e-6ba8ce00cbdc None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:27:45.856737 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:27:45.938219 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190ed43b-83e1-41d1-bf5e-6ba8ce00cbdc None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:27:45.980863 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46db8c3-0732-40ef-9ca0-f96e994cbf81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:27:45.981617 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46db8c3-0732-40ef-9ca0-f96e994cbf81 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:45.981967 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46db8c3-0732-40ef-9ca0-f96e994cbf81 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:45.982278 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46db8c3-0732-40ef-9ca0-f96e994cbf81 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:45.983251 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46db8c3-0732-40ef-9ca0-f96e994cbf81 None admin] RBAC: Adding query filter params (role.id=a3b793754e7b40b0bb21afe74eec3c18, scope.project.id=252d9673f08c479a98d04fee3f5856a4, user.id=59ec84a77cb449a1a22c86ac9873b8af) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:27:45.983595 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46db8c3-0732-40ef-9ca0-f96e994cbf81 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:27:45.985046 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46db8c3-0732-40ef-9ca0-f96e994cbf81 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:27:45.991604 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 629/1260] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:27:45 2022] GET /identity/v3/role_assignments?user.id=59ec84a77cb449a1a22c86ac9873b8af&scope.project.id=252d9673f08c479a98d04fee3f5856a4&role.id=a3b793754e7b40b0bb21afe74eec3c18 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:27:48.237754 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3048a639-60e6-4aae-ba21-a1d0650f23aa None None] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:48.238303 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3048a639-60e6-4aae-ba21-a1d0650f23aa None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:48.238429 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3048a639-60e6-4aae-ba21-a1d0650f23aa None None] PATH_INFO: `/` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:48.239676 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 632/1261] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:27:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:27:48.244910 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52d6a10-016a-4839-8edb-6337a50e48f2 None None] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:48.245337 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52d6a10-016a-4839-8edb-6337a50e48f2 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:48.245482 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52d6a10-016a-4839-8edb-6337a50e48f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:48.262343 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.auth.core [None req-e52d6a10-016a-4839-8edb-6337a50e48f2 None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143587) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:27:48.263529 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-e52d6a10-016a-4839-8edb-6337a50e48f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:27:48.329668 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 630/1262] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:27:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:27:48.352470 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3048a639-60e6-4aae-ba21-a1d0650f23aa None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:27:48.365345 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:27:48.475027 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3048a639-60e6-4aae-ba21-a1d0650f23aa None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:27:48.508351 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5f0ff7-db1b-4041-9c03-a03f36288aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:27:48.509069 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5f0ff7-db1b-4041-9c03-a03f36288aed None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:48.509351 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5f0ff7-db1b-4041-9c03-a03f36288aed None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:48.509626 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5f0ff7-db1b-4041-9c03-a03f36288aed None admin] PATH_INFO: `/v3/roles/admin` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:48.519751 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5f0ff7-db1b-4041-9c03-a03f36288aed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:27:48.527221 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-5b5f0ff7-db1b-4041-9c03-a03f36288aed None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:27:48.527221 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:27:48.527221 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:27:48.527221 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:27:48.527221 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:27:48.527221 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:27:48.527221 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:27:48.527221 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:27:48.527221 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:27:48.527221 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:27:48.527221 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:27:48.527221 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:27:48.527221 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:27:48.527221 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:27:48.527221 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:27:48.527221 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:27:48.528284 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:27:48.528284 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:27:48.528284 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:27:48.528284 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:27:48.528284 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:27:48.528284 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:27:48.528284 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:27:48.528284 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:27:48.528284 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:27:48.528284 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:27:48.528284 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:27:48.528284 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:27:48.528284 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:27:48.528284 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:27:48.528284 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:27:48.528284 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:27:48.529147 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:27:48.529147 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:27:48.529147 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:27:48.529147 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:27:48.529147 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:27:48.529147 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:27:48.529147 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:27:48.529147 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:27:48.529147 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:27:48.529147 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:27:48.529147 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:27:48.529147 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:27:48.529147 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 633/1263] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:27:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:27:48.535177 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52d6a10-016a-4839-8edb-6337a50e48f2 None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:27:48.554011 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:27:48.633572 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52d6a10-016a-4839-8edb-6337a50e48f2 None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:27:48.667352 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe216b1c-b316-4454-874d-c24226471e84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:27:48.668019 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe216b1c-b316-4454-874d-c24226471e84 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:48.668301 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe216b1c-b316-4454-874d-c24226471e84 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:48.668579 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe216b1c-b316-4454-874d-c24226471e84 None admin] PATH_INFO: `/v3/roles` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:48.669326 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe216b1c-b316-4454-874d-c24226471e84 None admin] RBAC: Adding query filter params (name=admin) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:27:48.669635 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe216b1c-b316-4454-874d-c24226471e84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:27:48.671052 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe216b1c-b316-4454-874d-c24226471e84 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:27:48.679804 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 631/1264] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:27:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:27:48.685555 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5f0ff7-db1b-4041-9c03-a03f36288aed None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:27:48.752297 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:27:48.785337 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5f0ff7-db1b-4041-9c03-a03f36288aed None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:27:48.818895 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bae422-dbe2-4620-a8a0-a7edf0399456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:27:48.819672 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bae422-dbe2-4620-a8a0-a7edf0399456 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:48.819950 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bae422-dbe2-4620-a8a0-a7edf0399456 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:48.820223 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bae422-dbe2-4620-a8a0-a7edf0399456 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:48.830633 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bae422-dbe2-4620-a8a0-a7edf0399456 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:27:48.832354 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bae422-dbe2-4620-a8a0-a7edf0399456 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:27:48.849768 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-d5bae422-dbe2-4620-a8a0-a7edf0399456 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:27:48.849768 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:27:48.849768 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:27:48.849768 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:27:48.849768 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:27:48.849768 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:27:48.849768 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:27:48.849768 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:27:48.849768 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:27:48.849768 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:27:48.849768 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:27:48.849768 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:27:48.849768 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:27:48.849768 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:27:48.849768 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:27:48.850787 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:27:48.850787 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:27:48.850787 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:27:48.850787 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:27:48.850787 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:27:48.850787 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:27:48.850787 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:27:48.850787 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:27:48.850787 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:27:48.850787 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:27:48.850787 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:27:48.850787 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:27:48.850787 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:27:48.850787 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:27:48.850787 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:27:48.850787 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:27:48.850787 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:27:48.851885 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:27:48.851885 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:27:48.851885 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:27:48.851885 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:27:48.851885 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 634/1265] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:27:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:27:48.856692 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe216b1c-b316-4454-874d-c24226471e84 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:27:48.941637 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:27:48.960765 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe216b1c-b316-4454-874d-c24226471e84 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:27:48.993655 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4503ae2-d896-43db-beeb-4e58322cff54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:27:48.994345 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4503ae2-d896-43db-beeb-4e58322cff54 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:48.994630 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4503ae2-d896-43db-beeb-4e58322cff54 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:48.994906 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4503ae2-d896-43db-beeb-4e58322cff54 None admin] PATH_INFO: `/v3/domains` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:48.995627 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4503ae2-d896-43db-beeb-4e58322cff54 None admin] RBAC: Adding query filter params (name=Default) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:27:48.995937 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4503ae2-d896-43db-beeb-4e58322cff54 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:27:48.997312 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4503ae2-d896-43db-beeb-4e58322cff54 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:27:49.009653 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 632/1266] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:27:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:27:49.015343 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bae422-dbe2-4620-a8a0-a7edf0399456 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:27:49.113983 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bae422-dbe2-4620-a8a0-a7edf0399456 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:27:49.132810 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:27:49.147082 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc227c08-fd73-42cc-a4c9-4bcda36ce2c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:27:49.147869 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc227c08-fd73-42cc-a4c9-4bcda36ce2c6 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:49.148150 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc227c08-fd73-42cc-a4c9-4bcda36ce2c6 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:49.148422 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc227c08-fd73-42cc-a4c9-4bcda36ce2c6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:49.155784 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc227c08-fd73-42cc-a4c9-4bcda36ce2c6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:27:49.157223 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc227c08-fd73-42cc-a4c9-4bcda36ce2c6 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:27:49.172324 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-fc227c08-fd73-42cc-a4c9-4bcda36ce2c6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 14:27:49.172324 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:27:49.172324 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:27:49.172324 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:27:49.172324 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:27:49.172324 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:27:49.172324 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:27:49.172324 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:27:49.172324 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:27:49.172324 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:27:49.172324 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:27:49.172324 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:27:49.172324 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:27:49.172324 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:27:49.172324 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:27:49.172324 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:27:49.173388 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:27:49.173388 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:27:49.173388 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:27:49.173388 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:27:49.173388 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:27:49.173388 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:27:49.173388 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:27:49.173388 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:27:49.173388 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:27:49.173388 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:27:49.173388 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:27:49.173388 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:27:49.173388 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:27:49.173388 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:27:49.173388 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:27:49.173388 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:27:49.173388 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:27:49.173388 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:27:49.174671 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:27:49.174671 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:27:49.174671 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:27:49.174671 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:27:49.174671 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:27:49.174671 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:27:49.174671 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:27:49.174671 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:27:49.174671 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 14:27:49.174671 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:27:49.174671 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 635/1267] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:27:49 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:27:49.180052 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4503ae2-d896-43db-beeb-4e58322cff54 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:27:49.279267 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4503ae2-d896-43db-beeb-4e58322cff54 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:27:49.313293 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc64f689-b7bd-45cb-80c4-312d346d0f4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:27:49.313982 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc64f689-b7bd-45cb-80c4-312d346d0f4f None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:49.314267 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc64f689-b7bd-45cb-80c4-312d346d0f4f None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:49.314543 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc64f689-b7bd-45cb-80c4-312d346d0f4f None admin] PATH_INFO: `/v3/users` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:49.315419 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc64f689-b7bd-45cb-80c4-312d346d0f4f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:27:49.315726 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc64f689-b7bd-45cb-80c4-312d346d0f4f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:27:49.317159 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc64f689-b7bd-45cb-80c4-312d346d0f4f None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:27:49.325553 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:27:49.334349 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 633/1268] 10.222.0.57 () {56 vars in 1143 bytes} [Thu May 19 14:27:49 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:27:49.339955 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc227c08-fd73-42cc-a4c9-4bcda36ce2c6 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:27:49.444190 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc227c08-fd73-42cc-a4c9-4bcda36ce2c6 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:27:49.477362 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3392c1d-f2a3-423d-ba37-8e87830256d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:27:49.478165 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3392c1d-f2a3-423d-ba37-8e87830256d5 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:49.478447 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3392c1d-f2a3-423d-ba37-8e87830256d5 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:49.478721 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3392c1d-f2a3-423d-ba37-8e87830256d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:49.489289 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3392c1d-f2a3-423d-ba37-8e87830256d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:27:49.490754 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3392c1d-f2a3-423d-ba37-8e87830256d5 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:27:49.507349 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-e3392c1d-f2a3-423d-ba37-8e87830256d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:27:49.507349 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:27:49.507349 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:27:49.507349 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:27:49.507349 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:27:49.507349 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:27:49.507349 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:27:49.507349 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:27:49.507349 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:27:49.507349 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:27:49.507349 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:27:49.507349 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:27:49.507349 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:27:49.507349 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:27:49.507349 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:27:49.508302 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:27:49.508302 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:27:49.508302 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:27:49.508302 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:27:49.508302 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:27:49.508302 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:27:49.508302 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:27:49.508302 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:27:49.508302 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:27:49.508302 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:27:49.508302 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:27:49.508302 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:27:49.508302 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:27:49.508302 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:27:49.508302 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:27:49.508302 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:27:49.508302 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:27:49.509321 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:27:49.509321 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:27:49.509321 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:27:49.509321 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:27:49.509988 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 636/1269] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:27:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:27:49.514185 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc64f689-b7bd-45cb-80c4-312d346d0f4f None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:27:49.614530 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc64f689-b7bd-45cb-80c4-312d346d0f4f None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:27:49.648571 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fb6f57-f56e-4229-bd4b-09e4683f6e0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:27:49.649215 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fb6f57-f56e-4229-bd4b-09e4683f6e0b None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:49.649493 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fb6f57-f56e-4229-bd4b-09e4683f6e0b None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:49.649765 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fb6f57-f56e-4229-bd4b-09e4683f6e0b None admin] PATH_INFO: `/v3/domains` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:49.650509 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fb6f57-f56e-4229-bd4b-09e4683f6e0b None admin] RBAC: Adding query filter params (name=Default) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:27:49.650811 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fb6f57-f56e-4229-bd4b-09e4683f6e0b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:27:49.652180 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fb6f57-f56e-4229-bd4b-09e4683f6e0b None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:27:49.665600 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 634/1270] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:27:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:27:49.671173 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3392c1d-f2a3-423d-ba37-8e87830256d5 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:27:49.705224 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:27:49.774585 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3392c1d-f2a3-423d-ba37-8e87830256d5 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:27:49.808293 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d911ae-ecc8-440f-996b-3fbd9dc6aead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:27:49.809073 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d911ae-ecc8-440f-996b-3fbd9dc6aead None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:49.809352 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d911ae-ecc8-440f-996b-3fbd9dc6aead None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:49.809624 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d911ae-ecc8-440f-996b-3fbd9dc6aead None admin] PATH_INFO: `/v3/projects/service` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:49.816679 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d911ae-ecc8-440f-996b-3fbd9dc6aead None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:27:49.818131 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d911ae-ecc8-440f-996b-3fbd9dc6aead None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:27:49.832237 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-f3d911ae-ecc8-440f-996b-3fbd9dc6aead None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:27:49.832237 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:27:49.832237 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:27:49.832237 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:27:49.832237 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:27:49.832237 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:27:49.832237 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:27:49.832237 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:27:49.832237 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:27:49.832237 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:27:49.832237 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:27:49.832237 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:27:49.832237 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:27:49.832237 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:27:49.832237 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:27:49.833220 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:27:49.833220 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:27:49.833220 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:27:49.833220 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:27:49.833220 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:27:49.833220 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:27:49.833220 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:27:49.833220 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:27:49.833220 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:27:49.833220 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:27:49.833220 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:27:49.833220 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:27:49.833220 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:27:49.833220 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:27:49.833220 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:27:49.833220 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:27:49.833220 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:27:49.834162 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:27:49.834162 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:27:49.834162 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:27:49.834162 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:27:49.834162 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:27:49.834162 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:27:49.834162 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:27:49.834162 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:27:49.834162 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 637/1271] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:27:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:27:49.839227 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fb6f57-f56e-4229-bd4b-09e4683f6e0b None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:27:49.897862 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:27:49.943267 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fb6f57-f56e-4229-bd4b-09e4683f6e0b None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:27:49.976890 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c26c3e-bb03-47c8-885a-b1d0f1c87c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:27:49.977468 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c26c3e-bb03-47c8-885a-b1d0f1c87c72 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:49.977746 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c26c3e-bb03-47c8-885a-b1d0f1c87c72 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:49.978045 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c26c3e-bb03-47c8-885a-b1d0f1c87c72 None admin] PATH_INFO: `/v3/projects` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:49.978819 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c26c3e-bb03-47c8-885a-b1d0f1c87c72 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:27:49.979127 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c26c3e-bb03-47c8-885a-b1d0f1c87c72 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:27:49.980556 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c26c3e-bb03-47c8-885a-b1d0f1c87c72 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:27:49.992875 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 635/1272] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:27:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:27:49.998866 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d911ae-ecc8-440f-996b-3fbd9dc6aead None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:27:50.088070 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:27:50.097142 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d911ae-ecc8-440f-996b-3fbd9dc6aead None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:27:50.130398 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01289be5-c0f3-48a8-896c-d383c181c311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:27:50.131275 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01289be5-c0f3-48a8-896c-d383c181c311 None admin] REQUEST_METHOD: `PUT` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:50.131556 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01289be5-c0f3-48a8-896c-d383c181c311 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:50.131835 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01289be5-c0f3-48a8-896c-d383c181c311 None admin] PATH_INFO: `/v3/projects/252d9673f08c479a98d04fee3f5856a4/users/59ec84a77cb449a1a22c86ac9873b8af/roles/a3b793754e7b40b0bb21afe74eec3c18` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:50.133759 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01289be5-c0f3-48a8-896c-d383c181c311 None admin] RBAC: Authorizing `identity:create_grant(project_id=252d9673f08c479a98d04fee3f5856a4, user_id=59ec84a77cb449a1a22c86ac9873b8af, role_id=a3b793754e7b40b0bb21afe74eec3c18)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:27:50.135206 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01289be5-c0f3-48a8-896c-d383c181c311 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:27:50.147814 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 638/1273] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 14:27:49 2022] PUT /identity/v3/projects/252d9673f08c479a98d04fee3f5856a4/users/59ec84a77cb449a1a22c86ac9873b8af/roles/a3b793754e7b40b0bb21afe74eec3c18 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 14:27:52.372401 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7880abdf-6978-414b-a2bc-ad12a1fc2b1c None None] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:52.372977 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7880abdf-6978-414b-a2bc-ad12a1fc2b1c None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:52.373361 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7880abdf-6978-414b-a2bc-ad12a1fc2b1c None None] PATH_INFO: `/` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:52.375553 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 636/1274] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:27:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:27:52.383138 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c857e077-9cd9-4730-841f-c05d62dcc7bb None None] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:52.383579 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c857e077-9cd9-4730-841f-c05d62dcc7bb None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:52.383730 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c857e077-9cd9-4730-841f-c05d62dcc7bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:52.404816 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.auth.core [None req-c857e077-9cd9-4730-841f-c05d62dcc7bb None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143586) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:27:52.408392 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.fernet_utils [None req-c857e077-9cd9-4730-841f-c05d62dcc7bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143586) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:27:52.474908 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 639/1275] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:27:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:27:52.498228 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7880abdf-6978-414b-a2bc-ad12a1fc2b1c None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:27:52.599043 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7880abdf-6978-414b-a2bc-ad12a1fc2b1c None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:27:52.615611 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:27:52.632270 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea08a11-9f13-4355-9525-bf972a57a507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:27:52.633050 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea08a11-9f13-4355-9525-bf972a57a507 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:52.633335 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea08a11-9f13-4355-9525-bf972a57a507 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:52.633615 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea08a11-9f13-4355-9525-bf972a57a507 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:52.644150 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea08a11-9f13-4355-9525-bf972a57a507 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:27:52.653349 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-7ea08a11-9f13-4355-9525-bf972a57a507 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:27:52.653349 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:27:52.653349 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:27:52.653349 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:27:52.653349 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:27:52.653349 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:27:52.653349 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:27:52.653349 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:27:52.653349 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:27:52.653349 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:27:52.653349 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:27:52.653349 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:27:52.653349 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:27:52.653349 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:27:52.653349 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:27:52.653349 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:27:52.654423 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:27:52.654423 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:27:52.654423 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:27:52.654423 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:27:52.654423 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:27:52.654423 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:27:52.654423 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:27:52.654423 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:27:52.654423 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:27:52.654423 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:27:52.654423 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:27:52.654423 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:27:52.654423 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:27:52.654423 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:27:52.654423 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:27:52.654423 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:27:52.655193 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:27:52.655193 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:27:52.655193 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:27:52.655193 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:27:52.655193 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:27:52.655193 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:27:52.655193 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:27:52.655193 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:27:52.655193 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:27:52.655193 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:27:52.655193 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:27:52.655193 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:27:52.655193 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 637/1276] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:27:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:27:52.660042 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c857e077-9cd9-4730-841f-c05d62dcc7bb None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:27:52.737415 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:27:52.756733 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c857e077-9cd9-4730-841f-c05d62dcc7bb None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:27:52.791419 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79dd0fd-59b6-4f9c-81fc-0ed4f5c29623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:27:52.792075 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79dd0fd-59b6-4f9c-81fc-0ed4f5c29623 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:52.792356 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79dd0fd-59b6-4f9c-81fc-0ed4f5c29623 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:52.792629 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79dd0fd-59b6-4f9c-81fc-0ed4f5c29623 None admin] PATH_INFO: `/v3/roles` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:52.793367 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79dd0fd-59b6-4f9c-81fc-0ed4f5c29623 None admin] RBAC: Adding query filter params (name=admin) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:27:52.793670 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79dd0fd-59b6-4f9c-81fc-0ed4f5c29623 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:27:52.795058 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79dd0fd-59b6-4f9c-81fc-0ed4f5c29623 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:27:52.803376 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 640/1277] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:27:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:27:52.809079 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea08a11-9f13-4355-9525-bf972a57a507 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:27:52.907952 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea08a11-9f13-4355-9525-bf972a57a507 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:27:52.940351 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7818420a-a13b-4993-9ec2-c11165e05f03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:27:52.941113 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7818420a-a13b-4993-9ec2-c11165e05f03 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:52.941396 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7818420a-a13b-4993-9ec2-c11165e05f03 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:52.941670 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7818420a-a13b-4993-9ec2-c11165e05f03 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:52.952673 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7818420a-a13b-4993-9ec2-c11165e05f03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:27:52.954162 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7818420a-a13b-4993-9ec2-c11165e05f03 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:27:52.972286 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-7818420a-a13b-4993-9ec2-c11165e05f03 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:27:52.972286 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:27:52.972286 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:27:52.972286 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:27:52.972286 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:27:52.972286 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:27:52.972286 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:27:52.972286 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:27:52.972286 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:27:52.972286 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:27:52.972286 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:27:52.972286 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:27:52.972286 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:27:52.972286 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:27:52.972286 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:27:52.973266 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:27:52.973266 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:27:52.973266 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:27:52.973266 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:27:52.973266 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:27:52.973266 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:27:52.973266 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:27:52.973266 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:27:52.973266 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:27:52.973266 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:27:52.973266 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:27:52.973266 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:27:52.973266 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:27:52.973266 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:27:52.973266 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:27:52.973266 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:27:52.973266 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:27:52.974274 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:27:52.974274 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:27:52.974274 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:27:52.974274 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:27:52.974274 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 638/1278] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:27:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:27:52.979207 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79dd0fd-59b6-4f9c-81fc-0ed4f5c29623 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:27:53.077206 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79dd0fd-59b6-4f9c-81fc-0ed4f5c29623 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:27:53.110841 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7978bce5-9e6b-4ac0-8993-1634d9ee9e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:27:53.111493 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7978bce5-9e6b-4ac0-8993-1634d9ee9e93 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:53.111927 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7978bce5-9e6b-4ac0-8993-1634d9ee9e93 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:53.112310 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7978bce5-9e6b-4ac0-8993-1634d9ee9e93 None admin] PATH_INFO: `/v3/domains` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:53.113032 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7978bce5-9e6b-4ac0-8993-1634d9ee9e93 None admin] RBAC: Adding query filter params (name=Default) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:27:53.113342 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7978bce5-9e6b-4ac0-8993-1634d9ee9e93 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:27:53.114729 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7978bce5-9e6b-4ac0-8993-1634d9ee9e93 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:27:53.126114 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 641/1279] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:27:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:27:53.132459 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7818420a-a13b-4993-9ec2-c11165e05f03 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:27:53.162996 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:27:53.231147 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7818420a-a13b-4993-9ec2-c11165e05f03 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:27:53.266522 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e64851e-2c92-4372-ad08-456efc173c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:27:53.267311 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e64851e-2c92-4372-ad08-456efc173c4d None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:53.267591 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e64851e-2c92-4372-ad08-456efc173c4d None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:53.267973 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e64851e-2c92-4372-ad08-456efc173c4d None admin] PATH_INFO: `/v3/users/placement` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:53.275250 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e64851e-2c92-4372-ad08-456efc173c4d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:27:53.276676 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e64851e-2c92-4372-ad08-456efc173c4d None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:27:53.290531 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-3e64851e-2c92-4372-ad08-456efc173c4d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 14:27:53.290531 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:27:53.290531 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:27:53.290531 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:27:53.290531 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:27:53.290531 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:27:53.290531 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:27:53.290531 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:27:53.290531 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:27:53.290531 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:27:53.290531 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:27:53.290531 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:27:53.290531 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:27:53.290531 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:27:53.290531 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:27:53.290531 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:27:53.291601 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:27:53.291601 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:27:53.291601 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:27:53.291601 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:27:53.291601 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:27:53.291601 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:27:53.291601 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:27:53.291601 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:27:53.291601 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:27:53.291601 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:27:53.291601 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:27:53.291601 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:27:53.291601 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:27:53.291601 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:27:53.291601 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:27:53.291601 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:27:53.291601 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:27:53.291601 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:27:53.292868 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:27:53.292868 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:27:53.292868 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:27:53.292868 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:27:53.292868 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:27:53.292868 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:27:53.292868 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:27:53.292868 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:27:53.292868 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 14:27:53.292868 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:27:53.292868 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 639/1280] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:27:53 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:27:53.297526 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7978bce5-9e6b-4ac0-8993-1634d9ee9e93 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:27:53.301899 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:27:53.404350 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7978bce5-9e6b-4ac0-8993-1634d9ee9e93 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:27:53.437574 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89a8252-2ced-4799-93df-d8170e7497da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:27:53.438201 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89a8252-2ced-4799-93df-d8170e7497da None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:53.438487 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89a8252-2ced-4799-93df-d8170e7497da None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:53.438762 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89a8252-2ced-4799-93df-d8170e7497da None admin] PATH_INFO: `/v3/users` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:53.439596 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89a8252-2ced-4799-93df-d8170e7497da None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:27:53.440432 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89a8252-2ced-4799-93df-d8170e7497da None admin] RBAC: Authorizing `identity:list_users()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:27:53.441973 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89a8252-2ced-4799-93df-d8170e7497da None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:27:53.458747 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 642/1281] 10.222.0.57 () {56 vars in 1143 bytes} [Thu May 19 14:27:53 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:27:53.464501 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e64851e-2c92-4372-ad08-456efc173c4d None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:27:53.554167 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:27:53.563917 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e64851e-2c92-4372-ad08-456efc173c4d None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:27:53.597422 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0402aebd-00e8-4cca-a585-9ce0e984c098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:27:53.598084 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0402aebd-00e8-4cca-a585-9ce0e984c098 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:53.598370 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0402aebd-00e8-4cca-a585-9ce0e984c098 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:53.598650 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0402aebd-00e8-4cca-a585-9ce0e984c098 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:53.668493 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0402aebd-00e8-4cca-a585-9ce0e984c098 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:27:53.670003 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0402aebd-00e8-4cca-a585-9ce0e984c098 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:27:53.757499 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:27:53.767506 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 640/1282] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:27:53 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:27:53.773349 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89a8252-2ced-4799-93df-d8170e7497da None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:27:53.834041 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:27:53.878987 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89a8252-2ced-4799-93df-d8170e7497da None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:27:53.912676 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e13cc4-5d24-4cf5-a011-bf5566e431e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:27:53.913462 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e13cc4-5d24-4cf5-a011-bf5566e431e1 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:53.913748 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e13cc4-5d24-4cf5-a011-bf5566e431e1 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:53.914055 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e13cc4-5d24-4cf5-a011-bf5566e431e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:53.923915 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e13cc4-5d24-4cf5-a011-bf5566e431e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:27:53.925567 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e13cc4-5d24-4cf5-a011-bf5566e431e1 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:27:53.942651 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-50e13cc4-5d24-4cf5-a011-bf5566e431e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:27:53.942651 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:27:53.942651 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:27:53.942651 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:27:53.942651 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:27:53.942651 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:27:53.942651 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:27:53.942651 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:27:53.942651 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:27:53.942651 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:27:53.942651 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:27:53.942651 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:27:53.942651 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:27:53.942651 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:27:53.942651 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:27:53.943625 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:27:53.943625 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:27:53.943625 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:27:53.943625 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:27:53.943625 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:27:53.943625 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:27:53.943625 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:27:53.943625 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:27:53.943625 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:27:53.943625 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:27:53.943625 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:27:53.943625 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:27:53.943625 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:27:53.943625 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:27:53.943625 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:27:53.943625 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:27:53.943625 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:27:53.944547 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:27:53.944547 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:27:53.944547 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:27:53.944547 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:27:53.944547 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 643/1283] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:27:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:27:53.949534 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0402aebd-00e8-4cca-a585-9ce0e984c098 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:27:54.046962 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0402aebd-00e8-4cca-a585-9ce0e984c098 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:27:54.082120 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c04509-ca3c-4508-9994-57a56d34e0fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:27:54.082763 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c04509-ca3c-4508-9994-57a56d34e0fc None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:54.083047 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c04509-ca3c-4508-9994-57a56d34e0fc None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:54.083324 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c04509-ca3c-4508-9994-57a56d34e0fc None admin] PATH_INFO: `/v3/domains` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:54.084036 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c04509-ca3c-4508-9994-57a56d34e0fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:27:54.084342 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c04509-ca3c-4508-9994-57a56d34e0fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:27:54.085708 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c04509-ca3c-4508-9994-57a56d34e0fc None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:27:54.096966 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 641/1284] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:27:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:27:54.102578 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e13cc4-5d24-4cf5-a011-bf5566e431e1 None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:27:54.205279 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e13cc4-5d24-4cf5-a011-bf5566e431e1 None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:27:54.219515 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:27:54.239596 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db7e809-9d3b-40d4-887c-077d60e8c0fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:27:54.240380 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db7e809-9d3b-40d4-887c-077d60e8c0fc None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:54.240659 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db7e809-9d3b-40d4-887c-077d60e8c0fc None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:54.240929 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db7e809-9d3b-40d4-887c-077d60e8c0fc None admin] PATH_INFO: `/v3/projects/service` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:54.247531 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db7e809-9d3b-40d4-887c-077d60e8c0fc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:27:54.249004 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db7e809-9d3b-40d4-887c-077d60e8c0fc None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:27:54.262643 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application [None req-8db7e809-9d3b-40d4-887c-077d60e8c0fc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:27:54.262643 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:27:54.262643 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:27:54.262643 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:27:54.262643 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:27:54.262643 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:27:54.262643 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:27:54.262643 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:27:54.262643 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:27:54.262643 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:27:54.262643 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:27:54.262643 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:27:54.262643 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:27:54.262643 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:27:54.262643 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:27:54.263640 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:27:54.263640 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:27:54.263640 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:27:54.263640 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:27:54.263640 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:27:54.263640 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:27:54.263640 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:27:54.263640 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:27:54.263640 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application with Lock( May 19 14:27:54.263640 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:27:54.263640 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._enter() May 19 14:27:54.263640 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:27:54.263640 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:27:54.263640 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:27:54.263640 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.creator() May 19 14:27:54.263640 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:27:54.263640 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application created_value = creator( May 19 14:27:54.264536 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:27:54.264536 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:27:54.264536 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:27:54.264536 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:27:54.264536 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:27:54.264536 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:27:54.264536 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:27:54.264536 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: ERROR keystone.server.flask.application May 19 14:27:54.264536 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 644/1285] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:27:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:27:54.270353 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c04509-ca3c-4508-9994-57a56d34e0fc None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:27:54.320008 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:27:54.373352 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c04509-ca3c-4508-9994-57a56d34e0fc None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:27:54.407650 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158d05e6-8959-466e-a3b4-66385966c1ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:27:54.408304 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158d05e6-8959-466e-a3b4-66385966c1ef None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:54.408607 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158d05e6-8959-466e-a3b4-66385966c1ef None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:54.408881 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158d05e6-8959-466e-a3b4-66385966c1ef None admin] PATH_INFO: `/v3/projects` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:54.409668 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158d05e6-8959-466e-a3b4-66385966c1ef None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:27:54.410008 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158d05e6-8959-466e-a3b4-66385966c1ef None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:27:54.411451 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158d05e6-8959-466e-a3b4-66385966c1ef None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:27:54.423164 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 642/1286] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:27:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:27:54.429235 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db7e809-9d3b-40d4-887c-077d60e8c0fc None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:27:54.527865 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db7e809-9d3b-40d4-887c-077d60e8c0fc None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:27:54.560831 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0484f98-52c0-4dae-916a-0ab0763654d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:27:54.561486 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0484f98-52c0-4dae-916a-0ab0763654d5 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:54.561771 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0484f98-52c0-4dae-916a-0ab0763654d5 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:54.562254 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0484f98-52c0-4dae-916a-0ab0763654d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:54.563095 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0484f98-52c0-4dae-916a-0ab0763654d5 None admin] RBAC: Adding query filter params (role.id=a3b793754e7b40b0bb21afe74eec3c18, scope.project.id=252d9673f08c479a98d04fee3f5856a4, user.id=59ec84a77cb449a1a22c86ac9873b8af) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:27:54.563418 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0484f98-52c0-4dae-916a-0ab0763654d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:27:54.565479 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0484f98-52c0-4dae-916a-0ab0763654d5 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:27:54.571409 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 645/1287] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:27:54 2022] GET /identity/v3/role_assignments?user.id=59ec84a77cb449a1a22c86ac9873b8af&scope.project.id=252d9673f08c479a98d04fee3f5856a4&role.id=a3b793754e7b40b0bb21afe74eec3c18 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:27:56.819160 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268aa02a-8e5c-4b2f-82eb-e85c3c178169 None None] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:56.819677 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268aa02a-8e5c-4b2f-82eb-e85c3c178169 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:56.819767 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268aa02a-8e5c-4b2f-82eb-e85c3c178169 None None] PATH_INFO: `/` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:56.821075 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 643/1288] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:27:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:27:56.826251 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44b51ef-6920-481b-979f-7c79bf849f6f None None] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:56.826693 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44b51ef-6920-481b-979f-7c79bf849f6f None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:56.826820 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44b51ef-6920-481b-979f-7c79bf849f6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:56.844679 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.auth.core [None req-a44b51ef-6920-481b-979f-7c79bf849f6f None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143586) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:27:56.846495 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.fernet_utils [None req-a44b51ef-6920-481b-979f-7c79bf849f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143586) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:27:56.875703 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:27:56.913181 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 646/1289] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:27:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:27:56.935110 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268aa02a-8e5c-4b2f-82eb-e85c3c178169 None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:27:57.035299 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268aa02a-8e5c-4b2f-82eb-e85c3c178169 None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:27:57.046458 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:27:57.069214 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eda0c4f-9e29-4789-908c-7e5162141907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:27:57.070035 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eda0c4f-9e29-4789-908c-7e5162141907 None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:57.070314 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eda0c4f-9e29-4789-908c-7e5162141907 None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:57.070585 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eda0c4f-9e29-4789-908c-7e5162141907 None admin] PATH_INFO: `/v3/services/placement` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:57.071226 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eda0c4f-9e29-4789-908c-7e5162141907 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:27:57.072592 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eda0c4f-9e29-4789-908c-7e5162141907 None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:27:57.084703 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-3eda0c4f-9e29-4789-908c-7e5162141907 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 19 14:27:57.084703 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:27:57.084703 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:27:57.084703 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:27:57.084703 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:27:57.084703 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:27:57.084703 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:27:57.084703 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:27:57.084703 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 14:27:57.084703 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:27:57.084703 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:27:57.084703 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:27:57.084703 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:27:57.084703 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:27:57.084703 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:27:57.084703 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:27:57.084703 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:27:57.085926 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:27:57.085926 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:27:57.085926 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:27:57.085926 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:27:57.085926 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:27:57.085926 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:27:57.085926 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:27:57.085926 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:27:57.085926 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:27:57.085926 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:27:57.085926 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:27:57.085926 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:27:57.085926 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:27:57.085926 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:27:57.085926 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:27:57.085926 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:27:57.086739 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:27:57.086739 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:27:57.086739 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:27:57.086739 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:27:57.086739 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:27:57.086739 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:27:57.086739 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:27:57.086739 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:27:57.086739 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:27:57.086739 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:27:57.086739 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:27:57.086739 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:27:57.086739 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 14:27:57.086739 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:27:57.086739 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 644/1290] 10.222.0.57 () {56 vars in 1117 bytes} [Thu May 19 14:27:56 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:27:57.092144 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44b51ef-6920-481b-979f-7c79bf849f6f None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:27:57.190470 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44b51ef-6920-481b-979f-7c79bf849f6f None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:27:57.224394 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7d5f04-91b7-4660-a5ee-f86c5612787e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:27:57.225048 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7d5f04-91b7-4660-a5ee-f86c5612787e None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:57.225330 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7d5f04-91b7-4660-a5ee-f86c5612787e None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:57.225609 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7d5f04-91b7-4660-a5ee-f86c5612787e None admin] PATH_INFO: `/v3/services` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:57.226340 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7d5f04-91b7-4660-a5ee-f86c5612787e None admin] RBAC: Adding query filter params (name=placement) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:27:57.226650 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7d5f04-91b7-4660-a5ee-f86c5612787e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:27:57.228013 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7d5f04-91b7-4660-a5ee-f86c5612787e None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:27:57.232734 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 647/1291] 10.222.0.57 () {56 vars in 1116 bytes} [Thu May 19 14:27:57 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:27:57.238743 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eda0c4f-9e29-4789-908c-7e5162141907 None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:27:57.336093 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eda0c4f-9e29-4789-908c-7e5162141907 None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:27:57.374364 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b73f4aa-f1b6-4b3b-bdb3-c564ccb6ad6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:27:57.375028 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b73f4aa-f1b6-4b3b-bdb3-c564ccb6ad6b None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:57.375314 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b73f4aa-f1b6-4b3b-bdb3-c564ccb6ad6b None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:57.375593 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b73f4aa-f1b6-4b3b-bdb3-c564ccb6ad6b None admin] PATH_INFO: `/v3/services` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:57.376302 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b73f4aa-f1b6-4b3b-bdb3-c564ccb6ad6b None admin] RBAC: Adding query filter params (type=placement) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:27:57.376611 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b73f4aa-f1b6-4b3b-bdb3-c564ccb6ad6b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:27:57.378630 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b73f4aa-f1b6-4b3b-bdb3-c564ccb6ad6b None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:27:57.383856 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 645/1292] 10.222.0.57 () {56 vars in 1116 bytes} [Thu May 19 14:27:57 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:27:59.607270 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5c7d8f-a8b2-4661-8860-c4fba6bfb77d None None] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:59.607807 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5c7d8f-a8b2-4661-8860-c4fba6bfb77d None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:59.607807 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5c7d8f-a8b2-4661-8860-c4fba6bfb77d None None] PATH_INFO: `/` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:59.609175 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 648/1293] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:27:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:27:59.614415 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fe0a59-1a7f-458d-9efe-cbc8bb36994f None None] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:59.614814 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fe0a59-1a7f-458d-9efe-cbc8bb36994f None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:59.614975 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fe0a59-1a7f-458d-9efe-cbc8bb36994f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:59.632185 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.auth.core [None req-e3fe0a59-1a7f-458d-9efe-cbc8bb36994f None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143587) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:27:59.633368 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-e3fe0a59-1a7f-458d-9efe-cbc8bb36994f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:27:59.663249 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:27:59.700951 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 646/1294] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:27:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:27:59.724656 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5c7d8f-a8b2-4661-8860-c4fba6bfb77d None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:27:59.755567 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:27:59.828257 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5c7d8f-a8b2-4661-8860-c4fba6bfb77d None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:27:59.862954 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0f3a2a-0d83-4fa9-b1f9-0c0a492f8f4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:27:59.863548 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0f3a2a-0d83-4fa9-b1f9-0c0a492f8f4a None admin] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:59.863835 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0f3a2a-0d83-4fa9-b1f9-0c0a492f8f4a None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:59.864115 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0f3a2a-0d83-4fa9-b1f9-0c0a492f8f4a None admin] PATH_INFO: `/v3/services` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:59.864840 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0f3a2a-0d83-4fa9-b1f9-0c0a492f8f4a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:27:59.866240 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0f3a2a-0d83-4fa9-b1f9-0c0a492f8f4a None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:27:59.880466 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 649/1295] 10.222.0.57 () {60 vars in 1141 bytes} [Thu May 19 14:27:59 2022] POST /identity/v3/services => generated 246 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 14:28:02.141386 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8e8b57-4e97-4fc9-8aee-d754cde961f1 None None] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:28:02.141935 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8e8b57-4e97-4fc9-8aee-d754cde961f1 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:28:02.142048 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8e8b57-4e97-4fc9-8aee-d754cde961f1 None None] PATH_INFO: `/` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:28:02.144117 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 647/1296] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:28:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:28:02.149575 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8be652-dd04-46b1-b836-b81dd5011100 None None] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:28:02.150024 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8be652-dd04-46b1-b836-b81dd5011100 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:28:02.150180 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8be652-dd04-46b1-b836-b81dd5011100 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:28:02.166614 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.auth.core [None req-0b8be652-dd04-46b1-b836-b81dd5011100 None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143586) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:28:02.167789 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.fernet_utils [None req-0b8be652-dd04-46b1-b836-b81dd5011100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143586) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:28:02.239737 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 650/1297] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:28:02 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:28:02.263024 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8e8b57-4e97-4fc9-8aee-d754cde961f1 None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:28:02.364017 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8e8b57-4e97-4fc9-8aee-d754cde961f1 None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:28:02.398946 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69e8827-d449-4217-bf56-67994ca17ffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:28:02.399656 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69e8827-d449-4217-bf56-67994ca17ffe None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:28:02.399934 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69e8827-d449-4217-bf56-67994ca17ffe None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:28:02.400207 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69e8827-d449-4217-bf56-67994ca17ffe None admin] PATH_INFO: `/v3/services/placement` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:28:02.400851 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69e8827-d449-4217-bf56-67994ca17ffe None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:28:02.402249 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69e8827-d449-4217-bf56-67994ca17ffe None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:28:02.413751 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-a69e8827-d449-4217-bf56-67994ca17ffe None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 19 14:28:02.413751 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:28:02.413751 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:28:02.413751 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:28:02.413751 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:28:02.413751 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:28:02.413751 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:28:02.413751 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:28:02.413751 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 14:28:02.413751 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:28:02.413751 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:28:02.413751 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:28:02.413751 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:28:02.413751 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:28:02.413751 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:28:02.413751 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:28:02.413751 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:28:02.414884 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:28:02.414884 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:28:02.414884 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:28:02.414884 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:28:02.414884 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:28:02.414884 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:28:02.414884 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:28:02.414884 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:28:02.414884 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:28:02.414884 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:28:02.414884 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:28:02.414884 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:28:02.414884 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:28:02.414884 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:28:02.414884 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:28:02.414884 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:28:02.415914 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:28:02.415914 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:28:02.415914 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:28:02.415914 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:28:02.415914 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:28:02.415914 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:28:02.415914 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:28:02.415914 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:28:02.415914 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:28:02.415914 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:28:02.415914 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:28:02.415914 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:28:02.415914 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 14:28:02.415914 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:28:02.415914 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 648/1298] 10.222.0.57 () {56 vars in 1117 bytes} [Thu May 19 14:28:02 2022] GET /identity/v3/services/placement => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:28:02.421069 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8be652-dd04-46b1-b836-b81dd5011100 None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:28:02.440003 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:28:02.519991 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8be652-dd04-46b1-b836-b81dd5011100 None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:28:02.553436 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7b6e0f-b17f-49c3-8c3c-33468dc8e2aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:28:02.554133 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7b6e0f-b17f-49c3-8c3c-33468dc8e2aa None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:28:02.554423 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7b6e0f-b17f-49c3-8c3c-33468dc8e2aa None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:28:02.554700 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7b6e0f-b17f-49c3-8c3c-33468dc8e2aa None admin] PATH_INFO: `/v3/services` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:28:02.555401 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7b6e0f-b17f-49c3-8c3c-33468dc8e2aa None admin] RBAC: Adding query filter params (name=placement) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:28:02.555709 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7b6e0f-b17f-49c3-8c3c-33468dc8e2aa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:28:02.557077 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7b6e0f-b17f-49c3-8c3c-33468dc8e2aa None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:28:02.562175 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 651/1299] 10.222.0.57 () {56 vars in 1116 bytes} [Thu May 19 14:28:02 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:28:02.568056 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69e8827-d449-4217-bf56-67994ca17ffe None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:28:02.600199 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:28:02.668472 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69e8827-d449-4217-bf56-67994ca17ffe None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:28:02.701574 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0734fd59-5216-437e-bafc-f0c5133012fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:28:02.702246 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0734fd59-5216-437e-bafc-f0c5133012fa None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:28:02.702532 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0734fd59-5216-437e-bafc-f0c5133012fa None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:28:02.702812 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0734fd59-5216-437e-bafc-f0c5133012fa None admin] PATH_INFO: `/v3/endpoints` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:28:02.703601 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0734fd59-5216-437e-bafc-f0c5133012fa None admin] RBAC: Adding query filter params (interface=public, service_id=5b244277da784b4199f1726f1de2b765, region_id=RegionOne) {{(pid=143587) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:28:02.703916 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0734fd59-5216-437e-bafc-f0c5133012fa None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:28:02.705346 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0734fd59-5216-437e-bafc-f0c5133012fa None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:28:02.710536 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 649/1300] 10.222.0.57 () {56 vars in 1251 bytes} [Thu May 19 14:28:02 2022] GET /identity/v3/endpoints?service_id=5b244277da784b4199f1726f1de2b765&interface=public®ion_id=RegionOne => generated 194 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:28:02.716152 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7b6e0f-b17f-49c3-8c3c-33468dc8e2aa None admin] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:28:02.797214 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:28:02.815567 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7b6e0f-b17f-49c3-8c3c-33468dc8e2aa None admin] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:28:02.848967 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af042a45-598c-4d72-94a3-9704a6c3c0d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:28:02.849624 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af042a45-598c-4d72-94a3-9704a6c3c0d9 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:28:02.849937 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af042a45-598c-4d72-94a3-9704a6c3c0d9 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:28:02.850218 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af042a45-598c-4d72-94a3-9704a6c3c0d9 None admin] PATH_INFO: `/v3/services` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:28:02.850898 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af042a45-598c-4d72-94a3-9704a6c3c0d9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:28:02.852253 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af042a45-598c-4d72-94a3-9704a6c3c0d9 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:28:02.858891 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 652/1301] 10.222.0.57 () {56 vars in 1087 bytes} [Thu May 19 14:28:02 2022] GET /identity/v3/services => generated 2191 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 14:28:05.076749 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2ed701-12c0-484d-a8b0-0fdb0f10d2bd None None] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:28:05.077249 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2ed701-12c0-484d-a8b0-0fdb0f10d2bd None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:28:05.077249 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2ed701-12c0-484d-a8b0-0fdb0f10d2bd None None] PATH_INFO: `/` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:28:05.078615 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 650/1302] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:28:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:28:05.084699 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e07ddce-9960-4af0-9f2b-df53cee32cb1 None None] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:28:05.085103 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e07ddce-9960-4af0-9f2b-df53cee32cb1 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:28:05.085263 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e07ddce-9960-4af0-9f2b-df53cee32cb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:28:05.103797 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.auth.core [None req-0e07ddce-9960-4af0-9f2b-df53cee32cb1 None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143586) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:28:05.104982 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.fernet_utils [None req-0e07ddce-9960-4af0-9f2b-df53cee32cb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143586) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:28:05.170828 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 653/1303] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:28:05 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:28:05.194244 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2ed701-12c0-484d-a8b0-0fdb0f10d2bd None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:28:05.287750 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:28:05.295231 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2ed701-12c0-484d-a8b0-0fdb0f10d2bd None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:28:05.327428 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58716b6f-a0f1-4126-b5d9-e6b602ced93e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:28:05.328149 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58716b6f-a0f1-4126-b5d9-e6b602ced93e None admin] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:28:05.328429 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58716b6f-a0f1-4126-b5d9-e6b602ced93e None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:28:05.328701 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58716b6f-a0f1-4126-b5d9-e6b602ced93e None admin] PATH_INFO: `/v3/services/placement` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:28:05.329346 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58716b6f-a0f1-4126-b5d9-e6b602ced93e None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:28:05.331390 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58716b6f-a0f1-4126-b5d9-e6b602ced93e None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:28:05.343171 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application [None req-58716b6f-a0f1-4126-b5d9-e6b602ced93e None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 19 14:28:05.343171 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:28:05.343171 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:28:05.343171 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:28:05.343171 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:28:05.343171 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:28:05.343171 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:28:05.343171 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:28:05.343171 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 14:28:05.343171 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:28:05.343171 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:28:05.343171 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:28:05.343171 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:28:05.343171 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:28:05.343171 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:28:05.343171 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:28:05.343171 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:28:05.344276 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:28:05.344276 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:28:05.344276 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:28:05.344276 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:28:05.344276 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:28:05.344276 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:28:05.344276 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:28:05.344276 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:28:05.344276 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:28:05.344276 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:28:05.344276 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:28:05.344276 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:28:05.344276 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:28:05.344276 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:28:05.344276 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:28:05.344276 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:28:05.345247 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:28:05.345247 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application with Lock( May 19 14:28:05.345247 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:28:05.345247 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self._enter() May 19 14:28:05.345247 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:28:05.345247 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:28:05.345247 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:28:05.345247 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application return self.creator() May 19 14:28:05.345247 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:28:05.345247 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application created_value = creator( May 19 14:28:05.345247 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:28:05.345247 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:28:05.345247 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 14:28:05.345247 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: ERROR keystone.server.flask.application May 19 14:28:05.345247 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 651/1304] 10.222.0.57 () {56 vars in 1117 bytes} [Thu May 19 14:28:05 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:28:05.351286 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e07ddce-9960-4af0-9f2b-df53cee32cb1 None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:28:05.427720 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:28:05.450473 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e07ddce-9960-4af0-9f2b-df53cee32cb1 None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:28:05.483989 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53faa91e-053c-459a-b69f-564fa1faf433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:28:05.484644 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53faa91e-053c-459a-b69f-564fa1faf433 None admin] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:28:05.484927 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53faa91e-053c-459a-b69f-564fa1faf433 None admin] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:28:05.485206 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53faa91e-053c-459a-b69f-564fa1faf433 None admin] PATH_INFO: `/v3/services` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:28:05.485942 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53faa91e-053c-459a-b69f-564fa1faf433 None admin] RBAC: Adding query filter params (name=placement) {{(pid=143586) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:28:05.486346 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53faa91e-053c-459a-b69f-564fa1faf433 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:28:05.487732 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53faa91e-053c-459a-b69f-564fa1faf433 None admin] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:28:05.492782 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 654/1305] 10.222.0.57 () {56 vars in 1116 bytes} [Thu May 19 14:28:05 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:28:05.498701 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58716b6f-a0f1-4126-b5d9-e6b602ced93e None admin] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:28:05.602997 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58716b6f-a0f1-4126-b5d9-e6b602ced93e None admin] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:28:05.635573 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5e8f33-de9f-4499-b770-9b89b9d92d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f2f0290e1b04ce48a90ce03cb93d060', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:28:05.636222 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5e8f33-de9f-4499-b770-9b89b9d92d7f None admin] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:28:05.636504 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5e8f33-de9f-4499-b770-9b89b9d92d7f None admin] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:28:05.636781 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5e8f33-de9f-4499-b770-9b89b9d92d7f None admin] PATH_INFO: `/v3/endpoints` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:28:05.637514 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5e8f33-de9f-4499-b770-9b89b9d92d7f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:28:05.638910 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5e8f33-de9f-4499-b770-9b89b9d92d7f None admin] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:28:05.659893 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 652/1306] 10.222.0.57 () {60 vars in 1144 bytes} [Thu May 19 14:28:05 2022] POST /identity/v3/endpoints => generated 331 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:28:15.489785 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=30 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:28:15.653406 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:29:10.837295 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55deda3-111b-4884-a75a-78c34d372fd2 None None] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:29:10.837777 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55deda3-111b-4884-a75a-78c34d372fd2 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:29:10.838272 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55deda3-111b-4884-a75a-78c34d372fd2 None None] PATH_INFO: `/` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:29:10.841564 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 655/1307] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:29:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:29:10.847066 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb397ed2-a6d6-4735-ba20-41c37355ab64 None None] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:29:10.847941 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb397ed2-a6d6-4735-ba20-41c37355ab64 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:29:10.848365 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb397ed2-a6d6-4735-ba20-41c37355ab64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:29:10.970448 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.auth.core [None req-eb397ed2-a6d6-4735-ba20-41c37355ab64 None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143587) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:29:10.972430 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-eb397ed2-a6d6-4735-ba20-41c37355ab64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:29:11.188275 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 653/1308] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:29:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 342 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:29:11.236641 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9dd88f-f3c6-4fc3-82e1-29a04d0914ff None None] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:29:11.238166 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9dd88f-f3c6-4fc3-82e1-29a04d0914ff None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:29:11.238756 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9dd88f-f3c6-4fc3-82e1-29a04d0914ff None None] PATH_INFO: `/` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:29:11.240432 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 656/1309] 10.222.0.57 () {54 vars in 964 bytes} [Thu May 19 14:29:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:29:11.246634 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02ec8da-2a78-480f-8dff-d0b31bff1437 None None] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:29:11.247156 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02ec8da-2a78-480f-8dff-d0b31bff1437 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:29:11.247673 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02ec8da-2a78-480f-8dff-d0b31bff1437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:29:11.395026 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.auth.core [None req-e02ec8da-2a78-480f-8dff-d0b31bff1437 None None] MFA Rules not processed for user `0ed9caca45c9449b84582da44a1262ef`. Rule list: `[]` (Enabled: `True`). {{(pid=143587) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:29:11.396923 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-e02ec8da-2a78-480f-8dff-d0b31bff1437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:29:11.424675 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:29:11.495995 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 654/1310] 10.222.0.57 () {58 vars in 1061 bytes} [Thu May 19 14:29:11 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:29:11.506307 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9dd88f-f3c6-4fc3-82e1-29a04d0914ff None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:29:11.508534 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.fernet_utils [None req-1b9dd88f-f3c6-4fc3-82e1-29a04d0914ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143586) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:29:11.689761 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9dd88f-f3c6-4fc3-82e1-29a04d0914ff None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:29:11.778312 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e491cce-ab7e-4855-9720-0b0cedb36354 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed9caca45c9449b84582da44a1262ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '252d9673f08c479a98d04fee3f5856a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:29:11.779032 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e491cce-ab7e-4855-9720-0b0cedb36354 service neutron] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:29:11.779292 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e491cce-ab7e-4855-9720-0b0cedb36354 service neutron] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:29:11.780092 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e491cce-ab7e-4855-9720-0b0cedb36354 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:29:11.781948 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.fernet_utils [None req-8e491cce-ab7e-4855-9720-0b0cedb36354 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143586) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:29:11.844162 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:29:11.909698 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e491cce-ab7e-4855-9720-0b0cedb36354 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:29:11.911202 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e491cce-ab7e-4855-9720-0b0cedb36354 service neutron] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:29:11.942286 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 657/1311] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 14:29:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:29:14.558446 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f19d21e-93cf-42a9-9b89-f86ae124b648 None None] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:29:14.558906 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f19d21e-93cf-42a9-9b89-f86ae124b648 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:29:14.559063 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f19d21e-93cf-42a9-9b89-f86ae124b648 None None] PATH_INFO: `/` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:29:14.561231 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 655/1312] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:29:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:29:14.568403 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72295e4-301b-40e9-bc75-ab12d84c3205 None None] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:29:14.568936 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72295e4-301b-40e9-bc75-ab12d84c3205 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:29:14.568936 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72295e4-301b-40e9-bc75-ab12d84c3205 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:29:14.667986 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.auth.core [None req-e72295e4-301b-40e9-bc75-ab12d84c3205 None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143586) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:29:14.669370 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.fernet_utils [None req-e72295e4-301b-40e9-bc75-ab12d84c3205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143586) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:29:14.778096 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 658/1313] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:29:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:29:14.818115 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f19d21e-93cf-42a9-9b89-f86ae124b648 None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:29:14.819804 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-0f19d21e-93cf-42a9-9b89-f86ae124b648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:29:14.931402 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f19d21e-93cf-42a9-9b89-f86ae124b648 None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:29:15.013883 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:29:15.017277 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf71a9d-d0d9-4beb-92f6-b7db12dfa9ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed9caca45c9449b84582da44a1262ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '252d9673f08c479a98d04fee3f5856a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:29:15.017995 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf71a9d-d0d9-4beb-92f6-b7db12dfa9ca service neutron] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:29:15.018260 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf71a9d-d0d9-4beb-92f6-b7db12dfa9ca service neutron] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:29:15.018608 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf71a9d-d0d9-4beb-92f6-b7db12dfa9ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:29:15.020568 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-3cf71a9d-d0d9-4beb-92f6-b7db12dfa9ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:29:15.147474 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf71a9d-d0d9-4beb-92f6-b7db12dfa9ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:29:15.148952 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf71a9d-d0d9-4beb-92f6-b7db12dfa9ca service neutron] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:29:15.170960 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 656/1314] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 14:29:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:29:17.786925 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02d0814-82c3-488d-a7a5-0b2747d76b8d None None] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:29:17.787450 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02d0814-82c3-488d-a7a5-0b2747d76b8d None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:29:17.787570 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02d0814-82c3-488d-a7a5-0b2747d76b8d None None] PATH_INFO: `/` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:29:17.788876 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 659/1315] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:29:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:29:17.794268 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa70c96-1f99-450f-94ca-867b20665105 None None] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:29:17.794690 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa70c96-1f99-450f-94ca-867b20665105 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:29:17.794841 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa70c96-1f99-450f-94ca-867b20665105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:29:17.865233 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.auth.core [None req-9aa70c96-1f99-450f-94ca-867b20665105 None None] MFA Rules not processed for user `21249d0b4cb3470e83a9f256cbe52f18`. Rule list: `[]` (Enabled: `True`). {{(pid=143587) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:29:17.866501 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-9aa70c96-1f99-450f-94ca-867b20665105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:29:17.920541 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:29:17.967691 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 657/1316] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:29:17 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 14:29:18.007571 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02d0814-82c3-488d-a7a5-0b2747d76b8d None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:29:18.009097 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.fernet_utils [None req-d02d0814-82c3-488d-a7a5-0b2747d76b8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143586) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:29:18.018977 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:29:18.114823 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02d0814-82c3-488d-a7a5-0b2747d76b8d None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:29:18.195123 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a00ae4-9736-406e-a23b-5d4ae836a9f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed9caca45c9449b84582da44a1262ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '252d9673f08c479a98d04fee3f5856a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:29:18.195705 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a00ae4-9736-406e-a23b-5d4ae836a9f5 service neutron] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:29:18.195990 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a00ae4-9736-406e-a23b-5d4ae836a9f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:29:18.196269 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a00ae4-9736-406e-a23b-5d4ae836a9f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:29:18.198085 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.fernet_utils [None req-44a00ae4-9736-406e-a23b-5d4ae836a9f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143586) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:29:18.268747 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:29:18.321209 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a00ae4-9736-406e-a23b-5d4ae836a9f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:29:18.322722 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a00ae4-9736-406e-a23b-5d4ae836a9f5 service neutron] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:29:18.344287 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 660/1317] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 14:29:18 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:29:21.178842 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9a1631-c3ad-4611-84e7-2ffb0d13c46e None None] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:29:21.179408 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9a1631-c3ad-4611-84e7-2ffb0d13c46e None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:29:21.179408 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9a1631-c3ad-4611-84e7-2ffb0d13c46e None None] PATH_INFO: `/` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:29:21.180826 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 658/1318] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:29:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:29:21.186468 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9552d140-af51-4bda-a36e-8adf07fa28ce None None] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:29:21.186877 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9552d140-af51-4bda-a36e-8adf07fa28ce None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:29:21.187032 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9552d140-af51-4bda-a36e-8adf07fa28ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:29:21.252138 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.auth.core [None req-9552d140-af51-4bda-a36e-8adf07fa28ce None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143586) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:29:21.253365 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.fernet_utils [None req-9552d140-af51-4bda-a36e-8adf07fa28ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143586) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:29:21.352460 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 661/1319] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:29:21 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:29:21.392388 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9a1631-c3ad-4611-84e7-2ffb0d13c46e None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:29:21.394300 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-3e9a1631-c3ad-4611-84e7-2ffb0d13c46e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:29:21.497170 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9a1631-c3ad-4611-84e7-2ffb0d13c46e None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:29:21.569495 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135c4d26-09a7-4730-92b3-f56c68ddb3a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed9caca45c9449b84582da44a1262ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '252d9673f08c479a98d04fee3f5856a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:29:21.570170 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135c4d26-09a7-4730-92b3-f56c68ddb3a6 service neutron] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:29:21.570456 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135c4d26-09a7-4730-92b3-f56c68ddb3a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:29:21.570734 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135c4d26-09a7-4730-92b3-f56c68ddb3a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:29:21.572395 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-135c4d26-09a7-4730-92b3-f56c68ddb3a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:29:21.580479 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:29:21.682968 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135c4d26-09a7-4730-92b3-f56c68ddb3a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:29:21.684430 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135c4d26-09a7-4730-92b3-f56c68ddb3a6 service neutron] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:29:21.703248 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 659/1320] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 14:29:21 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:29:24.659697 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d515bd-9bb1-4e4a-a89a-8ae56d4e07a9 None None] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:29:24.660154 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d515bd-9bb1-4e4a-a89a-8ae56d4e07a9 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:29:24.660233 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d515bd-9bb1-4e4a-a89a-8ae56d4e07a9 None None] PATH_INFO: `/` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:29:24.661759 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 662/1321] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:29:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:29:24.666981 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c72ccf-e0b9-4844-b7e0-879e3e0886cf None None] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:29:24.667381 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c72ccf-e0b9-4844-b7e0-879e3e0886cf None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:29:24.667536 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c72ccf-e0b9-4844-b7e0-879e3e0886cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:29:24.732936 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.auth.core [None req-01c72ccf-e0b9-4844-b7e0-879e3e0886cf None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143587) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:29:24.734168 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-01c72ccf-e0b9-4844-b7e0-879e3e0886cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:29:24.791256 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:29:24.830851 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 660/1322] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:29:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:29:24.867491 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d515bd-9bb1-4e4a-a89a-8ae56d4e07a9 None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:29:24.869030 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.fernet_utils [None req-18d515bd-9bb1-4e4a-a89a-8ae56d4e07a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143586) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:29:24.888988 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:29:24.976354 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d515bd-9bb1-4e4a-a89a-8ae56d4e07a9 None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:29:25.052245 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1281f028-954a-41ca-9f35-616823e627e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed9caca45c9449b84582da44a1262ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '252d9673f08c479a98d04fee3f5856a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:29:25.052895 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1281f028-954a-41ca-9f35-616823e627e1 service neutron] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:29:25.053175 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1281f028-954a-41ca-9f35-616823e627e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:29:25.053447 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1281f028-954a-41ca-9f35-616823e627e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:29:25.055125 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.fernet_utils [None req-1281f028-954a-41ca-9f35-616823e627e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143586) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:29:25.127707 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:29:25.166451 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1281f028-954a-41ca-9f35-616823e627e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:29:25.167885 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1281f028-954a-41ca-9f35-616823e627e1 service neutron] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:29:25.188680 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 663/1323] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 14:29:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:29:27.490239 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4871a0-444d-4c73-8331-00016697e721 None None] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:29:27.490731 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4871a0-444d-4c73-8331-00016697e721 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:29:27.490837 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4871a0-444d-4c73-8331-00016697e721 None None] PATH_INFO: `/` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:29:27.492335 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 661/1324] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:29:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:29:27.497420 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb4af01-55b1-425a-8305-3347f2b325d3 None None] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:29:27.498433 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb4af01-55b1-425a-8305-3347f2b325d3 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:29:27.498536 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb4af01-55b1-425a-8305-3347f2b325d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:29:27.567645 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.auth.core [None req-5fb4af01-55b1-425a-8305-3347f2b325d3 None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143586) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:29:27.568632 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.fernet_utils [None req-5fb4af01-55b1-425a-8305-3347f2b325d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143586) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:29:27.684550 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 664/1325] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:29:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:29:27.721521 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4871a0-444d-4c73-8331-00016697e721 None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:29:27.723166 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-3a4871a0-444d-4c73-8331-00016697e721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:29:27.827270 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4871a0-444d-4c73-8331-00016697e721 None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:29:27.899888 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1804a38f-58ef-4232-a009-0189ded2c3e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed9caca45c9449b84582da44a1262ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '252d9673f08c479a98d04fee3f5856a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:29:27.900536 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1804a38f-58ef-4232-a009-0189ded2c3e0 service neutron] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:29:27.900815 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1804a38f-58ef-4232-a009-0189ded2c3e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:29:27.901088 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1804a38f-58ef-4232-a009-0189ded2c3e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:29:27.902795 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-1804a38f-58ef-4232-a009-0189ded2c3e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:29:27.921692 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:29:28.016367 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1804a38f-58ef-4232-a009-0189ded2c3e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:29:28.017821 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1804a38f-58ef-4232-a009-0189ded2c3e0 service neutron] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:29:28.036963 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 662/1326] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 14:29:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:29:30.316008 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d6095a-5a25-4966-aed9-4eb7a20d53c2 None None] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:29:30.316554 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d6095a-5a25-4966-aed9-4eb7a20d53c2 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:29:30.316554 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d6095a-5a25-4966-aed9-4eb7a20d53c2 None None] PATH_INFO: `/` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:29:30.317948 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 665/1327] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:29:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:29:30.323228 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b111834-ea57-42e7-a959-5d27ffba6f33 None None] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:29:30.323668 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b111834-ea57-42e7-a959-5d27ffba6f33 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:29:30.323792 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b111834-ea57-42e7-a959-5d27ffba6f33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:29:30.390081 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.auth.core [None req-5b111834-ea57-42e7-a959-5d27ffba6f33 None None] MFA Rules not processed for user `21249d0b4cb3470e83a9f256cbe52f18`. Rule list: `[]` (Enabled: `True`). {{(pid=143587) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:29:30.391296 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-5b111834-ea57-42e7-a959-5d27ffba6f33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:29:30.455368 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:29:30.487308 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 663/1328] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:29:30 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:29:30.540705 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da5cf02-e173-4fe8-af06-5245d9b6737d None None] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:29:30.540999 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da5cf02-e173-4fe8-af06-5245d9b6737d None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:29:30.541283 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da5cf02-e173-4fe8-af06-5245d9b6737d None None] PATH_INFO: `/` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:29:30.542695 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 666/1329] 10.222.0.57 () {54 vars in 964 bytes} [Thu May 19 14:29:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:29:30.549167 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe08f477-1442-4085-8be7-eab8c2f5c02a None None] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:29:30.549489 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe08f477-1442-4085-8be7-eab8c2f5c02a None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:29:30.549773 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe08f477-1442-4085-8be7-eab8c2f5c02a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:29:30.624376 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.auth.core [None req-fe08f477-1442-4085-8be7-eab8c2f5c02a None None] MFA Rules not processed for user `0ed9caca45c9449b84582da44a1262ef`. Rule list: `[]` (Enabled: `True`). {{(pid=143587) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:29:30.625588 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-fe08f477-1442-4085-8be7-eab8c2f5c02a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:29:30.700827 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 664/1330] 10.222.0.57 () {58 vars in 1061 bytes} [Thu May 19 14:29:30 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:29:30.709817 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da5cf02-e173-4fe8-af06-5245d9b6737d None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:29:30.711420 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.fernet_utils [None req-0da5cf02-e173-4fe8-af06-5245d9b6737d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143586) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:29:30.743594 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:29:30.816316 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da5cf02-e173-4fe8-af06-5245d9b6737d None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:29:30.889003 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78d71c0-f30d-4c25-b5b4-26afb8dce648 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed9caca45c9449b84582da44a1262ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '252d9673f08c479a98d04fee3f5856a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:29:30.889563 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78d71c0-f30d-4c25-b5b4-26afb8dce648 service neutron] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:29:30.889863 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78d71c0-f30d-4c25-b5b4-26afb8dce648 service neutron] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:29:30.890145 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78d71c0-f30d-4c25-b5b4-26afb8dce648 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:29:30.891777 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.fernet_utils [None req-f78d71c0-f30d-4c25-b5b4-26afb8dce648 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143586) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:29:30.967619 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:29:30.999753 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78d71c0-f30d-4c25-b5b4-26afb8dce648 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:29:31.001222 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78d71c0-f30d-4c25-b5b4-26afb8dce648 service neutron] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:29:31.018820 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 667/1331] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 14:29:30 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:29:33.879134 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a43aeb7-c279-4869-8f69-c389f23fe64c None None] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:29:33.879661 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a43aeb7-c279-4869-8f69-c389f23fe64c None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:29:33.879749 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a43aeb7-c279-4869-8f69-c389f23fe64c None None] PATH_INFO: `/` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:29:33.881090 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 665/1332] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:29:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:29:33.886480 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e77fb01-a79d-45fc-88de-78eed63c31ad None None] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:29:33.886871 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e77fb01-a79d-45fc-88de-78eed63c31ad None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:29:33.887049 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e77fb01-a79d-45fc-88de-78eed63c31ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:29:33.951559 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.auth.core [None req-8e77fb01-a79d-45fc-88de-78eed63c31ad None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143586) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:29:33.952688 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.fernet_utils [None req-8e77fb01-a79d-45fc-88de-78eed63c31ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143586) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:29:34.049850 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 668/1333] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:29:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:29:34.085592 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a43aeb7-c279-4869-8f69-c389f23fe64c None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:29:34.087154 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-2a43aeb7-c279-4869-8f69-c389f23fe64c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:29:34.157271 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:29:34.190835 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a43aeb7-c279-4869-8f69-c389f23fe64c None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:29:34.262571 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c96246-4f47-4bb9-b75a-7667c5e85b7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed9caca45c9449b84582da44a1262ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '252d9673f08c479a98d04fee3f5856a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:29:34.263324 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c96246-4f47-4bb9-b75a-7667c5e85b7b service neutron] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:29:34.263508 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c96246-4f47-4bb9-b75a-7667c5e85b7b service neutron] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:29:34.263779 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c96246-4f47-4bb9-b75a-7667c5e85b7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:29:34.265403 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-24c96246-4f47-4bb9-b75a-7667c5e85b7b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:29:34.373900 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c96246-4f47-4bb9-b75a-7667c5e85b7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:29:34.375348 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c96246-4f47-4bb9-b75a-7667c5e85b7b service neutron] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:29:34.385032 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:29:34.393830 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 666/1334] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 14:29:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:29:36.675849 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8b0bc3-6307-4b0f-9a26-719f2419dbff None None] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:29:36.676309 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8b0bc3-6307-4b0f-9a26-719f2419dbff None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:29:36.676401 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8b0bc3-6307-4b0f-9a26-719f2419dbff None None] PATH_INFO: `/` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:29:36.677873 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 669/1335] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:29:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:29:36.683200 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23efbffb-6a4a-4608-836e-64ebe9a3654d None None] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:29:36.683601 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23efbffb-6a4a-4608-836e-64ebe9a3654d None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:29:36.683771 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23efbffb-6a4a-4608-836e-64ebe9a3654d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:29:36.749454 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.auth.core [None req-23efbffb-6a4a-4608-836e-64ebe9a3654d None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143587) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:29:36.750698 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-23efbffb-6a4a-4608-836e-64ebe9a3654d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:29:36.846961 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 667/1336] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:29:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:29:36.888027 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8b0bc3-6307-4b0f-9a26-719f2419dbff None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:29:36.889562 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.fernet_utils [None req-cd8b0bc3-6307-4b0f-9a26-719f2419dbff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143586) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:29:36.930675 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:29:36.995125 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8b0bc3-6307-4b0f-9a26-719f2419dbff None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:29:37.076777 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1825bd40-b195-4620-9cf6-501966afb17b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed9caca45c9449b84582da44a1262ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '252d9673f08c479a98d04fee3f5856a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:29:37.077427 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1825bd40-b195-4620-9cf6-501966afb17b service neutron] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:29:37.077713 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1825bd40-b195-4620-9cf6-501966afb17b service neutron] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:29:37.078016 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1825bd40-b195-4620-9cf6-501966afb17b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:29:37.079672 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.fernet_utils [None req-1825bd40-b195-4620-9cf6-501966afb17b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143586) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:29:37.164620 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:29:37.191156 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1825bd40-b195-4620-9cf6-501966afb17b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:29:37.192529 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1825bd40-b195-4620-9cf6-501966afb17b service neutron] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:29:37.210459 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 670/1337] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 14:29:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:29:39.956188 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838acc71-599f-4e7a-91e3-e2d5579ba808 None None] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:29:39.956730 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838acc71-599f-4e7a-91e3-e2d5579ba808 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:29:39.956730 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838acc71-599f-4e7a-91e3-e2d5579ba808 None None] PATH_INFO: `/` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:29:39.958091 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 668/1338] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:29:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:29:39.963328 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3969ccb-898f-44a5-ab54-58a84c3793da None None] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:29:39.963761 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3969ccb-898f-44a5-ab54-58a84c3793da None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:29:39.964143 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3969ccb-898f-44a5-ab54-58a84c3793da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:29:40.029294 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.auth.core [None req-f3969ccb-898f-44a5-ab54-58a84c3793da None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143586) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:29:40.030571 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.fernet_utils [None req-f3969ccb-898f-44a5-ab54-58a84c3793da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143586) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:29:40.126323 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 671/1339] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:29:39 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:29:40.162647 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838acc71-599f-4e7a-91e3-e2d5579ba808 None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:29:40.164177 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-838acc71-599f-4e7a-91e3-e2d5579ba808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:29:40.239973 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:29:40.267794 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838acc71-599f-4e7a-91e3-e2d5579ba808 None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:29:40.341286 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cfddbb-e9e4-4cda-bf9a-e8b25417cb2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed9caca45c9449b84582da44a1262ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '252d9673f08c479a98d04fee3f5856a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:29:40.342057 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cfddbb-e9e4-4cda-bf9a-e8b25417cb2a service neutron] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:29:40.342350 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cfddbb-e9e4-4cda-bf9a-e8b25417cb2a service neutron] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:29:40.342629 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cfddbb-e9e4-4cda-bf9a-e8b25417cb2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:29:40.344300 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-66cfddbb-e9e4-4cda-bf9a-e8b25417cb2a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:29:40.456619 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cfddbb-e9e4-4cda-bf9a-e8b25417cb2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:29:40.458103 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cfddbb-e9e4-4cda-bf9a-e8b25417cb2a service neutron] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:29:40.477050 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 669/1340] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 14:29:40 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:29:44.852008 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3469ea33-6db0-4c5b-a01c-ce1c93ea5a09 None None] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:29:44.852526 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3469ea33-6db0-4c5b-a01c-ce1c93ea5a09 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:29:44.852526 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3469ea33-6db0-4c5b-a01c-ce1c93ea5a09 None None] PATH_INFO: `/` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:29:44.853889 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 672/1341] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:29:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:29:44.859922 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f403414b-15cc-455a-93de-763c60dc14d0 None None] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:29:44.860381 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f403414b-15cc-455a-93de-763c60dc14d0 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:29:44.860481 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f403414b-15cc-455a-93de-763c60dc14d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:29:44.866579 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=100 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:29:44.932503 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.auth.core [None req-f403414b-15cc-455a-93de-763c60dc14d0 None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143587) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:29:44.933650 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-f403414b-15cc-455a-93de-763c60dc14d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:29:45.038824 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 670/1342] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:29:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:29:45.077940 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3469ea33-6db0-4c5b-a01c-ce1c93ea5a09 None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:29:45.080114 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.fernet_utils [None req-3469ea33-6db0-4c5b-a01c-ce1c93ea5a09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143586) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:29:45.131786 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:29:45.178828 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3469ea33-6db0-4c5b-a01c-ce1c93ea5a09 None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:29:45.182412 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0e04fe-ca4a-4cd6-8cdb-e3e5b2bee192 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed9caca45c9449b84582da44a1262ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '252d9673f08c479a98d04fee3f5856a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:29:45.183055 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0e04fe-ca4a-4cd6-8cdb-e3e5b2bee192 service neutron] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:29:45.183334 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0e04fe-ca4a-4cd6-8cdb-e3e5b2bee192 service neutron] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:29:45.183607 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0e04fe-ca4a-4cd6-8cdb-e3e5b2bee192 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:29:45.195074 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0e04fe-ca4a-4cd6-8cdb-e3e5b2bee192 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:29:45.196500 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0e04fe-ca4a-4cd6-8cdb-e3e5b2bee192 service neutron] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:29:45.207338 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 673/1343] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 14:29:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:29:48.089087 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4b73b7-42c4-4379-8fbb-b6e61156ada0 None None] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:29:48.089610 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4b73b7-42c4-4379-8fbb-b6e61156ada0 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:29:48.089610 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4b73b7-42c4-4379-8fbb-b6e61156ada0 None None] PATH_INFO: `/` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:29:48.091174 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 671/1344] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:29:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:29:48.096341 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78aaea7-4270-4d31-a2db-21bb513e0242 None None] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:29:48.096806 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78aaea7-4270-4d31-a2db-21bb513e0242 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:29:48.096992 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78aaea7-4270-4d31-a2db-21bb513e0242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:29:48.125802 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.auth.core [None req-f78aaea7-4270-4d31-a2db-21bb513e0242 None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143586) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:29:48.127005 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.fernet_utils [None req-f78aaea7-4270-4d31-a2db-21bb513e0242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143586) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:29:48.201760 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 674/1345] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:29:48 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:29:48.238871 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4b73b7-42c4-4379-8fbb-b6e61156ada0 None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:29:48.250616 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4b73b7-42c4-4379-8fbb-b6e61156ada0 None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:29:48.254231 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071052d6-18f3-41b3-8b65-c8a8e6936bcd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed9caca45c9449b84582da44a1262ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '252d9673f08c479a98d04fee3f5856a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:29:48.254868 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071052d6-18f3-41b3-8b65-c8a8e6936bcd service neutron] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:29:48.255153 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071052d6-18f3-41b3-8b65-c8a8e6936bcd service neutron] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:29:48.255433 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071052d6-18f3-41b3-8b65-c8a8e6936bcd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:29:48.264754 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071052d6-18f3-41b3-8b65-c8a8e6936bcd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:29:48.266227 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071052d6-18f3-41b3-8b65-c8a8e6936bcd service neutron] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:29:48.275638 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 672/1346] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 14:29:48 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:29:52.594843 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cb32d5-6b16-470f-8cc1-ace0bb00f204 None None] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:29:52.595339 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cb32d5-6b16-470f-8cc1-ace0bb00f204 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:29:52.595339 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cb32d5-6b16-470f-8cc1-ace0bb00f204 None None] PATH_INFO: `/` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:29:52.596688 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 675/1347] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:29:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:29:52.601980 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a5c6f1-b9f3-401f-9396-585567fbdcda None None] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:29:52.602415 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a5c6f1-b9f3-401f-9396-585567fbdcda None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:29:52.602537 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a5c6f1-b9f3-401f-9396-585567fbdcda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:29:52.621128 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.auth.core [None req-00a5c6f1-b9f3-401f-9396-585567fbdcda None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143587) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:29:52.622964 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-00a5c6f1-b9f3-401f-9396-585567fbdcda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:29:52.631224 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 673/1348] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:29:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:29:52.670277 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cb32d5-6b16-470f-8cc1-ace0bb00f204 None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:29:52.681134 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cb32d5-6b16-470f-8cc1-ace0bb00f204 None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:29:52.685820 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1076ce5-340d-4efa-b99f-2af23bad0f29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed9caca45c9449b84582da44a1262ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '252d9673f08c479a98d04fee3f5856a4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:29:52.686570 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1076ce5-340d-4efa-b99f-2af23bad0f29 service neutron] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:29:52.686928 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1076ce5-340d-4efa-b99f-2af23bad0f29 service neutron] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:29:52.687278 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1076ce5-340d-4efa-b99f-2af23bad0f29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:29:52.696670 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1076ce5-340d-4efa-b99f-2af23bad0f29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:29:52.698220 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1076ce5-340d-4efa-b99f-2af23bad0f29 service neutron] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:29:52.708090 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 676/1349] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 14:29:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 14:29:58.186477 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e49a0f3-6548-4b73-a620-d79bab258474 None None] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:29:58.187343 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e49a0f3-6548-4b73-a620-d79bab258474 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:29:58.187772 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e49a0f3-6548-4b73-a620-d79bab258474 None None] PATH_INFO: `/` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:29:58.189650 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 674/1350] 10.222.0.57 () {54 vars in 921 bytes} [Thu May 19 14:29:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:29:58.195209 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23045b3-81bf-4662-9537-f069e18c9619 None None] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:29:58.195635 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23045b3-81bf-4662-9537-f069e18c9619 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:29:58.196046 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23045b3-81bf-4662-9537-f069e18c9619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:29:58.272986 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.auth.core [None req-b23045b3-81bf-4662-9537-f069e18c9619 None None] MFA Rules not processed for user `59ec84a77cb449a1a22c86ac9873b8af`. Rule list: `[]` (Enabled: `True`). {{(pid=143586) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:29:58.274405 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.fernet_utils [None req-b23045b3-81bf-4662-9537-f069e18c9619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143586) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:29:58.307674 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=100 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:29:58.359810 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 677/1351] 10.222.0.57 () {58 vars in 1018 bytes} [Thu May 19 14:29:58 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:30:02.615950 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=72 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:30:03.205766 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42391401-8641-422f-bf89-d41219cc0c79 None None] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:30:03.206329 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42391401-8641-422f-bf89-d41219cc0c79 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:30:03.206772 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42391401-8641-422f-bf89-d41219cc0c79 None None] PATH_INFO: `/` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:30:03.208589 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 675/1352] 10.222.0.57 () {54 vars in 921 bytes} [Thu May 19 14:30:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:30:03.214345 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bb5a9b-3803-4b1e-a4ba-b700bde17bd5 None None] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:30:03.216192 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bb5a9b-3803-4b1e-a4ba-b700bde17bd5 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:30:03.217011 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bb5a9b-3803-4b1e-a4ba-b700bde17bd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:30:03.235721 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.auth.core [None req-00bb5a9b-3803-4b1e-a4ba-b700bde17bd5 None None] MFA Rules not processed for user `59ec84a77cb449a1a22c86ac9873b8af`. Rule list: `[]` (Enabled: `True`). {{(pid=143586) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:30:03.236934 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.fernet_utils [None req-00bb5a9b-3803-4b1e-a4ba-b700bde17bd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143586) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:30:03.245640 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 678/1353] 10.222.0.57 () {58 vars in 1018 bytes} [Thu May 19 14:30:03 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:30:04.721979 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fc9db4-531f-4f2e-96ce-e677409d57dc None None] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:30:04.722526 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fc9db4-531f-4f2e-96ce-e677409d57dc None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:30:04.722682 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fc9db4-531f-4f2e-96ce-e677409d57dc None None] PATH_INFO: `/` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:30:04.724364 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 676/1354] 10.222.0.57 () {54 vars in 921 bytes} [Thu May 19 14:30:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:30:04.729000 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5399ca77-e703-4ffd-8f22-ba64332de1c2 None None] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:30:04.729394 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5399ca77-e703-4ffd-8f22-ba64332de1c2 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:30:04.729707 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5399ca77-e703-4ffd-8f22-ba64332de1c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:30:04.748298 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.auth.core [None req-5399ca77-e703-4ffd-8f22-ba64332de1c2 None None] MFA Rules not processed for user `59ec84a77cb449a1a22c86ac9873b8af`. Rule list: `[]` (Enabled: `True`). {{(pid=143586) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:30:04.749493 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.fernet_utils [None req-5399ca77-e703-4ffd-8f22-ba64332de1c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143586) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:30:04.757991 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 679/1355] 10.222.0.57 () {58 vars in 1018 bytes} [Thu May 19 14:30:04 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 14:30:08.673580 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cb4d0b-5d14-4450-ae25-1ea2ba98be3c None None] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:30:08.674123 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cb4d0b-5d14-4450-ae25-1ea2ba98be3c None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:30:08.674201 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cb4d0b-5d14-4450-ae25-1ea2ba98be3c None None] PATH_INFO: `/` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:30:08.675594 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 677/1356] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:30:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:30:08.681453 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d74b6dc-e0a9-477b-949d-925c9600e78c None None] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:30:08.681960 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d74b6dc-e0a9-477b-949d-925c9600e78c None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:30:08.682622 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d74b6dc-e0a9-477b-949d-925c9600e78c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:30:08.701797 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.auth.core [None req-2d74b6dc-e0a9-477b-949d-925c9600e78c None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143586) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:30:08.703209 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.fernet_utils [None req-2d74b6dc-e0a9-477b-949d-925c9600e78c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143586) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:30:08.718345 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 680/1357] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:30:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:30:08.770207 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee65e20-d64c-463e-8a93-654829ce4bdd None None] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:30:08.770841 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee65e20-d64c-463e-8a93-654829ce4bdd None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:30:08.771362 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee65e20-d64c-463e-8a93-654829ce4bdd None None] PATH_INFO: `/` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:30:08.773021 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 678/1358] 10.222.0.57 () {54 vars in 961 bytes} [Thu May 19 14:30:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:30:08.792149 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc16e358-67a3-4f4c-b950-d49e22dda6cd None None] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:30:08.794250 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc16e358-67a3-4f4c-b950-d49e22dda6cd None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:30:08.794834 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc16e358-67a3-4f4c-b950-d49e22dda6cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:30:08.857592 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.auth.core [None req-dc16e358-67a3-4f4c-b950-d49e22dda6cd None None] MFA Rules not processed for user `5f28de22c2ee4c53b6d14d5b0ddf7fe9`. Rule list: `[]` (Enabled: `True`). {{(pid=143586) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:30:08.859527 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.fernet_utils [None req-dc16e358-67a3-4f4c-b950-d49e22dda6cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143586) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:30:08.950790 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 681/1359] 10.222.0.57 () {58 vars in 1058 bytes} [Thu May 19 14:30:08 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:30:08.958542 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee65e20-d64c-463e-8a93-654829ce4bdd None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:30:08.973797 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee65e20-d64c-463e-8a93-654829ce4bdd None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:30:08.979217 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddde38d6-96ef-40cf-b656-6a7d4bf3ab81 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f28de22c2ee4c53b6d14d5b0ddf7fe9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '252d9673f08c479a98d04fee3f5856a4', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:30:08.979966 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddde38d6-96ef-40cf-b656-6a7d4bf3ab81 service nova] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:30:08.980277 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddde38d6-96ef-40cf-b656-6a7d4bf3ab81 service nova] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:30:08.980587 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddde38d6-96ef-40cf-b656-6a7d4bf3ab81 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:30:08.989511 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddde38d6-96ef-40cf-b656-6a7d4bf3ab81 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:30:08.991703 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddde38d6-96ef-40cf-b656-6a7d4bf3ab81 service nova] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:30:09.002952 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 679/1360] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 14:30:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:30:11.483035 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900e35cf-de4c-4deb-aa39-24460a93e6e3 None None] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:30:11.483618 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900e35cf-de4c-4deb-aa39-24460a93e6e3 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:30:11.483618 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900e35cf-de4c-4deb-aa39-24460a93e6e3 None None] PATH_INFO: `/` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:30:11.484973 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 682/1361] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:30:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:30:11.490488 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5973527-fc6a-47c3-bb19-21bd1cf3e548 None None] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:30:11.490925 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5973527-fc6a-47c3-bb19-21bd1cf3e548 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:30:11.491102 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5973527-fc6a-47c3-bb19-21bd1cf3e548 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:30:11.509901 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.auth.core [None req-e5973527-fc6a-47c3-bb19-21bd1cf3e548 None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143587) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:30:11.512386 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-e5973527-fc6a-47c3-bb19-21bd1cf3e548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:30:11.520530 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 680/1362] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:30:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:30:11.566133 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900e35cf-de4c-4deb-aa39-24460a93e6e3 None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:30:11.581661 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900e35cf-de4c-4deb-aa39-24460a93e6e3 None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:30:11.587129 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8c1191-fb57-4b02-a69a-170369570e69 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f28de22c2ee4c53b6d14d5b0ddf7fe9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '252d9673f08c479a98d04fee3f5856a4', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:30:11.588240 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8c1191-fb57-4b02-a69a-170369570e69 service nova] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:30:11.588349 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8c1191-fb57-4b02-a69a-170369570e69 service nova] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:30:11.588548 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8c1191-fb57-4b02-a69a-170369570e69 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:30:11.597562 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8c1191-fb57-4b02-a69a-170369570e69 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:30:11.599236 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8c1191-fb57-4b02-a69a-170369570e69 service nova] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:30:11.609614 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 683/1363] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 14:30:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:30:14.302592 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bff706-945e-439b-8f62-e99c79cc5bee None None] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:30:14.303172 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bff706-945e-439b-8f62-e99c79cc5bee None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:30:14.303172 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bff706-945e-439b-8f62-e99c79cc5bee None None] PATH_INFO: `/` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:30:14.304977 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 681/1364] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:30:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:30:14.310326 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a2dbcf-74b5-40d0-9f2c-542171c259c7 None None] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:30:14.310767 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a2dbcf-74b5-40d0-9f2c-542171c259c7 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:30:14.311085 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a2dbcf-74b5-40d0-9f2c-542171c259c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:30:14.331800 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.auth.core [None req-25a2dbcf-74b5-40d0-9f2c-542171c259c7 None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143586) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:30:14.333227 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.fernet_utils [None req-25a2dbcf-74b5-40d0-9f2c-542171c259c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143586) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:30:14.345879 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 684/1365] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:30:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:30:14.392524 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff5eb4a-91b3-401e-b30b-7db4941e41ab None None] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:30:14.392881 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff5eb4a-91b3-401e-b30b-7db4941e41ab None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:30:14.393239 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff5eb4a-91b3-401e-b30b-7db4941e41ab None None] PATH_INFO: `/` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:30:14.394750 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 682/1366] 10.222.0.57 () {54 vars in 961 bytes} [Thu May 19 14:30:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:30:14.400426 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c45528d-8f98-48c5-82a4-38fd33749194 None None] REQUEST_METHOD: `POST` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:30:14.400925 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c45528d-8f98-48c5-82a4-38fd33749194 None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:30:14.401308 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c45528d-8f98-48c5-82a4-38fd33749194 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:30:14.421423 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.auth.core [None req-7c45528d-8f98-48c5-82a4-38fd33749194 None None] MFA Rules not processed for user `5f28de22c2ee4c53b6d14d5b0ddf7fe9`. Rule list: `[]` (Enabled: `True`). {{(pid=143586) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:30:14.422979 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.fernet_utils [None req-7c45528d-8f98-48c5-82a4-38fd33749194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143586) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:30:14.433149 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 685/1367] 10.222.0.57 () {58 vars in 1058 bytes} [Thu May 19 14:30:14 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:30:14.441133 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff5eb4a-91b3-401e-b30b-7db4941e41ab None None] Authenticating user token {{(pid=143587) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:30:14.456748 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff5eb4a-91b3-401e-b30b-7db4941e41ab None None] Validating token access rules against request {{(pid=143587) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:30:14.465038 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a2bbe5-d5ee-42b3-bf42-0c7a72fd9e35 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f28de22c2ee4c53b6d14d5b0ddf7fe9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '252d9673f08c479a98d04fee3f5856a4', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143587) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:30:14.465989 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a2bbe5-d5ee-42b3-bf42-0c7a72fd9e35 service nova] REQUEST_METHOD: `GET` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:30:14.466455 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a2bbe5-d5ee-42b3-bf42-0c7a72fd9e35 service nova] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:30:14.466735 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a2bbe5-d5ee-42b3-bf42-0c7a72fd9e35 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:30:14.475873 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a2bbe5-d5ee-42b3-bf42-0c7a72fd9e35 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:30:14.477473 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a2bbe5-d5ee-42b3-bf42-0c7a72fd9e35 service nova] RBAC: Authorization granted {{(pid=143587) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:30:14.488578 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 683/1368] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 14:30:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:30:15.451398 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3b7ba0-299e-4df9-921a-f5df3425b71f None None] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:30:15.451964 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3b7ba0-299e-4df9-921a-f5df3425b71f None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:30:15.451964 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3b7ba0-299e-4df9-921a-f5df3425b71f None None] PATH_INFO: `/` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:30:15.453533 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 686/1369] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:30:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:30:15.459132 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f530e72-d32d-46ea-b9dd-738322422178 None None] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:30:15.459641 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f530e72-d32d-46ea-b9dd-738322422178 None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:30:15.459738 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f530e72-d32d-46ea-b9dd-738322422178 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:30:15.479857 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.auth.core [None req-3f530e72-d32d-46ea-b9dd-738322422178 None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143587) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:30:15.481225 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-3f530e72-d32d-46ea-b9dd-738322422178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:30:15.489561 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 684/1370] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:30:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:30:16.882870 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3424643-9a87-46df-ada2-a8166de06afb None None] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:30:16.883413 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3424643-9a87-46df-ada2-a8166de06afb None None] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:30:16.883413 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3424643-9a87-46df-ada2-a8166de06afb None None] PATH_INFO: `/` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:30:16.884772 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 687/1371] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:30:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:30:16.890664 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a41b44-3bf7-40a2-8b0c-cca0a1a0bb2a None None] REQUEST_METHOD: `POST` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:30:16.891104 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a41b44-3bf7-40a2-8b0c-cca0a1a0bb2a None None] SCRIPT_NAME: `/identity` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:30:16.891202 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a41b44-3bf7-40a2-8b0c-cca0a1a0bb2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143587) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:30:16.910252 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.auth.core [None req-c1a41b44-3bf7-40a2-8b0c-cca0a1a0bb2a None None] MFA Rules not processed for user `4f2f0290e1b04ce48a90ce03cb93d060`. Rule list: `[]` (Enabled: `True`). {{(pid=143587) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:30:16.911638 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG keystone.common.fernet_utils [None req-c1a41b44-3bf7-40a2-8b0c-cca0a1a0bb2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143587) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:30:16.922137 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: [pid: 143587|app: 0|req: 685/1372] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:30:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:30:16.966748 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3424643-9a87-46df-ada2-a8166de06afb None None] Authenticating user token {{(pid=143586) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:30:16.981525 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3424643-9a87-46df-ada2-a8166de06afb None None] Validating token access rules against request {{(pid=143586) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:30:16.986918 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d0d4c0-097b-4844-95a6-76718088a8a2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f28de22c2ee4c53b6d14d5b0ddf7fe9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '252d9673f08c479a98d04fee3f5856a4', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143586) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:30:16.987554 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d0d4c0-097b-4844-95a6-76718088a8a2 service nova] REQUEST_METHOD: `GET` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:30:16.987835 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d0d4c0-097b-4844-95a6-76718088a8a2 service nova] SCRIPT_NAME: `/identity` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:30:16.988109 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d0d4c0-097b-4844-95a6-76718088a8a2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=143586) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:30:16.996630 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d0d4c0-097b-4844-95a6-76718088a8a2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:30:16.998287 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d0d4c0-097b-4844-95a6-76718088a8a2 service nova] RBAC: Authorization granted {{(pid=143586) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:30:17.008078 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: [pid: 143586|app: 0|req: 688/1373] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 14:30:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:30:26.904652 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: DEBUG dbcounter [-] [143587] Writing DB stats keystone:SELECT=21 {{(pid=143587) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:30:27.003609 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: DEBUG dbcounter [-] [143586] Writing DB stats keystone:SELECT=95 {{(pid=143586) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:30:38.817653 dv-cinder-842541-1-iscsi systemd[1]: Stopping Devstack devstack@keystone.service... May 19 14:30:38.817835 dv-cinder-842541-1-iscsi devstack@keystone.service[143576]: Thu May 19 14:30:38 2022 - graceful shutdown triggered... May 19 14:30:38.818273 dv-cinder-842541-1-iscsi devstack@keystone.service[143586]: Gracefully killing worker 1 (pid: 143586)... May 19 14:30:38.818620 dv-cinder-842541-1-iscsi devstack@keystone.service[143587]: Gracefully killing worker 2 (pid: 143587)... May 19 14:30:39.818985 dv-cinder-842541-1-iscsi devstack@keystone.service[143576]: worker 1 buried after 1 seconds May 19 14:30:39.818985 dv-cinder-842541-1-iscsi devstack@keystone.service[143576]: worker 2 buried after 1 seconds May 19 14:30:39.818985 dv-cinder-842541-1-iscsi devstack@keystone.service[143576]: goodbye to uWSGI. May 19 14:30:39.820529 dv-cinder-842541-1-iscsi systemd[1]: devstack@keystone.service: Succeeded. May 19 14:30:39.821078 dv-cinder-842541-1-iscsi systemd[1]: Stopped Devstack devstack@keystone.service. May 19 14:36:33.977308 dv-cinder-842541-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... May 19 14:36:33.984850 dv-cinder-842541-1-iscsi devstack@keystone.service[218616]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini May 19 14:36:33.986850 dv-cinder-842541-1-iscsi devstack@keystone.service[218616]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu May 19 14:36:33 2022] *** May 19 14:36:33.986850 dv-cinder-842541-1-iscsi devstack@keystone.service[218616]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 May 19 14:36:33.986850 dv-cinder-842541-1-iscsi devstack@keystone.service[218616]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 May 19 14:36:33.986850 dv-cinder-842541-1-iscsi devstack@keystone.service[218616]: nodename: dv-cinder-842541-1-iscsi May 19 14:36:33.986850 dv-cinder-842541-1-iscsi devstack@keystone.service[218616]: machine: x86_64 May 19 14:36:33.986850 dv-cinder-842541-1-iscsi devstack@keystone.service[218616]: clock source: unix May 19 14:36:33.986850 dv-cinder-842541-1-iscsi devstack@keystone.service[218616]: pcre jit disabled May 19 14:36:33.987497 dv-cinder-842541-1-iscsi devstack@keystone.service[218616]: detected number of CPU cores: 4 May 19 14:36:33.987497 dv-cinder-842541-1-iscsi devstack@keystone.service[218616]: current working directory: / May 19 14:36:33.987497 dv-cinder-842541-1-iscsi devstack@keystone.service[218616]: detected binary path: /usr/bin/uwsgi-core May 19 14:36:33.987497 dv-cinder-842541-1-iscsi devstack@keystone.service[218616]: your processes number limit is 47680 May 19 14:36:33.987497 dv-cinder-842541-1-iscsi devstack@keystone.service[218616]: your memory page size is 4096 bytes May 19 14:36:33.987497 dv-cinder-842541-1-iscsi devstack@keystone.service[218616]: detected max file descriptor number: 2048 May 19 14:36:33.987497 dv-cinder-842541-1-iscsi devstack@keystone.service[218616]: lock engine: pthread robust mutexes May 19 14:36:33.987497 dv-cinder-842541-1-iscsi devstack@keystone.service[218616]: thunder lock: enabled May 19 14:36:33.987497 dv-cinder-842541-1-iscsi devstack@keystone.service[218616]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 May 19 14:36:33.987497 dv-cinder-842541-1-iscsi devstack@keystone.service[218616]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] May 19 14:36:34.033739 dv-cinder-842541-1-iscsi devstack@keystone.service[218616]: Python main interpreter initialized at 0x559d68fa29c0 May 19 14:36:34.033739 dv-cinder-842541-1-iscsi devstack@keystone.service[218616]: python threads support enabled May 19 14:36:34.033739 dv-cinder-842541-1-iscsi devstack@keystone.service[218616]: your server socket listen backlog is limited to 100 connections May 19 14:36:34.033739 dv-cinder-842541-1-iscsi devstack@keystone.service[218616]: your mercy for graceful operations on workers is 90 seconds May 19 14:36:34.034220 dv-cinder-842541-1-iscsi devstack@keystone.service[218616]: mapped 403077 bytes (393 KB) for 2 cores May 19 14:36:34.034220 dv-cinder-842541-1-iscsi devstack@keystone.service[218616]: *** Operational MODE: preforking *** May 19 14:36:34.034496 dv-cinder-842541-1-iscsi devstack@keystone.service[218616]: *** uWSGI is running in multiple interpreter mode *** May 19 14:36:34.034496 dv-cinder-842541-1-iscsi devstack@keystone.service[218616]: spawned uWSGI master process (pid: 218616) May 19 14:36:34.034705 dv-cinder-842541-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. May 19 14:36:34.035873 dv-cinder-842541-1-iscsi devstack@keystone.service[218616]: spawned uWSGI worker 1 (pid: 218624, cores: 1) May 19 14:36:34.035873 dv-cinder-842541-1-iscsi devstack@keystone.service[218616]: spawned uWSGI worker 2 (pid: 218625, cores: 1) May 19 14:36:34.035873 dv-cinder-842541-1-iscsi devstack@keystone.service[218616]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... May 19 14:36:39.814904 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 14:36:39.814904 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: from cryptography.utils import int_from_bytes May 19 14:36:39.815797 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 14:36:39.815797 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: from cryptography.utils import int_from_bytes May 19 14:36:39.815993 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 14:36:39.815993 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: from cryptography.utils import int_from_bytes May 19 14:36:39.816488 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 14:36:39.816488 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: from cryptography.utils import int_from_bytes May 19 14:36:39.865827 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 19 14:36:39.866043 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 19 14:36:39.866215 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] command line args: [] {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 19 14:36:39.866483 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 19 14:36:39.866659 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] ================================================================================ {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 19 14:36:39.866911 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] admin_token = **** {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:36:39.867070 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 19 14:36:39.867170 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] config_dir = [] {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:36:39.867259 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 19 14:36:39.867349 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:36:39.867443 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] command line args: [] {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 19 14:36:39.867562 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] config_source = [] {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:36:39.867663 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] debug = True {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:36:39.867755 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 19 14:36:39.867838 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:36:39.868609 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] ================================================================================ {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 19 14:36:39.868703 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] admin_token = **** {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:36:39.868779 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:36:39.868928 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] config_dir = [] {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:36:39.869085 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] insecure_debug = False {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:36:39.869210 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:36:39.869333 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] config_source = [] {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:36:39.869442 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:36:39.869551 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] debug = True {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:36:39.869665 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:36:39.869779 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:36:39.869969 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] list_limit = None {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:36:39.870088 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:36:39.870203 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] log_config_append = None {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:36:39.870316 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] insecure_debug = False {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:36:39.870429 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:36:39.870558 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:36:39.870665 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] log_dir = None {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:36:39.870665 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] log_file = None {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:36:39.870890 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:36:39.870983 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:36:39.871072 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] list_limit = None {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:36:39.871072 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] log_config_append = None {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:36:39.871289 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:36:39.871382 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:36:39.871472 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:36:39.871555 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] log_dir = None {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:36:39.871643 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:36:39.871734 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] log_file = None {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:36:39.871828 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:36:39.871918 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:36:39.872011 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:36:39.872100 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:36:39.872188 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:36:39.872276 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:36:39.872369 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(user_domain)s %(project_domain)s {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:36:39.872457 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:36:39.872547 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:36:39.872634 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:36:39.872723 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:36:39.872816 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:36:39.872914 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:36:39.873002 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:36:39.873096 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:36:39.873184 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(user_domain)s %(project_domain)s {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:36:39.873333 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:36:39.873481 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:36:39.873574 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] notification_format = cadf {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:36:39.873663 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:36:39.873857 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:36:39.873968 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:36:39.874108 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:36:39.874363 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:36:39.874458 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.57/identity {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:36:39.874547 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] publish_errors = False {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:36:39.874634 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] notification_format = cadf {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:36:39.874770 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:36:39.874872 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:36:39.875005 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:36:39.875186 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:36:39.875365 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.57/identity {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:36:39.875471 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:36:39.875574 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] publish_errors = False {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:36:39.875664 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:36:39.875777 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:36:39.875874 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] standard_threads = False {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:36:39.876018 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:36:39.876153 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] strict_password_check = False {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:36:39.876292 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:36:39.876425 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:36:39.876559 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:36:39.876682 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] use_eventlog = False {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:36:39.876814 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:36:39.876930 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] use_journal = False {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:36:39.877049 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] standard_threads = False {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:36:39.877183 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] use_json = False {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:36:39.877183 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] use_stderr = False {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:36:39.877435 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] strict_password_check = False {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:36:39.877570 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] use_syslog = False {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:36:39.877697 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:36:39.877833 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] watch_log_file = False {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:36:39.877993 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] use_eventlog = False {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:36:39.877993 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] use_journal = False {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:36:39.878262 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.878389 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] use_json = False {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:36:39.878522 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.878661 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] use_stderr = False {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:36:39.878661 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] use_syslog = False {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:36:39.878906 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.879040 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] watch_log_file = False {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:36:39.879174 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.879174 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.879430 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.879573 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.879706 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.879832 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.879959 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.880093 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.880240 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.880368 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.880494 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.880620 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.880752 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.880885 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.881025 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.881165 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.881298 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.881431 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.881558 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.881696 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.881829 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.881994 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.882130 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.882264 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.882402 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.882542 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.882676 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.882809 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.882942 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.883077 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.883210 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.883364 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.883498 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.883633 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.883789 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.884480 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.884615 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.884873 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.885024 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.885160 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.885296 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.885440 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.885576 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.885801 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.885975 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.886112 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.886248 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.886383 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.886518 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.886659 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.886794 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.886928 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.887062 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.887197 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.887332 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.887472 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.887606 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.887740 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.887740 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.888000 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.888132 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.888268 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.888404 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.888538 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.888672 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.888805 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.888939 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.889083 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.889220 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.889354 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.889489 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.889623 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.889757 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.889924 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.890062 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.890196 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.890330 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.890464 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.890601 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.890735 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.890870 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.891004 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.891144 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.891278 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.891435 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.891653 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.891803 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.891939 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.892067 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.892200 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.892340 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.892474 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.892609 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.892743 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.892878 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.893011 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.893155 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.893297 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.893430 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.893565 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.893692 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.893833 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.894031 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.894166 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.894300 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.894434 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.894568 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.894696 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.894845 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.894995 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.895123 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.895256 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.895399 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.895534 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.895672 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.895806 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.895939 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.896066 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.896205 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.896339 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.896473 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.896607 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.896743 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.896883 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.897031 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.897031 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.897300 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.897300 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.897561 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.897699 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.897834 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.897988 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.898133 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.898267 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.898401 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.898534 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.898668 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.898802 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.898942 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.898942 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.899194 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.899317 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] cache.enabled = True {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.899489 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.899783 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.899935 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.900086 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.900229 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.900357 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.900490 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.900630 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.900770 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.900905 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.901034 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.901176 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.901310 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] cache.enabled = True {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.901444 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.901583 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.901717 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.901869 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.902000 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.902138 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.902272 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.902412 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.902541 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.902674 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.902808 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.902941 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.903076 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.903217 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.903351 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.903477 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.903604 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.903729 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.903868 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.904001 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.904130 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.904263 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.904390 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.904522 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.904662 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.904795 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.904929 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.905062 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.905196 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.905330 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.905478 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.905612 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.905745 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.905898 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.906028 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.906168 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.906309 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.906444 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.906444 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.906689 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.906822 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.906955 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.907095 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.907230 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.907363 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.907491 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] auth.external = None {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.907624 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.907750 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] auth.mapped = None {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.907889 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.908017 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.908164 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.908308 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.908308 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] auth.password = None {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.908522 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.908661 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] auth.token = None {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.908802 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.909010 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.909128 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] auth.external = None {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.909261 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] catalog.caching = True {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.909395 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] auth.mapped = None {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.909530 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.909657 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.909794 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.909962 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.910092 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.910218 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] auth.password = None {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.910353 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.910492 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] auth.token = None {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.910620 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.910753 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.910880 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] credential.caching = True {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.911012 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] catalog.caching = True {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.911146 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] credential.driver = sql {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.911285 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.911422 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.911549 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.911682 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.911816 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.911951 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.912087 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.912215 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.912353 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.912482 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.912614 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] credential.caching = True {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.912749 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.912884 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] credential.driver = sql {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.913018 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.913152 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.913279 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.913412 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.913541 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.913679 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.913812 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.913983 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.914122 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.914256 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.914391 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.914531 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.914673 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.914800 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.914928 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.915064 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.915192 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] federation.caching = True {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.915323 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.915438 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.915551 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.915666 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] federation.driver = sql {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.915780 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.915901 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.916022 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.916137 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.916245 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.916359 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.916473 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.916588 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.916708 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] federation.caching = True {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.916861 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.917002 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.917119 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.917232 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] federation.driver = sql {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.917340 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.917439 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.917524 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.917612 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.917704 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.917798 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.917908 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] identity.caching = True {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.918005 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.918089 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.918180 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.918278 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.918361 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.918462 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.918462 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.918620 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.918702 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] identity.driver = sql {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.918789 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.918880 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.918975 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.919059 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.919152 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] identity.caching = True {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.919247 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.919330 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.919418 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.919512 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.919596 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.919678 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.919765 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.919848 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.919941 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.920037 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] identity.driver = sql {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.920126 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.920208 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.920304 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.920304 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.920452 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.920547 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.920637 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.920733 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.920733 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.920733 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.920962 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.921053 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.921142 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.921224 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.921312 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.921394 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.921486 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.921588 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.921677 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.921760 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.921959 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.922097 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.922230 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.922367 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.922502 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.922638 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.922638 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.922885 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.923020 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.923167 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.923295 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.923427 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.923560 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.923693 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.923820 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.923958 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.924085 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.924210 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.924348 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.924481 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.924609 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.924747 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.924874 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] ldap.password = **** {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.925006 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.925141 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.925274 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.925407 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.925544 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.925677 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.925810 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.925968 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.926118 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.926251 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.926395 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.926523 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.926655 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.926782 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.926914 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.927040 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.927179 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.927313 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.927446 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.927574 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.927707 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.927847 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.927974 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] ldap.password = **** {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.928111 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.928246 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.928379 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.928506 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.928646 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.928773 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.928906 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] ldap.user = None {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.929039 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.929180 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.929308 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.929443 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.929577 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.929711 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.929873 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.930004 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.930142 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.930296 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.930430 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.930558 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.931200 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.931333 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.931473 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.931614 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.931810 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.931971 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.932106 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.932234 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.932370 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.932585 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.932721 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] ldap.user = None {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.932856 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.932984 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.933111 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.933252 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.933400 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.933536 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.933670 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.933804 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.933965 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.934101 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.934242 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.934391 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.934562 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.934698 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.934826 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.934959 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.935096 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.935223 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.935364 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.935491 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.935619 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.935744 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.935880 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.936064 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.936193 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.936319 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.936445 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.936587 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.936734 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.936868 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.937002 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] policy.driver = sql {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.937135 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.937264 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.937397 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.937536 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.937672 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.937800 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.937986 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.938127 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] receipt.caching = True {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.938127 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.938378 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.938512 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.938647 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.938781 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.938908 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.939035 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.939173 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.939308 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.939431 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.939563 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] resource.caching = True {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.939691 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.939825 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.939961 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.940088 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] resource.driver = sql {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.940215 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] policy.driver = sql {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.940348 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.940481 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.940615 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.940755 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.940883 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.941009 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.941152 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] revoke.caching = True {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.941285 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] receipt.caching = True {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.941424 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.941564 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.941691 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.941825 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.941984 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] role.cache_time = None {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.942118 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.942257 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] role.caching = True {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.942397 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.942525 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] role.driver = sql {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.942652 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.942785 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] role.list_limit = None {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.942921 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] resource.caching = True {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.943058 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.943193 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.943321 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.943449 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] resource.driver = sql {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.943582 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.943709 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.943845 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.943986 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.944114 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.944255 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.944383 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.944517 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] revoke.caching = True {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.944653 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.944781 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.944914 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.945048 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.945176 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.945309 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] role.cache_time = None {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.945449 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.945577 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] role.caching = True {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.945703 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.945836 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] role.driver = sql {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.945996 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.946131 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] role.list_limit = None {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.946288 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.946459 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.946459 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.947216 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.947351 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.947478 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.947617 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.947733 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.947869 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.948001 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.948001 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.948242 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.948381 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.948509 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.948637 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.948782 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.948916 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.949050 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.949190 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.949323 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.949458 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.949591 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.949718 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.949870 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.950013 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.950148 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.950284 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.950412 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.950546 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.950682 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.950816 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.950816 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.951063 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.951190 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.951324 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.951461 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.951594 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.951722 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] token.cache_time = None {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.951855 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.951989 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] token.caching = True {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.952116 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.952255 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.952391 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.952519 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] token.provider = fernet {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.952645 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.952772 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.952904 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.953044 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.953179 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.953312 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.953440 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.953574 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.953712 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.953914 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.954055 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.954191 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.954324 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.954452 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] trust.driver = sql {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.954579 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.954715 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.954849 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.954977 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.955103 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.955239 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.955372 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] token.cache_time = None {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.955513 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.955641 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] token.caching = True {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.955641 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.955880 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.955880 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.956130 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] token.provider = fernet {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.956266 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.956402 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.956402 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.956643 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.956776 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.956909 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] database.connection = **** {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.957046 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.957182 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.957317 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.957451 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.957657 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.957793 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.957944 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] trust.driver = sql {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.958073 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.958073 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.958335 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.958470 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.958598 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.958739 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.958872 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.959008 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.959148 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.959275 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.959407 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.959535 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.959677 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.959811 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.960206 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.960343 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.960470 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.960645 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.960775 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] database.connection = **** {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.960898 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.961035 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.961163 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.961288 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.961414 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.961549 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.961676 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.961809 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.962566 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.962566 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=218625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 19 14:36:39.962825 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.962825 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.962825 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.962825 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.962825 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.962825 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.962825 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.962825 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.962825 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.963896 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.963896 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.963896 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.963896 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.963896 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:36:39.963896 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=218624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 19 14:36:39.984772 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=218625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:36:39.985005 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=218625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:36:39.985234 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=218625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:36:39.987249 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=218624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:36:39.987486 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=218624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:36:39.987716 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=218624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:36:39.994295 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=218625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:36:39.996746 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=218624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:36:40.022564 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=218625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:36:40.022794 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=218625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:36:40.023506 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=218624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:36:40.023746 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=218624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:36:40.077253 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=218624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:36:40.083587 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=218625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:36:40.130184 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=218624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:36:40.130414 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=218624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:36:40.132464 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=218625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:36:40.132703 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=218625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:36:40.197576 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=218625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:36:40.197822 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=218625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:36:40.198071 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=218625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:36:40.198294 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=218625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:36:40.198435 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=218624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:36:40.198546 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=218625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:36:40.198680 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=218624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:36:40.198781 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=218625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:36:40.198911 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=218624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:36:40.199139 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=218624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:36:40.199366 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=218624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:36:40.199589 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=218624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:36:40.222402 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=218625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:36:40.222628 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=218625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:36:40.222741 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=218624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:36:40.222855 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=218625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:36:40.222958 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=218624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:36:40.223080 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=218625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:36:40.223171 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=218624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:36:40.223297 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=218625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:36:40.223390 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=218624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:36:40.223510 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=218625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:36:40.223610 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=218624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:36:40.223722 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=218625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:36:40.223829 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=218624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:36:40.223921 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=218624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:36:40.235480 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=218625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:36:40.236702 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=218624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:36:40.250415 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=218624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:36:40.250586 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=218625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:36:40.250677 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=218624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:36:40.250769 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=218625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:36:40.250869 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=218624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:36:40.250961 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=218625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:36:40.251076 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=218624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:36:40.251170 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=218625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:36:40.251295 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=218624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:36:40.251402 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=218625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:36:40.251492 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=218624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:36:40.251577 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=218625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:36:40.251709 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=218624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:36:40.251803 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=218625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:36:40.269513 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=218624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.269689 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=218625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.269793 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=218624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 14:36:40.269933 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=218625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 14:36:40.270096 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=218624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.270269 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=218625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.270371 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=218624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 14:36:40.270463 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=218625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 14:36:40.270554 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=218624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.270726 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=218625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.270819 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=218624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.270986 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=218625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.271079 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=218624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.271242 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=218625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.271348 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=218624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.271495 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=218625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.282965 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=218624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.283255 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=218624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.283386 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=218625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.283528 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=218624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.283650 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=218625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.283801 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=218624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.283906 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=218625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.284184 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=218625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.292113 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=218624) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:36:40.292370 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=218625) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:36:40.295431 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=218624) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:36:40.295652 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=218625) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:36:40.295745 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=218624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.295929 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=218625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.296026 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=218624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.296207 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=218625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.296370 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=218624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.296594 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=218625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.296684 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=218624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.296876 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=218625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.296966 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=218624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.297138 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=218625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.297238 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=218624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.297408 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=218625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.297518 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=218624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.297677 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=218625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.297779 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=218624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.297971 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=218625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.298070 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=218624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.298242 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=218624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.298333 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=218625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.298507 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=218625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.325333 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=218625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.325457 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=218624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.326961 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=218625) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:36:40.327057 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=218624) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:36:40.327238 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=218625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.327372 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=218624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.332513 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=218625) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:36:40.332616 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=218624) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:36:40.332787 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=218625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.332894 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=218624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.333067 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=218624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.333249 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=218625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.341190 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=218625) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:36:40.341394 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=218624) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:36:40.341527 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=218625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.341681 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=218624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.346131 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=218625) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:36:40.346345 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=218624) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:36:40.346449 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=218625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.346632 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=218624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.346734 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=218625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.346917 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=218624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.347024 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=218625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.347198 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=218625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.347291 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=218624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.347469 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=218625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.347561 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=218624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.347735 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=218625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.347840 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=218624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.348001 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=218625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.348093 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=218624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.348256 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=218624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.366665 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=218625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.366939 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=218625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.367034 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=218624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.367252 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=218624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.371541 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=218625) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:36:40.371953 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=218624) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:36:40.375153 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=218625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.375449 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=218625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.375604 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=218624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.375906 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=218624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.380367 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=218625) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:36:40.380926 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=218624) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:36:40.384540 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=218625) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:36:40.385090 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=218624) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:36:40.388356 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=218625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.388656 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=218625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.388905 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=218624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.388999 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=218625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.389208 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=218624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.389301 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=218625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.389484 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=218625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.389594 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=218624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.389761 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=218625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.389878 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=218624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.390082 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=218624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.390374 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=218624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.409622 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=218625) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:36:40.409980 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=218624) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:36:40.410096 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=218625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.410273 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=218625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.410365 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=218624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.410544 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=218625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.410642 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=218624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.410880 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=218624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.418977 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=218625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.419517 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=218624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.420678 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=218625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.420956 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=218625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.421248 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=218624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.421536 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=218624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.424029 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=218625) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:36:40.424305 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=218625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.424593 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=218625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.424686 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=218624) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:36:40.424872 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=218625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.424965 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=218624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.425149 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=218625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.425258 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=218624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.425436 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=218624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.425730 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=218624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.439030 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=218625) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:36:40.439315 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=218625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.439594 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=218625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.439796 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=218624) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:36:40.439898 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=218625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.440085 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=218624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.440187 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=218625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.440369 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=218624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.440476 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=218625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.440653 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=218624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.440747 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=218625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.440936 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=218624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.441214 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=218624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.441502 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=218624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.460380 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=218625) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:36:40.461207 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=218624) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:36:40.464274 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=218625) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:36:40.465073 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=218624) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:36:40.467623 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=218625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.468414 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=218624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.469207 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=218625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.470050 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=218624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.470865 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=218625) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:36:40.471145 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=218625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.471423 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=218625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.471685 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=218624) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:36:40.471978 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=218624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.472263 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=218624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.478814 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=218625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.479729 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=218624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.480425 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=218625) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:36:40.481360 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=218624) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:36:40.483669 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=218625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.483960 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=218625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.484248 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=218625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.484525 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=218625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.484631 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=218624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.484909 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=218624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.485199 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=218624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.485481 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=218624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.493700 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=218625) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:36:40.494010 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=218625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.494291 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=218625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.494912 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=218624) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:36:40.495210 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=218624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.495499 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=218624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.502337 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=218625) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:36:40.502620 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=218625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.502899 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=218625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.503178 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=218625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.503449 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=218625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.503548 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=218624) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:36:40.503725 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=218625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.503820 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=218624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.504013 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=218625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.504107 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=218624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.504280 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=218625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.504382 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=218624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.504553 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=218625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.504645 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=218624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.504818 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=218625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.504925 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=218624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.505092 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=218625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.505186 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=218624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.505350 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=218625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.505443 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=218624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.505631 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=218625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.505725 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=218624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.505912 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=218625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.506009 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=218624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.506225 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=218624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.506492 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=218624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.506773 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=218624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.507039 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=218624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:36:40.545575 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x559d68fa29c0 pid: 218625 (default app) May 19 14:36:40.546920 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x559d68fa29c0 pid: 218624 (default app) May 19 14:36:40.548994 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaa3ad6-ca22-482d-a4f3-ec6f76a07a26 None None] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:40.549283 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaa3ad6-ca22-482d-a4f3-ec6f76a07a26 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:40.549560 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaa3ad6-ca22-482d-a4f3-ec6f76a07a26 None None] PATH_INFO: `/v3` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:40.553748 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 1/1] 10.222.0.57 () {50 vars in 791 bytes} [Thu May 19 14:36:40 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:36:48.572328 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db1630d-0441-415b-8519-936c0321b4ff None None] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:48.572887 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db1630d-0441-415b-8519-936c0321b4ff None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:48.572887 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db1630d-0441-415b-8519-936c0321b4ff None None] PATH_INFO: `/` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:48.577122 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 1/2] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:36:48 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:36:48.582973 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ff2cb0-214f-4d32-815a-aef3ead770bd None None] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:48.583431 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ff2cb0-214f-4d32-815a-aef3ead770bd None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:48.583535 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ff2cb0-214f-4d32-815a-aef3ead770bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:48.654003 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: INFO dbcounter [None req-05ff2cb0-214f-4d32-815a-aef3ead770bd None None] Registered counter for database keystone May 19 14:36:48.675410 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG oslo_db.sqlalchemy.engines [None req-05ff2cb0-214f-4d32-815a-aef3ead770bd None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=218625) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 19 14:36:48.681763 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writer thread running {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 19 14:36:48.867593 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG passlib.handlers.bcrypt [None req-05ff2cb0-214f-4d32-815a-aef3ead770bd None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=218625) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 19 14:36:48.867939 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG passlib.handlers.bcrypt [None req-05ff2cb0-214f-4d32-815a-aef3ead770bd None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=218625) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 19 14:36:48.925152 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.auth.core [None req-05ff2cb0-214f-4d32-815a-aef3ead770bd None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:36:48.926855 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.fernet_utils [None req-05ff2cb0-214f-4d32-815a-aef3ead770bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:36:49.053662 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 2/3] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:36:48 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 472 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) May 19 14:36:49.078090 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db1630d-0441-415b-8519-936c0321b4ff None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:49.149080 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: INFO dbcounter [None req-9db1630d-0441-415b-8519-936c0321b4ff None None] Registered counter for database keystone May 19 14:36:49.169431 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG oslo_db.sqlalchemy.engines [None req-9db1630d-0441-415b-8519-936c0321b4ff None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=218624) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 19 14:36:49.175801 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writer thread running {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 19 14:36:49.297572 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db1630d-0441-415b-8519-936c0321b4ff None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:49.302750 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cfeb2a-70df-4977-ba7a-bfe1ef3044d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3cba789bd6f4fdaaa058dac80cb6fbe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:49.303423 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cfeb2a-70df-4977-ba7a-bfe1ef3044d5 admin admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:49.303711 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cfeb2a-70df-4977-ba7a-bfe1ef3044d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:49.303985 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cfeb2a-70df-4977-ba7a-bfe1ef3044d5 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:49.312936 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cfeb2a-70df-4977-ba7a-bfe1ef3044d5 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:49.369329 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: WARNING keystone.common.rbac_enforcer.enforcer [None req-b8cfeb2a-70df-4977-ba7a-bfe1ef3044d5 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 19 14:36:49.369664 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cfeb2a-70df-4977-ba7a-bfe1ef3044d5 admin admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:49.379558 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 2/4] 10.222.0.57 () {58 vars in 1324 bytes} [Thu May 19 14:36:49 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 14:36:49.386002 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ff2cb0-214f-4d32-815a-aef3ead770bd None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:49.400674 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ff2cb0-214f-4d32-815a-aef3ead770bd None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:49.405125 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dded56-62be-43c1-9502-de7be4dd09e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3cba789bd6f4fdaaa058dac80cb6fbe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:49.406288 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dded56-62be-43c1-9502-de7be4dd09e4 admin admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:49.406582 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dded56-62be-43c1-9502-de7be4dd09e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:49.406867 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dded56-62be-43c1-9502-de7be4dd09e4 admin admin] PATH_INFO: `/v3/projects/c3cba789bd6f4fdaaa058dac80cb6fbe` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:49.408436 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dded56-62be-43c1-9502-de7be4dd09e4 admin admin] RBAC: Authorizing `identity:get_project(project_id=c3cba789bd6f4fdaaa058dac80cb6fbe)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:49.568517 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: WARNING keystone.common.rbac_enforcer.enforcer [None req-42dded56-62be-43c1-9502-de7be4dd09e4 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 19 14:36:49.568869 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dded56-62be-43c1-9502-de7be4dd09e4 admin admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:49.574256 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 3/5] 10.222.0.57 () {56 vars in 1207 bytes} [Thu May 19 14:36:49 2022] GET /identity/v3/projects/c3cba789bd6f4fdaaa058dac80cb6fbe => generated 344 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:36:51.783483 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e4728f-639a-4262-aea7-2747c10d3de0 None None] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:51.784064 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e4728f-639a-4262-aea7-2747c10d3de0 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:51.784064 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e4728f-639a-4262-aea7-2747c10d3de0 None None] PATH_INFO: `/` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:51.788239 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 3/6] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:36:51 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:36:51.795994 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5686f0-ffa5-4781-ab12-908ea5b3796e None None] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:51.796441 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5686f0-ffa5-4781-ab12-908ea5b3796e None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:51.796540 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5686f0-ffa5-4781-ab12-908ea5b3796e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:51.816144 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.auth.core [None req-9e5686f0-ffa5-4781-ab12-908ea5b3796e None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:36:51.817666 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.fernet_utils [None req-9e5686f0-ffa5-4781-ab12-908ea5b3796e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:36:51.825646 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 4/7] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:36:51 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 19 14:36:51.848642 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e4728f-639a-4262-aea7-2747c10d3de0 None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:51.861898 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e4728f-639a-4262-aea7-2747c10d3de0 None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:51.866025 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d6e559-58cd-41f0-a53b-94f6b29fbd0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3cba789bd6f4fdaaa058dac80cb6fbe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:51.866666 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d6e559-58cd-41f0-a53b-94f6b29fbd0a admin admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:51.866955 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d6e559-58cd-41f0-a53b-94f6b29fbd0a admin admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:51.867237 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d6e559-58cd-41f0-a53b-94f6b29fbd0a admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:51.875849 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d6e559-58cd-41f0-a53b-94f6b29fbd0a admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:51.877625 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d6e559-58cd-41f0-a53b-94f6b29fbd0a admin admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:51.886881 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 4/8] 10.222.0.57 () {58 vars in 1324 bytes} [Thu May 19 14:36:51 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 14:36:51.892537 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5686f0-ffa5-4781-ab12-908ea5b3796e None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:51.902042 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5686f0-ffa5-4781-ab12-908ea5b3796e None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:51.906479 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05c294b-142c-49bf-a681-61236a5e5c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3cba789bd6f4fdaaa058dac80cb6fbe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:51.907339 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05c294b-142c-49bf-a681-61236a5e5c4a admin admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:51.907623 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05c294b-142c-49bf-a681-61236a5e5c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:51.907994 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05c294b-142c-49bf-a681-61236a5e5c4a admin admin] PATH_INFO: `/v3/users/d8062aba2d374babbc9d68178a9546e3` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:51.909413 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05c294b-142c-49bf-a681-61236a5e5c4a admin admin] RBAC: Authorizing `identity:get_user(user_id=d8062aba2d374babbc9d68178a9546e3)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:51.911268 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05c294b-142c-49bf-a681-61236a5e5c4a admin admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:51.912834 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 5/9] 10.222.0.57 () {56 vars in 1198 bytes} [Thu May 19 14:36:51 2022] GET /identity/v3/users/d8062aba2d374babbc9d68178a9546e3 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:36:54.220607 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f838583e-5969-4be8-9aa8-3fac75d3aec5 None None] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:54.221153 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f838583e-5969-4be8-9aa8-3fac75d3aec5 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:54.221153 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f838583e-5969-4be8-9aa8-3fac75d3aec5 None None] PATH_INFO: `/` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:54.222560 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 5/10] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:36:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:36:54.228383 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262e72d3-9b3f-4762-98f8-39a7a1e4445d None None] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:54.228825 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262e72d3-9b3f-4762-98f8-39a7a1e4445d None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:54.228927 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262e72d3-9b3f-4762-98f8-39a7a1e4445d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:54.247012 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.auth.core [None req-262e72d3-9b3f-4762-98f8-39a7a1e4445d None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:36:54.248199 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.fernet_utils [None req-262e72d3-9b3f-4762-98f8-39a7a1e4445d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:36:54.257805 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770a72d9-0c93-4b56-960a-95c9916edf3e None None] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:54.258140 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770a72d9-0c93-4b56-960a-95c9916edf3e None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:54.258420 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770a72d9-0c93-4b56-960a-95c9916edf3e None None] PATH_INFO: `/` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:54.260405 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 6/11] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:36:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:36:54.265594 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c66a5e-e62e-4fec-bf7d-70d35fe7abc8 None None] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:54.265909 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c66a5e-e62e-4fec-bf7d-70d35fe7abc8 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:54.266189 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c66a5e-e62e-4fec-bf7d-70d35fe7abc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:54.306229 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG passlib.handlers.bcrypt [None req-c6c66a5e-e62e-4fec-bf7d-70d35fe7abc8 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=218624) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 19 14:36:54.306744 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG passlib.handlers.bcrypt [None req-c6c66a5e-e62e-4fec-bf7d-70d35fe7abc8 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=218624) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 19 14:36:54.333978 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 6/12] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:36:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:36:54.353720 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.auth.core [None req-c6c66a5e-e62e-4fec-bf7d-70d35fe7abc8 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:36:54.356355 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-c6c66a5e-e62e-4fec-bf7d-70d35fe7abc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:36:54.358194 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262e72d3-9b3f-4762-98f8-39a7a1e4445d None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:54.382311 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:54.457344 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 7/13] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:36:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:36:54.463757 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262e72d3-9b3f-4762-98f8-39a7a1e4445d None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:54.479925 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c66a5e-e62e-4fec-bf7d-70d35fe7abc8 None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:54.499614 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f5ad1c-691f-4703-a811-14e2e050e008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:54.500426 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f5ad1c-691f-4703-a811-14e2e050e008 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:54.500722 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f5ad1c-691f-4703-a811-14e2e050e008 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:54.501002 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f5ad1c-691f-4703-a811-14e2e050e008 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:54.513204 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f5ad1c-691f-4703-a811-14e2e050e008 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:36:54.524737 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-07f5ad1c-691f-4703-a811-14e2e050e008 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:36:54.524737 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:36:54.524737 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:36:54.524737 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:36:54.524737 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:36:54.524737 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:36:54.524737 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:36:54.524737 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:36:54.524737 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:36:54.524737 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:36:54.524737 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:36:54.524737 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:36:54.524737 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:36:54.524737 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:36:54.524737 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:36:54.524737 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:36:54.525810 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:36:54.525810 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:36:54.525810 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:36:54.525810 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:36:54.525810 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:36:54.525810 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:36:54.525810 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:36:54.525810 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:36:54.525810 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:36:54.525810 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:36:54.525810 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:36:54.525810 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:36:54.525810 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:36:54.525810 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:36:54.525810 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:36:54.525810 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:36:54.526756 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:36:54.526756 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:36:54.526756 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:36:54.526756 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:36:54.526756 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:36:54.526756 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:36:54.526756 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:36:54.526756 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:36:54.526756 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:36:54.526756 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:36:54.526756 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:36:54.526756 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:36:54.526756 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 7/14] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:36:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:36:54.532014 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f5ad1c-691f-4703-a811-14e2e050e008 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:54.621154 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:54.641276 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f5ad1c-691f-4703-a811-14e2e050e008 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:54.677572 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8953096f-5819-49ee-b15d-429215c6184e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:54.678288 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8953096f-5819-49ee-b15d-429215c6184e None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:54.678576 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8953096f-5819-49ee-b15d-429215c6184e None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:54.678858 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8953096f-5819-49ee-b15d-429215c6184e None admin] PATH_INFO: `/v3/roles` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:54.679631 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8953096f-5819-49ee-b15d-429215c6184e None admin] RBAC: Adding query filter params (name=admin) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:36:54.679944 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8953096f-5819-49ee-b15d-429215c6184e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:54.682150 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8953096f-5819-49ee-b15d-429215c6184e None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:54.694287 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 8/15] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:36:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:36:54.698041 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c66a5e-e62e-4fec-bf7d-70d35fe7abc8 None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:54.700525 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8953096f-5819-49ee-b15d-429215c6184e None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:54.700663 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:54.738481 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcb27df-ac48-4d66-95e2-a01d420b0bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:54.739405 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcb27df-ac48-4d66-95e2-a01d420b0bec None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:54.739697 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcb27df-ac48-4d66-95e2-a01d420b0bec None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:54.739980 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcb27df-ac48-4d66-95e2-a01d420b0bec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:54.803861 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8953096f-5819-49ee-b15d-429215c6184e None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:54.808893 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcb27df-ac48-4d66-95e2-a01d420b0bec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:54.810475 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcb27df-ac48-4d66-95e2-a01d420b0bec None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:54.839496 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8c0330-152b-497f-8fa4-4d3badf97dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:54.840376 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8c0330-152b-497f-8fa4-4d3badf97dba None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:54.840658 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8c0330-152b-497f-8fa4-4d3badf97dba None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:54.840934 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8c0330-152b-497f-8fa4-4d3badf97dba None admin] PATH_INFO: `/v3/users/d8062aba2d374babbc9d68178a9546e3` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:54.842194 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8c0330-152b-497f-8fa4-4d3badf97dba None admin] RBAC: Authorizing `identity:get_user(user_id=d8062aba2d374babbc9d68178a9546e3)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:54.843647 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8c0330-152b-497f-8fa4-4d3badf97dba None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:54.845300 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 9/16] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:36:54 2022] GET /identity/v3/users/d8062aba2d374babbc9d68178a9546e3 => generated 247 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:36:54.850646 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8c0330-152b-497f-8fa4-4d3badf97dba None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:54.865510 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:54.911394 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 8/17] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:36:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 14:36:54.916995 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcb27df-ac48-4d66-95e2-a01d420b0bec None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:54.919784 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:54.953614 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8c0330-152b-497f-8fa4-4d3badf97dba None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:54.991145 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e5a231-2b92-4244-919b-b7248affe114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:54.991946 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e5a231-2b92-4244-919b-b7248affe114 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:54.992243 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e5a231-2b92-4244-919b-b7248affe114 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:54.992525 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e5a231-2b92-4244-919b-b7248affe114 None admin] PATH_INFO: `/v3/domains/default` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:54.994501 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e5a231-2b92-4244-919b-b7248affe114 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:54.995842 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e5a231-2b92-4244-919b-b7248affe114 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:54.997639 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 10/18] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:36:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:36:55.003070 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e5a231-2b92-4244-919b-b7248affe114 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:55.022045 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcb27df-ac48-4d66-95e2-a01d420b0bec None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:55.055342 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71401a1-2673-4e2c-93d3-4cb0b1cd2614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:55.057012 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71401a1-2673-4e2c-93d3-4cb0b1cd2614 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:55.057289 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71401a1-2673-4e2c-93d3-4cb0b1cd2614 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:55.057565 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71401a1-2673-4e2c-93d3-4cb0b1cd2614 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:55.077901 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71401a1-2673-4e2c-93d3-4cb0b1cd2614 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:55.079406 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71401a1-2673-4e2c-93d3-4cb0b1cd2614 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:55.085799 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:55.100515 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-f71401a1-2673-4e2c-93d3-4cb0b1cd2614 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:36:55.100515 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:36:55.100515 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:36:55.100515 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:36:55.100515 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:36:55.100515 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:36:55.100515 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:36:55.100515 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:36:55.100515 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:36:55.100515 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:36:55.100515 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:36:55.100515 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:36:55.100515 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:36:55.100515 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:36:55.100515 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:36:55.101503 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:36:55.101503 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:36:55.101503 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:36:55.101503 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:36:55.101503 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:36:55.101503 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:36:55.101503 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:36:55.101503 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:36:55.101503 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:36:55.101503 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:36:55.101503 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:36:55.101503 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:36:55.101503 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:36:55.101503 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:36:55.101503 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:36:55.101503 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:36:55.101503 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:36:55.102647 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:36:55.102647 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:36:55.102647 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:36:55.102647 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:36:55.102647 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 9/19] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:36:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:36:55.107227 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e5a231-2b92-4244-919b-b7248affe114 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:55.107796 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71401a1-2673-4e2c-93d3-4cb0b1cd2614 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:55.143236 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cbbb19-ccb4-4fca-8800-069c4ae928ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:55.143898 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cbbb19-ccb4-4fca-8800-069c4ae928ad None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:55.144186 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cbbb19-ccb4-4fca-8800-069c4ae928ad None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:55.144465 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cbbb19-ccb4-4fca-8800-069c4ae928ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:55.145306 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cbbb19-ccb4-4fca-8800-069c4ae928ad None admin] RBAC: Adding query filter params (role.id=c2935b1a25ca46fcba419fe3d1715330, scope.domain.id=default, user.id=d8062aba2d374babbc9d68178a9546e3) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:36:55.145619 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cbbb19-ccb4-4fca-8800-069c4ae928ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:55.147123 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cbbb19-ccb4-4fca-8800-069c4ae928ad None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:55.154034 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 11/20] 10.222.0.57 () {56 vars in 1322 bytes} [Thu May 19 14:36:55 2022] GET /identity/v3/role_assignments?user.id=d8062aba2d374babbc9d68178a9546e3&scope.domain.id=default&role.id=c2935b1a25ca46fcba419fe3d1715330 => generated 233 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:36:55.186276 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:55.234899 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71401a1-2673-4e2c-93d3-4cb0b1cd2614 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:55.273163 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1396f6-4d3a-42ed-8cba-8b8500855488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:55.273735 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1396f6-4d3a-42ed-8cba-8b8500855488 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:55.274051 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1396f6-4d3a-42ed-8cba-8b8500855488 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:55.276993 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1396f6-4d3a-42ed-8cba-8b8500855488 None admin] PATH_INFO: `/v3/domains` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:55.276993 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1396f6-4d3a-42ed-8cba-8b8500855488 None admin] RBAC: Adding query filter params (name=Default) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:36:55.276993 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1396f6-4d3a-42ed-8cba-8b8500855488 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:55.276993 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1396f6-4d3a-42ed-8cba-8b8500855488 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:55.300764 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 10/21] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:36:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:36:57.757152 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5a50ef-c9ea-46bd-8854-35b435e6a0f5 None None] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:57.757707 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5a50ef-c9ea-46bd-8854-35b435e6a0f5 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:57.757707 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5a50ef-c9ea-46bd-8854-35b435e6a0f5 None None] PATH_INFO: `/` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:57.759357 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 12/22] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:36:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:36:57.764957 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa959219-4140-451e-8690-8a82aff3fb3e None None] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:57.765870 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa959219-4140-451e-8690-8a82aff3fb3e None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:57.766370 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa959219-4140-451e-8690-8a82aff3fb3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:57.794863 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.auth.core [None req-fa959219-4140-451e-8690-8a82aff3fb3e None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:36:57.796428 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-fa959219-4140-451e-8690-8a82aff3fb3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:36:57.885459 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 11/23] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:36:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:36:57.919960 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5a50ef-c9ea-46bd-8854-35b435e6a0f5 None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:58.070687 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5a50ef-c9ea-46bd-8854-35b435e6a0f5 None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:58.123502 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2d0050-5927-4fc0-a994-dc4b027c187c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:58.124314 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2d0050-5927-4fc0-a994-dc4b027c187c None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:58.124607 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2d0050-5927-4fc0-a994-dc4b027c187c None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:58.124889 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2d0050-5927-4fc0-a994-dc4b027c187c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:58.140770 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2d0050-5927-4fc0-a994-dc4b027c187c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:36:58.148865 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-da2d0050-5927-4fc0-a994-dc4b027c187c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:36:58.148865 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:36:58.148865 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:36:58.148865 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:36:58.148865 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:36:58.148865 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:36:58.148865 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:36:58.148865 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:36:58.148865 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:36:58.148865 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:36:58.148865 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:36:58.148865 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:36:58.148865 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:36:58.148865 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:36:58.148865 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:36:58.148865 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:36:58.150638 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:36:58.150638 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:36:58.150638 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:36:58.150638 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:36:58.150638 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:36:58.150638 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:36:58.150638 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:36:58.150638 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:36:58.150638 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:36:58.150638 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:36:58.150638 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:36:58.150638 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:36:58.150638 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:36:58.150638 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:36:58.150638 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:36:58.150638 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:36:58.151856 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:36:58.151856 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:36:58.151856 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:36:58.151856 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:36:58.151856 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:36:58.151856 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:36:58.151856 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:36:58.151856 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:36:58.151856 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:36:58.151856 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:36:58.151856 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:36:58.151856 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:36:58.151856 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 13/24] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:36:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:36:58.158947 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa959219-4140-451e-8690-8a82aff3fb3e None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:58.234812 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:58.311218 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa959219-4140-451e-8690-8a82aff3fb3e None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:58.357881 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8804a183-f7bd-4740-8828-19a1ae2e4937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:58.358516 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8804a183-f7bd-4740-8828-19a1ae2e4937 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:58.358805 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8804a183-f7bd-4740-8828-19a1ae2e4937 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:58.359087 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8804a183-f7bd-4740-8828-19a1ae2e4937 None admin] PATH_INFO: `/v3/roles` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:58.359897 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8804a183-f7bd-4740-8828-19a1ae2e4937 None admin] RBAC: Adding query filter params (name=admin) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:36:58.360221 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8804a183-f7bd-4740-8828-19a1ae2e4937 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:58.361706 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8804a183-f7bd-4740-8828-19a1ae2e4937 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:58.380809 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 12/25] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:36:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:36:58.387760 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2d0050-5927-4fc0-a994-dc4b027c187c None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:58.397472 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:58.531105 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2d0050-5927-4fc0-a994-dc4b027c187c None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:58.580506 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c5efe7-a695-43f2-bb80-04bbafed489d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:58.581403 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c5efe7-a695-43f2-bb80-04bbafed489d None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:58.581690 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c5efe7-a695-43f2-bb80-04bbafed489d None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:58.582003 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c5efe7-a695-43f2-bb80-04bbafed489d None admin] PATH_INFO: `/v3/users/d8062aba2d374babbc9d68178a9546e3` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:58.583649 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbc8f69-ebf2-40e5-896e-32fcbe7506f8 None None] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:58.583938 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbc8f69-ebf2-40e5-896e-32fcbe7506f8 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:58.584216 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbc8f69-ebf2-40e5-896e-32fcbe7506f8 None None] PATH_INFO: `/` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:58.585866 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 13/26] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:36:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:36:58.591063 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c5efe7-a695-43f2-bb80-04bbafed489d None admin] RBAC: Authorizing `identity:get_user(user_id=d8062aba2d374babbc9d68178a9546e3)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:58.592256 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6085847f-32c6-41bf-8cb9-c9707f933add None None] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:58.592549 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6085847f-32c6-41bf-8cb9-c9707f933add None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:58.592834 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6085847f-32c6-41bf-8cb9-c9707f933add None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:58.596375 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c5efe7-a695-43f2-bb80-04bbafed489d None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:58.598298 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 14/27] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:36:58 2022] GET /identity/v3/users/d8062aba2d374babbc9d68178a9546e3 => generated 247 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:36:58.603594 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c5efe7-a695-43f2-bb80-04bbafed489d None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:58.617261 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.auth.core [None req-6085847f-32c6-41bf-8cb9-c9707f933add None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:36:58.619789 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-6085847f-32c6-41bf-8cb9-c9707f933add None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:36:58.701803 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 14/28] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:36:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:36:58.707280 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd3daac-c165-4d24-a8b9-1488e152aff9 None None] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:58.707578 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd3daac-c165-4d24-a8b9-1488e152aff9 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:58.707864 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd3daac-c165-4d24-a8b9-1488e152aff9 None None] PATH_INFO: `/` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:58.710539 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 15/29] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:36:58 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:36:58.711611 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a99249-10b5-4b00-80e2-c32259b81a78 None None] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:58.711987 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a99249-10b5-4b00-80e2-c32259b81a78 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:58.712414 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a99249-10b5-4b00-80e2-c32259b81a78 None None] PATH_INFO: `/` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:58.716485 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 16/30] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:36:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:36:58.717462 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7998a49d-e1cf-4e5a-b766-7aff9e931679 None None] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:58.717758 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7998a49d-e1cf-4e5a-b766-7aff9e931679 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:58.718156 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7998a49d-e1cf-4e5a-b766-7aff9e931679 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:58.730960 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:58.748421 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.auth.core [None req-7998a49d-e1cf-4e5a-b766-7aff9e931679 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:36:58.749804 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-7998a49d-e1cf-4e5a-b766-7aff9e931679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:36:58.767353 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c5efe7-a695-43f2-bb80-04bbafed489d None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:58.788094 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:58.808858 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ce02ff-73c3-4951-80f4-4a9c927d93a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:58.810202 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ce02ff-73c3-4951-80f4-4a9c927d93a7 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:58.810703 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ce02ff-73c3-4951-80f4-4a9c927d93a7 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:58.811186 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ce02ff-73c3-4951-80f4-4a9c927d93a7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:58.812674 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ce02ff-73c3-4951-80f4-4a9c927d93a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:58.814575 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ce02ff-73c3-4951-80f4-4a9c927d93a7 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:58.816879 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 15/31] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:36:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:36:58.823478 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8324b962-2d5a-4f4c-85f7-e2d0d4e51a6b None None] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:58.824030 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8324b962-2d5a-4f4c-85f7-e2d0d4e51a6b None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:58.824516 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8324b962-2d5a-4f4c-85f7-e2d0d4e51a6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:58.833143 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 17/32] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:36:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:36:58.834128 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7998a49d-e1cf-4e5a-b766-7aff9e931679 None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:58.845068 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.auth.core [None req-8324b962-2d5a-4f4c-85f7-e2d0d4e51a6b None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:36:58.846790 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.fernet_utils [None req-8324b962-2d5a-4f4c-85f7-e2d0d4e51a6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:36:58.924290 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 16/33] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:36:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:36:58.925601 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8324b962-2d5a-4f4c-85f7-e2d0d4e51a6b None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:58.950188 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7998a49d-e1cf-4e5a-b766-7aff9e931679 None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:58.989829 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2de77f-8eab-4b53-80a4-209495c0f894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:58.990532 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2de77f-8eab-4b53-80a4-209495c0f894 None admin] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:58.990817 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2de77f-8eab-4b53-80a4-209495c0f894 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:58.991090 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2de77f-8eab-4b53-80a4-209495c0f894 None admin] PATH_INFO: `/v3/roles` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:58.991841 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2de77f-8eab-4b53-80a4-209495c0f894 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:58.993781 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2de77f-8eab-4b53-80a4-209495c0f894 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:59.004652 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:59.039483 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8324b962-2d5a-4f4c-85f7-e2d0d4e51a6b None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:59.077161 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d9eaab-8c11-4e41-9758-3d83b40b63a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:59.078068 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d9eaab-8c11-4e41-9758-3d83b40b63a1 None admin] REQUEST_METHOD: `PUT` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:59.078359 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d9eaab-8c11-4e41-9758-3d83b40b63a1 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:59.078648 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d9eaab-8c11-4e41-9758-3d83b40b63a1 None admin] PATH_INFO: `/v3/domains/default/users/d8062aba2d374babbc9d68178a9546e3/roles/c2935b1a25ca46fcba419fe3d1715330` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:59.080684 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d9eaab-8c11-4e41-9758-3d83b40b63a1 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=d8062aba2d374babbc9d68178a9546e3, role_id=c2935b1a25ca46fcba419fe3d1715330)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:59.082182 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d9eaab-8c11-4e41-9758-3d83b40b63a1 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:59.119615 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 18/34] 10.222.0.57 () {60 vars in 1131 bytes} [Thu May 19 14:36:58 2022] POST /identity/v3/roles => generated 225 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:36:59.120605 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2de77f-8eab-4b53-80a4-209495c0f894 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:59.171896 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:59.207897 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 17/35] 10.222.0.57 () {58 vars in 1361 bytes} [Thu May 19 14:36:58 2022] PUT /identity/v3/domains/default/users/d8062aba2d374babbc9d68178a9546e3/roles/c2935b1a25ca46fcba419fe3d1715330 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:36:59.210503 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46c87d5-1024-44c7-b5e3-a1592d320941 None None] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:59.211069 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46c87d5-1024-44c7-b5e3-a1592d320941 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:59.211602 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46c87d5-1024-44c7-b5e3-a1592d320941 None None] PATH_INFO: `/` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:59.217529 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 18/36] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:36:59 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:36:59.218874 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46c87d5-1024-44c7-b5e3-a1592d320941 None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:59.263705 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2de77f-8eab-4b53-80a4-209495c0f894 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:59.304344 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcdb265-7282-465d-8907-8f2d235a4e0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:59.305120 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcdb265-7282-465d-8907-8f2d235a4e0b None admin] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:59.305506 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcdb265-7282-465d-8907-8f2d235a4e0b None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:59.306006 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcdb265-7282-465d-8907-8f2d235a4e0b None admin] PATH_INFO: `/v3/roles` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:59.307162 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcdb265-7282-465d-8907-8f2d235a4e0b None admin] RBAC: Authorizing `identity:create_role()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:59.308402 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcdb265-7282-465d-8907-8f2d235a4e0b None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:59.326741 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 19/37] 10.222.0.57 () {60 vars in 1131 bytes} [Thu May 19 14:36:59 2022] POST /identity/v3/roles => generated 219 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:36:59.328104 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9455755d-84db-4d06-a5e2-abb86930cee6 None None] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:59.328404 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9455755d-84db-4d06-a5e2-abb86930cee6 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:59.328689 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9455755d-84db-4d06-a5e2-abb86930cee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:59.349132 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46c87d5-1024-44c7-b5e3-a1592d320941 None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:59.367244 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.auth.core [None req-9455755d-84db-4d06-a5e2-abb86930cee6 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:36:59.369560 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-9455755d-84db-4d06-a5e2-abb86930cee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:36:59.386622 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:59.393583 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc15996c-717c-4d4b-ac75-63bc36a34d83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:59.394847 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc15996c-717c-4d4b-ac75-63bc36a34d83 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:59.395370 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc15996c-717c-4d4b-ac75-63bc36a34d83 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:59.395888 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc15996c-717c-4d4b-ac75-63bc36a34d83 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:59.410617 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc15996c-717c-4d4b-ac75-63bc36a34d83 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:59.412774 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc15996c-717c-4d4b-ac75-63bc36a34d83 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:59.438827 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-cc15996c-717c-4d4b-ac75-63bc36a34d83 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:36:59.438827 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:36:59.438827 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:36:59.438827 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:36:59.438827 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:36:59.438827 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:36:59.438827 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:36:59.438827 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:36:59.438827 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:36:59.438827 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:36:59.438827 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:36:59.438827 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:36:59.438827 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:36:59.438827 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:36:59.438827 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:36:59.439848 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:36:59.439848 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:36:59.439848 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:36:59.439848 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:36:59.439848 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:36:59.439848 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:36:59.439848 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:36:59.439848 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:36:59.439848 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:36:59.439848 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:36:59.439848 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:36:59.439848 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:36:59.439848 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:36:59.439848 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:36:59.439848 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:36:59.439848 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:36:59.439848 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:36:59.440831 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:36:59.440831 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:36:59.440831 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:36:59.440831 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:36:59.443430 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 19/38] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:36:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:36:59.452015 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc15996c-717c-4d4b-ac75-63bc36a34d83 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:59.565792 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 20/39] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:36:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:36:59.592519 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9455755d-84db-4d06-a5e2-abb86930cee6 None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:59.618804 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:59.620366 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc15996c-717c-4d4b-ac75-63bc36a34d83 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:59.662560 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5766b350-c93f-41f0-8199-04f76e736a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:59.662912 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5766b350-c93f-41f0-8199-04f76e736a55 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:59.663116 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5766b350-c93f-41f0-8199-04f76e736a55 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:59.663408 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5766b350-c93f-41f0-8199-04f76e736a55 None admin] PATH_INFO: `/v3/domains` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:59.664153 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5766b350-c93f-41f0-8199-04f76e736a55 None admin] RBAC: Adding query filter params (name=Default) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:36:59.664465 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5766b350-c93f-41f0-8199-04f76e736a55 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:59.666047 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5766b350-c93f-41f0-8199-04f76e736a55 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:59.688168 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 20/40] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:36:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:36:59.693945 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5766b350-c93f-41f0-8199-04f76e736a55 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:59.745105 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:59.748939 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9455755d-84db-4d06-a5e2-abb86930cee6 None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:59.797725 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93118f5-3d29-4765-98b0-a7c4cb670d84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:59.798409 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93118f5-3d29-4765-98b0-a7c4cb670d84 None admin] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:59.798711 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93118f5-3d29-4765-98b0-a7c4cb670d84 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:59.799069 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93118f5-3d29-4765-98b0-a7c4cb670d84 None admin] PATH_INFO: `/v3/roles` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:59.799842 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93118f5-3d29-4765-98b0-a7c4cb670d84 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:59.801344 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93118f5-3d29-4765-98b0-a7c4cb670d84 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:59.828121 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 21/41] 10.222.0.57 () {60 vars in 1131 bytes} [Thu May 19 14:36:59 2022] POST /identity/v3/roles => generated 223 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:36:59.840102 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5766b350-c93f-41f0-8199-04f76e736a55 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:59.893907 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4322a7b-4adb-4d85-aa18-6d81fc845a25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:59.894622 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4322a7b-4adb-4d85-aa18-6d81fc845a25 None admin] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:59.894912 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4322a7b-4adb-4d85-aa18-6d81fc845a25 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:59.895199 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4322a7b-4adb-4d85-aa18-6d81fc845a25 None admin] PATH_INFO: `/v3/projects` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:59.895948 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4322a7b-4adb-4d85-aa18-6d81fc845a25 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:59.897567 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4322a7b-4adb-4d85-aa18-6d81fc845a25 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:59.935061 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 21/42] 10.222.0.57 () {60 vars in 1141 bytes} [Thu May 19 14:36:59 2022] POST /identity/v3/projects => generated 301 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 14:37:00.032619 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cec5ba-2219-49d5-b47b-4c5a4be46257 None None] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:00.033088 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cec5ba-2219-49d5-b47b-4c5a4be46257 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:00.033498 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cec5ba-2219-49d5-b47b-4c5a4be46257 None None] PATH_INFO: `/` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:00.035135 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 22/43] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:37:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:37:00.041539 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a012d7e7-df66-4611-bebc-efd6e39cfa67 None None] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:00.041884 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a012d7e7-df66-4611-bebc-efd6e39cfa67 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:00.042152 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a012d7e7-df66-4611-bebc-efd6e39cfa67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:00.064939 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.auth.core [None req-a012d7e7-df66-4611-bebc-efd6e39cfa67 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:37:00.066694 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.fernet_utils [None req-a012d7e7-df66-4611-bebc-efd6e39cfa67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:37:00.150486 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 22/44] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:37:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:37:00.173577 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cec5ba-2219-49d5-b47b-4c5a4be46257 None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:00.267607 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:00.290821 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cec5ba-2219-49d5-b47b-4c5a4be46257 None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:00.325130 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0a5058-695b-408f-b82a-8fc6957bf6d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:00.325958 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0a5058-695b-408f-b82a-8fc6957bf6d6 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:00.326249 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0a5058-695b-408f-b82a-8fc6957bf6d6 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:00.326527 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0a5058-695b-408f-b82a-8fc6957bf6d6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:00.327969 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0a5058-695b-408f-b82a-8fc6957bf6d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:00.329438 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0a5058-695b-408f-b82a-8fc6957bf6d6 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:00.331143 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 23/45] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:37:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:00.337164 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a012d7e7-df66-4611-bebc-efd6e39cfa67 None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:00.366558 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:00.440873 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a012d7e7-df66-4611-bebc-efd6e39cfa67 None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:00.474646 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0b0f08-011f-4a72-bd31-ab3fddf9301f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:00.475303 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0b0f08-011f-4a72-bd31-ab3fddf9301f None admin] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:00.475590 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0b0f08-011f-4a72-bd31-ab3fddf9301f None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:00.475868 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0b0f08-011f-4a72-bd31-ab3fddf9301f None admin] PATH_INFO: `/v3/projects` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:00.476587 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0b0f08-011f-4a72-bd31-ab3fddf9301f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:00.478356 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0b0f08-011f-4a72-bd31-ab3fddf9301f None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:00.502791 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 23/46] 10.222.0.57 () {60 vars in 1141 bytes} [Thu May 19 14:37:00 2022] POST /identity/v3/projects => generated 312 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:37:01.752695 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857e16b0-9608-4897-8ece-0ffb791af2b6 None None] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:01.753254 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857e16b0-9608-4897-8ece-0ffb791af2b6 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:01.753254 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857e16b0-9608-4897-8ece-0ffb791af2b6 None None] PATH_INFO: `/` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:01.754680 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 24/47] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:37:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:37:01.760705 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027ae747-e6b5-4347-ae8a-53714d6d9ae2 None None] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:01.761160 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027ae747-e6b5-4347-ae8a-53714d6d9ae2 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:01.761269 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027ae747-e6b5-4347-ae8a-53714d6d9ae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:01.780047 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.auth.core [None req-027ae747-e6b5-4347-ae8a-53714d6d9ae2 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:37:01.781272 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.fernet_utils [None req-027ae747-e6b5-4347-ae8a-53714d6d9ae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:37:01.849705 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 24/48] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:37:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:37:01.873374 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857e16b0-9608-4897-8ece-0ffb791af2b6 None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:01.975637 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857e16b0-9608-4897-8ece-0ffb791af2b6 None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:02.010345 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f6d2d5-627e-4b36-846b-abe9f01359fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:02.011069 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f6d2d5-627e-4b36-846b-abe9f01359fc None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:02.011362 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f6d2d5-627e-4b36-846b-abe9f01359fc None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:02.011646 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f6d2d5-627e-4b36-846b-abe9f01359fc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:02.023273 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f6d2d5-627e-4b36-846b-abe9f01359fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:37:02.032182 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-c8f6d2d5-627e-4b36-846b-abe9f01359fc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:37:02.032182 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:37:02.032182 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:37:02.032182 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:37:02.032182 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:37:02.032182 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:37:02.032182 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:37:02.032182 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:37:02.032182 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:37:02.032182 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:37:02.032182 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:37:02.032182 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:37:02.032182 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:37:02.032182 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:37:02.032182 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:37:02.032182 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:37:02.033346 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:37:02.033346 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:37:02.033346 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:37:02.033346 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:37:02.033346 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:37:02.033346 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:37:02.033346 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:37:02.033346 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:37:02.033346 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:37:02.033346 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:37:02.033346 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:37:02.033346 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:37:02.033346 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:37:02.033346 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:37:02.033346 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:37:02.033346 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:37:02.034361 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:37:02.034361 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:37:02.034361 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:37:02.034361 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:37:02.034361 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:37:02.034361 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:37:02.034361 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:37:02.034361 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:37:02.034361 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:37:02.034361 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:37:02.034361 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:37:02.034361 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:37:02.034361 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 25/49] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:37:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:37:02.039225 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027ae747-e6b5-4347-ae8a-53714d6d9ae2 None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:02.048615 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:02.146361 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027ae747-e6b5-4347-ae8a-53714d6d9ae2 None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:02.184002 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae274a7d-fb6a-49d1-a033-7a31e3e58ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:02.184693 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae274a7d-fb6a-49d1-a033-7a31e3e58ae8 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:02.184984 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae274a7d-fb6a-49d1-a033-7a31e3e58ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:02.185265 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae274a7d-fb6a-49d1-a033-7a31e3e58ae8 None admin] PATH_INFO: `/v3/roles` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:02.186050 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae274a7d-fb6a-49d1-a033-7a31e3e58ae8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:37:02.186366 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae274a7d-fb6a-49d1-a033-7a31e3e58ae8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:02.187770 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae274a7d-fb6a-49d1-a033-7a31e3e58ae8 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:02.196996 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 25/50] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:37:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:02.202471 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f6d2d5-627e-4b36-846b-abe9f01359fc None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:02.211541 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:02.306478 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f6d2d5-627e-4b36-846b-abe9f01359fc None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:02.341264 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6977ef01-7113-45ef-abcd-feec0a80727a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:02.342181 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6977ef01-7113-45ef-abcd-feec0a80727a None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:02.342473 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6977ef01-7113-45ef-abcd-feec0a80727a None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:02.342757 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6977ef01-7113-45ef-abcd-feec0a80727a None admin] PATH_INFO: `/v3/users/d8062aba2d374babbc9d68178a9546e3` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:02.344197 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6977ef01-7113-45ef-abcd-feec0a80727a None admin] RBAC: Authorizing `identity:get_user(user_id=d8062aba2d374babbc9d68178a9546e3)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:02.345675 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6977ef01-7113-45ef-abcd-feec0a80727a None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:02.347388 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 26/51] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:37:02 2022] GET /identity/v3/users/d8062aba2d374babbc9d68178a9546e3 => generated 247 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:02.352852 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae274a7d-fb6a-49d1-a033-7a31e3e58ae8 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:02.420398 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:02.456310 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae274a7d-fb6a-49d1-a033-7a31e3e58ae8 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:02.490261 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e773f70-51f5-4085-8b0b-943958c629b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:02.491064 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e773f70-51f5-4085-8b0b-943958c629b9 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:02.491357 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e773f70-51f5-4085-8b0b-943958c629b9 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:02.491640 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e773f70-51f5-4085-8b0b-943958c629b9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:02.493364 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e773f70-51f5-4085-8b0b-943958c629b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:02.494977 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e773f70-51f5-4085-8b0b-943958c629b9 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:02.496622 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 26/52] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:37:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:37:02.502423 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6977ef01-7113-45ef-abcd-feec0a80727a None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:02.578851 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:02.606944 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6977ef01-7113-45ef-abcd-feec0a80727a None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:02.641217 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba17812-08ba-4c0f-8d04-be00930d83a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:02.641923 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba17812-08ba-4c0f-8d04-be00930d83a1 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:02.642251 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba17812-08ba-4c0f-8d04-be00930d83a1 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:02.642526 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba17812-08ba-4c0f-8d04-be00930d83a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:02.643373 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba17812-08ba-4c0f-8d04-be00930d83a1 None admin] RBAC: Adding query filter params (role.id=c2935b1a25ca46fcba419fe3d1715330, scope.domain.id=default, user.id=d8062aba2d374babbc9d68178a9546e3) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:37:02.643694 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba17812-08ba-4c0f-8d04-be00930d83a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:02.645180 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba17812-08ba-4c0f-8d04-be00930d83a1 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:02.654692 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 27/53] 10.222.0.57 () {56 vars in 1322 bytes} [Thu May 19 14:37:02 2022] GET /identity/v3/role_assignments?user.id=d8062aba2d374babbc9d68178a9546e3&scope.domain.id=default&role.id=c2935b1a25ca46fcba419fe3d1715330 => generated 530 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:02.772478 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949f8949-4f19-49cb-843e-9cd8bf7a6650 None None] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:02.773198 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949f8949-4f19-49cb-843e-9cd8bf7a6650 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:02.773198 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949f8949-4f19-49cb-843e-9cd8bf7a6650 None None] PATH_INFO: `/` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:02.775205 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 27/54] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:37:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:37:02.783443 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07cafc3-e70a-444e-8fcc-1764ec65346e None None] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:02.784018 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07cafc3-e70a-444e-8fcc-1764ec65346e None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:02.784271 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07cafc3-e70a-444e-8fcc-1764ec65346e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:02.804692 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.auth.core [None req-a07cafc3-e70a-444e-8fcc-1764ec65346e None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:37:02.806742 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-a07cafc3-e70a-444e-8fcc-1764ec65346e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:37:02.879461 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 28/55] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:37:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:37:02.902740 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949f8949-4f19-49cb-843e-9cd8bf7a6650 None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:03.005379 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949f8949-4f19-49cb-843e-9cd8bf7a6650 None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:03.038940 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee3e1a5-0034-42b9-b876-6fbf32d734eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:03.040280 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee3e1a5-0034-42b9-b876-6fbf32d734eb None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:03.040573 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee3e1a5-0034-42b9-b876-6fbf32d734eb None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:03.040999 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee3e1a5-0034-42b9-b876-6fbf32d734eb None admin] PATH_INFO: `/v3/domains/default` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:03.042229 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee3e1a5-0034-42b9-b876-6fbf32d734eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:03.043717 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee3e1a5-0034-42b9-b876-6fbf32d734eb None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:03.045357 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 28/56] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:37:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:03.050988 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07cafc3-e70a-444e-8fcc-1764ec65346e None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:03.117090 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:03.150306 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07cafc3-e70a-444e-8fcc-1764ec65346e None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:03.184250 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd46c89-83ac-4715-aa8c-7c33f6a8169d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:03.184918 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd46c89-83ac-4715-aa8c-7c33f6a8169d None admin] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:03.185205 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd46c89-83ac-4715-aa8c-7c33f6a8169d None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:03.185482 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd46c89-83ac-4715-aa8c-7c33f6a8169d None admin] PATH_INFO: `/v3/projects` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:03.186230 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd46c89-83ac-4715-aa8c-7c33f6a8169d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:03.187714 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd46c89-83ac-4715-aa8c-7c33f6a8169d None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:03.214055 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 29/57] 10.222.0.57 () {60 vars in 1140 bytes} [Thu May 19 14:37:03 2022] POST /identity/v3/projects => generated 298 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:37:05.464648 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9140e35e-0a5a-446e-88ba-3a2e6eb14958 None None] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:05.465171 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9140e35e-0a5a-446e-88ba-3a2e6eb14958 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:05.465171 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9140e35e-0a5a-446e-88ba-3a2e6eb14958 None None] PATH_INFO: `/` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:05.466577 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 29/58] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:37:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:37:05.471744 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65642c16-fb11-4c26-ac5a-94a2d2470a0d None None] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:05.472120 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65642c16-fb11-4c26-ac5a-94a2d2470a0d None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:05.472312 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65642c16-fb11-4c26-ac5a-94a2d2470a0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:05.488851 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.auth.core [None req-65642c16-fb11-4c26-ac5a-94a2d2470a0d None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:37:05.490073 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-65642c16-fb11-4c26-ac5a-94a2d2470a0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:37:05.554680 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 30/59] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:37:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:37:05.579826 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9140e35e-0a5a-446e-88ba-3a2e6eb14958 None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:05.583721 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:05.681493 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9140e35e-0a5a-446e-88ba-3a2e6eb14958 None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:05.714754 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0923661e-621b-48e2-8f40-42ccaea3c61a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:05.715485 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0923661e-621b-48e2-8f40-42ccaea3c61a None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:05.715778 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0923661e-621b-48e2-8f40-42ccaea3c61a None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:05.716063 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0923661e-621b-48e2-8f40-42ccaea3c61a None admin] PATH_INFO: `/v3/domains/default` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:05.717266 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0923661e-621b-48e2-8f40-42ccaea3c61a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:05.718764 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0923661e-621b-48e2-8f40-42ccaea3c61a None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:05.720388 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 30/60] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:37:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:05.725934 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65642c16-fb11-4c26-ac5a-94a2d2470a0d None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:05.776398 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:05.827825 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65642c16-fb11-4c26-ac5a-94a2d2470a0d None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:05.861622 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c09c62c-4afb-4d0d-934d-3b8d93d70da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:05.862323 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c09c62c-4afb-4d0d-934d-3b8d93d70da3 None admin] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:05.862628 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c09c62c-4afb-4d0d-934d-3b8d93d70da3 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:05.862885 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c09c62c-4afb-4d0d-934d-3b8d93d70da3 None admin] PATH_INFO: `/v3/users` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:05.863605 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c09c62c-4afb-4d0d-934d-3b8d93d70da3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:05.865074 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c09c62c-4afb-4d0d-934d-3b8d93d70da3 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:05.890826 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 31/61] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 19 14:37:05 2022] POST /identity/v3/users => generated 275 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:37:09.362737 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b396ee75-8c61-47b5-a005-0f1c7b89790e None None] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:09.363254 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b396ee75-8c61-47b5-a005-0f1c7b89790e None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:09.363335 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b396ee75-8c61-47b5-a005-0f1c7b89790e None None] PATH_INFO: `/` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:09.364706 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 31/62] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:37:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:37:09.376239 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2441350-e8f1-45fc-93cf-c4a61d5d202f None None] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:09.376749 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2441350-e8f1-45fc-93cf-c4a61d5d202f None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:09.376840 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2441350-e8f1-45fc-93cf-c4a61d5d202f None None] PATH_INFO: `/` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:09.378254 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 32/63] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:37:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:37:09.384165 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f4b116-7afb-459a-afbc-e0242ca7fd18 None None] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:09.384330 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4929388f-26aa-4e44-bbdb-92892dae9393 None None] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:09.384543 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4929388f-26aa-4e44-bbdb-92892dae9393 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:09.384999 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f4b116-7afb-459a-afbc-e0242ca7fd18 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:09.385294 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f4b116-7afb-459a-afbc-e0242ca7fd18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:09.385500 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4929388f-26aa-4e44-bbdb-92892dae9393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:09.413333 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.auth.core [None req-a6f4b116-7afb-459a-afbc-e0242ca7fd18 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:37:09.414994 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.fernet_utils [None req-a6f4b116-7afb-459a-afbc-e0242ca7fd18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:37:09.422340 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.auth.core [None req-4929388f-26aa-4e44-bbdb-92892dae9393 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:37:09.429038 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-4929388f-26aa-4e44-bbdb-92892dae9393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:37:09.539231 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 32/64] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:37:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:37:09.572767 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 33/65] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:37:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:37:09.582942 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f4b116-7afb-459a-afbc-e0242ca7fd18 None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:09.585373 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:09.598064 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4929388f-26aa-4e44-bbdb-92892dae9393 None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:09.636945 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:09.711615 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4929388f-26aa-4e44-bbdb-92892dae9393 None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:09.758143 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f4b116-7afb-459a-afbc-e0242ca7fd18 None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:09.760384 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2d1dff-24f2-4428-83a0-177baf09e832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:09.761487 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2d1dff-24f2-4428-83a0-177baf09e832 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:09.761782 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2d1dff-24f2-4428-83a0-177baf09e832 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:09.762093 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2d1dff-24f2-4428-83a0-177baf09e832 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:09.779593 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2d1dff-24f2-4428-83a0-177baf09e832 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:37:09.790668 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-6d2d1dff-24f2-4428-83a0-177baf09e832 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:37:09.790668 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:37:09.790668 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:37:09.790668 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:37:09.790668 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:37:09.790668 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:37:09.790668 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:37:09.790668 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:37:09.790668 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:37:09.790668 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:37:09.790668 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:37:09.790668 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:37:09.790668 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:37:09.790668 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:37:09.790668 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:37:09.790668 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:37:09.791661 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:37:09.791661 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:37:09.791661 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:37:09.791661 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:37:09.791661 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:37:09.791661 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:37:09.791661 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:37:09.791661 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:37:09.791661 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:37:09.791661 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:37:09.791661 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:37:09.791661 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:37:09.791661 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:37:09.791661 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:37:09.791661 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:37:09.791661 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:37:09.796074 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:37:09.796074 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:37:09.796074 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:37:09.796074 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:37:09.796074 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:37:09.796074 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:37:09.796074 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:37:09.796074 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:37:09.796074 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:37:09.796074 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:37:09.796074 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:37:09.796074 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:37:09.796074 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 34/66] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:37:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:37:09.796074 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e84daa6-117c-480e-b80f-a37870141ec1 None None] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:09.798643 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e84daa6-117c-480e-b80f-a37870141ec1 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:09.798643 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e84daa6-117c-480e-b80f-a37870141ec1 None None] PATH_INFO: `/` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:09.798643 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 35/67] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:37:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:37:09.801253 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8654f8ff-315c-45aa-8972-37e2880f168b None None] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:09.801556 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8654f8ff-315c-45aa-8972-37e2880f168b None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:09.801877 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8654f8ff-315c-45aa-8972-37e2880f168b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:09.808583 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67f3a54-5966-4cd1-942d-8fc350ad4fc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:09.809678 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67f3a54-5966-4cd1-942d-8fc350ad4fc3 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:09.810227 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67f3a54-5966-4cd1-942d-8fc350ad4fc3 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:09.810729 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67f3a54-5966-4cd1-942d-8fc350ad4fc3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:09.823289 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67f3a54-5966-4cd1-942d-8fc350ad4fc3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:37:09.823738 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.auth.core [None req-8654f8ff-315c-45aa-8972-37e2880f168b None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:37:09.824992 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-8654f8ff-315c-45aa-8972-37e2880f168b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:37:09.831588 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-c67f3a54-5966-4cd1-942d-8fc350ad4fc3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:37:09.831588 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:37:09.831588 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:37:09.831588 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:37:09.831588 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:37:09.831588 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:37:09.831588 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:37:09.831588 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:37:09.831588 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:37:09.831588 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:37:09.831588 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:37:09.831588 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:37:09.831588 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:37:09.831588 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:37:09.831588 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:37:09.831588 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:37:09.832555 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:37:09.832555 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:37:09.832555 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:37:09.832555 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:37:09.832555 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:37:09.832555 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:37:09.832555 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:37:09.832555 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:37:09.832555 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:37:09.832555 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:37:09.832555 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:37:09.832555 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:37:09.832555 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:37:09.832555 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:37:09.832555 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:37:09.832555 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:37:09.833431 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:37:09.833431 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:37:09.833431 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:37:09.833431 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:37:09.833431 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:37:09.833431 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:37:09.833431 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:37:09.833431 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:37:09.833431 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:37:09.833431 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:37:09.833431 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:37:09.833431 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:37:09.833431 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 33/68] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:37:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:37:09.834200 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67f3a54-5966-4cd1-942d-8fc350ad4fc3 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:09.899933 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 36/69] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:37:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:37:09.901747 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7a8be8-8bc9-4439-a82e-8435715e12a7 None None] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:09.902078 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7a8be8-8bc9-4439-a82e-8435715e12a7 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:09.902363 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7a8be8-8bc9-4439-a82e-8435715e12a7 None None] PATH_INFO: `/` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:09.903837 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 37/70] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:37:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:37:09.908649 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7a8be8-8bc9-4439-a82e-8435715e12a7 None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:09.919534 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:09.944084 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:09.971015 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67f3a54-5966-4cd1-942d-8fc350ad4fc3 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:10.014478 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc55203b-36eb-4279-8e20-afabbe6eab0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:10.015582 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc55203b-36eb-4279-8e20-afabbe6eab0f None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:10.015878 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc55203b-36eb-4279-8e20-afabbe6eab0f None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:10.016170 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc55203b-36eb-4279-8e20-afabbe6eab0f None admin] PATH_INFO: `/v3/roles` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:10.016926 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc55203b-36eb-4279-8e20-afabbe6eab0f None admin] RBAC: Adding query filter params (name=admin) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:37:10.017244 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc55203b-36eb-4279-8e20-afabbe6eab0f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:10.018808 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc55203b-36eb-4279-8e20-afabbe6eab0f None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:10.030446 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7a8be8-8bc9-4439-a82e-8435715e12a7 None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:10.032434 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 34/71] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:37:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:10.033661 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6097f87-9a47-45fb-9d2b-90084f2f5946 None None] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:10.033996 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6097f87-9a47-45fb-9d2b-90084f2f5946 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:10.034284 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6097f87-9a47-45fb-9d2b-90084f2f5946 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:10.051027 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.auth.core [None req-f6097f87-9a47-45fb-9d2b-90084f2f5946 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:37:10.052231 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.fernet_utils [None req-f6097f87-9a47-45fb-9d2b-90084f2f5946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:37:10.073080 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf4ea7b-8e7f-49b2-95ec-294be0e1210a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:10.073834 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf4ea7b-8e7f-49b2-95ec-294be0e1210a None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:10.074154 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf4ea7b-8e7f-49b2-95ec-294be0e1210a None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:10.074438 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf4ea7b-8e7f-49b2-95ec-294be0e1210a None admin] PATH_INFO: `/v3/roles` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:10.075205 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf4ea7b-8e7f-49b2-95ec-294be0e1210a None admin] RBAC: Adding query filter params (name=member) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:37:10.075522 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf4ea7b-8e7f-49b2-95ec-294be0e1210a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:10.076937 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf4ea7b-8e7f-49b2-95ec-294be0e1210a None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:10.086283 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 38/72] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:37:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:10.087319 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf4ea7b-8e7f-49b2-95ec-294be0e1210a None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:10.135092 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 35/73] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:37:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:37:10.136321 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6097f87-9a47-45fb-9d2b-90084f2f5946 None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:10.190694 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:10.194498 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:10.204780 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf4ea7b-8e7f-49b2-95ec-294be0e1210a None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:10.239230 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6097f87-9a47-45fb-9d2b-90084f2f5946 None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:10.239417 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef6db16-52d6-43bf-b4b6-750ec622706f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:10.240036 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef6db16-52d6-43bf-b4b6-750ec622706f None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:10.240327 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef6db16-52d6-43bf-b4b6-750ec622706f None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:10.240608 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef6db16-52d6-43bf-b4b6-750ec622706f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:10.252254 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef6db16-52d6-43bf-b4b6-750ec622706f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:37:10.261239 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-aef6db16-52d6-43bf-b4b6-750ec622706f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:37:10.261239 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:37:10.261239 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:37:10.261239 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:37:10.261239 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:37:10.261239 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:37:10.261239 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:37:10.261239 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:37:10.261239 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:37:10.261239 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:37:10.261239 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:37:10.261239 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:37:10.261239 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:37:10.261239 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:37:10.261239 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:37:10.261239 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:37:10.262433 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:37:10.262433 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:37:10.262433 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:37:10.262433 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:37:10.262433 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:37:10.262433 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:37:10.262433 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:37:10.262433 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:37:10.262433 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:37:10.262433 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:37:10.262433 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:37:10.262433 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:37:10.262433 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:37:10.262433 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:37:10.262433 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:37:10.262433 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:37:10.263524 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:37:10.263524 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:37:10.263524 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:37:10.263524 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:37:10.263524 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:37:10.263524 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:37:10.263524 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:37:10.263524 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:37:10.263524 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:37:10.263524 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:37:10.263524 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:37:10.263524 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:37:10.263524 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 39/74] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 19 14:37:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:37:10.264523 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef6db16-52d6-43bf-b4b6-750ec622706f None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:10.284053 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a29fdf9-25f8-423b-b277-6e73ec4e0ca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:10.284943 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a29fdf9-25f8-423b-b277-6e73ec4e0ca0 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:10.285225 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a29fdf9-25f8-423b-b277-6e73ec4e0ca0 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:10.285501 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a29fdf9-25f8-423b-b277-6e73ec4e0ca0 None admin] PATH_INFO: `/v3/users/d8062aba2d374babbc9d68178a9546e3` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:10.286752 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a29fdf9-25f8-423b-b277-6e73ec4e0ca0 None admin] RBAC: Authorizing `identity:get_user(user_id=d8062aba2d374babbc9d68178a9546e3)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:10.288225 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a29fdf9-25f8-423b-b277-6e73ec4e0ca0 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:10.290414 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 36/75] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:37:10 2022] GET /identity/v3/users/d8062aba2d374babbc9d68178a9546e3 => generated 247 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:10.295594 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b79bc3-a762-43e8-92dd-4f550694d1d3 None None] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:10.295594 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b79bc3-a762-43e8-92dd-4f550694d1d3 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:10.295594 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b79bc3-a762-43e8-92dd-4f550694d1d3 None None] PATH_INFO: `/` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:10.295594 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 37/76] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:37:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:37:10.295594 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b79bc3-a762-43e8-92dd-4f550694d1d3 None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:10.372234 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef6db16-52d6-43bf-b4b6-750ec622706f None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:10.399412 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b79bc3-a762-43e8-92dd-4f550694d1d3 None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:10.406348 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe9c866-e562-4809-9eac-256d7df396a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:10.407231 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe9c866-e562-4809-9eac-256d7df396a2 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:10.407514 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe9c866-e562-4809-9eac-256d7df396a2 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:10.407791 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe9c866-e562-4809-9eac-256d7df396a2 None admin] PATH_INFO: `/v3/users/d15c701d832345a08a0951afb6b6d914` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:10.423037 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe9c866-e562-4809-9eac-256d7df396a2 None admin] RBAC: Authorizing `identity:get_user(user_id=d15c701d832345a08a0951afb6b6d914)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:10.423201 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:10.424569 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe9c866-e562-4809-9eac-256d7df396a2 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:10.426325 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 40/77] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:37:10 2022] GET /identity/v3/users/d15c701d832345a08a0951afb6b6d914 => generated 275 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:10.427101 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe9c866-e562-4809-9eac-256d7df396a2 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:10.436088 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:10.440734 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297ca185-c98c-417c-86f8-f6c3fcaaf3e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:10.441540 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297ca185-c98c-417c-86f8-f6c3fcaaf3e4 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:10.441829 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297ca185-c98c-417c-86f8-f6c3fcaaf3e4 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:10.442137 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297ca185-c98c-417c-86f8-f6c3fcaaf3e4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:10.453397 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297ca185-c98c-417c-86f8-f6c3fcaaf3e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:37:10.461444 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-297ca185-c98c-417c-86f8-f6c3fcaaf3e4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:37:10.461444 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:37:10.461444 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:37:10.461444 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:37:10.461444 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:37:10.461444 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:37:10.461444 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:37:10.461444 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:37:10.461444 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:37:10.461444 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:37:10.461444 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:37:10.461444 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:37:10.461444 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:37:10.461444 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:37:10.461444 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:37:10.461444 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:37:10.463237 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:37:10.463237 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:37:10.463237 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:37:10.463237 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:37:10.463237 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:37:10.463237 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:37:10.463237 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:37:10.463237 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:37:10.463237 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:37:10.463237 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:37:10.463237 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:37:10.463237 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:37:10.463237 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:37:10.463237 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:37:10.463237 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:37:10.463237 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:37:10.464783 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:37:10.464783 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:37:10.464783 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:37:10.464783 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:37:10.464783 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:37:10.464783 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:37:10.464783 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:37:10.464783 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:37:10.464783 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:37:10.464783 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:37:10.464783 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:37:10.464783 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:37:10.464783 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 38/78] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:37:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:37:10.464783 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297ca185-c98c-417c-86f8-f6c3fcaaf3e4 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:10.534569 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe9c866-e562-4809-9eac-256d7df396a2 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:10.564666 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297ca185-c98c-417c-86f8-f6c3fcaaf3e4 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:10.569776 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45e3a20-11a6-4d1b-b562-90a80a0956b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:10.570479 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45e3a20-11a6-4d1b-b562-90a80a0956b2 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:10.570830 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45e3a20-11a6-4d1b-b562-90a80a0956b2 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:10.571327 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45e3a20-11a6-4d1b-b562-90a80a0956b2 None admin] PATH_INFO: `/v3/roles` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:10.572103 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45e3a20-11a6-4d1b-b562-90a80a0956b2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:37:10.572421 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45e3a20-11a6-4d1b-b562-90a80a0956b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:10.573847 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45e3a20-11a6-4d1b-b562-90a80a0956b2 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:10.582180 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 41/79] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:37:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:10.583234 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33df035-446b-4d64-b17e-2a6bb5818b34 None None] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:10.583530 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33df035-446b-4d64-b17e-2a6bb5818b34 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:10.583811 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33df035-446b-4d64-b17e-2a6bb5818b34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:10.599216 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d57076e-f662-4c23-a7cb-1a9821107345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:10.599870 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d57076e-f662-4c23-a7cb-1a9821107345 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:10.600159 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d57076e-f662-4c23-a7cb-1a9821107345 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:10.600280 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.auth.core [None req-e33df035-446b-4d64-b17e-2a6bb5818b34 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:37:10.600439 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d57076e-f662-4c23-a7cb-1a9821107345 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:10.601452 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-e33df035-446b-4d64-b17e-2a6bb5818b34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:37:10.653050 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:10.667432 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:10.673375 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 42/80] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:37:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:37:10.674331 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33df035-446b-4d64-b17e-2a6bb5818b34 None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:10.692429 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d57076e-f662-4c23-a7cb-1a9821107345 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:10.693993 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d57076e-f662-4c23-a7cb-1a9821107345 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:10.778443 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33df035-446b-4d64-b17e-2a6bb5818b34 None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:10.797580 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 39/81] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:37:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:37:10.798841 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d57076e-f662-4c23-a7cb-1a9821107345 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:10.818269 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948db647-5789-422c-97b7-89acae693701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:10.818924 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948db647-5789-422c-97b7-89acae693701 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:10.819211 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948db647-5789-422c-97b7-89acae693701 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:10.819489 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948db647-5789-422c-97b7-89acae693701 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:10.880347 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:10.884776 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:10.888381 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948db647-5789-422c-97b7-89acae693701 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:10.890498 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948db647-5789-422c-97b7-89acae693701 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:10.901032 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d57076e-f662-4c23-a7cb-1a9821107345 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:10.935541 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880beb70-68c8-4ca2-87ff-8eec02a25b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:10.936218 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880beb70-68c8-4ca2-87ff-8eec02a25b7a None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:10.936512 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880beb70-68c8-4ca2-87ff-8eec02a25b7a None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:10.936796 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880beb70-68c8-4ca2-87ff-8eec02a25b7a None admin] PATH_INFO: `/v3/roles` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:10.937551 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880beb70-68c8-4ca2-87ff-8eec02a25b7a None admin] RBAC: Adding query filter params (name=member) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:37:10.937894 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880beb70-68c8-4ca2-87ff-8eec02a25b7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:10.939324 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880beb70-68c8-4ca2-87ff-8eec02a25b7a None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:10.948692 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 40/82] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:37:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:10.949537 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880beb70-68c8-4ca2-87ff-8eec02a25b7a None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:10.989316 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 43/83] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:37:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:37:10.990217 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948db647-5789-422c-97b7-89acae693701 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:11.049423 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880beb70-68c8-4ca2-87ff-8eec02a25b7a None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:11.083557 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982c63af-38ce-4bcb-a73b-08449fd4c756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:11.084436 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982c63af-38ce-4bcb-a73b-08449fd4c756 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:11.084722 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982c63af-38ce-4bcb-a73b-08449fd4c756 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:11.085251 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982c63af-38ce-4bcb-a73b-08449fd4c756 None admin] PATH_INFO: `/v3/users/d15c701d832345a08a0951afb6b6d914` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:11.086562 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982c63af-38ce-4bcb-a73b-08449fd4c756 None admin] RBAC: Authorizing `identity:get_user(user_id=d15c701d832345a08a0951afb6b6d914)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:11.088216 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982c63af-38ce-4bcb-a73b-08449fd4c756 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:11.089887 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 41/84] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:37:10 2022] GET /identity/v3/users/d15c701d832345a08a0951afb6b6d914 => generated 275 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:11.094639 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982c63af-38ce-4bcb-a73b-08449fd4c756 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:11.094971 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:11.099390 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948db647-5789-422c-97b7-89acae693701 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:11.107592 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:11.137983 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecad38e-054d-4034-be34-b2002d52d9f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:11.138786 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecad38e-054d-4034-be34-b2002d52d9f7 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:11.139078 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecad38e-054d-4034-be34-b2002d52d9f7 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:11.139370 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecad38e-054d-4034-be34-b2002d52d9f7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:11.140613 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecad38e-054d-4034-be34-b2002d52d9f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:11.142112 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecad38e-054d-4034-be34-b2002d52d9f7 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:11.143755 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 44/85] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:37:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:11.146700 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecad38e-054d-4034-be34-b2002d52d9f7 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:11.194348 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982c63af-38ce-4bcb-a73b-08449fd4c756 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:11.228564 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de04cef3-2399-4fea-95ca-59fb12312b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:11.229436 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de04cef3-2399-4fea-95ca-59fb12312b1a None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:11.229727 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de04cef3-2399-4fea-95ca-59fb12312b1a None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:11.230041 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de04cef3-2399-4fea-95ca-59fb12312b1a None admin] PATH_INFO: `/v3/projects/ff0814efdfab4314800f7e0e2109a949` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:11.231250 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de04cef3-2399-4fea-95ca-59fb12312b1a None admin] RBAC: Authorizing `identity:get_project(project_id=ff0814efdfab4314800f7e0e2109a949)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:11.232737 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de04cef3-2399-4fea-95ca-59fb12312b1a None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:11.234661 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 42/86] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:37:11 2022] GET /identity/v3/projects/ff0814efdfab4314800f7e0e2109a949 => generated 298 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:11.235453 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de04cef3-2399-4fea-95ca-59fb12312b1a None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:11.251808 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecad38e-054d-4034-be34-b2002d52d9f7 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:11.284826 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a86391e-5769-489d-b31f-25ef5ecfe737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:11.285699 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a86391e-5769-489d-b31f-25ef5ecfe737 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:11.286021 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a86391e-5769-489d-b31f-25ef5ecfe737 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:11.286303 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a86391e-5769-489d-b31f-25ef5ecfe737 None admin] PATH_INFO: `/v3/users/d15c701d832345a08a0951afb6b6d914` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:11.287500 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a86391e-5769-489d-b31f-25ef5ecfe737 None admin] RBAC: Authorizing `identity:get_user(user_id=d15c701d832345a08a0951afb6b6d914)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:11.288940 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a86391e-5769-489d-b31f-25ef5ecfe737 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:11.291237 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 45/87] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:37:11 2022] GET /identity/v3/users/d15c701d832345a08a0951afb6b6d914 => generated 275 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:11.292056 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a86391e-5769-489d-b31f-25ef5ecfe737 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:11.321070 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:11.327312 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:11.341572 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de04cef3-2399-4fea-95ca-59fb12312b1a None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:11.375684 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df9de7c-47bd-4c16-8c3d-94476b46b4fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:11.376891 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df9de7c-47bd-4c16-8c3d-94476b46b4fc None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:11.377098 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df9de7c-47bd-4c16-8c3d-94476b46b4fc None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:11.377374 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df9de7c-47bd-4c16-8c3d-94476b46b4fc None admin] PATH_INFO: `/v3/projects/ff0814efdfab4314800f7e0e2109a949` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:11.378588 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df9de7c-47bd-4c16-8c3d-94476b46b4fc None admin] RBAC: Authorizing `identity:get_project(project_id=ff0814efdfab4314800f7e0e2109a949)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:11.380057 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df9de7c-47bd-4c16-8c3d-94476b46b4fc None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:11.381773 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 43/88] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:37:11 2022] GET /identity/v3/projects/ff0814efdfab4314800f7e0e2109a949 => generated 298 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:11.386917 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df9de7c-47bd-4c16-8c3d-94476b46b4fc None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:11.395874 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a86391e-5769-489d-b31f-25ef5ecfe737 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:11.433998 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d86bb31-2bd7-46e8-810d-e43350e44042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:11.434683 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d86bb31-2bd7-46e8-810d-e43350e44042 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:11.434926 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d86bb31-2bd7-46e8-810d-e43350e44042 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:11.435205 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d86bb31-2bd7-46e8-810d-e43350e44042 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:11.483692 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df9de7c-47bd-4c16-8c3d-94476b46b4fc None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:11.504306 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d86bb31-2bd7-46e8-810d-e43350e44042 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:11.505787 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d86bb31-2bd7-46e8-810d-e43350e44042 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:11.517724 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9190009f-3f9d-4b83-8381-9b5ac74647ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:11.518436 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9190009f-3f9d-4b83-8381-9b5ac74647ba None admin] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:11.518729 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9190009f-3f9d-4b83-8381-9b5ac74647ba None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:11.519010 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9190009f-3f9d-4b83-8381-9b5ac74647ba None admin] PATH_INFO: `/v3/users` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:11.519734 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9190009f-3f9d-4b83-8381-9b5ac74647ba None admin] RBAC: Authorizing `identity:create_user()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:11.521196 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9190009f-3f9d-4b83-8381-9b5ac74647ba None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:11.539227 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:11.548091 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 44/89] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 19 14:37:11 2022] POST /identity/v3/users => generated 289 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:37:11.548919 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9190009f-3f9d-4b83-8381-9b5ac74647ba None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:11.552345 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:11.635616 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 46/90] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:37:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:37:11.636264 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d86bb31-2bd7-46e8-810d-e43350e44042 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:11.668900 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9190009f-3f9d-4b83-8381-9b5ac74647ba None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:11.708294 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee71981b-396f-4822-88fc-c70e614cbd6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:11.708975 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee71981b-396f-4822-88fc-c70e614cbd6c None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:11.709265 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee71981b-396f-4822-88fc-c70e614cbd6c None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:11.709548 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee71981b-396f-4822-88fc-c70e614cbd6c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:11.710820 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee71981b-396f-4822-88fc-c70e614cbd6c None admin] RBAC: Adding query filter params (role.id=c2935b1a25ca46fcba419fe3d1715330, scope.project.id=ff0814efdfab4314800f7e0e2109a949, user.id=d8062aba2d374babbc9d68178a9546e3) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:37:11.711142 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee71981b-396f-4822-88fc-c70e614cbd6c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:11.713438 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee71981b-396f-4822-88fc-c70e614cbd6c None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:11.721362 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 45/91] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:37:11 2022] GET /identity/v3/role_assignments?user.id=d8062aba2d374babbc9d68178a9546e3&scope.project.id=ff0814efdfab4314800f7e0e2109a949&role.id=c2935b1a25ca46fcba419fe3d1715330 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:11.722280 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee71981b-396f-4822-88fc-c70e614cbd6c None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:11.764564 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d86bb31-2bd7-46e8-810d-e43350e44042 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:11.814656 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b7ee66-6dd4-41e0-af8f-ce48245ba723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:11.815723 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b7ee66-6dd4-41e0-af8f-ce48245ba723 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:11.816264 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b7ee66-6dd4-41e0-af8f-ce48245ba723 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:11.816775 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b7ee66-6dd4-41e0-af8f-ce48245ba723 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:11.820710 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:11.824827 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:11.876731 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee71981b-396f-4822-88fc-c70e614cbd6c None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:11.913229 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b7ee66-6dd4-41e0-af8f-ce48245ba723 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:11.914734 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b7ee66-6dd4-41e0-af8f-ce48245ba723 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:11.927242 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9f04aa-2a79-4032-8069-6464d0c2f44b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:11.927970 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9f04aa-2a79-4032-8069-6464d0c2f44b None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:11.928268 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9f04aa-2a79-4032-8069-6464d0c2f44b None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:11.928554 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9f04aa-2a79-4032-8069-6464d0c2f44b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:11.929420 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9f04aa-2a79-4032-8069-6464d0c2f44b None admin] RBAC: Adding query filter params (role.id=7ebce5564f40456e94a3616961ed73ff, scope.project.id=ff0814efdfab4314800f7e0e2109a949, user.id=d15c701d832345a08a0951afb6b6d914) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:37:11.929744 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9f04aa-2a79-4032-8069-6464d0c2f44b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:11.932268 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9f04aa-2a79-4032-8069-6464d0c2f44b None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:11.940136 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 46/92] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:37:11 2022] GET /identity/v3/role_assignments?user.id=d15c701d832345a08a0951afb6b6d914&scope.project.id=ff0814efdfab4314800f7e0e2109a949&role.id=7ebce5564f40456e94a3616961ed73ff => generated 259 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:11.941138 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9f04aa-2a79-4032-8069-6464d0c2f44b None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:12.109013 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 47/93] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:37:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:37:12.115099 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b7ee66-6dd4-41e0-af8f-ce48245ba723 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:12.123435 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9f04aa-2a79-4032-8069-6464d0c2f44b None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:12.160854 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:12.170598 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e103b9-9d9d-4a19-87b0-0d9482e77d8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:12.171775 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e103b9-9d9d-4a19-87b0-0d9482e77d8e None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:12.172278 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e103b9-9d9d-4a19-87b0-0d9482e77d8e None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:12.172765 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e103b9-9d9d-4a19-87b0-0d9482e77d8e None admin] PATH_INFO: `/v3/projects/ff0814efdfab4314800f7e0e2109a949` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:12.174365 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e103b9-9d9d-4a19-87b0-0d9482e77d8e None admin] RBAC: Authorizing `identity:get_project(project_id=ff0814efdfab4314800f7e0e2109a949)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:12.176398 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e103b9-9d9d-4a19-87b0-0d9482e77d8e None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:12.178816 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 47/94] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:37:11 2022] GET /identity/v3/projects/ff0814efdfab4314800f7e0e2109a949 => generated 298 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:12.186572 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e103b9-9d9d-4a19-87b0-0d9482e77d8e None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:12.205325 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:12.340580 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b7ee66-6dd4-41e0-af8f-ce48245ba723 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:12.391405 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e103b9-9d9d-4a19-87b0-0d9482e77d8e None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:12.416925 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2ed457-b564-4bed-9fc1-1336d25a7305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:12.417886 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2ed457-b564-4bed-9fc1-1336d25a7305 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:12.418135 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2ed457-b564-4bed-9fc1-1336d25a7305 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:12.418420 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2ed457-b564-4bed-9fc1-1336d25a7305 None admin] PATH_INFO: `/v3/projects/c539a4fcbe8c4250a79893326416b622` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:12.421070 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2ed457-b564-4bed-9fc1-1336d25a7305 None admin] RBAC: Authorizing `identity:get_project(project_id=c539a4fcbe8c4250a79893326416b622)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:12.422592 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2ed457-b564-4bed-9fc1-1336d25a7305 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:12.424488 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 48/95] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:37:12 2022] GET /identity/v3/projects/c539a4fcbe8c4250a79893326416b622 => generated 312 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:37:12.430304 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2ed457-b564-4bed-9fc1-1336d25a7305 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:12.437243 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d971290-1d4b-41c0-a2af-4069afdabb15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:12.438262 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d971290-1d4b-41c0-a2af-4069afdabb15 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:12.438755 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d971290-1d4b-41c0-a2af-4069afdabb15 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:12.439353 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d971290-1d4b-41c0-a2af-4069afdabb15 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:12.440491 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d971290-1d4b-41c0-a2af-4069afdabb15 None admin] RBAC: Adding query filter params (role.id=b894e00b17e34f0cbde0ab3d760e2433, scope.project.id=ff0814efdfab4314800f7e0e2109a949, user.id=d15c701d832345a08a0951afb6b6d914) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:37:12.441002 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d971290-1d4b-41c0-a2af-4069afdabb15 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:12.442749 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d971290-1d4b-41c0-a2af-4069afdabb15 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:12.449173 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 48/96] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:37:12 2022] GET /identity/v3/role_assignments?user.id=d15c701d832345a08a0951afb6b6d914&scope.project.id=ff0814efdfab4314800f7e0e2109a949&role.id=b894e00b17e34f0cbde0ab3d760e2433 => generated 259 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:37:12.583173 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2ed457-b564-4bed-9fc1-1336d25a7305 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:12.585179 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:12.636204 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cf47c0-39b3-411a-a094-c0b836815ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:12.637252 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cf47c0-39b3-411a-a094-c0b836815ce4 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:12.637736 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cf47c0-39b3-411a-a094-c0b836815ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:12.638249 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cf47c0-39b3-411a-a094-c0b836815ce4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:12.639310 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cf47c0-39b3-411a-a094-c0b836815ce4 None admin] RBAC: Adding query filter params (role.id=7ebce5564f40456e94a3616961ed73ff, scope.project.id=c539a4fcbe8c4250a79893326416b622, user.id=d15c701d832345a08a0951afb6b6d914) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:37:12.639821 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cf47c0-39b3-411a-a094-c0b836815ce4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:12.641489 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cf47c0-39b3-411a-a094-c0b836815ce4 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:12.654658 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 49/97] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:37:12 2022] GET /identity/v3/role_assignments?user.id=d15c701d832345a08a0951afb6b6d914&scope.project.id=c539a4fcbe8c4250a79893326416b622&role.id=7ebce5564f40456e94a3616961ed73ff => generated 259 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:37:14.746232 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002e2c67-a501-4911-abd2-516acd989eb2 None None] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:14.747278 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002e2c67-a501-4911-abd2-516acd989eb2 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:14.747776 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002e2c67-a501-4911-abd2-516acd989eb2 None None] PATH_INFO: `/` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:14.749673 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 49/98] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:37:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:37:14.755087 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa9d678-207b-471d-a030-e4a2ac33fcfa None None] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:14.755930 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa9d678-207b-471d-a030-e4a2ac33fcfa None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:14.756434 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa9d678-207b-471d-a030-e4a2ac33fcfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:14.779904 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.auth.core [None req-dfa9d678-207b-471d-a030-e4a2ac33fcfa None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:37:14.781501 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-dfa9d678-207b-471d-a030-e4a2ac33fcfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:37:14.787923 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac52126-436d-44bb-8b98-f60cdb2576b5 None None] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:14.788568 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac52126-436d-44bb-8b98-f60cdb2576b5 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:14.789075 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac52126-436d-44bb-8b98-f60cdb2576b5 None None] PATH_INFO: `/` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:14.792469 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 50/99] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:37:14 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:37:14.803727 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea07949-a0ae-4857-a453-2230ce3b0bcb None None] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:14.804357 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea07949-a0ae-4857-a453-2230ce3b0bcb None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:14.804840 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea07949-a0ae-4857-a453-2230ce3b0bcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:14.831961 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.auth.core [None req-fea07949-a0ae-4857-a453-2230ce3b0bcb None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:37:14.835476 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.fernet_utils [None req-fea07949-a0ae-4857-a453-2230ce3b0bcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:37:14.952319 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 50/100] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:37:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:37:14.958642 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:14.968358 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 51/101] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:37:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:37:14.993928 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa9d678-207b-471d-a030-e4a2ac33fcfa None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:15.003162 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea07949-a0ae-4857-a453-2230ce3b0bcb None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:15.130239 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:15.139666 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa9d678-207b-471d-a030-e4a2ac33fcfa None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:15.209468 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2644d79-99e8-4e9e-acf6-7a9ff85cbb0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:15.212820 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2644d79-99e8-4e9e-acf6-7a9ff85cbb0f None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:15.215890 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2644d79-99e8-4e9e-acf6-7a9ff85cbb0f None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:15.217269 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2644d79-99e8-4e9e-acf6-7a9ff85cbb0f None admin] PATH_INFO: `/v3/roles/member` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:15.228603 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea07949-a0ae-4857-a453-2230ce3b0bcb None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:15.245529 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2644d79-99e8-4e9e-acf6-7a9ff85cbb0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:37:15.261447 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-b2644d79-99e8-4e9e-acf6-7a9ff85cbb0f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:37:15.261447 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:37:15.261447 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:37:15.261447 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:37:15.261447 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:37:15.261447 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:37:15.261447 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:37:15.261447 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:37:15.261447 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:37:15.261447 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:37:15.261447 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:37:15.261447 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:37:15.261447 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:37:15.261447 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:37:15.261447 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:37:15.261447 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:37:15.262518 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:37:15.262518 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:37:15.262518 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:37:15.262518 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:37:15.262518 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:37:15.262518 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:37:15.262518 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:37:15.262518 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:37:15.262518 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:37:15.262518 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:37:15.262518 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:37:15.262518 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:37:15.262518 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:37:15.262518 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:37:15.262518 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:37:15.262518 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:37:15.263443 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:37:15.263443 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:37:15.263443 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:37:15.263443 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:37:15.263443 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:37:15.263443 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:37:15.263443 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:37:15.263443 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:37:15.263443 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:37:15.263443 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:37:15.263443 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:37:15.263443 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:37:15.268881 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 51/102] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:37:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:37:15.276912 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d7512a-ff4e-46ec-97c8-c8769811d186 None None] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:15.277495 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d7512a-ff4e-46ec-97c8-c8769811d186 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:15.278022 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d7512a-ff4e-46ec-97c8-c8769811d186 None None] PATH_INFO: `/` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:15.280657 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 52/103] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:37:15 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:37:15.291630 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6862b9a-beb4-432b-a81c-d17dd7cae8f9 None None] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:15.291945 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6862b9a-beb4-432b-a81c-d17dd7cae8f9 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:15.292216 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6862b9a-beb4-432b-a81c-d17dd7cae8f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:15.301694 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eb0b8b-77f5-4195-b672-13a624ba7490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:15.302905 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eb0b8b-77f5-4195-b672-13a624ba7490 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:15.303494 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eb0b8b-77f5-4195-b672-13a624ba7490 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:15.303984 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eb0b8b-77f5-4195-b672-13a624ba7490 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:15.323027 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04eb0b8b-77f5-4195-b672-13a624ba7490 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:37:15.330948 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-04eb0b8b-77f5-4195-b672-13a624ba7490 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:37:15.330948 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:37:15.330948 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:37:15.330948 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:37:15.330948 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:37:15.330948 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:37:15.330948 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:37:15.330948 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:37:15.330948 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:37:15.330948 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:37:15.330948 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:37:15.330948 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:37:15.330948 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:37:15.330948 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:37:15.330948 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:37:15.330948 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:37:15.331922 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:37:15.331922 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:37:15.331922 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:37:15.331922 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:37:15.331922 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:37:15.331922 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:37:15.331922 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:37:15.331922 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:37:15.331922 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:37:15.331922 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:37:15.331922 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:37:15.331922 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:37:15.331922 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:37:15.331922 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:37:15.331922 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:37:15.331922 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:37:15.332794 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:37:15.332794 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:37:15.332794 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:37:15.332794 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:37:15.332794 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:37:15.332794 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:37:15.332794 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:37:15.332794 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:37:15.332794 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:37:15.332794 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:37:15.332794 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:37:15.332794 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:37:15.332794 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 52/104] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:37:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:37:15.333545 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eb0b8b-77f5-4195-b672-13a624ba7490 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:15.334385 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.auth.core [None req-f6862b9a-beb4-432b-a81c-d17dd7cae8f9 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:37:15.335920 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-f6862b9a-beb4-432b-a81c-d17dd7cae8f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:37:15.431771 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:15.488230 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eb0b8b-77f5-4195-b672-13a624ba7490 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:15.503596 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 53/105] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:37:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:37:15.504529 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6862b9a-beb4-432b-a81c-d17dd7cae8f9 None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:15.533284 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c2274f-7ef9-495d-b02a-f78283f0c185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:15.534386 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c2274f-7ef9-495d-b02a-f78283f0c185 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:15.534945 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c2274f-7ef9-495d-b02a-f78283f0c185 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:15.535451 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c2274f-7ef9-495d-b02a-f78283f0c185 None admin] PATH_INFO: `/v3/roles` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:15.536421 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c2274f-7ef9-495d-b02a-f78283f0c185 None admin] RBAC: Adding query filter params (name=member) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:37:15.536945 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c2274f-7ef9-495d-b02a-f78283f0c185 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:15.538596 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c2274f-7ef9-495d-b02a-f78283f0c185 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:15.549426 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 53/106] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:37:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:15.550670 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c2274f-7ef9-495d-b02a-f78283f0c185 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:15.670268 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:15.693141 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6862b9a-beb4-432b-a81c-d17dd7cae8f9 None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:15.724527 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c2274f-7ef9-495d-b02a-f78283f0c185 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:15.737526 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bfa01d-716f-4655-817d-d8d7162c2bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:15.738616 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bfa01d-716f-4655-817d-d8d7162c2bcf None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:15.739124 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bfa01d-716f-4655-817d-d8d7162c2bcf None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:15.739650 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bfa01d-716f-4655-817d-d8d7162c2bcf None admin] PATH_INFO: `/v3/roles` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:15.740619 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bfa01d-716f-4655-817d-d8d7162c2bcf None admin] RBAC: Adding query filter params (name=reader) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:37:15.741193 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bfa01d-716f-4655-817d-d8d7162c2bcf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:15.743124 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bfa01d-716f-4655-817d-d8d7162c2bcf None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:15.745495 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:15.753479 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 54/107] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:37:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:15.754658 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bfa01d-716f-4655-817d-d8d7162c2bcf None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:15.770163 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557d5afe-05ac-42bf-9dea-f60875dc6a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:15.770977 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557d5afe-05ac-42bf-9dea-f60875dc6a81 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:15.771265 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557d5afe-05ac-42bf-9dea-f60875dc6a81 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:15.771554 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557d5afe-05ac-42bf-9dea-f60875dc6a81 None admin] PATH_INFO: `/v3/domains/default` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:15.773020 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557d5afe-05ac-42bf-9dea-f60875dc6a81 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:15.776218 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557d5afe-05ac-42bf-9dea-f60875dc6a81 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:15.778906 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 54/108] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:37:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:15.780212 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0739e98-d7bb-49d3-9760-0a11c30c4506 None None] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:15.780509 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0739e98-d7bb-49d3-9760-0a11c30c4506 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:15.780792 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0739e98-d7bb-49d3-9760-0a11c30c4506 None None] PATH_INFO: `/` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:15.782917 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 55/109] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:37:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:37:15.788124 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0739e98-d7bb-49d3-9760-0a11c30c4506 None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:15.869516 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bfa01d-716f-4655-817d-d8d7162c2bcf None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:15.908506 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382212d9-ddec-409f-b4e1-54685f73f4fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:15.909077 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0739e98-d7bb-49d3-9760-0a11c30c4506 None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:15.909421 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382212d9-ddec-409f-b4e1-54685f73f4fd None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:15.909710 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382212d9-ddec-409f-b4e1-54685f73f4fd None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:15.910023 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382212d9-ddec-409f-b4e1-54685f73f4fd None admin] PATH_INFO: `/v3/users/d15c701d832345a08a0951afb6b6d914` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:15.911264 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382212d9-ddec-409f-b4e1-54685f73f4fd None admin] RBAC: Authorizing `identity:get_user(user_id=d15c701d832345a08a0951afb6b6d914)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:15.912730 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382212d9-ddec-409f-b4e1-54685f73f4fd None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:15.914653 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 55/110] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:37:15 2022] GET /identity/v3/users/d15c701d832345a08a0951afb6b6d914 => generated 275 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:15.915307 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382212d9-ddec-409f-b4e1-54685f73f4fd None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:15.936304 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:15.951804 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dad0820-9048-469f-adaa-d1cc2f6f2f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:15.952693 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dad0820-9048-469f-adaa-d1cc2f6f2f27 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:15.952981 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dad0820-9048-469f-adaa-d1cc2f6f2f27 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:15.953261 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dad0820-9048-469f-adaa-d1cc2f6f2f27 None admin] PATH_INFO: `/v3/users/bc06d3d555b24e59a3957bf501a9d140` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:15.968424 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dad0820-9048-469f-adaa-d1cc2f6f2f27 None admin] RBAC: Authorizing `identity:get_user(user_id=bc06d3d555b24e59a3957bf501a9d140)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:15.970075 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dad0820-9048-469f-adaa-d1cc2f6f2f27 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:15.971769 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 56/111] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:37:15 2022] GET /identity/v3/users/bc06d3d555b24e59a3957bf501a9d140 => generated 289 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:15.971769 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf99842-8def-4d36-b849-33ce4167fd04 None None] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:15.971769 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf99842-8def-4d36-b849-33ce4167fd04 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:15.971769 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf99842-8def-4d36-b849-33ce4167fd04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:15.993575 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.auth.core [None req-4cf99842-8def-4d36-b849-33ce4167fd04 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:37:15.995036 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.fernet_utils [None req-4cf99842-8def-4d36-b849-33ce4167fd04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:37:16.036623 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:16.041737 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382212d9-ddec-409f-b4e1-54685f73f4fd None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:16.081931 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 57/112] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:37:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:37:16.081931 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1155df5a-b0bb-4f80-b5a5-88a1e61c85f1 None None] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:16.081931 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1155df5a-b0bb-4f80-b5a5-88a1e61c85f1 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:16.081931 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1155df5a-b0bb-4f80-b5a5-88a1e61c85f1 None None] PATH_INFO: `/` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:16.081931 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 58/113] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:37:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:37:16.082495 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1155df5a-b0bb-4f80-b5a5-88a1e61c85f1 None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:16.088817 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30494c06-1557-4b19-9d51-75a9d22c4f19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:16.089716 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30494c06-1557-4b19-9d51-75a9d22c4f19 None admin] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:16.090166 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30494c06-1557-4b19-9d51-75a9d22c4f19 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:16.090574 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30494c06-1557-4b19-9d51-75a9d22c4f19 None admin] PATH_INFO: `/v3/projects` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:16.091467 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30494c06-1557-4b19-9d51-75a9d22c4f19 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:16.093239 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30494c06-1557-4b19-9d51-75a9d22c4f19 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:16.123660 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 56/114] 10.222.0.57 () {60 vars in 1141 bytes} [Thu May 19 14:37:15 2022] POST /identity/v3/projects => generated 302 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:37:16.129986 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30494c06-1557-4b19-9d51-75a9d22c4f19 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:16.219526 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1155df5a-b0bb-4f80-b5a5-88a1e61c85f1 None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:16.232410 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:16.262669 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa93bc8-6d47-42e0-9a1c-6a7980fd31ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:16.263570 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa93bc8-6d47-42e0-9a1c-6a7980fd31ad None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:16.263864 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa93bc8-6d47-42e0-9a1c-6a7980fd31ad None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:16.264155 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa93bc8-6d47-42e0-9a1c-6a7980fd31ad None admin] PATH_INFO: `/v3/projects/ff0814efdfab4314800f7e0e2109a949` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:16.265436 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa93bc8-6d47-42e0-9a1c-6a7980fd31ad None admin] RBAC: Authorizing `identity:get_project(project_id=ff0814efdfab4314800f7e0e2109a949)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:16.267125 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa93bc8-6d47-42e0-9a1c-6a7980fd31ad None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:16.268835 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 59/115] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:37:16 2022] GET /identity/v3/projects/ff0814efdfab4314800f7e0e2109a949 => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:16.270181 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e890ef-9150-40f0-aeba-e3504a1bf58a None None] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:16.270470 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e890ef-9150-40f0-aeba-e3504a1bf58a None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:16.270753 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e890ef-9150-40f0-aeba-e3504a1bf58a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:16.271427 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30494c06-1557-4b19-9d51-75a9d22c4f19 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:16.290087 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.auth.core [None req-c2e890ef-9150-40f0-aeba-e3504a1bf58a None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:37:16.291543 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.fernet_utils [None req-c2e890ef-9150-40f0-aeba-e3504a1bf58a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:37:16.323778 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4666d1-39b5-47fe-937b-b302747a6834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:16.324357 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4666d1-39b5-47fe-937b-b302747a6834 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:16.324644 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4666d1-39b5-47fe-937b-b302747a6834 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:16.324922 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4666d1-39b5-47fe-937b-b302747a6834 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:16.332160 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:16.377158 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 60/116] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:37:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:37:16.378370 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e890ef-9150-40f0-aeba-e3504a1bf58a None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:16.403151 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4666d1-39b5-47fe-937b-b302747a6834 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:16.404658 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4666d1-39b5-47fe-937b-b302747a6834 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:16.475023 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:16.489616 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e890ef-9150-40f0-aeba-e3504a1bf58a None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:16.516509 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 57/117] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:37:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:37:16.517723 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e1936f-0345-462a-b3d7-d6e844def4ba None None] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:16.518049 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e1936f-0345-462a-b3d7-d6e844def4ba None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:16.518342 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e1936f-0345-462a-b3d7-d6e844def4ba None None] PATH_INFO: `/` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:16.523544 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 58/118] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:37:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:37:16.523544 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e1936f-0345-462a-b3d7-d6e844def4ba None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:16.528126 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6975598-a544-42f1-a7b4-4e9dc3b153f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:16.528936 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6975598-a544-42f1-a7b4-4e9dc3b153f7 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:16.529223 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6975598-a544-42f1-a7b4-4e9dc3b153f7 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:16.529500 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6975598-a544-42f1-a7b4-4e9dc3b153f7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:16.542627 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6975598-a544-42f1-a7b4-4e9dc3b153f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:37:16.550369 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-d6975598-a544-42f1-a7b4-4e9dc3b153f7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:37:16.550369 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:37:16.550369 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:37:16.550369 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:37:16.550369 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:37:16.550369 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:37:16.550369 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:37:16.550369 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:37:16.550369 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:37:16.550369 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:37:16.550369 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:37:16.550369 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:37:16.550369 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:37:16.550369 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:37:16.550369 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:37:16.550369 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:37:16.551568 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:37:16.551568 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:37:16.551568 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:37:16.551568 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:37:16.551568 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:37:16.551568 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:37:16.551568 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:37:16.551568 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:37:16.551568 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:37:16.551568 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:37:16.551568 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:37:16.551568 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:37:16.551568 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:37:16.551568 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:37:16.551568 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:37:16.551568 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:37:16.552483 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:37:16.552483 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:37:16.552483 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:37:16.552483 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:37:16.552483 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:37:16.552483 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:37:16.552483 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:37:16.552483 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:37:16.552483 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:37:16.552483 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:37:16.552483 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:37:16.552483 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:37:16.552483 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 61/119] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:37:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:37:16.552483 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6975598-a544-42f1-a7b4-4e9dc3b153f7 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:16.588562 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:16.641935 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e1936f-0345-462a-b3d7-d6e844def4ba None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:16.666972 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6975598-a544-42f1-a7b4-4e9dc3b153f7 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:16.679875 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ee2c14-a940-4219-a2f1-39e23f2deee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:16.680803 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ee2c14-a940-4219-a2f1-39e23f2deee5 None admin] REQUEST_METHOD: `PUT` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:16.681187 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ee2c14-a940-4219-a2f1-39e23f2deee5 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:16.681355 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ee2c14-a940-4219-a2f1-39e23f2deee5 None admin] PATH_INFO: `/v3/projects/ff0814efdfab4314800f7e0e2109a949/users/d15c701d832345a08a0951afb6b6d914/roles/7ebce5564f40456e94a3616961ed73ff` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:16.683371 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ee2c14-a940-4219-a2f1-39e23f2deee5 None admin] RBAC: Authorizing `identity:create_grant(project_id=ff0814efdfab4314800f7e0e2109a949, user_id=d15c701d832345a08a0951afb6b6d914, role_id=7ebce5564f40456e94a3616961ed73ff)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:16.684999 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ee2c14-a940-4219-a2f1-39e23f2deee5 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:16.700280 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 59/120] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 14:37:16 2022] PUT /identity/v3/projects/ff0814efdfab4314800f7e0e2109a949/users/d15c701d832345a08a0951afb6b6d914/roles/7ebce5564f40456e94a3616961ed73ff => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:37:16.710471 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ee2c14-a940-4219-a2f1-39e23f2deee5 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:16.714862 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fca75c-1349-4336-bef3-3f19383fc991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:16.715690 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fca75c-1349-4336-bef3-3f19383fc991 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:16.716025 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fca75c-1349-4336-bef3-3f19383fc991 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:16.716310 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fca75c-1349-4336-bef3-3f19383fc991 None admin] PATH_INFO: `/v3/roles/member` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:16.729991 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fca75c-1349-4336-bef3-3f19383fc991 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:37:16.738185 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-62fca75c-1349-4336-bef3-3f19383fc991 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:37:16.738185 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:37:16.738185 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:37:16.738185 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:37:16.738185 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:37:16.738185 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:37:16.738185 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:37:16.738185 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:37:16.738185 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:37:16.738185 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:37:16.738185 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:37:16.738185 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:37:16.738185 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:37:16.738185 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:37:16.738185 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:37:16.738185 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:37:16.739324 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:37:16.739324 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:37:16.739324 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:37:16.739324 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:37:16.739324 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:37:16.739324 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:37:16.739324 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:37:16.739324 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:37:16.739324 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:37:16.739324 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:37:16.739324 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:37:16.739324 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:37:16.739324 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:37:16.739324 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:37:16.739324 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:37:16.739324 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:37:16.745715 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:37:16.745715 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:37:16.745715 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:37:16.745715 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:37:16.745715 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:37:16.745715 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:37:16.745715 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:37:16.745715 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:37:16.745715 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:37:16.745715 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:37:16.745715 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:37:16.745715 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:37:16.745715 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 62/121] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:37:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:37:16.745715 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aafc706-5054-4560-b860-bc4088079022 None None] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:16.746772 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aafc706-5054-4560-b860-bc4088079022 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:16.746772 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aafc706-5054-4560-b860-bc4088079022 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:16.757159 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:16.765232 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.auth.core [None req-1aafc706-5054-4560-b860-bc4088079022 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:37:16.766617 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.fernet_utils [None req-1aafc706-5054-4560-b860-bc4088079022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:37:16.853952 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 63/122] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:37:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:37:16.853952 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aafc706-5054-4560-b860-bc4088079022 None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:16.893974 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:16.906595 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ee2c14-a940-4219-a2f1-39e23f2deee5 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:16.949987 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d5d217-f3ce-4c13-86a3-87bce353f686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:16.951270 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d5d217-f3ce-4c13-86a3-87bce353f686 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:16.951623 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d5d217-f3ce-4c13-86a3-87bce353f686 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:16.951913 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d5d217-f3ce-4c13-86a3-87bce353f686 None admin] PATH_INFO: `/v3/projects/ff0814efdfab4314800f7e0e2109a949` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:16.954475 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d5d217-f3ce-4c13-86a3-87bce353f686 None admin] RBAC: Authorizing `identity:get_project(project_id=ff0814efdfab4314800f7e0e2109a949)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:16.955393 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d5d217-f3ce-4c13-86a3-87bce353f686 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:16.957158 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 60/123] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:37:16 2022] GET /identity/v3/projects/ff0814efdfab4314800f7e0e2109a949 => generated 298 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:16.957978 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d5d217-f3ce-4c13-86a3-87bce353f686 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:16.998597 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aafc706-5054-4560-b860-bc4088079022 None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:17.035040 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca7ae58-374c-4fb8-858f-b4c10bc25bd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:17.035873 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca7ae58-374c-4fb8-858f-b4c10bc25bd2 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:17.036247 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca7ae58-374c-4fb8-858f-b4c10bc25bd2 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:17.036598 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca7ae58-374c-4fb8-858f-b4c10bc25bd2 None admin] PATH_INFO: `/v3/roles` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:17.037478 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca7ae58-374c-4fb8-858f-b4c10bc25bd2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:37:17.038060 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca7ae58-374c-4fb8-858f-b4c10bc25bd2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:17.040502 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca7ae58-374c-4fb8-858f-b4c10bc25bd2 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:17.054218 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 64/124] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:37:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:17.054218 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca7ae58-374c-4fb8-858f-b4c10bc25bd2 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:17.084363 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:17.097087 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d5d217-f3ce-4c13-86a3-87bce353f686 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:17.151104 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6829fc51-cc6b-4df4-8322-e133452c87ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:17.154110 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6829fc51-cc6b-4df4-8322-e133452c87ea None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:17.155589 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6829fc51-cc6b-4df4-8322-e133452c87ea None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:17.156134 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6829fc51-cc6b-4df4-8322-e133452c87ea None admin] PATH_INFO: `/v3/roles` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:17.157445 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6829fc51-cc6b-4df4-8322-e133452c87ea None admin] RBAC: Adding query filter params (name=member) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:37:17.158290 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6829fc51-cc6b-4df4-8322-e133452c87ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:17.159968 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6829fc51-cc6b-4df4-8322-e133452c87ea None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:17.175919 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 61/125] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:37:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:17.176436 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6829fc51-cc6b-4df4-8322-e133452c87ea None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:17.182431 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:17.210808 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca7ae58-374c-4fb8-858f-b4c10bc25bd2 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:17.250623 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f9a5a4-d9dd-4e6f-a406-42c6ee049304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:17.251097 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f9a5a4-d9dd-4e6f-a406-42c6ee049304 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:17.251390 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f9a5a4-d9dd-4e6f-a406-42c6ee049304 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:17.251725 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f9a5a4-d9dd-4e6f-a406-42c6ee049304 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:17.264450 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f9a5a4-d9dd-4e6f-a406-42c6ee049304 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:37:17.273747 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-b1f9a5a4-d9dd-4e6f-a406-42c6ee049304 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:37:17.273747 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:37:17.273747 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:37:17.273747 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:37:17.273747 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:37:17.273747 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:37:17.273747 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:37:17.273747 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:37:17.273747 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:37:17.273747 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:37:17.273747 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:37:17.273747 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:37:17.273747 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:37:17.273747 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:37:17.273747 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:37:17.273747 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:37:17.275785 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:37:17.275785 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:37:17.275785 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:37:17.275785 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:37:17.275785 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:37:17.275785 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:37:17.275785 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:37:17.275785 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:37:17.275785 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:37:17.275785 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:37:17.275785 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:37:17.275785 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:37:17.275785 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:37:17.275785 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:37:17.275785 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:37:17.275785 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:37:17.276680 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:37:17.276680 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:37:17.276680 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:37:17.276680 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:37:17.276680 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:37:17.276680 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:37:17.276680 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:37:17.276680 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:37:17.276680 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:37:17.276680 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:37:17.276680 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:37:17.276680 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:37:17.276680 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 65/126] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 19 14:37:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:37:17.276680 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f9a5a4-d9dd-4e6f-a406-42c6ee049304 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:17.323707 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6829fc51-cc6b-4df4-8322-e133452c87ea None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:17.373565 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aeea871-d396-4da4-a9ae-23ce642ce82d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:17.374487 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeea871-d396-4da4-a9ae-23ce642ce82d None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:17.375053 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeea871-d396-4da4-a9ae-23ce642ce82d None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:17.375439 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeea871-d396-4da4-a9ae-23ce642ce82d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:17.378697 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aeea871-d396-4da4-a9ae-23ce642ce82d None admin] RBAC: Adding query filter params (role.id=6129e931488746afb2145fa463c08395, scope.project.id=ff0814efdfab4314800f7e0e2109a949, user.id=bc06d3d555b24e59a3957bf501a9d140) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:37:17.379422 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aeea871-d396-4da4-a9ae-23ce642ce82d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:17.381797 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aeea871-d396-4da4-a9ae-23ce642ce82d None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:17.392198 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 62/127] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:37:17 2022] GET /identity/v3/role_assignments?user.id=bc06d3d555b24e59a3957bf501a9d140&scope.project.id=ff0814efdfab4314800f7e0e2109a949&role.id=6129e931488746afb2145fa463c08395 => generated 259 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:17.393281 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aeea871-d396-4da4-a9ae-23ce642ce82d None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:17.426288 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f9a5a4-d9dd-4e6f-a406-42c6ee049304 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:17.429452 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:17.510536 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e4c77e-b6ac-44c7-8d22-7e1dd2e2eb5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:17.511457 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e4c77e-b6ac-44c7-8d22-7e1dd2e2eb5a None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:17.511745 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e4c77e-b6ac-44c7-8d22-7e1dd2e2eb5a None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:17.512029 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e4c77e-b6ac-44c7-8d22-7e1dd2e2eb5a None admin] PATH_INFO: `/v3/users/d8062aba2d374babbc9d68178a9546e3` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:17.516819 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e4c77e-b6ac-44c7-8d22-7e1dd2e2eb5a None admin] RBAC: Authorizing `identity:get_user(user_id=d8062aba2d374babbc9d68178a9546e3)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:17.520973 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e4c77e-b6ac-44c7-8d22-7e1dd2e2eb5a None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:17.523057 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 66/128] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:37:17 2022] GET /identity/v3/users/d8062aba2d374babbc9d68178a9546e3 => generated 247 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:17.523689 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e4c77e-b6ac-44c7-8d22-7e1dd2e2eb5a None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:17.559719 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:17.615351 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aeea871-d396-4da4-a9ae-23ce642ce82d None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:17.683840 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbf79f3-939d-4e78-ad6f-ad75246e29f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:17.684929 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbf79f3-939d-4e78-ad6f-ad75246e29f7 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:17.685223 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbf79f3-939d-4e78-ad6f-ad75246e29f7 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:17.685506 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbf79f3-939d-4e78-ad6f-ad75246e29f7 None admin] PATH_INFO: `/v3/users/d15c701d832345a08a0951afb6b6d914` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:17.686509 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e4c77e-b6ac-44c7-8d22-7e1dd2e2eb5a None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:17.686972 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbf79f3-939d-4e78-ad6f-ad75246e29f7 None admin] RBAC: Authorizing `identity:get_user(user_id=d15c701d832345a08a0951afb6b6d914)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:17.688802 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbf79f3-939d-4e78-ad6f-ad75246e29f7 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:17.693924 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 63/129] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:37:17 2022] GET /identity/v3/users/d15c701d832345a08a0951afb6b6d914 => generated 275 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:17.695138 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbf79f3-939d-4e78-ad6f-ad75246e29f7 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:17.767214 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7c799c-d9fc-4402-9a8b-569d8885acf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:17.768278 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7c799c-d9fc-4402-9a8b-569d8885acf6 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:17.768818 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7c799c-d9fc-4402-9a8b-569d8885acf6 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:17.769337 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7c799c-d9fc-4402-9a8b-569d8885acf6 None admin] PATH_INFO: `/v3/roles` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:17.770472 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7c799c-d9fc-4402-9a8b-569d8885acf6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:37:17.771043 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7c799c-d9fc-4402-9a8b-569d8885acf6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:17.772738 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7c799c-d9fc-4402-9a8b-569d8885acf6 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:17.792571 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 67/130] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:37:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:17.793800 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7c799c-d9fc-4402-9a8b-569d8885acf6 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:17.830630 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:17.845549 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbf79f3-939d-4e78-ad6f-ad75246e29f7 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:17.892638 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f86922-a5e4-4f98-aca8-d748bc31c3e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:17.894619 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f86922-a5e4-4f98-aca8-d748bc31c3e3 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:17.895135 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f86922-a5e4-4f98-aca8-d748bc31c3e3 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:17.895867 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f86922-a5e4-4f98-aca8-d748bc31c3e3 None admin] PATH_INFO: `/v3/projects/ff0814efdfab4314800f7e0e2109a949` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:17.897603 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f86922-a5e4-4f98-aca8-d748bc31c3e3 None admin] RBAC: Authorizing `identity:get_project(project_id=ff0814efdfab4314800f7e0e2109a949)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:17.901488 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f86922-a5e4-4f98-aca8-d748bc31c3e3 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:17.903714 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 64/131] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:37:17 2022] GET /identity/v3/projects/ff0814efdfab4314800f7e0e2109a949 => generated 298 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:17.905021 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f86922-a5e4-4f98-aca8-d748bc31c3e3 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:17.949495 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:17.990403 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7c799c-d9fc-4402-9a8b-569d8885acf6 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:18.036818 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bc4795-3e8d-4652-bfa8-d783ec7cb1d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:18.037872 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bc4795-3e8d-4652-bfa8-d783ec7cb1d2 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:18.038187 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bc4795-3e8d-4652-bfa8-d783ec7cb1d2 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:18.038475 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bc4795-3e8d-4652-bfa8-d783ec7cb1d2 None admin] PATH_INFO: `/v3/projects/c539a4fcbe8c4250a79893326416b622` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:18.041239 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bc4795-3e8d-4652-bfa8-d783ec7cb1d2 None admin] RBAC: Authorizing `identity:get_project(project_id=c539a4fcbe8c4250a79893326416b622)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:18.042935 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bc4795-3e8d-4652-bfa8-d783ec7cb1d2 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:18.044992 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 68/132] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:37:17 2022] GET /identity/v3/projects/c539a4fcbe8c4250a79893326416b622 => generated 312 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:18.045697 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bc4795-3e8d-4652-bfa8-d783ec7cb1d2 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:18.065978 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f86922-a5e4-4f98-aca8-d748bc31c3e3 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:18.101438 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a810653e-0482-45d4-a453-ef7fceb2e806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:18.103403 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a810653e-0482-45d4-a453-ef7fceb2e806 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:18.103696 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a810653e-0482-45d4-a453-ef7fceb2e806 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:18.103980 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a810653e-0482-45d4-a453-ef7fceb2e806 None admin] PATH_INFO: `/v3/users/d15c701d832345a08a0951afb6b6d914` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:18.105244 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a810653e-0482-45d4-a453-ef7fceb2e806 None admin] RBAC: Authorizing `identity:get_user(user_id=d15c701d832345a08a0951afb6b6d914)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:18.106756 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a810653e-0482-45d4-a453-ef7fceb2e806 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:18.108439 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 65/133] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:37:17 2022] GET /identity/v3/users/d15c701d832345a08a0951afb6b6d914 => generated 275 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:18.109291 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a810653e-0482-45d4-a453-ef7fceb2e806 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:18.168374 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:18.267405 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a810653e-0482-45d4-a453-ef7fceb2e806 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:18.279810 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bc4795-3e8d-4652-bfa8-d783ec7cb1d2 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:18.322433 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84a40a4-67ba-4b4a-81af-8dc1599aa975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:18.323343 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84a40a4-67ba-4b4a-81af-8dc1599aa975 None admin] REQUEST_METHOD: `PUT` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:18.323655 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84a40a4-67ba-4b4a-81af-8dc1599aa975 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:18.323940 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84a40a4-67ba-4b4a-81af-8dc1599aa975 None admin] PATH_INFO: `/v3/projects/c539a4fcbe8c4250a79893326416b622/users/d15c701d832345a08a0951afb6b6d914/roles/7ebce5564f40456e94a3616961ed73ff` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:18.327433 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84a40a4-67ba-4b4a-81af-8dc1599aa975 None admin] RBAC: Authorizing `identity:create_grant(project_id=c539a4fcbe8c4250a79893326416b622, user_id=d15c701d832345a08a0951afb6b6d914, role_id=7ebce5564f40456e94a3616961ed73ff)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:18.328916 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84a40a4-67ba-4b4a-81af-8dc1599aa975 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:18.350471 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 66/134] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 14:37:18 2022] PUT /identity/v3/projects/c539a4fcbe8c4250a79893326416b622/users/d15c701d832345a08a0951afb6b6d914/roles/7ebce5564f40456e94a3616961ed73ff => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:37:18.351761 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84a40a4-67ba-4b4a-81af-8dc1599aa975 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:18.359323 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f1ed7d-5156-4f55-b11a-4039ae54b1f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:18.360265 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f1ed7d-5156-4f55-b11a-4039ae54b1f9 None admin] REQUEST_METHOD: `PUT` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:18.360564 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f1ed7d-5156-4f55-b11a-4039ae54b1f9 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:18.360896 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f1ed7d-5156-4f55-b11a-4039ae54b1f9 None admin] PATH_INFO: `/v3/projects/ff0814efdfab4314800f7e0e2109a949/users/d8062aba2d374babbc9d68178a9546e3/roles/c2935b1a25ca46fcba419fe3d1715330` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:18.364069 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f1ed7d-5156-4f55-b11a-4039ae54b1f9 None admin] RBAC: Authorizing `identity:create_grant(project_id=ff0814efdfab4314800f7e0e2109a949, user_id=d8062aba2d374babbc9d68178a9546e3, role_id=c2935b1a25ca46fcba419fe3d1715330)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:18.366785 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f1ed7d-5156-4f55-b11a-4039ae54b1f9 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:18.393312 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 69/135] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 14:37:18 2022] PUT /identity/v3/projects/ff0814efdfab4314800f7e0e2109a949/users/d8062aba2d374babbc9d68178a9546e3/roles/c2935b1a25ca46fcba419fe3d1715330 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:37:18.393652 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:18.475404 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:18.493913 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84a40a4-67ba-4b4a-81af-8dc1599aa975 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:18.534482 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74686d06-216d-45e5-8e40-5986db3873a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:18.535288 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74686d06-216d-45e5-8e40-5986db3873a1 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:18.535578 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74686d06-216d-45e5-8e40-5986db3873a1 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:18.535860 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74686d06-216d-45e5-8e40-5986db3873a1 None admin] PATH_INFO: `/v3/projects/ff0814efdfab4314800f7e0e2109a949` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:18.537144 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74686d06-216d-45e5-8e40-5986db3873a1 None admin] RBAC: Authorizing `identity:get_project(project_id=ff0814efdfab4314800f7e0e2109a949)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:18.538667 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74686d06-216d-45e5-8e40-5986db3873a1 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:18.540661 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 67/136] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:37:18 2022] GET /identity/v3/projects/ff0814efdfab4314800f7e0e2109a949 => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:18.547327 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f1ed7d-5156-4f55-b11a-4039ae54b1f9 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:18.698839 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f1ed7d-5156-4f55-b11a-4039ae54b1f9 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:18.739062 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48947d8-f245-481d-9761-2adfc1bcbbdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:18.739972 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48947d8-f245-481d-9761-2adfc1bcbbdb None admin] REQUEST_METHOD: `PUT` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:18.740260 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48947d8-f245-481d-9761-2adfc1bcbbdb None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:18.740538 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48947d8-f245-481d-9761-2adfc1bcbbdb None admin] PATH_INFO: `/v3/projects/ff0814efdfab4314800f7e0e2109a949/users/d15c701d832345a08a0951afb6b6d914/roles/b894e00b17e34f0cbde0ab3d760e2433` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:18.744352 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48947d8-f245-481d-9761-2adfc1bcbbdb None admin] RBAC: Authorizing `identity:create_grant(project_id=ff0814efdfab4314800f7e0e2109a949, user_id=d15c701d832345a08a0951afb6b6d914, role_id=b894e00b17e34f0cbde0ab3d760e2433)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:18.744614 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48947d8-f245-481d-9761-2adfc1bcbbdb None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:18.764870 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 70/137] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 14:37:18 2022] PUT /identity/v3/projects/ff0814efdfab4314800f7e0e2109a949/users/d15c701d832345a08a0951afb6b6d914/roles/b894e00b17e34f0cbde0ab3d760e2433 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:37:18.949373 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbf72ce-bafc-496e-80c0-9489f5b0071f None None] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:18.949780 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbf72ce-bafc-496e-80c0-9489f5b0071f None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:18.950008 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbf72ce-bafc-496e-80c0-9489f5b0071f None None] PATH_INFO: `/` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:18.951531 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 68/138] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:37:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:37:18.957268 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19809442-573d-47d2-b56f-e5053fddb754 None None] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:18.957925 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19809442-573d-47d2-b56f-e5053fddb754 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:18.957925 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19809442-573d-47d2-b56f-e5053fddb754 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:18.981130 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.auth.core [None req-19809442-573d-47d2-b56f-e5053fddb754 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:37:18.982443 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.fernet_utils [None req-19809442-573d-47d2-b56f-e5053fddb754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:37:19.052160 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:19.064390 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 71/139] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:37:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:37:19.091335 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbf72ce-bafc-496e-80c0-9489f5b0071f None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:19.202214 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbf72ce-bafc-496e-80c0-9489f5b0071f None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:19.237339 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345ab6b1-135a-4a37-8df2-cb1b2f71fb65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:19.238192 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345ab6b1-135a-4a37-8df2-cb1b2f71fb65 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:19.238469 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345ab6b1-135a-4a37-8df2-cb1b2f71fb65 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:19.238745 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345ab6b1-135a-4a37-8df2-cb1b2f71fb65 None admin] PATH_INFO: `/v3/domains/default` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:19.239945 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345ab6b1-135a-4a37-8df2-cb1b2f71fb65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:19.241402 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345ab6b1-135a-4a37-8df2-cb1b2f71fb65 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:19.244150 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 69/140] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:37:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:19.249510 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19809442-573d-47d2-b56f-e5053fddb754 None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:19.353651 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19809442-573d-47d2-b56f-e5053fddb754 None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:19.389140 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc17124-236c-4318-b934-203a5e7ced0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:19.389832 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc17124-236c-4318-b934-203a5e7ced0a None admin] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:19.390146 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc17124-236c-4318-b934-203a5e7ced0a None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:19.390429 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc17124-236c-4318-b934-203a5e7ced0a None admin] PATH_INFO: `/v3/users` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:19.391227 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc17124-236c-4318-b934-203a5e7ced0a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:19.392703 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc17124-236c-4318-b934-203a5e7ced0a None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:19.417288 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 72/141] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 19 14:37:19 2022] POST /identity/v3/users => generated 283 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:37:19.818770 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d616b831-977c-4e04-8cce-8271f975a6e7 None None] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:19.819503 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d616b831-977c-4e04-8cce-8271f975a6e7 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:19.820020 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d616b831-977c-4e04-8cce-8271f975a6e7 None None] PATH_INFO: `/` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:19.822415 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 70/142] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:37:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:37:19.828160 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd80d6ef-2fb6-49ad-ae9a-f21dfba3fb03 None None] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:19.828740 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd80d6ef-2fb6-49ad-ae9a-f21dfba3fb03 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:19.829237 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd80d6ef-2fb6-49ad-ae9a-f21dfba3fb03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:19.851899 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.auth.core [None req-bd80d6ef-2fb6-49ad-ae9a-f21dfba3fb03 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:37:19.853485 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.fernet_utils [None req-bd80d6ef-2fb6-49ad-ae9a-f21dfba3fb03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:37:19.908089 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:19.938147 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 73/143] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:37:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:37:19.960787 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d616b831-977c-4e04-8cce-8271f975a6e7 None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:19.988562 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:20.088869 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d616b831-977c-4e04-8cce-8271f975a6e7 None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:20.129122 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843aaa66-0dcf-4739-87c1-057e22dbc6ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:20.130017 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843aaa66-0dcf-4739-87c1-057e22dbc6ca None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:20.130313 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843aaa66-0dcf-4739-87c1-057e22dbc6ca None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:20.130597 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843aaa66-0dcf-4739-87c1-057e22dbc6ca None admin] PATH_INFO: `/v3/roles/member` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:20.144819 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843aaa66-0dcf-4739-87c1-057e22dbc6ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:37:20.153663 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-843aaa66-0dcf-4739-87c1-057e22dbc6ca None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:37:20.153663 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:37:20.153663 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:37:20.153663 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:37:20.153663 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:37:20.153663 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:37:20.153663 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:37:20.153663 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:37:20.153663 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:37:20.153663 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:37:20.153663 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:37:20.153663 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:37:20.153663 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:37:20.153663 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:37:20.153663 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:37:20.153663 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:37:20.154698 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:37:20.154698 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:37:20.154698 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:37:20.154698 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:37:20.154698 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:37:20.154698 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:37:20.154698 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:37:20.154698 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:37:20.154698 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:37:20.154698 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:37:20.154698 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:37:20.154698 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:37:20.154698 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:37:20.154698 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:37:20.154698 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:37:20.154698 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:37:20.155592 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:37:20.155592 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:37:20.155592 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:37:20.155592 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:37:20.155592 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:37:20.155592 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:37:20.155592 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:37:20.155592 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:37:20.155592 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:37:20.155592 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:37:20.155592 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:37:20.155592 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:37:20.159677 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 71/144] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:37:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:37:20.161420 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd80d6ef-2fb6-49ad-ae9a-f21dfba3fb03 None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:20.291617 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd80d6ef-2fb6-49ad-ae9a-f21dfba3fb03 None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:20.335517 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54dccd2-8c2b-477d-85d8-6ff13327f826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:20.335517 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54dccd2-8c2b-477d-85d8-6ff13327f826 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:20.335517 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54dccd2-8c2b-477d-85d8-6ff13327f826 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:20.335517 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54dccd2-8c2b-477d-85d8-6ff13327f826 None admin] PATH_INFO: `/v3/roles` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:20.336546 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54dccd2-8c2b-477d-85d8-6ff13327f826 None admin] RBAC: Adding query filter params (name=member) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:37:20.336546 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54dccd2-8c2b-477d-85d8-6ff13327f826 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:20.336546 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54dccd2-8c2b-477d-85d8-6ff13327f826 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:20.344852 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 74/145] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:37:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:20.350597 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843aaa66-0dcf-4739-87c1-057e22dbc6ca None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:20.454578 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:20.476407 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843aaa66-0dcf-4739-87c1-057e22dbc6ca None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:20.515258 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c04475-a339-46f8-b64b-8b07f5a609db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:20.516148 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c04475-a339-46f8-b64b-8b07f5a609db None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:20.516440 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c04475-a339-46f8-b64b-8b07f5a609db None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:20.516728 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c04475-a339-46f8-b64b-8b07f5a609db None admin] PATH_INFO: `/v3/users/d15c701d832345a08a0951afb6b6d914` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:20.518026 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c04475-a339-46f8-b64b-8b07f5a609db None admin] RBAC: Authorizing `identity:get_user(user_id=d15c701d832345a08a0951afb6b6d914)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:20.519562 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c04475-a339-46f8-b64b-8b07f5a609db None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:20.521549 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 72/146] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:37:20 2022] GET /identity/v3/users/d15c701d832345a08a0951afb6b6d914 => generated 275 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:20.527709 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54dccd2-8c2b-477d-85d8-6ff13327f826 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:20.571357 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:20.636477 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54dccd2-8c2b-477d-85d8-6ff13327f826 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:20.658702 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3739c867-74e9-4ab1-a8db-45fe0ab5fd52 None None] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:20.659036 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3739c867-74e9-4ab1-a8db-45fe0ab5fd52 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:20.659279 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3739c867-74e9-4ab1-a8db-45fe0ab5fd52 None None] PATH_INFO: `/` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:20.660805 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 73/147] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:37:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:37:20.666767 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3be64b0-3233-4328-b1d8-a0b59a4ec435 None None] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:20.667064 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3be64b0-3233-4328-b1d8-a0b59a4ec435 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:20.667353 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3be64b0-3233-4328-b1d8-a0b59a4ec435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:20.693364 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.auth.core [None req-a3be64b0-3233-4328-b1d8-a0b59a4ec435 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:37:20.694622 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-a3be64b0-3233-4328-b1d8-a0b59a4ec435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:37:20.700786 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4c5c37-b1ce-40dc-83ad-6af46509a385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:20.701730 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4c5c37-b1ce-40dc-83ad-6af46509a385 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:20.702207 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4c5c37-b1ce-40dc-83ad-6af46509a385 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:20.702674 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4c5c37-b1ce-40dc-83ad-6af46509a385 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:20.812329 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 74/148] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:37:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:37:20.840676 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4c5c37-b1ce-40dc-83ad-6af46509a385 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:20.842224 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4c5c37-b1ce-40dc-83ad-6af46509a385 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:20.846989 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3be64b0-3233-4328-b1d8-a0b59a4ec435 None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:20.888967 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:20.958798 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 75/149] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:37:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:37:20.965136 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4c5c37-b1ce-40dc-83ad-6af46509a385 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:20.991481 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:21.017177 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3be64b0-3233-4328-b1d8-a0b59a4ec435 None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:21.063281 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd058351-1c5b-4a9d-b8e4-fc62a7c47e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:21.064095 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd058351-1c5b-4a9d-b8e4-fc62a7c47e63 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:21.064381 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd058351-1c5b-4a9d-b8e4-fc62a7c47e63 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:21.064658 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd058351-1c5b-4a9d-b8e4-fc62a7c47e63 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:21.077507 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd058351-1c5b-4a9d-b8e4-fc62a7c47e63 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:37:21.088459 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-bd058351-1c5b-4a9d-b8e4-fc62a7c47e63 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:37:21.088459 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:37:21.088459 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:37:21.088459 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:37:21.088459 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:37:21.088459 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:37:21.088459 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:37:21.088459 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:37:21.088459 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:37:21.088459 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:37:21.088459 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:37:21.088459 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:37:21.088459 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:37:21.088459 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:37:21.088459 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:37:21.088459 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:37:21.089519 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:37:21.089519 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:37:21.089519 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:37:21.089519 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:37:21.089519 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:37:21.089519 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:37:21.089519 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:37:21.089519 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:37:21.089519 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:37:21.089519 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:37:21.089519 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:37:21.089519 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:37:21.089519 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:37:21.089519 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:37:21.089519 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:37:21.089519 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:37:21.090432 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:37:21.090432 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:37:21.090432 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:37:21.090432 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:37:21.090432 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:37:21.090432 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:37:21.090432 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:37:21.090432 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:37:21.090432 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:37:21.090432 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:37:21.090432 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:37:21.090432 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:37:21.090432 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 75/150] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:37:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:37:21.097959 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd058351-1c5b-4a9d-b8e4-fc62a7c47e63 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:21.177417 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4c5c37-b1ce-40dc-83ad-6af46509a385 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:21.239175 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62233e8-4709-4719-a617-555b3f4799d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:21.240065 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62233e8-4709-4719-a617-555b3f4799d8 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:21.240356 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62233e8-4709-4719-a617-555b3f4799d8 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:21.240640 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62233e8-4709-4719-a617-555b3f4799d8 None admin] PATH_INFO: `/v3/projects/ff0814efdfab4314800f7e0e2109a949` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:21.246647 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62233e8-4709-4719-a617-555b3f4799d8 None admin] RBAC: Authorizing `identity:get_project(project_id=ff0814efdfab4314800f7e0e2109a949)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:21.248146 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62233e8-4709-4719-a617-555b3f4799d8 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:21.250108 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 76/151] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:37:20 2022] GET /identity/v3/projects/ff0814efdfab4314800f7e0e2109a949 => generated 298 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:37:21.255725 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62233e8-4709-4719-a617-555b3f4799d8 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:21.270747 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:21.288900 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd058351-1c5b-4a9d-b8e4-fc62a7c47e63 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:21.357171 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527e7282-87a7-42ce-91f3-09acf281e939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:21.358051 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527e7282-87a7-42ce-91f3-09acf281e939 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:21.358343 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527e7282-87a7-42ce-91f3-09acf281e939 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:21.358622 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527e7282-87a7-42ce-91f3-09acf281e939 None admin] PATH_INFO: `/v3/roles` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:21.359398 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527e7282-87a7-42ce-91f3-09acf281e939 None admin] RBAC: Adding query filter params (name=reader) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:37:21.359710 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527e7282-87a7-42ce-91f3-09acf281e939 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:21.361312 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527e7282-87a7-42ce-91f3-09acf281e939 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:21.376259 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 76/152] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:37:21 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:37:21.382311 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527e7282-87a7-42ce-91f3-09acf281e939 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:21.388013 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:21.406774 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62233e8-4709-4719-a617-555b3f4799d8 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:21.445221 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823299e8-0496-4c99-9f2e-d028c672dc2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:21.446434 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823299e8-0496-4c99-9f2e-d028c672dc2d None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:21.446968 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823299e8-0496-4c99-9f2e-d028c672dc2d None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:21.447444 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823299e8-0496-4c99-9f2e-d028c672dc2d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:21.448333 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823299e8-0496-4c99-9f2e-d028c672dc2d None admin] RBAC: Adding query filter params (role.id=7ebce5564f40456e94a3616961ed73ff, scope.project.id=ff0814efdfab4314800f7e0e2109a949, user.id=d15c701d832345a08a0951afb6b6d914) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:37:21.448656 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823299e8-0496-4c99-9f2e-d028c672dc2d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:21.450404 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823299e8-0496-4c99-9f2e-d028c672dc2d None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:21.457337 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 77/153] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:37:21 2022] GET /identity/v3/role_assignments?user.id=d15c701d832345a08a0951afb6b6d914&scope.project.id=ff0814efdfab4314800f7e0e2109a949&role.id=7ebce5564f40456e94a3616961ed73ff => generated 608 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:21.581407 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527e7282-87a7-42ce-91f3-09acf281e939 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:21.633481 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bf571e-f517-4559-9c0f-b207a069eae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:21.635026 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bf571e-f517-4559-9c0f-b207a069eae3 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:21.635550 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bf571e-f517-4559-9c0f-b207a069eae3 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:21.636033 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bf571e-f517-4559-9c0f-b207a069eae3 None admin] PATH_INFO: `/v3/users/bc06d3d555b24e59a3957bf501a9d140` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:21.642866 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bf571e-f517-4559-9c0f-b207a069eae3 None admin] RBAC: Authorizing `identity:get_user(user_id=bc06d3d555b24e59a3957bf501a9d140)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:21.644719 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bf571e-f517-4559-9c0f-b207a069eae3 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:21.646732 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 77/154] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:37:21 2022] GET /identity/v3/users/bc06d3d555b24e59a3957bf501a9d140 => generated 289 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:37:21.652805 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823299e8-0496-4c99-9f2e-d028c672dc2d None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:21.795601 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:21.817085 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823299e8-0496-4c99-9f2e-d028c672dc2d None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:21.862074 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5350c6f8-0f02-4fd8-9c06-38b23a45b112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:21.862957 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5350c6f8-0f02-4fd8-9c06-38b23a45b112 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:21.863245 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5350c6f8-0f02-4fd8-9c06-38b23a45b112 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:21.863526 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5350c6f8-0f02-4fd8-9c06-38b23a45b112 None admin] PATH_INFO: `/v3/projects/ff0814efdfab4314800f7e0e2109a949` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:21.864782 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5350c6f8-0f02-4fd8-9c06-38b23a45b112 None admin] RBAC: Authorizing `identity:get_project(project_id=ff0814efdfab4314800f7e0e2109a949)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:21.866301 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5350c6f8-0f02-4fd8-9c06-38b23a45b112 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:21.868747 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 78/155] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:37:21 2022] GET /identity/v3/projects/ff0814efdfab4314800f7e0e2109a949 => generated 298 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:21.874788 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bf571e-f517-4559-9c0f-b207a069eae3 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:21.989247 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:22.031249 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bf571e-f517-4559-9c0f-b207a069eae3 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:22.079774 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571eaba8-8a1c-4abe-8abd-1a571ecf3b06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:22.081080 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571eaba8-8a1c-4abe-8abd-1a571ecf3b06 None admin] REQUEST_METHOD: `PUT` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:22.081588 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571eaba8-8a1c-4abe-8abd-1a571ecf3b06 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:22.082124 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571eaba8-8a1c-4abe-8abd-1a571ecf3b06 None admin] PATH_INFO: `/v3/projects/ff0814efdfab4314800f7e0e2109a949/users/bc06d3d555b24e59a3957bf501a9d140/roles/6129e931488746afb2145fa463c08395` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:22.087969 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571eaba8-8a1c-4abe-8abd-1a571ecf3b06 None admin] RBAC: Authorizing `identity:create_grant(project_id=ff0814efdfab4314800f7e0e2109a949, user_id=bc06d3d555b24e59a3957bf501a9d140, role_id=6129e931488746afb2145fa463c08395)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:22.089746 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571eaba8-8a1c-4abe-8abd-1a571ecf3b06 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:22.112377 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 78/156] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 14:37:21 2022] PUT /identity/v3/projects/ff0814efdfab4314800f7e0e2109a949/users/bc06d3d555b24e59a3957bf501a9d140/roles/6129e931488746afb2145fa463c08395 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:37:22.330990 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045a778d-fae9-4783-88ac-805426e5b438 None None] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:22.332018 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045a778d-fae9-4783-88ac-805426e5b438 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:22.332522 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045a778d-fae9-4783-88ac-805426e5b438 None None] PATH_INFO: `/` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:22.334472 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 79/157] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:37:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:37:22.342480 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015ebbd9-6e21-4dd7-87ca-add129b3e788 None None] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:22.343029 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015ebbd9-6e21-4dd7-87ca-add129b3e788 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:22.343029 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015ebbd9-6e21-4dd7-87ca-add129b3e788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:22.374915 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.auth.core [None req-015ebbd9-6e21-4dd7-87ca-add129b3e788 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:37:22.376140 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-015ebbd9-6e21-4dd7-87ca-add129b3e788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:37:22.464486 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 79/158] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:37:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:37:22.506394 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045a778d-fae9-4783-88ac-805426e5b438 None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:22.657079 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045a778d-fae9-4783-88ac-805426e5b438 None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:22.702722 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43794c5-cb38-4dc9-bb44-af85326ab862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:22.703537 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43794c5-cb38-4dc9-bb44-af85326ab862 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:22.703822 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43794c5-cb38-4dc9-bb44-af85326ab862 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:22.704095 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43794c5-cb38-4dc9-bb44-af85326ab862 None admin] PATH_INFO: `/v3/roles/member` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:22.716563 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:22.725828 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43794c5-cb38-4dc9-bb44-af85326ab862 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:37:22.734349 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-f43794c5-cb38-4dc9-bb44-af85326ab862 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:37:22.734349 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:37:22.734349 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:37:22.734349 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:37:22.734349 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:37:22.734349 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:37:22.734349 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:37:22.734349 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:37:22.734349 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:37:22.734349 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:37:22.734349 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:37:22.734349 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:37:22.734349 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:37:22.734349 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:37:22.734349 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:37:22.734349 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:37:22.735387 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:37:22.735387 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:37:22.735387 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:37:22.735387 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:37:22.735387 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:37:22.735387 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:37:22.735387 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:37:22.735387 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:37:22.735387 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:37:22.735387 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:37:22.735387 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:37:22.735387 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:37:22.735387 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:37:22.735387 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:37:22.735387 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:37:22.735387 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:37:22.736292 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:37:22.736292 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:37:22.736292 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:37:22.736292 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:37:22.736292 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:37:22.736292 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:37:22.736292 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:37:22.736292 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:37:22.736292 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:37:22.736292 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:37:22.736292 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:37:22.736292 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:37:22.736292 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 80/159] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:37:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:37:22.746985 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015ebbd9-6e21-4dd7-87ca-add129b3e788 None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:22.821979 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:22.867950 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015ebbd9-6e21-4dd7-87ca-add129b3e788 None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:22.871662 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f822a8-2ba5-4a80-bcb2-b6ca7cbe0f14 None None] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:22.871959 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f822a8-2ba5-4a80-bcb2-b6ca7cbe0f14 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:22.872243 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f822a8-2ba5-4a80-bcb2-b6ca7cbe0f14 None None] PATH_INFO: `/` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:22.873799 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 81/160] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:37:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:37:22.879012 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15af348-8db2-4268-bd9b-0340eb50e821 None None] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:22.879300 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15af348-8db2-4268-bd9b-0340eb50e821 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:22.879578 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15af348-8db2-4268-bd9b-0340eb50e821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:22.915370 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.auth.core [None req-b15af348-8db2-4268-bd9b-0340eb50e821 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:37:22.916504 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.fernet_utils [None req-b15af348-8db2-4268-bd9b-0340eb50e821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:37:22.919106 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fcdb32-9e2d-4149-922f-7b56c8d4dc05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:22.919785 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fcdb32-9e2d-4149-922f-7b56c8d4dc05 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:22.920066 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fcdb32-9e2d-4149-922f-7b56c8d4dc05 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:22.920339 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fcdb32-9e2d-4149-922f-7b56c8d4dc05 None admin] PATH_INFO: `/v3/roles` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:22.921106 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fcdb32-9e2d-4149-922f-7b56c8d4dc05 None admin] RBAC: Adding query filter params (name=member) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:37:22.921412 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fcdb32-9e2d-4149-922f-7b56c8d4dc05 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:22.922846 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fcdb32-9e2d-4149-922f-7b56c8d4dc05 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:22.944394 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 80/161] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:37:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:22.951632 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fcdb32-9e2d-4149-922f-7b56c8d4dc05 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:23.078569 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 82/162] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:37:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:37:23.130665 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fcdb32-9e2d-4149-922f-7b56c8d4dc05 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:23.134110 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15af348-8db2-4268-bd9b-0340eb50e821 None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:23.156234 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:23.183795 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a7d0dc-a032-425a-8787-722228c0705c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:23.184990 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a7d0dc-a032-425a-8787-722228c0705c None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:23.185457 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a7d0dc-a032-425a-8787-722228c0705c None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:23.185944 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a7d0dc-a032-425a-8787-722228c0705c None admin] PATH_INFO: `/v3/users/d15c701d832345a08a0951afb6b6d914` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:23.187556 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a7d0dc-a032-425a-8787-722228c0705c None admin] RBAC: Authorizing `identity:get_user(user_id=d15c701d832345a08a0951afb6b6d914)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:23.189397 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a7d0dc-a032-425a-8787-722228c0705c None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:23.191717 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 81/163] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:37:22 2022] GET /identity/v3/users/d15c701d832345a08a0951afb6b6d914 => generated 275 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:37:23.199068 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a7d0dc-a032-425a-8787-722228c0705c None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:23.302824 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15af348-8db2-4268-bd9b-0340eb50e821 None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:23.321413 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a7d0dc-a032-425a-8787-722228c0705c None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:23.331302 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:23.340639 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0304cf4d-7172-4582-8409-b1e2c0ae8f94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:23.341627 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0304cf4d-7172-4582-8409-b1e2c0ae8f94 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:23.341761 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0304cf4d-7172-4582-8409-b1e2c0ae8f94 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:23.342096 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0304cf4d-7172-4582-8409-b1e2c0ae8f94 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:23.353192 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0304cf4d-7172-4582-8409-b1e2c0ae8f94 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:37:23.361046 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-0304cf4d-7172-4582-8409-b1e2c0ae8f94 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:37:23.361046 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:37:23.361046 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:37:23.361046 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:37:23.361046 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:37:23.361046 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:37:23.361046 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:37:23.361046 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:37:23.361046 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:37:23.361046 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:37:23.361046 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:37:23.361046 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:37:23.361046 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:37:23.361046 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:37:23.361046 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:37:23.361046 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:37:23.362287 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:37:23.362287 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:37:23.362287 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:37:23.362287 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:37:23.362287 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:37:23.362287 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:37:23.362287 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:37:23.362287 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:37:23.362287 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:37:23.362287 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:37:23.362287 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:37:23.362287 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:37:23.362287 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:37:23.362287 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:37:23.362287 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:37:23.362287 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:37:23.363280 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:37:23.363280 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:37:23.363280 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:37:23.363280 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:37:23.363280 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:37:23.363280 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:37:23.363280 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:37:23.363280 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:37:23.363280 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:37:23.363280 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:37:23.363280 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:37:23.363280 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:37:23.363280 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 83/164] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:37:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:37:23.368494 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0304cf4d-7172-4582-8409-b1e2c0ae8f94 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:23.415705 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e174bea6-b733-4702-aa47-4a26370f54dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:23.416543 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e174bea6-b733-4702-aa47-4a26370f54dc None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:23.416932 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e174bea6-b733-4702-aa47-4a26370f54dc None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:23.417371 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e174bea6-b733-4702-aa47-4a26370f54dc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:23.545761 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:23.554837 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0304cf4d-7172-4582-8409-b1e2c0ae8f94 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:23.560482 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e174bea6-b733-4702-aa47-4a26370f54dc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:23.562147 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e174bea6-b733-4702-aa47-4a26370f54dc None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:23.612017 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7c5557-f972-4c0d-bbea-0df930c4a016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:23.612709 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7c5557-f972-4c0d-bbea-0df930c4a016 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:23.613047 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7c5557-f972-4c0d-bbea-0df930c4a016 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:23.613331 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7c5557-f972-4c0d-bbea-0df930c4a016 None admin] PATH_INFO: `/v3/roles` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:23.614126 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7c5557-f972-4c0d-bbea-0df930c4a016 None admin] RBAC: Adding query filter params (name=admin) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:37:23.614433 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7c5557-f972-4c0d-bbea-0df930c4a016 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:23.615833 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7c5557-f972-4c0d-bbea-0df930c4a016 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:23.626698 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 84/165] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:37:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:37:23.628041 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195ed3b7-e7d1-4543-a16a-7b01d5b6086d None None] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:23.628397 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195ed3b7-e7d1-4543-a16a-7b01d5b6086d None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:23.628680 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195ed3b7-e7d1-4543-a16a-7b01d5b6086d None None] PATH_INFO: `/` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:23.630049 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 85/166] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:37:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:37:23.634779 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195ed3b7-e7d1-4543-a16a-7b01d5b6086d None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:23.760633 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 82/167] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:37:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 562 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:37:23.761707 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9874f264-c4c7-4b2f-affc-89fbd25a010b None None] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:23.770623 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9874f264-c4c7-4b2f-affc-89fbd25a010b None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:23.770941 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9874f264-c4c7-4b2f-affc-89fbd25a010b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:23.771405 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:23.811328 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.auth.core [None req-9874f264-c4c7-4b2f-affc-89fbd25a010b None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:37:23.812164 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-9874f264-c4c7-4b2f-affc-89fbd25a010b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:37:23.878562 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195ed3b7-e7d1-4543-a16a-7b01d5b6086d None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:23.950889 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a334edf4-d97a-45d5-bf3b-84ddf8de2c5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:23.951786 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a334edf4-d97a-45d5-bf3b-84ddf8de2c5f None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:23.952069 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a334edf4-d97a-45d5-bf3b-84ddf8de2c5f None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:23.952339 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a334edf4-d97a-45d5-bf3b-84ddf8de2c5f None admin] PATH_INFO: `/v3/users/d8062aba2d374babbc9d68178a9546e3` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:23.953564 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a334edf4-d97a-45d5-bf3b-84ddf8de2c5f None admin] RBAC: Authorizing `identity:get_user(user_id=d8062aba2d374babbc9d68178a9546e3)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:23.955072 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a334edf4-d97a-45d5-bf3b-84ddf8de2c5f None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:23.962348 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 86/168] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:37:23 2022] GET /identity/v3/users/d8062aba2d374babbc9d68178a9546e3 => generated 247 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:23.963280 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a334edf4-d97a-45d5-bf3b-84ddf8de2c5f None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:23.970701 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 83/169] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:37:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:37:23.971617 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9874f264-c4c7-4b2f-affc-89fbd25a010b None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:24.014475 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:24.212194 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9874f264-c4c7-4b2f-affc-89fbd25a010b None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:24.228890 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a334edf4-d97a-45d5-bf3b-84ddf8de2c5f None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:24.274700 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5410293c-c1d9-44d3-961b-ffe17d9886a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:24.275089 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:24.275258 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5410293c-c1d9-44d3-961b-ffe17d9886a7 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:24.275547 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5410293c-c1d9-44d3-961b-ffe17d9886a7 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:24.275826 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5410293c-c1d9-44d3-961b-ffe17d9886a7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:24.331229 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46ec853-15d1-40ed-ba88-45168b980bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:24.332123 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46ec853-15d1-40ed-ba88-45168b980bfe None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:24.332670 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46ec853-15d1-40ed-ba88-45168b980bfe None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:24.332670 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46ec853-15d1-40ed-ba88-45168b980bfe None admin] PATH_INFO: `/v3/projects/c539a4fcbe8c4250a79893326416b622` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:24.334476 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46ec853-15d1-40ed-ba88-45168b980bfe None admin] RBAC: Authorizing `identity:get_project(project_id=c539a4fcbe8c4250a79893326416b622)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:24.336222 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46ec853-15d1-40ed-ba88-45168b980bfe None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:24.338018 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 87/170] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:37:23 2022] GET /identity/v3/projects/c539a4fcbe8c4250a79893326416b622 => generated 312 bytes in 375 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:24.338868 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46ec853-15d1-40ed-ba88-45168b980bfe None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:24.407439 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5410293c-c1d9-44d3-961b-ffe17d9886a7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:24.409288 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5410293c-c1d9-44d3-961b-ffe17d9886a7 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:24.450735 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:24.517895 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46ec853-15d1-40ed-ba88-45168b980bfe None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:24.554656 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528e1e27-9f36-4c28-abcc-572255e0c7d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:24.556377 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528e1e27-9f36-4c28-abcc-572255e0c7d9 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:24.556782 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528e1e27-9f36-4c28-abcc-572255e0c7d9 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:24.557173 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528e1e27-9f36-4c28-abcc-572255e0c7d9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:24.570035 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528e1e27-9f36-4c28-abcc-572255e0c7d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:37:24.579313 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-528e1e27-9f36-4c28-abcc-572255e0c7d9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:37:24.579313 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:37:24.579313 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:37:24.579313 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:37:24.579313 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:37:24.579313 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:37:24.579313 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:37:24.579313 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:37:24.579313 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:37:24.579313 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:37:24.579313 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:37:24.579313 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:37:24.579313 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:37:24.579313 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:37:24.579313 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:37:24.579313 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:37:24.580467 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:37:24.580467 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:37:24.580467 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:37:24.580467 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:37:24.580467 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:37:24.580467 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:37:24.580467 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:37:24.580467 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:37:24.580467 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:37:24.580467 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:37:24.580467 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:37:24.580467 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:37:24.580467 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:37:24.580467 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:37:24.580467 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:37:24.580467 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:37:24.581384 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:37:24.581384 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:37:24.581384 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:37:24.581384 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:37:24.581384 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:37:24.581384 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:37:24.581384 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:37:24.581384 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:37:24.581384 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:37:24.581384 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:37:24.581384 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:37:24.581384 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:37:24.581384 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 88/171] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 19 14:37:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:37:24.583965 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a03450c-a895-4a3c-b5b4-5967c40369af None None] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:24.583965 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a03450c-a895-4a3c-b5b4-5967c40369af None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:24.583965 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a03450c-a895-4a3c-b5b4-5967c40369af None None] PATH_INFO: `/` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:24.585750 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 89/172] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:37:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:37:24.586778 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a03450c-a895-4a3c-b5b4-5967c40369af None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:24.646382 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:24.652554 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 84/173] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:37:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 681 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:37:24.653450 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5410293c-c1d9-44d3-961b-ffe17d9886a7 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:24.738370 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a03450c-a895-4a3c-b5b4-5967c40369af None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:24.776500 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5410293c-c1d9-44d3-961b-ffe17d9886a7 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:24.797270 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:24.812105 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d05eeb-f211-474b-8443-2226b9eebaa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:24.812787 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748cc505-35bc-4727-873d-003692f2f8ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:24.812950 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d05eeb-f211-474b-8443-2226b9eebaa3 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:24.817961 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d05eeb-f211-474b-8443-2226b9eebaa3 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:24.818127 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748cc505-35bc-4727-873d-003692f2f8ca None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:24.818127 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748cc505-35bc-4727-873d-003692f2f8ca None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:24.818127 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748cc505-35bc-4727-873d-003692f2f8ca None admin] PATH_INFO: `/v3/roles` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:24.818127 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748cc505-35bc-4727-873d-003692f2f8ca None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:37:24.818127 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748cc505-35bc-4727-873d-003692f2f8ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:24.818127 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748cc505-35bc-4727-873d-003692f2f8ca None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:24.819047 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d05eeb-f211-474b-8443-2226b9eebaa3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:24.820212 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d05eeb-f211-474b-8443-2226b9eebaa3 None admin] RBAC: Adding query filter params (role.id=7ebce5564f40456e94a3616961ed73ff, scope.project.id=c539a4fcbe8c4250a79893326416b622, user.id=d15c701d832345a08a0951afb6b6d914) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:37:24.821273 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d05eeb-f211-474b-8443-2226b9eebaa3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:24.822981 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d05eeb-f211-474b-8443-2226b9eebaa3 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:24.827235 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 85/174] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:37:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:24.828764 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffda7dbf-4ed3-44c1-b2ce-48fe7a12008a None None] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:24.829295 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffda7dbf-4ed3-44c1-b2ce-48fe7a12008a None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:24.829790 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffda7dbf-4ed3-44c1-b2ce-48fe7a12008a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:24.835813 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 90/175] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:37:24 2022] GET /identity/v3/role_assignments?user.id=d15c701d832345a08a0951afb6b6d914&scope.project.id=c539a4fcbe8c4250a79893326416b622&role.id=7ebce5564f40456e94a3616961ed73ff => generated 608 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:24.837581 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d05eeb-f211-474b-8443-2226b9eebaa3 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:24.867040 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.auth.core [None req-ffda7dbf-4ed3-44c1-b2ce-48fe7a12008a None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:37:24.868820 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-ffda7dbf-4ed3-44c1-b2ce-48fe7a12008a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:37:24.956045 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d05eeb-f211-474b-8443-2226b9eebaa3 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:24.956888 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 86/176] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:37:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:37:24.965125 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffda7dbf-4ed3-44c1-b2ce-48fe7a12008a None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:24.985456 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:25.020053 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12ccd02-7adf-41e5-971a-336bd8aba218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:25.021476 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12ccd02-7adf-41e5-971a-336bd8aba218 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:25.022024 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12ccd02-7adf-41e5-971a-336bd8aba218 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:25.022534 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12ccd02-7adf-41e5-971a-336bd8aba218 None admin] PATH_INFO: `/v3/projects/ff0814efdfab4314800f7e0e2109a949` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:25.024277 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12ccd02-7adf-41e5-971a-336bd8aba218 None admin] RBAC: Authorizing `identity:get_project(project_id=ff0814efdfab4314800f7e0e2109a949)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:25.026241 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12ccd02-7adf-41e5-971a-336bd8aba218 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:25.028471 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 91/177] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:37:24 2022] GET /identity/v3/projects/ff0814efdfab4314800f7e0e2109a949 => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:25.030095 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e084065c-dfd5-41d3-9559-ed04cf6532ae None None] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:25.030601 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e084065c-dfd5-41d3-9559-ed04cf6532ae None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:25.031106 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e084065c-dfd5-41d3-9559-ed04cf6532ae None None] PATH_INFO: `/` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:25.032658 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 92/178] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:37:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:37:25.033987 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13062f05-a238-44af-a63d-edef4b6218a2 None None] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:25.034539 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13062f05-a238-44af-a63d-edef4b6218a2 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:25.035031 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13062f05-a238-44af-a63d-edef4b6218a2 None None] PATH_INFO: `/` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:25.036555 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 93/179] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:37:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:37:25.037505 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13062f05-a238-44af-a63d-edef4b6218a2 None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:25.072298 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffda7dbf-4ed3-44c1-b2ce-48fe7a12008a None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:25.088923 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:25.118838 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61fcfc0-4550-499f-b91a-d550d175b916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:25.119478 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61fcfc0-4550-499f-b91a-d550d175b916 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:25.119927 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61fcfc0-4550-499f-b91a-d550d175b916 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:25.120219 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61fcfc0-4550-499f-b91a-d550d175b916 None admin] PATH_INFO: `/v3/users/d15c701d832345a08a0951afb6b6d914` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:25.122017 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61fcfc0-4550-499f-b91a-d550d175b916 None admin] RBAC: Authorizing `identity:get_user(user_id=d15c701d832345a08a0951afb6b6d914)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:25.123907 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61fcfc0-4550-499f-b91a-d550d175b916 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:25.125769 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 87/180] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:37:24 2022] GET /identity/v3/users/d15c701d832345a08a0951afb6b6d914 => generated 275 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:25.126927 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2f5f29-cd13-45c1-b24f-97f1e984d2e4 None None] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:25.127218 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2f5f29-cd13-45c1-b24f-97f1e984d2e4 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:25.127496 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2f5f29-cd13-45c1-b24f-97f1e984d2e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:25.146034 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.auth.core [None req-8a2f5f29-cd13-45c1-b24f-97f1e984d2e4 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:37:25.147374 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-8a2f5f29-cd13-45c1-b24f-97f1e984d2e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:37:25.185257 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13062f05-a238-44af-a63d-edef4b6218a2 None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:25.230505 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454effd7-ef48-4698-85ec-385dc57b72d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:25.231700 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454effd7-ef48-4698-85ec-385dc57b72d6 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:25.232200 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454effd7-ef48-4698-85ec-385dc57b72d6 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:25.232719 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454effd7-ef48-4698-85ec-385dc57b72d6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:25.243685 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 88/181] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:37:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:37:25.245156 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b307f4b-36e1-477a-9988-75cc0e5ab27c None None] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:25.245449 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b307f4b-36e1-477a-9988-75cc0e5ab27c None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:25.245722 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b307f4b-36e1-477a-9988-75cc0e5ab27c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:25.250566 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454effd7-ef48-4698-85ec-385dc57b72d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:37:25.259500 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-454effd7-ef48-4698-85ec-385dc57b72d6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:37:25.259500 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:37:25.259500 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:37:25.259500 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:37:25.259500 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:37:25.259500 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:37:25.259500 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:37:25.259500 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:37:25.259500 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:37:25.259500 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:37:25.259500 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:37:25.259500 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:37:25.259500 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:37:25.259500 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:37:25.259500 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:37:25.259500 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:37:25.260525 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:37:25.260525 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:37:25.260525 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:37:25.260525 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:37:25.260525 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:37:25.260525 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:37:25.260525 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:37:25.260525 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:37:25.260525 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:37:25.260525 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:37:25.260525 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:37:25.260525 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:37:25.260525 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:37:25.260525 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:37:25.260525 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:37:25.260525 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:37:25.261466 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:37:25.261466 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:37:25.261466 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:37:25.261466 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:37:25.261466 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:37:25.261466 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:37:25.261466 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:37:25.261466 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:37:25.261466 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:37:25.261466 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:37:25.261466 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:37:25.261466 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:37:25.261466 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 94/182] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:37:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:37:25.263289 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454effd7-ef48-4698-85ec-385dc57b72d6 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:25.271547 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.auth.core [None req-3b307f4b-36e1-477a-9988-75cc0e5ab27c None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:37:25.272752 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-3b307f4b-36e1-477a-9988-75cc0e5ab27c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:37:25.308203 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:25.357971 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 89/183] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:37:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:37:25.357971 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b307f4b-36e1-477a-9988-75cc0e5ab27c None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:25.377897 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454effd7-ef48-4698-85ec-385dc57b72d6 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:25.407267 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:25.415120 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776b3e77-cc29-47ec-84d0-d4e8f95233e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:25.415895 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776b3e77-cc29-47ec-84d0-d4e8f95233e9 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:25.416184 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776b3e77-cc29-47ec-84d0-d4e8f95233e9 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:25.416458 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776b3e77-cc29-47ec-84d0-d4e8f95233e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:25.417306 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776b3e77-cc29-47ec-84d0-d4e8f95233e9 None admin] RBAC: Adding query filter params (role.id=c2935b1a25ca46fcba419fe3d1715330, scope.project.id=ff0814efdfab4314800f7e0e2109a949, user.id=d8062aba2d374babbc9d68178a9546e3) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:37:25.417619 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776b3e77-cc29-47ec-84d0-d4e8f95233e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:25.419108 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776b3e77-cc29-47ec-84d0-d4e8f95233e9 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:25.425255 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 95/184] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:37:25 2022] GET /identity/v3/role_assignments?user.id=d8062aba2d374babbc9d68178a9546e3&scope.project.id=ff0814efdfab4314800f7e0e2109a949&role.id=c2935b1a25ca46fcba419fe3d1715330 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:25.425255 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776b3e77-cc29-47ec-84d0-d4e8f95233e9 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:25.498878 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b307f4b-36e1-477a-9988-75cc0e5ab27c None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:25.543278 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4ffdc3-3fae-4b68-aef6-9ecf9eaeab53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:25.543944 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4ffdc3-3fae-4b68-aef6-9ecf9eaeab53 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:25.544232 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4ffdc3-3fae-4b68-aef6-9ecf9eaeab53 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:25.544522 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4ffdc3-3fae-4b68-aef6-9ecf9eaeab53 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:25.567597 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776b3e77-cc29-47ec-84d0-d4e8f95233e9 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:25.569493 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:25.607294 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21320b8-6a10-437e-b794-31e31f4e927c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:25.608458 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21320b8-6a10-437e-b794-31e31f4e927c None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:25.608988 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21320b8-6a10-437e-b794-31e31f4e927c None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:25.609329 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21320b8-6a10-437e-b794-31e31f4e927c None admin] PATH_INFO: `/v3/roles` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:25.610144 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21320b8-6a10-437e-b794-31e31f4e927c None admin] RBAC: Adding query filter params (name=admin) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:37:25.610638 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21320b8-6a10-437e-b794-31e31f4e927c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:25.612151 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21320b8-6a10-437e-b794-31e31f4e927c None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:25.623131 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 96/185] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:37:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:25.623549 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21320b8-6a10-437e-b794-31e31f4e927c None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:25.628748 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4ffdc3-3fae-4b68-aef6-9ecf9eaeab53 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:25.630276 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4ffdc3-3fae-4b68-aef6-9ecf9eaeab53 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:25.685312 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:25.744237 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21320b8-6a10-437e-b794-31e31f4e927c None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:25.751717 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 90/186] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:37:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:37:25.752397 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4ffdc3-3fae-4b68-aef6-9ecf9eaeab53 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:25.782186 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d083e48-e67b-49d7-999f-a3a20069c23a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:25.782986 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d083e48-e67b-49d7-999f-a3a20069c23a None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:25.783273 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d083e48-e67b-49d7-999f-a3a20069c23a None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:25.783548 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d083e48-e67b-49d7-999f-a3a20069c23a None admin] PATH_INFO: `/v3/domains/default` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:25.784717 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d083e48-e67b-49d7-999f-a3a20069c23a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:25.786206 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d083e48-e67b-49d7-999f-a3a20069c23a None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:25.787832 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 97/187] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:37:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:25.789079 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a581b13-d64b-44ba-ac27-50b26dac8a4b None None] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:25.789447 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a581b13-d64b-44ba-ac27-50b26dac8a4b None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:25.790260 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a581b13-d64b-44ba-ac27-50b26dac8a4b None None] PATH_INFO: `/` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:25.794087 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 98/188] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:37:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:37:25.794087 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a581b13-d64b-44ba-ac27-50b26dac8a4b None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:25.826035 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:25.854329 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4ffdc3-3fae-4b68-aef6-9ecf9eaeab53 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:25.889020 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e9c379-14a6-499d-a1bc-85a5fac28c19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:25.889828 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e9c379-14a6-499d-a1bc-85a5fac28c19 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:25.890144 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e9c379-14a6-499d-a1bc-85a5fac28c19 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:25.890420 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e9c379-14a6-499d-a1bc-85a5fac28c19 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:25.897901 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:25.900227 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a581b13-d64b-44ba-ac27-50b26dac8a4b None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:25.901429 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e9c379-14a6-499d-a1bc-85a5fac28c19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:37:25.910136 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-b7e9c379-14a6-499d-a1bc-85a5fac28c19 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:37:25.910136 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:37:25.910136 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:37:25.910136 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:37:25.910136 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:37:25.910136 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:37:25.910136 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:37:25.910136 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:37:25.910136 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:37:25.910136 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:37:25.910136 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:37:25.910136 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:37:25.910136 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:37:25.910136 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:37:25.910136 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:37:25.910136 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:37:25.911222 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:37:25.911222 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:37:25.911222 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:37:25.911222 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:37:25.911222 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:37:25.911222 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:37:25.911222 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:37:25.911222 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:37:25.911222 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:37:25.911222 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:37:25.911222 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:37:25.911222 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:37:25.911222 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:37:25.911222 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:37:25.911222 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:37:25.911222 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:37:25.912156 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:37:25.912156 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:37:25.912156 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:37:25.912156 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:37:25.912156 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:37:25.912156 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:37:25.912156 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:37:25.912156 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:37:25.912156 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:37:25.912156 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:37:25.912156 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:37:25.912156 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:37:25.912156 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 91/189] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 19 14:37:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:37:25.914002 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e9c379-14a6-499d-a1bc-85a5fac28c19 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:25.935013 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18323800-7778-4cfc-b017-beb3787b4d21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:25.935892 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18323800-7778-4cfc-b017-beb3787b4d21 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:25.936171 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18323800-7778-4cfc-b017-beb3787b4d21 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:25.936449 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18323800-7778-4cfc-b017-beb3787b4d21 None admin] PATH_INFO: `/v3/users/90ad82926efe47569ea5db9e4f598327` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:25.951185 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18323800-7778-4cfc-b017-beb3787b4d21 None admin] RBAC: Authorizing `identity:get_user(user_id=90ad82926efe47569ea5db9e4f598327)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:25.952790 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18323800-7778-4cfc-b017-beb3787b4d21 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:25.958930 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 99/190] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:37:25 2022] GET /identity/v3/users/90ad82926efe47569ea5db9e4f598327 => generated 283 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:25.958930 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18323800-7778-4cfc-b017-beb3787b4d21 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:26.017813 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e9c379-14a6-499d-a1bc-85a5fac28c19 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:26.045294 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:26.052897 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c9e946-71c8-4639-9071-e3bb6a9e974a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:26.053760 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c9e946-71c8-4639-9071-e3bb6a9e974a None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:26.054079 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c9e946-71c8-4639-9071-e3bb6a9e974a None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:26.054361 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c9e946-71c8-4639-9071-e3bb6a9e974a None admin] PATH_INFO: `/v3/projects/ff0814efdfab4314800f7e0e2109a949` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:26.056075 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18323800-7778-4cfc-b017-beb3787b4d21 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:26.056243 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c9e946-71c8-4639-9071-e3bb6a9e974a None admin] RBAC: Authorizing `identity:get_project(project_id=ff0814efdfab4314800f7e0e2109a949)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:26.057729 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c9e946-71c8-4639-9071-e3bb6a9e974a None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:26.059490 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 92/191] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:37:25 2022] GET /identity/v3/projects/ff0814efdfab4314800f7e0e2109a949 => generated 298 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:26.060650 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3175a3-b9cb-4b08-bcc1-816410f475d4 None None] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:26.060943 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3175a3-b9cb-4b08-bcc1-816410f475d4 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:26.061225 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3175a3-b9cb-4b08-bcc1-816410f475d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:26.077416 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.auth.core [None req-9d3175a3-b9cb-4b08-bcc1-816410f475d4 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:37:26.078661 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-9d3175a3-b9cb-4b08-bcc1-816410f475d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:37:26.091405 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b8472b-c355-4ae8-b83f-b63b43e29d43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:26.092076 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b8472b-c355-4ae8-b83f-b63b43e29d43 None admin] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:26.092359 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b8472b-c355-4ae8-b83f-b63b43e29d43 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:26.092633 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b8472b-c355-4ae8-b83f-b63b43e29d43 None admin] PATH_INFO: `/v3/users` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:26.093345 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b8472b-c355-4ae8-b83f-b63b43e29d43 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:26.094811 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b8472b-c355-4ae8-b83f-b63b43e29d43 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:26.115247 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 100/192] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 19 14:37:25 2022] POST /identity/v3/users => generated 297 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:37:26.116071 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b8472b-c355-4ae8-b83f-b63b43e29d43 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:26.157955 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:26.166952 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 93/193] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:37:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:37:26.168367 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3175a3-b9cb-4b08-bcc1-816410f475d4 None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:26.263965 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b8472b-c355-4ae8-b83f-b63b43e29d43 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:26.290699 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3175a3-b9cb-4b08-bcc1-816410f475d4 None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:26.305028 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e591af-e388-4863-8614-fd5e7c4c60b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:26.306028 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e591af-e388-4863-8614-fd5e7c4c60b9 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:26.306513 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e591af-e388-4863-8614-fd5e7c4c60b9 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:26.306787 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e591af-e388-4863-8614-fd5e7c4c60b9 None admin] PATH_INFO: `/v3/roles` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:26.307546 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e591af-e388-4863-8614-fd5e7c4c60b9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:37:26.307850 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e591af-e388-4863-8614-fd5e7c4c60b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:26.309253 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e591af-e388-4863-8614-fd5e7c4c60b9 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:26.318565 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 101/194] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:37:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:26.319546 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e591af-e388-4863-8614-fd5e7c4c60b9 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:26.333135 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:26.341169 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92b1d64-ce93-405c-9349-8ad6081fb313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:26.341916 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92b1d64-ce93-405c-9349-8ad6081fb313 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:26.342205 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92b1d64-ce93-405c-9349-8ad6081fb313 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:26.342479 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92b1d64-ce93-405c-9349-8ad6081fb313 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:26.427900 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:26.433061 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92b1d64-ce93-405c-9349-8ad6081fb313 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:26.435000 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92b1d64-ce93-405c-9349-8ad6081fb313 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:26.452440 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e591af-e388-4863-8614-fd5e7c4c60b9 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:26.497452 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de956b4-41df-4f7d-b7d0-5a009e2de4f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:26.498241 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de956b4-41df-4f7d-b7d0-5a009e2de4f8 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:26.498534 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de956b4-41df-4f7d-b7d0-5a009e2de4f8 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:26.498817 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de956b4-41df-4f7d-b7d0-5a009e2de4f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:26.500190 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de956b4-41df-4f7d-b7d0-5a009e2de4f8 None admin] RBAC: Adding query filter params (role.id=b894e00b17e34f0cbde0ab3d760e2433, scope.project.id=ff0814efdfab4314800f7e0e2109a949, user.id=d15c701d832345a08a0951afb6b6d914) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:37:26.500514 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de956b4-41df-4f7d-b7d0-5a009e2de4f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:26.502239 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de956b4-41df-4f7d-b7d0-5a009e2de4f8 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:26.509176 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 102/195] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:37:26 2022] GET /identity/v3/role_assignments?user.id=d15c701d832345a08a0951afb6b6d914&scope.project.id=ff0814efdfab4314800f7e0e2109a949&role.id=b894e00b17e34f0cbde0ab3d760e2433 => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:26.510087 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de956b4-41df-4f7d-b7d0-5a009e2de4f8 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:26.558038 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 94/196] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:37:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:37:26.559824 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92b1d64-ce93-405c-9349-8ad6081fb313 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:26.619234 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:26.694319 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de956b4-41df-4f7d-b7d0-5a009e2de4f8 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:26.728733 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:26.736838 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92b1d64-ce93-405c-9349-8ad6081fb313 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:26.762441 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199e71ea-1f3d-449b-8874-e57ad369be60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:26.763265 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199e71ea-1f3d-449b-8874-e57ad369be60 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:26.763551 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199e71ea-1f3d-449b-8874-e57ad369be60 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:26.763832 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199e71ea-1f3d-449b-8874-e57ad369be60 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:26.779513 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199e71ea-1f3d-449b-8874-e57ad369be60 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:37:26.787715 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-199e71ea-1f3d-449b-8874-e57ad369be60 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:37:26.787715 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:37:26.787715 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:37:26.787715 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:37:26.787715 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:37:26.787715 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:37:26.787715 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:37:26.787715 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:37:26.787715 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:37:26.787715 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:37:26.787715 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:37:26.787715 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:37:26.787715 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:37:26.787715 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:37:26.787715 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:37:26.787715 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:37:26.788797 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:37:26.788797 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:37:26.788797 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:37:26.788797 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:37:26.788797 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:37:26.788797 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:37:26.788797 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:37:26.788797 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:37:26.788797 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:37:26.788797 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:37:26.788797 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:37:26.788797 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:37:26.788797 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:37:26.788797 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:37:26.788797 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:37:26.788797 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:37:26.790301 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:37:26.790301 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:37:26.790301 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:37:26.790301 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:37:26.790301 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:37:26.790301 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:37:26.790301 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:37:26.790301 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:37:26.790301 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:37:26.790301 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:37:26.790301 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:37:26.790301 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:37:26.790301 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 103/197] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:37:26 2022] GET /identity/v3/roles/reader => generated 84 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:37:26.790301 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199e71ea-1f3d-449b-8874-e57ad369be60 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:26.798913 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271cb1cb-4f48-4dd8-a99c-987c5e974949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:26.799815 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271cb1cb-4f48-4dd8-a99c-987c5e974949 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:26.800097 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271cb1cb-4f48-4dd8-a99c-987c5e974949 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:26.800371 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271cb1cb-4f48-4dd8-a99c-987c5e974949 None admin] PATH_INFO: `/v3/users/90ad82926efe47569ea5db9e4f598327` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:26.802605 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271cb1cb-4f48-4dd8-a99c-987c5e974949 None admin] RBAC: Authorizing `identity:get_user(user_id=90ad82926efe47569ea5db9e4f598327)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:26.804147 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271cb1cb-4f48-4dd8-a99c-987c5e974949 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:26.806659 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 95/198] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:37:26 2022] GET /identity/v3/users/90ad82926efe47569ea5db9e4f598327 => generated 283 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:26.807244 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271cb1cb-4f48-4dd8-a99c-987c5e974949 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:26.926276 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199e71ea-1f3d-449b-8874-e57ad369be60 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:26.927127 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271cb1cb-4f48-4dd8-a99c-987c5e974949 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:26.963665 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:26.969910 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44116100-98fa-4ec1-b4fb-0737aa2312f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:26.970648 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44116100-98fa-4ec1-b4fb-0737aa2312f1 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:26.970935 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44116100-98fa-4ec1-b4fb-0737aa2312f1 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:26.971215 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44116100-98fa-4ec1-b4fb-0737aa2312f1 None admin] PATH_INFO: `/v3/roles` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:26.971984 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44116100-98fa-4ec1-b4fb-0737aa2312f1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:37:26.972319 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44116100-98fa-4ec1-b4fb-0737aa2312f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:26.973695 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44116100-98fa-4ec1-b4fb-0737aa2312f1 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:26.978541 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0625e43-4d26-4729-82a8-ba2e5a39196b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:26.979422 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0625e43-4d26-4729-82a8-ba2e5a39196b None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:26.979704 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0625e43-4d26-4729-82a8-ba2e5a39196b None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:26.979977 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0625e43-4d26-4729-82a8-ba2e5a39196b None admin] PATH_INFO: `/v3/projects/0c950d0347b741b68c581d47c8968d0d` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:26.981251 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0625e43-4d26-4729-82a8-ba2e5a39196b None admin] RBAC: Authorizing `identity:get_project(project_id=0c950d0347b741b68c581d47c8968d0d)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:26.982744 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0625e43-4d26-4729-82a8-ba2e5a39196b None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:26.984489 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 104/199] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:37:26 2022] GET /identity/v3/projects/0c950d0347b741b68c581d47c8968d0d => generated 302 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:26.985213 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 96/200] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:37:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:26.985340 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0625e43-4d26-4729-82a8-ba2e5a39196b None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:26.990959 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44116100-98fa-4ec1-b4fb-0737aa2312f1 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:27.031847 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:27.113263 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0625e43-4d26-4729-82a8-ba2e5a39196b None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:27.137689 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44116100-98fa-4ec1-b4fb-0737aa2312f1 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:27.159329 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd05a6c9-d6de-4c3a-b26f-5e24636c4d32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:27.160417 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd05a6c9-d6de-4c3a-b26f-5e24636c4d32 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:27.160938 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd05a6c9-d6de-4c3a-b26f-5e24636c4d32 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:27.161432 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd05a6c9-d6de-4c3a-b26f-5e24636c4d32 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:27.177479 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6b3787-ccc9-44ef-9cde-fdfd1029b743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:27.178189 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6b3787-ccc9-44ef-9cde-fdfd1029b743 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:27.178479 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6b3787-ccc9-44ef-9cde-fdfd1029b743 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:27.178762 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6b3787-ccc9-44ef-9cde-fdfd1029b743 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:27.179620 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6b3787-ccc9-44ef-9cde-fdfd1029b743 None admin] RBAC: Adding query filter params (role.id=c2935b1a25ca46fcba419fe3d1715330, scope.project.id=0c950d0347b741b68c581d47c8968d0d, user.id=90ad82926efe47569ea5db9e4f598327) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:37:27.180497 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6b3787-ccc9-44ef-9cde-fdfd1029b743 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:27.182000 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6b3787-ccc9-44ef-9cde-fdfd1029b743 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:27.188730 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 97/201] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:37:26 2022] GET /identity/v3/role_assignments?user.id=90ad82926efe47569ea5db9e4f598327&scope.project.id=0c950d0347b741b68c581d47c8968d0d&role.id=c2935b1a25ca46fcba419fe3d1715330 => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:27.189615 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6b3787-ccc9-44ef-9cde-fdfd1029b743 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:27.264929 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:27.269958 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd05a6c9-d6de-4c3a-b26f-5e24636c4d32 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:27.272720 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd05a6c9-d6de-4c3a-b26f-5e24636c4d32 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:27.295678 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:27.306689 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6b3787-ccc9-44ef-9cde-fdfd1029b743 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:27.364722 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffb6244-e98e-400b-9517-02db0889e4e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:27.365636 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffb6244-e98e-400b-9517-02db0889e4e4 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:27.365951 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffb6244-e98e-400b-9517-02db0889e4e4 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:27.366235 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffb6244-e98e-400b-9517-02db0889e4e4 None admin] PATH_INFO: `/v3/users/bc06d3d555b24e59a3957bf501a9d140` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:27.369878 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffb6244-e98e-400b-9517-02db0889e4e4 None admin] RBAC: Authorizing `identity:get_user(user_id=bc06d3d555b24e59a3957bf501a9d140)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:27.372578 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffb6244-e98e-400b-9517-02db0889e4e4 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:27.377569 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 98/202] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:37:27 2022] GET /identity/v3/users/bc06d3d555b24e59a3957bf501a9d140 => generated 289 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:27.382343 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffb6244-e98e-400b-9517-02db0889e4e4 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:27.440568 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 105/203] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:37:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:37:27.447498 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd05a6c9-d6de-4c3a-b26f-5e24636c4d32 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:27.596137 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffb6244-e98e-400b-9517-02db0889e4e4 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:27.651024 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:27.655226 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd05a6c9-d6de-4c3a-b26f-5e24636c4d32 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:27.697786 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b390338-f520-4c58-b5fa-2b3f4ec7548f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:27.698473 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b390338-f520-4c58-b5fa-2b3f4ec7548f None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:27.699151 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b390338-f520-4c58-b5fa-2b3f4ec7548f None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:27.699447 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b390338-f520-4c58-b5fa-2b3f4ec7548f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:27.701434 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a66c40-e785-4440-9a7c-ef7502fe0136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:27.702322 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a66c40-e785-4440-9a7c-ef7502fe0136 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:27.702606 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a66c40-e785-4440-9a7c-ef7502fe0136 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:27.702883 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a66c40-e785-4440-9a7c-ef7502fe0136 None admin] PATH_INFO: `/v3/projects/0c950d0347b741b68c581d47c8968d0d` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:27.705739 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a66c40-e785-4440-9a7c-ef7502fe0136 None admin] RBAC: Authorizing `identity:get_project(project_id=0c950d0347b741b68c581d47c8968d0d)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:27.707239 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a66c40-e785-4440-9a7c-ef7502fe0136 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:27.708989 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 106/204] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:37:27 2022] GET /identity/v3/projects/0c950d0347b741b68c581d47c8968d0d => generated 302 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:27.733330 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a66c40-e785-4440-9a7c-ef7502fe0136 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:27.768248 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:27.865177 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a66c40-e785-4440-9a7c-ef7502fe0136 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:27.865378 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b390338-f520-4c58-b5fa-2b3f4ec7548f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:27.866889 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b390338-f520-4c58-b5fa-2b3f4ec7548f None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:27.902464 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0ba9b6-5d64-43aa-8adc-3cbedf838160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:27.903231 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0ba9b6-5d64-43aa-8adc-3cbedf838160 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:27.903523 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0ba9b6-5d64-43aa-8adc-3cbedf838160 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:27.903806 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0ba9b6-5d64-43aa-8adc-3cbedf838160 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:27.904769 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0ba9b6-5d64-43aa-8adc-3cbedf838160 None admin] RBAC: Adding query filter params (role.id=b894e00b17e34f0cbde0ab3d760e2433, scope.project.id=0c950d0347b741b68c581d47c8968d0d, user.id=90ad82926efe47569ea5db9e4f598327) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:37:27.905087 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0ba9b6-5d64-43aa-8adc-3cbedf838160 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:27.906721 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0ba9b6-5d64-43aa-8adc-3cbedf838160 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:27.913055 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 107/205] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:37:27 2022] GET /identity/v3/role_assignments?user.id=90ad82926efe47569ea5db9e4f598327&scope.project.id=0c950d0347b741b68c581d47c8968d0d&role.id=b894e00b17e34f0cbde0ab3d760e2433 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:37:28.059045 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 99/206] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:37:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 677 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:37:28.064759 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0ba9b6-5d64-43aa-8adc-3cbedf838160 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:28.119897 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:28.216595 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0ba9b6-5d64-43aa-8adc-3cbedf838160 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:28.274571 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed0886e-9023-4247-ab89-650afcba1879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:28.275867 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed0886e-9023-4247-ab89-650afcba1879 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:28.276378 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed0886e-9023-4247-ab89-650afcba1879 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:28.276860 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed0886e-9023-4247-ab89-650afcba1879 None admin] PATH_INFO: `/v3/projects/ff0814efdfab4314800f7e0e2109a949` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:28.278594 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed0886e-9023-4247-ab89-650afcba1879 None admin] RBAC: Authorizing `identity:get_project(project_id=ff0814efdfab4314800f7e0e2109a949)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:28.280294 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed0886e-9023-4247-ab89-650afcba1879 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:28.285947 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 108/207] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:37:28 2022] GET /identity/v3/projects/ff0814efdfab4314800f7e0e2109a949 => generated 298 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:28.288027 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b390338-f520-4c58-b5fa-2b3f4ec7548f None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:28.398485 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:28.444919 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b390338-f520-4c58-b5fa-2b3f4ec7548f None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:28.482739 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230f8f1b-bf97-46c9-b62d-f640cb3fdc23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:28.483426 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230f8f1b-bf97-46c9-b62d-f640cb3fdc23 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:28.483765 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230f8f1b-bf97-46c9-b62d-f640cb3fdc23 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:28.484069 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230f8f1b-bf97-46c9-b62d-f640cb3fdc23 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:28.484925 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230f8f1b-bf97-46c9-b62d-f640cb3fdc23 None admin] RBAC: Adding query filter params (role.id=6129e931488746afb2145fa463c08395, scope.project.id=ff0814efdfab4314800f7e0e2109a949, user.id=bc06d3d555b24e59a3957bf501a9d140) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:37:28.485260 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230f8f1b-bf97-46c9-b62d-f640cb3fdc23 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:28.486764 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230f8f1b-bf97-46c9-b62d-f640cb3fdc23 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:28.494157 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 100/208] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:37:28 2022] GET /identity/v3/role_assignments?user.id=bc06d3d555b24e59a3957bf501a9d140&scope.project.id=ff0814efdfab4314800f7e0e2109a949&role.id=6129e931488746afb2145fa463c08395 => generated 608 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:28.927300 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5a4bf4-04c5-4735-a168-6251e95b66e3 None None] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:28.928349 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5a4bf4-04c5-4735-a168-6251e95b66e3 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:28.928847 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5a4bf4-04c5-4735-a168-6251e95b66e3 None None] PATH_INFO: `/` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:28.931358 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 109/209] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:37:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:37:28.936804 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956febbd-c26b-4a89-a711-6ae032bbba52 None None] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:28.937241 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956febbd-c26b-4a89-a711-6ae032bbba52 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:28.937419 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956febbd-c26b-4a89-a711-6ae032bbba52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:28.960585 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.auth.core [None req-956febbd-c26b-4a89-a711-6ae032bbba52 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:37:28.961890 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-956febbd-c26b-4a89-a711-6ae032bbba52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:37:28.975446 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb42a95-36f2-4664-9fc7-100f896b2ccc None None] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:28.976084 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb42a95-36f2-4664-9fc7-100f896b2ccc None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:28.976588 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb42a95-36f2-4664-9fc7-100f896b2ccc None None] PATH_INFO: `/` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:28.978550 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 110/210] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:37:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:37:28.983417 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2700a7d4-ded9-4980-abf6-df9fd13c8ad6 None None] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:28.983989 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2700a7d4-ded9-4980-abf6-df9fd13c8ad6 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:28.984478 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2700a7d4-ded9-4980-abf6-df9fd13c8ad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:29.007697 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.auth.core [None req-2700a7d4-ded9-4980-abf6-df9fd13c8ad6 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:37:29.013217 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.fernet_utils [None req-2700a7d4-ded9-4980-abf6-df9fd13c8ad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:37:29.042180 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 101/211] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:37:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:37:29.069954 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956febbd-c26b-4a89-a711-6ae032bbba52 None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:29.100235 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 111/212] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:37:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:37:29.123135 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2700a7d4-ded9-4980-abf6-df9fd13c8ad6 None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:29.139048 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:29.170315 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:29.193485 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956febbd-c26b-4a89-a711-6ae032bbba52 None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:29.242601 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71515db-691c-43b6-9e39-6c418d438332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:29.244493 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71515db-691c-43b6-9e39-6c418d438332 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:29.245122 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71515db-691c-43b6-9e39-6c418d438332 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:29.245607 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71515db-691c-43b6-9e39-6c418d438332 None admin] PATH_INFO: `/v3/domains/default` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:29.247904 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71515db-691c-43b6-9e39-6c418d438332 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:29.250277 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71515db-691c-43b6-9e39-6c418d438332 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:29.253421 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 102/213] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:37:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:37:29.261055 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71515db-691c-43b6-9e39-6c418d438332 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:29.261203 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2700a7d4-ded9-4980-abf6-df9fd13c8ad6 None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:29.299381 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6b7a41-daab-4e01-950f-3f549a52e9b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:29.301224 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6b7a41-daab-4e01-950f-3f549a52e9b6 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:29.301588 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6b7a41-daab-4e01-950f-3f549a52e9b6 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:29.301998 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6b7a41-daab-4e01-950f-3f549a52e9b6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:29.314739 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6b7a41-daab-4e01-950f-3f549a52e9b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:37:29.326250 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-cf6b7a41-daab-4e01-950f-3f549a52e9b6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:37:29.326250 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:37:29.326250 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:37:29.326250 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:37:29.326250 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:37:29.326250 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:37:29.326250 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:37:29.326250 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:37:29.326250 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:37:29.326250 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:37:29.326250 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:37:29.326250 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:37:29.326250 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:37:29.326250 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:37:29.326250 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:37:29.326250 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:37:29.327497 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:37:29.327497 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:37:29.327497 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:37:29.327497 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:37:29.327497 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:37:29.327497 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:37:29.327497 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:37:29.327497 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:37:29.327497 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:37:29.327497 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:37:29.327497 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:37:29.327497 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:37:29.327497 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:37:29.327497 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:37:29.327497 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:37:29.327497 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:37:29.328416 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:37:29.328416 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:37:29.328416 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:37:29.328416 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:37:29.328416 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:37:29.328416 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:37:29.328416 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:37:29.328416 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:37:29.328416 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:37:29.328416 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:37:29.328416 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:37:29.328416 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:37:29.332596 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 112/214] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:37:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:37:29.334003 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6b7a41-daab-4e01-950f-3f549a52e9b6 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:29.379838 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71515db-691c-43b6-9e39-6c418d438332 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:29.414255 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:29.426864 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb4e5d2-ed09-46fc-9791-7fdf154c91ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:29.427554 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb4e5d2-ed09-46fc-9791-7fdf154c91ef None admin] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:29.427845 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb4e5d2-ed09-46fc-9791-7fdf154c91ef None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:29.428125 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb4e5d2-ed09-46fc-9791-7fdf154c91ef None admin] PATH_INFO: `/v3/users` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:29.428858 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb4e5d2-ed09-46fc-9791-7fdf154c91ef None admin] RBAC: Authorizing `identity:create_user()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:29.431008 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb4e5d2-ed09-46fc-9791-7fdf154c91ef None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:29.433356 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:29.444652 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6b7a41-daab-4e01-950f-3f549a52e9b6 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:29.460275 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 103/215] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 19 14:37:29 2022] POST /identity/v3/users => generated 297 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 14:37:29.489255 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88becd6-0ff1-471d-afee-0da8376380ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:29.490030 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88becd6-0ff1-471d-afee-0da8376380ed None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:29.490383 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88becd6-0ff1-471d-afee-0da8376380ed None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:29.490646 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88becd6-0ff1-471d-afee-0da8376380ed None admin] PATH_INFO: `/v3/roles` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:29.491489 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88becd6-0ff1-471d-afee-0da8376380ed None admin] RBAC: Adding query filter params (name=member) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:37:29.491971 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88becd6-0ff1-471d-afee-0da8376380ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:29.493810 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88becd6-0ff1-471d-afee-0da8376380ed None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:29.504405 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 113/216] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:37:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:37:29.510270 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb4e5d2-ed09-46fc-9791-7fdf154c91ef None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:29.626761 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb4e5d2-ed09-46fc-9791-7fdf154c91ef None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:29.667382 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3511c99-4d05-470c-abb3-8ea95540466d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:29.668271 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3511c99-4d05-470c-abb3-8ea95540466d None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:29.668552 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3511c99-4d05-470c-abb3-8ea95540466d None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:29.668827 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3511c99-4d05-470c-abb3-8ea95540466d None admin] PATH_INFO: `/v3/users/bd1bb65a5946439a9f11bcbd467ddf68` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:29.684172 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3511c99-4d05-470c-abb3-8ea95540466d None admin] RBAC: Authorizing `identity:get_user(user_id=bd1bb65a5946439a9f11bcbd467ddf68)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:29.685792 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3511c99-4d05-470c-abb3-8ea95540466d None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:29.687896 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 104/217] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:37:29 2022] GET /identity/v3/users/bd1bb65a5946439a9f11bcbd467ddf68 => generated 297 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:29.693481 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88becd6-0ff1-471d-afee-0da8376380ed None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:29.811947 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88becd6-0ff1-471d-afee-0da8376380ed None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:29.848075 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea5c416-fe01-4518-9f31-f12261ac8bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:29.848807 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea5c416-fe01-4518-9f31-f12261ac8bdd None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:29.849090 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea5c416-fe01-4518-9f31-f12261ac8bdd None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:29.849361 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea5c416-fe01-4518-9f31-f12261ac8bdd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:29.873648 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:29.927574 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea5c416-fe01-4518-9f31-f12261ac8bdd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:29.929054 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea5c416-fe01-4518-9f31-f12261ac8bdd None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:29.957776 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fd7203-2198-4bc9-98e1-b0a42cc33419 None None] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:29.958344 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fd7203-2198-4bc9-98e1-b0a42cc33419 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:29.958344 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fd7203-2198-4bc9-98e1-b0a42cc33419 None None] PATH_INFO: `/` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:29.959737 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 105/218] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:37:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:37:29.969428 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc68f7db-f458-4bac-a92b-800c3774161e None None] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:29.970235 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc68f7db-f458-4bac-a92b-800c3774161e None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:29.971559 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc68f7db-f458-4bac-a92b-800c3774161e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:29.993524 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.auth.core [None req-fc68f7db-f458-4bac-a92b-800c3774161e None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:37:29.995616 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-fc68f7db-f458-4bac-a92b-800c3774161e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:37:30.034276 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:30.065310 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 114/219] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:37:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:37:30.071295 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea5c416-fe01-4518-9f31-f12261ac8bdd None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:30.089410 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 106/220] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:37:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:37:30.138821 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc68f7db-f458-4bac-a92b-800c3774161e None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:30.178406 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:30.281985 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea5c416-fe01-4518-9f31-f12261ac8bdd None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:30.297230 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc68f7db-f458-4bac-a92b-800c3774161e None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:30.342562 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a4e328-3976-4afc-8e12-1a890366353c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:30.343634 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a4e328-3976-4afc-8e12-1a890366353c None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:30.344562 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a4e328-3976-4afc-8e12-1a890366353c None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:30.345473 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a4e328-3976-4afc-8e12-1a890366353c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:30.349694 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf1213f-0381-44f7-9c75-772ce57426ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:30.350724 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf1213f-0381-44f7-9c75-772ce57426ac None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:30.351110 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf1213f-0381-44f7-9c75-772ce57426ac None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:30.351490 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf1213f-0381-44f7-9c75-772ce57426ac None admin] PATH_INFO: `/v3/projects/0c950d0347b741b68c581d47c8968d0d` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:30.352855 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf1213f-0381-44f7-9c75-772ce57426ac None admin] RBAC: Authorizing `identity:get_project(project_id=0c950d0347b741b68c581d47c8968d0d)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:30.354613 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf1213f-0381-44f7-9c75-772ce57426ac None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:30.356700 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 115/221] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:37:30 2022] GET /identity/v3/projects/0c950d0347b741b68c581d47c8968d0d => generated 302 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:37:30.360825 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a4e328-3976-4afc-8e12-1a890366353c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:37:30.369743 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-61a4e328-3976-4afc-8e12-1a890366353c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:37:30.369743 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:37:30.369743 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:37:30.369743 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:37:30.369743 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:37:30.369743 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:37:30.369743 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:37:30.369743 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:37:30.369743 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:37:30.369743 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:37:30.369743 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:37:30.369743 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:37:30.369743 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:37:30.369743 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:37:30.369743 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:37:30.369743 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:37:30.370829 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:37:30.370829 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:37:30.370829 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:37:30.370829 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:37:30.370829 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:37:30.370829 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:37:30.370829 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:37:30.370829 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:37:30.370829 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:37:30.370829 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:37:30.370829 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:37:30.370829 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:37:30.370829 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:37:30.370829 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:37:30.370829 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:37:30.370829 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:37:30.371797 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:37:30.371797 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:37:30.371797 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:37:30.371797 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:37:30.371797 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:37:30.371797 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:37:30.371797 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:37:30.371797 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:37:30.371797 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:37:30.371797 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:37:30.371797 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:37:30.371797 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:37:30.372524 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf1213f-0381-44f7-9c75-772ce57426ac None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:30.374876 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 107/222] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:37:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:37:30.382521 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a4e328-3976-4afc-8e12-1a890366353c None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:30.431119 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:30.563819 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf1213f-0381-44f7-9c75-772ce57426ac None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:30.576592 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a4e328-3976-4afc-8e12-1a890366353c None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:30.609163 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:30.625539 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9aab861-849a-4ff2-b8cf-c3afe3ad35ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:30.626264 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aab861-849a-4ff2-b8cf-c3afe3ad35ad None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:30.626526 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aab861-849a-4ff2-b8cf-c3afe3ad35ad None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:30.626804 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aab861-849a-4ff2-b8cf-c3afe3ad35ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:30.627645 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9aab861-849a-4ff2-b8cf-c3afe3ad35ad None admin] RBAC: Adding query filter params (role.id=7ebce5564f40456e94a3616961ed73ff, scope.project.id=0c950d0347b741b68c581d47c8968d0d, user.id=bd1bb65a5946439a9f11bcbd467ddf68) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:37:30.627957 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9aab861-849a-4ff2-b8cf-c3afe3ad35ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:30.628555 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7572bb1-c61b-41d3-9e40-8c230881c660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:30.629414 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9aab861-849a-4ff2-b8cf-c3afe3ad35ad None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:30.629686 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7572bb1-c61b-41d3-9e40-8c230881c660 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:30.630199 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7572bb1-c61b-41d3-9e40-8c230881c660 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:30.630688 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7572bb1-c61b-41d3-9e40-8c230881c660 None admin] PATH_INFO: `/v3/roles` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:30.631832 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7572bb1-c61b-41d3-9e40-8c230881c660 None admin] RBAC: Adding query filter params (name=admin) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:37:30.632531 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7572bb1-c61b-41d3-9e40-8c230881c660 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:30.634684 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7572bb1-c61b-41d3-9e40-8c230881c660 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:30.636324 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 116/223] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:37:30 2022] GET /identity/v3/role_assignments?user.id=bd1bb65a5946439a9f11bcbd467ddf68&scope.project.id=0c950d0347b741b68c581d47c8968d0d&role.id=7ebce5564f40456e94a3616961ed73ff => generated 259 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:30.637973 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da7f8ca-74ad-47eb-a1c6-d70da6308fc8 None None] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:30.638519 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da7f8ca-74ad-47eb-a1c6-d70da6308fc8 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:30.638989 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da7f8ca-74ad-47eb-a1c6-d70da6308fc8 None None] PATH_INFO: `/` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:30.645480 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 108/224] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:37:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:30.646175 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 117/225] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:37:30 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:37:30.647500 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b0b72d-754c-44d0-a228-59297da51e28 None None] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:30.647873 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b0b72d-754c-44d0-a228-59297da51e28 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:30.648286 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b0b72d-754c-44d0-a228-59297da51e28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:30.650843 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da7f8ca-74ad-47eb-a1c6-d70da6308fc8 None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:30.671159 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.auth.core [None req-07b0b72d-754c-44d0-a228-59297da51e28 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:37:30.671159 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-07b0b72d-754c-44d0-a228-59297da51e28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:37:30.746528 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 109/226] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:37:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:37:30.790831 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b0b72d-754c-44d0-a228-59297da51e28 None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:30.826860 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:30.850370 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da7f8ca-74ad-47eb-a1c6-d70da6308fc8 None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:30.918455 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d65b959-ece6-43dc-bf09-31b996d5f981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:30.919848 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d65b959-ece6-43dc-bf09-31b996d5f981 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:30.920397 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d65b959-ece6-43dc-bf09-31b996d5f981 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:30.920879 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d65b959-ece6-43dc-bf09-31b996d5f981 None admin] PATH_INFO: `/v3/users/90ad82926efe47569ea5db9e4f598327` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:30.922338 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d65b959-ece6-43dc-bf09-31b996d5f981 None admin] RBAC: Authorizing `identity:get_user(user_id=90ad82926efe47569ea5db9e4f598327)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:30.924056 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d65b959-ece6-43dc-bf09-31b996d5f981 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:30.926033 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 118/227] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:37:30 2022] GET /identity/v3/users/90ad82926efe47569ea5db9e4f598327 => generated 283 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:37:30.935050 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d65b959-ece6-43dc-bf09-31b996d5f981 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:30.954660 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b0b72d-754c-44d0-a228-59297da51e28 None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:30.998642 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403cb277-5f84-4d67-82bb-1688890ca4c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:30.999468 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403cb277-5f84-4d67-82bb-1688890ca4c2 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:30.999753 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403cb277-5f84-4d67-82bb-1688890ca4c2 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:31.000029 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403cb277-5f84-4d67-82bb-1688890ca4c2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:31.001460 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:31.030794 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403cb277-5f84-4d67-82bb-1688890ca4c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:37:31.047256 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-403cb277-5f84-4d67-82bb-1688890ca4c2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:37:31.047256 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:37:31.047256 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:37:31.047256 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:37:31.047256 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:37:31.047256 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:37:31.047256 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:37:31.047256 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:37:31.047256 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:37:31.047256 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:37:31.047256 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:37:31.047256 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:37:31.047256 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:37:31.047256 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:37:31.047256 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:37:31.047256 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:37:31.048297 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:37:31.048297 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:37:31.048297 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:37:31.048297 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:37:31.048297 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:37:31.048297 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:37:31.048297 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:37:31.048297 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:37:31.048297 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:37:31.048297 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:37:31.048297 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:37:31.048297 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:37:31.048297 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:37:31.048297 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:37:31.048297 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:37:31.048297 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:37:31.049199 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:37:31.049199 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:37:31.049199 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:37:31.049199 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:37:31.049199 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:37:31.049199 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:37:31.049199 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:37:31.049199 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:37:31.049199 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:37:31.049199 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:37:31.049199 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:37:31.049199 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:37:31.049199 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 110/228] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 19 14:37:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:37:31.068591 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d65b959-ece6-43dc-bf09-31b996d5f981 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:31.074966 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403cb277-5f84-4d67-82bb-1688890ca4c2 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:31.111387 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbb8b72-782a-46e3-88a3-15c01962265f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:31.112933 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbb8b72-782a-46e3-88a3-15c01962265f None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:31.113452 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbb8b72-782a-46e3-88a3-15c01962265f None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:31.113972 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbb8b72-782a-46e3-88a3-15c01962265f None admin] PATH_INFO: `/v3/projects/0c950d0347b741b68c581d47c8968d0d` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:31.115845 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbb8b72-782a-46e3-88a3-15c01962265f None admin] RBAC: Authorizing `identity:get_project(project_id=0c950d0347b741b68c581d47c8968d0d)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:31.118661 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbb8b72-782a-46e3-88a3-15c01962265f None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:31.121223 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 119/229] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:37:30 2022] GET /identity/v3/projects/0c950d0347b741b68c581d47c8968d0d => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:31.126453 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbb8b72-782a-46e3-88a3-15c01962265f None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:31.253068 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbb8b72-782a-46e3-88a3-15c01962265f None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:31.278959 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:31.295919 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:31.299149 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dae71cb-ecdf-40d3-892a-7f7001a10d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:31.300513 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dae71cb-ecdf-40d3-892a-7f7001a10d95 None admin] REQUEST_METHOD: `PUT` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:31.301756 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dae71cb-ecdf-40d3-892a-7f7001a10d95 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:31.302425 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dae71cb-ecdf-40d3-892a-7f7001a10d95 None admin] PATH_INFO: `/v3/projects/0c950d0347b741b68c581d47c8968d0d/users/90ad82926efe47569ea5db9e4f598327/roles/c2935b1a25ca46fcba419fe3d1715330` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:31.306790 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dae71cb-ecdf-40d3-892a-7f7001a10d95 None admin] RBAC: Authorizing `identity:create_grant(project_id=0c950d0347b741b68c581d47c8968d0d, user_id=90ad82926efe47569ea5db9e4f598327, role_id=c2935b1a25ca46fcba419fe3d1715330)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:31.308561 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dae71cb-ecdf-40d3-892a-7f7001a10d95 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:31.329456 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 120/230] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 14:37:31 2022] PUT /identity/v3/projects/0c950d0347b741b68c581d47c8968d0d/users/90ad82926efe47569ea5db9e4f598327/roles/c2935b1a25ca46fcba419fe3d1715330 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:37:31.342025 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403cb277-5f84-4d67-82bb-1688890ca4c2 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:31.383780 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b818d7-dcc4-40ad-9cfc-c3e115b290f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:31.385062 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b818d7-dcc4-40ad-9cfc-c3e115b290f1 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:31.385581 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b818d7-dcc4-40ad-9cfc-c3e115b290f1 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:31.386093 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b818d7-dcc4-40ad-9cfc-c3e115b290f1 None admin] PATH_INFO: `/v3/roles` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:31.387077 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b818d7-dcc4-40ad-9cfc-c3e115b290f1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:37:31.387591 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b818d7-dcc4-40ad-9cfc-c3e115b290f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:31.389196 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b818d7-dcc4-40ad-9cfc-c3e115b290f1 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:31.400163 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 111/231] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:37:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:31.406430 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dae71cb-ecdf-40d3-892a-7f7001a10d95 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:31.540258 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dae71cb-ecdf-40d3-892a-7f7001a10d95 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:31.594496 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6840d19e-6fab-422f-ba1b-2e50c338a880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:31.595752 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6840d19e-6fab-422f-ba1b-2e50c338a880 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:31.596240 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6840d19e-6fab-422f-ba1b-2e50c338a880 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:31.596713 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6840d19e-6fab-422f-ba1b-2e50c338a880 None admin] PATH_INFO: `/v3/users/90ad82926efe47569ea5db9e4f598327` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:31.603438 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6840d19e-6fab-422f-ba1b-2e50c338a880 None admin] RBAC: Authorizing `identity:get_user(user_id=90ad82926efe47569ea5db9e4f598327)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:31.605244 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6840d19e-6fab-422f-ba1b-2e50c338a880 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:31.607308 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 121/232] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:37:31 2022] GET /identity/v3/users/90ad82926efe47569ea5db9e4f598327 => generated 283 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:37:31.613005 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b818d7-dcc4-40ad-9cfc-c3e115b290f1 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:31.760289 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b818d7-dcc4-40ad-9cfc-c3e115b290f1 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:31.802757 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fd5b43-2b4f-4b1f-9431-3fc546b048c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:31.803979 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fd5b43-2b4f-4b1f-9431-3fc546b048c3 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:31.804470 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fd5b43-2b4f-4b1f-9431-3fc546b048c3 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:31.805574 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fd5b43-2b4f-4b1f-9431-3fc546b048c3 None admin] PATH_INFO: `/v3/projects/0c950d0347b741b68c581d47c8968d0d` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:31.807567 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fd5b43-2b4f-4b1f-9431-3fc546b048c3 None admin] RBAC: Authorizing `identity:get_project(project_id=0c950d0347b741b68c581d47c8968d0d)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:31.809372 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fd5b43-2b4f-4b1f-9431-3fc546b048c3 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:31.811348 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 112/233] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:37:31 2022] GET /identity/v3/projects/0c950d0347b741b68c581d47c8968d0d => generated 302 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:31.817292 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6840d19e-6fab-422f-ba1b-2e50c338a880 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:31.874688 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:31.952635 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6840d19e-6fab-422f-ba1b-2e50c338a880 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:32.010185 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0c4c31-92ef-4a94-a401-0006d3e0b1aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:32.011102 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0c4c31-92ef-4a94-a401-0006d3e0b1aa None admin] REQUEST_METHOD: `PUT` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:32.011389 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0c4c31-92ef-4a94-a401-0006d3e0b1aa None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:32.011668 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0c4c31-92ef-4a94-a401-0006d3e0b1aa None admin] PATH_INFO: `/v3/projects/0c950d0347b741b68c581d47c8968d0d/users/90ad82926efe47569ea5db9e4f598327/roles/b894e00b17e34f0cbde0ab3d760e2433` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:32.014011 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0c4c31-92ef-4a94-a401-0006d3e0b1aa None admin] RBAC: Authorizing `identity:create_grant(project_id=0c950d0347b741b68c581d47c8968d0d, user_id=90ad82926efe47569ea5db9e4f598327, role_id=b894e00b17e34f0cbde0ab3d760e2433)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:32.015498 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0c4c31-92ef-4a94-a401-0006d3e0b1aa None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:32.047757 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 122/234] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 14:37:31 2022] PUT /identity/v3/projects/0c950d0347b741b68c581d47c8968d0d/users/90ad82926efe47569ea5db9e4f598327/roles/b894e00b17e34f0cbde0ab3d760e2433 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:37:32.332012 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b392ae40-afeb-4281-8abd-85427e6c7e02 None None] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:32.332532 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b392ae40-afeb-4281-8abd-85427e6c7e02 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:32.332613 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b392ae40-afeb-4281-8abd-85427e6c7e02 None None] PATH_INFO: `/` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:32.334260 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 113/235] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:37:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:37:32.340559 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036ef4be-ac43-43ef-9a3a-1047f7008619 None None] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:32.340853 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036ef4be-ac43-43ef-9a3a-1047f7008619 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:32.341131 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036ef4be-ac43-43ef-9a3a-1047f7008619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:32.364426 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.auth.core [None req-036ef4be-ac43-43ef-9a3a-1047f7008619 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:37:32.365633 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.fernet_utils [None req-036ef4be-ac43-43ef-9a3a-1047f7008619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:37:32.444828 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 123/236] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:37:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:37:32.481022 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b392ae40-afeb-4281-8abd-85427e6c7e02 None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:32.494869 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:32.604167 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b392ae40-afeb-4281-8abd-85427e6c7e02 None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:32.644882 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e25a718-22ea-48a1-b537-8104b86032f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:32.645703 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e25a718-22ea-48a1-b537-8104b86032f5 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:32.646037 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e25a718-22ea-48a1-b537-8104b86032f5 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:32.646316 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e25a718-22ea-48a1-b537-8104b86032f5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:32.658881 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e25a718-22ea-48a1-b537-8104b86032f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:37:32.666579 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-0e25a718-22ea-48a1-b537-8104b86032f5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:37:32.666579 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:37:32.666579 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:37:32.666579 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:37:32.666579 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:37:32.666579 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:37:32.666579 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:37:32.666579 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:37:32.666579 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:37:32.666579 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:37:32.666579 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:37:32.666579 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:37:32.666579 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:37:32.666579 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:37:32.666579 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:37:32.666579 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:37:32.667608 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:37:32.667608 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:37:32.667608 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:37:32.667608 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:37:32.667608 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:37:32.667608 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:37:32.667608 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:37:32.667608 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:37:32.667608 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:37:32.667608 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:37:32.667608 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:37:32.667608 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:37:32.667608 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:37:32.667608 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:37:32.667608 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:37:32.667608 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:37:32.668508 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:37:32.668508 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:37:32.668508 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:37:32.668508 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:37:32.668508 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:37:32.668508 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:37:32.668508 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:37:32.668508 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:37:32.668508 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:37:32.668508 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:37:32.668508 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:37:32.668508 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:37:32.668508 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 114/237] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:37:32 2022] GET /identity/v3/roles/reader => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:37:32.675074 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036ef4be-ac43-43ef-9a3a-1047f7008619 None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:32.746968 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:32.838660 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e42eb9-3f03-4acb-873b-72baef110594 None None] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:32.838996 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e42eb9-3f03-4acb-873b-72baef110594 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:32.839229 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e42eb9-3f03-4acb-873b-72baef110594 None None] PATH_INFO: `/` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:32.840881 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 115/238] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:37:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:37:32.844118 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036ef4be-ac43-43ef-9a3a-1047f7008619 None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:32.846312 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ff08b6-dda8-48e2-a005-266bd8ee1352 None None] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:32.846607 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ff08b6-dda8-48e2-a005-266bd8ee1352 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:32.846893 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ff08b6-dda8-48e2-a005-266bd8ee1352 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:32.867454 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.auth.core [None req-d0ff08b6-dda8-48e2-a005-266bd8ee1352 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:37:32.869023 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-d0ff08b6-dda8-48e2-a005-266bd8ee1352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:37:32.919254 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3485b0cf-922f-470f-9802-703ce28f1828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:32.919940 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3485b0cf-922f-470f-9802-703ce28f1828 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:32.920228 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3485b0cf-922f-470f-9802-703ce28f1828 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:32.920511 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3485b0cf-922f-470f-9802-703ce28f1828 None admin] PATH_INFO: `/v3/roles` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:32.921271 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3485b0cf-922f-470f-9802-703ce28f1828 None admin] RBAC: Adding query filter params (name=reader) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:37:32.921578 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3485b0cf-922f-470f-9802-703ce28f1828 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:32.923028 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3485b0cf-922f-470f-9802-703ce28f1828 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:32.937302 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 124/239] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:37:32 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:32.940254 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:32.945001 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3485b0cf-922f-470f-9802-703ce28f1828 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:32.946751 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 116/240] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:37:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:37:32.975119 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ff08b6-dda8-48e2-a005-266bd8ee1352 None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:33.115314 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ff08b6-dda8-48e2-a005-266bd8ee1352 None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:33.130875 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3485b0cf-922f-470f-9802-703ce28f1828 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:33.134757 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:33.176237 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85374889-e67f-4817-8e62-2ffbd145e39b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:33.177062 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85374889-e67f-4817-8e62-2ffbd145e39b None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:33.177344 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85374889-e67f-4817-8e62-2ffbd145e39b None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:33.177615 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85374889-e67f-4817-8e62-2ffbd145e39b None admin] PATH_INFO: `/v3/domains/default` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:33.181294 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489273ca-b537-4fef-afe8-6a15c8d33ab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:33.181679 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85374889-e67f-4817-8e62-2ffbd145e39b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:33.182893 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489273ca-b537-4fef-afe8-6a15c8d33ab5 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:33.183241 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85374889-e67f-4817-8e62-2ffbd145e39b None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:33.183659 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489273ca-b537-4fef-afe8-6a15c8d33ab5 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:33.184179 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489273ca-b537-4fef-afe8-6a15c8d33ab5 None admin] PATH_INFO: `/v3/users/331dd46a8ec64f4598751c2eeacba71e` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:33.185219 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 117/241] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:37:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:33.194078 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85374889-e67f-4817-8e62-2ffbd145e39b None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:33.209221 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489273ca-b537-4fef-afe8-6a15c8d33ab5 None admin] RBAC: Authorizing `identity:get_user(user_id=331dd46a8ec64f4598751c2eeacba71e)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:33.212160 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489273ca-b537-4fef-afe8-6a15c8d33ab5 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:33.214285 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 125/242] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:37:32 2022] GET /identity/v3/users/331dd46a8ec64f4598751c2eeacba71e => generated 297 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:37:33.221905 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489273ca-b537-4fef-afe8-6a15c8d33ab5 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:33.324530 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:33.363048 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85374889-e67f-4817-8e62-2ffbd145e39b None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:33.363990 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489273ca-b537-4fef-afe8-6a15c8d33ab5 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:33.417676 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1b9544-6a48-4230-9f5f-34d291e8f7d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:33.418777 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1b9544-6a48-4230-9f5f-34d291e8f7d4 None admin] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:33.419265 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1b9544-6a48-4230-9f5f-34d291e8f7d4 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:33.419838 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1b9544-6a48-4230-9f5f-34d291e8f7d4 None admin] PATH_INFO: `/v3/users` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:33.420178 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb7702b-ba38-4caf-bbf3-24d9174be042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:33.420899 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb7702b-ba38-4caf-bbf3-24d9174be042 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:33.421187 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1b9544-6a48-4230-9f5f-34d291e8f7d4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:33.421317 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb7702b-ba38-4caf-bbf3-24d9174be042 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:33.421630 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb7702b-ba38-4caf-bbf3-24d9174be042 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:33.423224 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1b9544-6a48-4230-9f5f-34d291e8f7d4 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:33.452108 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:33.452678 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 118/243] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 19 14:37:33 2022] POST /identity/v3/users => generated 293 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 14:37:33.454549 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341e7958-f5f3-4dd4-9e86-794b6d81313f None None] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:33.455008 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341e7958-f5f3-4dd4-9e86-794b6d81313f None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:33.455521 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341e7958-f5f3-4dd4-9e86-794b6d81313f None None] PATH_INFO: `/` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:33.457107 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 119/244] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:37:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:37:33.469877 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9204cc3d-1a0c-49aa-96bc-4467fa8954ed None None] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:33.470460 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9204cc3d-1a0c-49aa-96bc-4467fa8954ed None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:33.470895 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9204cc3d-1a0c-49aa-96bc-4467fa8954ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:33.500970 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb7702b-ba38-4caf-bbf3-24d9174be042 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:33.502672 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb7702b-ba38-4caf-bbf3-24d9174be042 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:33.506570 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.auth.core [None req-9204cc3d-1a0c-49aa-96bc-4467fa8954ed None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:37:33.508602 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-9204cc3d-1a0c-49aa-96bc-4467fa8954ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:37:33.622444 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 126/245] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:37:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:37:33.627678 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb7702b-ba38-4caf-bbf3-24d9174be042 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:33.650233 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 120/246] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:37:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) May 19 14:37:33.684587 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9204cc3d-1a0c-49aa-96bc-4467fa8954ed None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:33.751929 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:33.761231 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:33.828525 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9204cc3d-1a0c-49aa-96bc-4467fa8954ed None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:33.832675 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb7702b-ba38-4caf-bbf3-24d9174be042 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:33.875485 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b05e3d-afbf-4328-8dfa-90a6bd255db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:33.876873 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b05e3d-afbf-4328-8dfa-90a6bd255db9 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:33.877168 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b05e3d-afbf-4328-8dfa-90a6bd255db9 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:33.877446 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b05e3d-afbf-4328-8dfa-90a6bd255db9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:33.888121 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef07be23-8942-4a9f-bbf4-d3e291ae1808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:33.889705 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef07be23-8942-4a9f-bbf4-d3e291ae1808 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:33.890218 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef07be23-8942-4a9f-bbf4-d3e291ae1808 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:33.890829 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef07be23-8942-4a9f-bbf4-d3e291ae1808 None admin] PATH_INFO: `/v3/projects/0c950d0347b741b68c581d47c8968d0d` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:33.892298 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b05e3d-afbf-4328-8dfa-90a6bd255db9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:37:33.892593 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef07be23-8942-4a9f-bbf4-d3e291ae1808 None admin] RBAC: Authorizing `identity:get_project(project_id=0c950d0347b741b68c581d47c8968d0d)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:33.894322 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef07be23-8942-4a9f-bbf4-d3e291ae1808 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:33.896362 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 127/247] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:37:33 2022] GET /identity/v3/projects/0c950d0347b741b68c581d47c8968d0d => generated 302 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:33.904380 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-61b05e3d-afbf-4328-8dfa-90a6bd255db9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:37:33.904380 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:37:33.904380 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:37:33.904380 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:37:33.904380 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:37:33.904380 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:37:33.904380 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:37:33.904380 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:37:33.904380 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:37:33.904380 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:37:33.904380 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:37:33.904380 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:37:33.904380 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:37:33.904380 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:37:33.904380 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:37:33.904380 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:37:33.905409 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:37:33.905409 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:37:33.905409 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:37:33.905409 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:37:33.905409 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:37:33.905409 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:37:33.905409 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:37:33.905409 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:37:33.905409 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:37:33.905409 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:37:33.905409 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:37:33.905409 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:37:33.905409 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:37:33.905409 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:37:33.905409 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:37:33.905409 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:37:33.906338 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:37:33.906338 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:37:33.906338 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:37:33.906338 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:37:33.906338 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:37:33.906338 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:37:33.906338 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:37:33.906338 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:37:33.906338 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:37:33.906338 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:37:33.906338 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:37:33.906338 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:37:33.907774 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 121/248] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:37:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:37:33.908887 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef07be23-8942-4a9f-bbf4-d3e291ae1808 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:33.917030 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b05e3d-afbf-4328-8dfa-90a6bd255db9 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:34.024734 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef07be23-8942-4a9f-bbf4-d3e291ae1808 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:34.033386 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b05e3d-afbf-4328-8dfa-90a6bd255db9 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:34.048960 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:34.054182 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:34.065373 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9849237c-f39d-4f5c-a713-a22b0d978f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:34.066562 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9849237c-f39d-4f5c-a713-a22b0d978f47 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:34.067174 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9849237c-f39d-4f5c-a713-a22b0d978f47 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:34.067667 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9849237c-f39d-4f5c-a713-a22b0d978f47 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:34.068714 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9849237c-f39d-4f5c-a713-a22b0d978f47 None admin] RBAC: Adding query filter params (role.id=6129e931488746afb2145fa463c08395, scope.project.id=0c950d0347b741b68c581d47c8968d0d, user.id=331dd46a8ec64f4598751c2eeacba71e) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:37:34.069350 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9849237c-f39d-4f5c-a713-a22b0d978f47 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:34.071266 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9849237c-f39d-4f5c-a713-a22b0d978f47 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:34.075353 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1b6a6a-8893-44a1-8017-be2f09c57e42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:34.076035 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1b6a6a-8893-44a1-8017-be2f09c57e42 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:34.076325 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1b6a6a-8893-44a1-8017-be2f09c57e42 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:34.076607 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1b6a6a-8893-44a1-8017-be2f09c57e42 None admin] PATH_INFO: `/v3/roles` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:34.077370 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1b6a6a-8893-44a1-8017-be2f09c57e42 None admin] RBAC: Adding query filter params (name=member) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:37:34.077685 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1b6a6a-8893-44a1-8017-be2f09c57e42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:34.079117 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1b6a6a-8893-44a1-8017-be2f09c57e42 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:34.079580 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 128/249] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:37:33 2022] GET /identity/v3/role_assignments?user.id=331dd46a8ec64f4598751c2eeacba71e&scope.project.id=0c950d0347b741b68c581d47c8968d0d&role.id=6129e931488746afb2145fa463c08395 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:34.091835 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 122/250] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:37:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:34.097700 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9849237c-f39d-4f5c-a713-a22b0d978f47 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:34.231493 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9849237c-f39d-4f5c-a713-a22b0d978f47 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:34.273471 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198a480d-416f-42c4-a744-e7aab2d690bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:34.275109 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198a480d-416f-42c4-a744-e7aab2d690bc None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:34.275622 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198a480d-416f-42c4-a744-e7aab2d690bc None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:34.276126 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198a480d-416f-42c4-a744-e7aab2d690bc None admin] PATH_INFO: `/v3/users/bd1bb65a5946439a9f11bcbd467ddf68` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:34.277662 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198a480d-416f-42c4-a744-e7aab2d690bc None admin] RBAC: Authorizing `identity:get_user(user_id=bd1bb65a5946439a9f11bcbd467ddf68)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:34.279443 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198a480d-416f-42c4-a744-e7aab2d690bc None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:34.281737 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 129/251] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:37:34 2022] GET /identity/v3/users/bd1bb65a5946439a9f11bcbd467ddf68 => generated 297 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:34.288591 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1b6a6a-8893-44a1-8017-be2f09c57e42 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:34.361993 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac0372a-ec0c-4484-a84c-19addc1f9371 None None] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:34.362768 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac0372a-ec0c-4484-a84c-19addc1f9371 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:34.363268 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac0372a-ec0c-4484-a84c-19addc1f9371 None None] PATH_INFO: `/` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:34.364924 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 130/252] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:37:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:37:34.371658 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0fbe8d-3498-4f33-bda2-457950c41ba7 None None] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:34.371966 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0fbe8d-3498-4f33-bda2-457950c41ba7 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:34.372351 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0fbe8d-3498-4f33-bda2-457950c41ba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:34.407782 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.auth.core [None req-ef0fbe8d-3498-4f33-bda2-457950c41ba7 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:37:34.410108 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.fernet_utils [None req-ef0fbe8d-3498-4f33-bda2-457950c41ba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:37:34.419164 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1b6a6a-8893-44a1-8017-be2f09c57e42 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:34.471668 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be428c9-343b-44c6-ac10-3f5ec203c032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:34.472639 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be428c9-343b-44c6-ac10-3f5ec203c032 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:34.472930 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be428c9-343b-44c6-ac10-3f5ec203c032 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:34.473207 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be428c9-343b-44c6-ac10-3f5ec203c032 None admin] PATH_INFO: `/v3/projects/0c950d0347b741b68c581d47c8968d0d` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:34.476197 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be428c9-343b-44c6-ac10-3f5ec203c032 None admin] RBAC: Authorizing `identity:get_project(project_id=0c950d0347b741b68c581d47c8968d0d)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:34.477761 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be428c9-343b-44c6-ac10-3f5ec203c032 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:34.480135 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 123/253] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:37:34 2022] GET /identity/v3/projects/0c950d0347b741b68c581d47c8968d0d => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:34.490712 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be428c9-343b-44c6-ac10-3f5ec203c032 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:34.496148 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:34.522412 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 131/254] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:37:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:37:34.553657 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:34.559117 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0fbe8d-3498-4f33-bda2-457950c41ba7 None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:34.704074 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be428c9-343b-44c6-ac10-3f5ec203c032 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:34.766978 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0fbe8d-3498-4f33-bda2-457950c41ba7 None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:34.793229 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a3b4a8-5b7f-4194-b83e-ec24a2753bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:34.794228 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a3b4a8-5b7f-4194-b83e-ec24a2753bb9 None admin] REQUEST_METHOD: `PUT` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:34.794513 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a3b4a8-5b7f-4194-b83e-ec24a2753bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:34.794790 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a3b4a8-5b7f-4194-b83e-ec24a2753bb9 None admin] PATH_INFO: `/v3/projects/0c950d0347b741b68c581d47c8968d0d/users/bd1bb65a5946439a9f11bcbd467ddf68/roles/7ebce5564f40456e94a3616961ed73ff` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:34.796807 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a3b4a8-5b7f-4194-b83e-ec24a2753bb9 None admin] RBAC: Authorizing `identity:create_grant(project_id=0c950d0347b741b68c581d47c8968d0d, user_id=bd1bb65a5946439a9f11bcbd467ddf68, role_id=7ebce5564f40456e94a3616961ed73ff)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:34.798320 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a3b4a8-5b7f-4194-b83e-ec24a2753bb9 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:34.830150 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35b18fb-935f-41e9-8ff2-75b67fcfb925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:34.831315 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35b18fb-935f-41e9-8ff2-75b67fcfb925 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:34.831855 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35b18fb-935f-41e9-8ff2-75b67fcfb925 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:34.832360 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35b18fb-935f-41e9-8ff2-75b67fcfb925 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:34.845871 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 124/255] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 14:37:34 2022] PUT /identity/v3/projects/0c950d0347b741b68c581d47c8968d0d/users/bd1bb65a5946439a9f11bcbd467ddf68/roles/7ebce5564f40456e94a3616961ed73ff => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:37:34.851515 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35b18fb-935f-41e9-8ff2-75b67fcfb925 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:37:34.870916 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-a35b18fb-935f-41e9-8ff2-75b67fcfb925 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:37:34.870916 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:37:34.870916 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:37:34.870916 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:37:34.870916 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:37:34.870916 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:37:34.870916 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:37:34.870916 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:37:34.870916 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:37:34.870916 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:37:34.870916 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:37:34.870916 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:37:34.870916 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:37:34.870916 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:37:34.870916 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:37:34.870916 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:37:34.872033 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:37:34.872033 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:37:34.872033 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:37:34.872033 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:37:34.872033 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:37:34.872033 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:37:34.872033 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:37:34.872033 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:37:34.872033 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:37:34.872033 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:37:34.872033 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:37:34.872033 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:37:34.872033 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:37:34.872033 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:37:34.872033 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:37:34.872033 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:37:34.872967 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:37:34.872967 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:37:34.872967 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:37:34.872967 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:37:34.872967 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:37:34.872967 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:37:34.872967 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:37:34.872967 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:37:34.872967 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:37:34.872967 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:37:34.872967 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:37:34.872967 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:37:34.875922 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 132/256] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:37:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:37:34.883652 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a3b4a8-5b7f-4194-b83e-ec24a2753bb9 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:35.015785 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a3b4a8-5b7f-4194-b83e-ec24a2753bb9 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:35.019091 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:35.063962 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4850505e-1e97-4bb1-b36f-33b8295d2a50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:35.064643 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4850505e-1e97-4bb1-b36f-33b8295d2a50 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:35.064925 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4850505e-1e97-4bb1-b36f-33b8295d2a50 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:35.065201 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4850505e-1e97-4bb1-b36f-33b8295d2a50 None admin] PATH_INFO: `/v3/roles` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:35.070133 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4850505e-1e97-4bb1-b36f-33b8295d2a50 None admin] RBAC: Adding query filter params (name=admin) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:37:35.071162 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4850505e-1e97-4bb1-b36f-33b8295d2a50 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:35.073155 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4850505e-1e97-4bb1-b36f-33b8295d2a50 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:35.083164 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 125/257] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:37:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:35.092698 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35b18fb-935f-41e9-8ff2-75b67fcfb925 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:35.170571 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7c9b94-62e4-4a44-8593-1ee614069d01 None None] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:35.170852 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7c9b94-62e4-4a44-8593-1ee614069d01 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:35.171159 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7c9b94-62e4-4a44-8593-1ee614069d01 None None] PATH_INFO: `/` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:35.172589 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 126/258] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:37:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:37:35.176444 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:35.178252 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55a177e-d52e-4342-b08b-84e837c13652 None None] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:35.178541 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55a177e-d52e-4342-b08b-84e837c13652 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:35.178822 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55a177e-d52e-4342-b08b-84e837c13652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:35.230483 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.auth.core [None req-a55a177e-d52e-4342-b08b-84e837c13652 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:37:35.231720 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-a55a177e-d52e-4342-b08b-84e837c13652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:37:35.241281 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35b18fb-935f-41e9-8ff2-75b67fcfb925 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:35.285868 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20199e70-28f7-421a-9c1a-cd8e696dc67d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:35.286763 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20199e70-28f7-421a-9c1a-cd8e696dc67d None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:35.287479 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20199e70-28f7-421a-9c1a-cd8e696dc67d None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:35.287772 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20199e70-28f7-421a-9c1a-cd8e696dc67d None admin] PATH_INFO: `/v3/users/90ad82926efe47569ea5db9e4f598327` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:35.289037 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20199e70-28f7-421a-9c1a-cd8e696dc67d None admin] RBAC: Authorizing `identity:get_user(user_id=90ad82926efe47569ea5db9e4f598327)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:35.290974 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20199e70-28f7-421a-9c1a-cd8e696dc67d None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:35.293547 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 133/259] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:37:35 2022] GET /identity/v3/users/90ad82926efe47569ea5db9e4f598327 => generated 283 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:35.299347 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20199e70-28f7-421a-9c1a-cd8e696dc67d None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:35.323724 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 127/260] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:37:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) May 19 14:37:35.350354 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55a177e-d52e-4342-b08b-84e837c13652 None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:35.442160 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20199e70-28f7-421a-9c1a-cd8e696dc67d None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:35.478262 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:35.514648 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432495c4-7559-4af1-822b-24dceebdd525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:35.515310 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432495c4-7559-4af1-822b-24dceebdd525 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:35.515536 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:35.515974 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432495c4-7559-4af1-822b-24dceebdd525 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:35.515974 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432495c4-7559-4af1-822b-24dceebdd525 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:35.539070 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55a177e-d52e-4342-b08b-84e837c13652 None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:35.608550 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432495c4-7559-4af1-822b-24dceebdd525 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:35.610049 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432495c4-7559-4af1-822b-24dceebdd525 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:35.647749 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1049dcd-6448-42f4-b8a6-e6a578a1889b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:35.649182 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1049dcd-6448-42f4-b8a6-e6a578a1889b None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:35.649778 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1049dcd-6448-42f4-b8a6-e6a578a1889b None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:35.649778 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1049dcd-6448-42f4-b8a6-e6a578a1889b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:35.670085 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1049dcd-6448-42f4-b8a6-e6a578a1889b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:37:35.686513 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-c1049dcd-6448-42f4-b8a6-e6a578a1889b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:37:35.686513 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:37:35.686513 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:37:35.686513 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:37:35.686513 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:37:35.686513 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:37:35.686513 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:37:35.686513 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:37:35.686513 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:37:35.686513 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:37:35.686513 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:37:35.686513 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:37:35.686513 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:37:35.686513 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:37:35.686513 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:37:35.686513 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:37:35.687571 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:37:35.687571 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:37:35.687571 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:37:35.687571 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:37:35.687571 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:37:35.687571 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:37:35.687571 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:37:35.687571 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:37:35.687571 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:37:35.687571 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:37:35.687571 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:37:35.687571 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:37:35.687571 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:37:35.687571 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:37:35.687571 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:37:35.687571 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:37:35.688555 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:37:35.688555 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:37:35.688555 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:37:35.688555 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:37:35.688555 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:37:35.688555 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:37:35.688555 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:37:35.688555 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:37:35.688555 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:37:35.688555 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:37:35.688555 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:37:35.688555 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:37:35.688555 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 128/261] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 19 14:37:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:37:35.698963 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1049dcd-6448-42f4-b8a6-e6a578a1889b None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:35.723208 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 134/262] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:37:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:37:35.731011 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432495c4-7559-4af1-822b-24dceebdd525 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:35.767445 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:35.884106 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1049dcd-6448-42f4-b8a6-e6a578a1889b None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:35.929678 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45621c13-e219-4216-9531-12d2f1bd6b34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:35.931002 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45621c13-e219-4216-9531-12d2f1bd6b34 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:35.931233 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45621c13-e219-4216-9531-12d2f1bd6b34 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:35.931503 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45621c13-e219-4216-9531-12d2f1bd6b34 None admin] PATH_INFO: `/v3/roles` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:35.932267 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45621c13-e219-4216-9531-12d2f1bd6b34 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:37:35.932581 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45621c13-e219-4216-9531-12d2f1bd6b34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:35.934009 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45621c13-e219-4216-9531-12d2f1bd6b34 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:35.949624 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 129/263] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:37:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:35.955067 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45621c13-e219-4216-9531-12d2f1bd6b34 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:35.964167 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432495c4-7559-4af1-822b-24dceebdd525 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:35.981576 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:36.062586 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93384b8-7a89-4e3c-8e54-22b27106a13c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:36.063877 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93384b8-7a89-4e3c-8e54-22b27106a13c None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:36.064370 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93384b8-7a89-4e3c-8e54-22b27106a13c None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:36.064858 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93384b8-7a89-4e3c-8e54-22b27106a13c None admin] PATH_INFO: `/v3/projects/0c950d0347b741b68c581d47c8968d0d` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:36.069509 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93384b8-7a89-4e3c-8e54-22b27106a13c None admin] RBAC: Authorizing `identity:get_project(project_id=0c950d0347b741b68c581d47c8968d0d)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:36.072562 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93384b8-7a89-4e3c-8e54-22b27106a13c None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:36.074884 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 135/264] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:37:35 2022] GET /identity/v3/projects/0c950d0347b741b68c581d47c8968d0d => generated 302 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:36.082911 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93384b8-7a89-4e3c-8e54-22b27106a13c None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:36.115653 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45621c13-e219-4216-9531-12d2f1bd6b34 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:36.152920 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeabb88a-31ff-4272-aaa8-c8d71ea666f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:36.153732 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeabb88a-31ff-4272-aaa8-c8d71ea666f9 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:36.154052 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeabb88a-31ff-4272-aaa8-c8d71ea666f9 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:36.154330 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeabb88a-31ff-4272-aaa8-c8d71ea666f9 None admin] PATH_INFO: `/v3/users/90ad82926efe47569ea5db9e4f598327` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:36.155565 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeabb88a-31ff-4272-aaa8-c8d71ea666f9 None admin] RBAC: Authorizing `identity:get_user(user_id=90ad82926efe47569ea5db9e4f598327)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:36.157074 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeabb88a-31ff-4272-aaa8-c8d71ea666f9 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:36.159122 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 130/265] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:37:35 2022] GET /identity/v3/users/90ad82926efe47569ea5db9e4f598327 => generated 283 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:37:36.170086 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeabb88a-31ff-4272-aaa8-c8d71ea666f9 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:36.335921 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:36.357397 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:36.372577 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93384b8-7a89-4e3c-8e54-22b27106a13c None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:36.386611 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeabb88a-31ff-4272-aaa8-c8d71ea666f9 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:36.453907 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f020da-0801-44c2-b782-26f97f8f9386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:36.454633 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f020da-0801-44c2-b782-26f97f8f9386 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:36.454995 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f020da-0801-44c2-b782-26f97f8f9386 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:36.455375 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f020da-0801-44c2-b782-26f97f8f9386 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:36.470573 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12e5ca8-2fcc-44a3-b06a-6598ada4a158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:36.471566 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12e5ca8-2fcc-44a3-b06a-6598ada4a158 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:36.472041 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12e5ca8-2fcc-44a3-b06a-6598ada4a158 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:36.472509 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12e5ca8-2fcc-44a3-b06a-6598ada4a158 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:36.473638 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12e5ca8-2fcc-44a3-b06a-6598ada4a158 None admin] RBAC: Adding query filter params (role.id=c2935b1a25ca46fcba419fe3d1715330, scope.project.id=0c950d0347b741b68c581d47c8968d0d, user.id=90ad82926efe47569ea5db9e4f598327) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:37:36.474174 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12e5ca8-2fcc-44a3-b06a-6598ada4a158 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:36.475825 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12e5ca8-2fcc-44a3-b06a-6598ada4a158 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:36.492124 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 136/266] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:37:36 2022] GET /identity/v3/role_assignments?user.id=90ad82926efe47569ea5db9e4f598327&scope.project.id=0c950d0347b741b68c581d47c8968d0d&role.id=c2935b1a25ca46fcba419fe3d1715330 => generated 608 bytes in 410 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:36.544778 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f020da-0801-44c2-b782-26f97f8f9386 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:36.546420 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f020da-0801-44c2-b782-26f97f8f9386 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:36.639072 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:36.662252 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 131/267] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:37:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 492 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 14:37:36.663863 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12e5ca8-2fcc-44a3-b06a-6598ada4a158 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:36.800552 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12e5ca8-2fcc-44a3-b06a-6598ada4a158 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:36.848337 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02170ae4-2168-4973-9013-45cc608c2e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:36.849277 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02170ae4-2168-4973-9013-45cc608c2e36 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:36.849569 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02170ae4-2168-4973-9013-45cc608c2e36 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:36.849866 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02170ae4-2168-4973-9013-45cc608c2e36 None admin] PATH_INFO: `/v3/projects/0c950d0347b741b68c581d47c8968d0d` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:36.851116 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02170ae4-2168-4973-9013-45cc608c2e36 None admin] RBAC: Authorizing `identity:get_project(project_id=0c950d0347b741b68c581d47c8968d0d)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:36.852788 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02170ae4-2168-4973-9013-45cc608c2e36 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:36.854668 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 137/268] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:37:36 2022] GET /identity/v3/projects/0c950d0347b741b68c581d47c8968d0d => generated 302 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:36.860693 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f020da-0801-44c2-b782-26f97f8f9386 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:36.937218 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a30d8b-e041-4b0e-ba47-7d93a9aff116 None None] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:36.937539 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a30d8b-e041-4b0e-ba47-7d93a9aff116 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:36.937781 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a30d8b-e041-4b0e-ba47-7d93a9aff116 None None] PATH_INFO: `/` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:36.939445 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 138/269] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:37:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:37:36.946865 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a57bc67-6166-491d-b2c9-2372bfc067d0 None None] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:36.947179 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a57bc67-6166-491d-b2c9-2372bfc067d0 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:36.947431 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a57bc67-6166-491d-b2c9-2372bfc067d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:36.968953 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.auth.core [None req-6a57bc67-6166-491d-b2c9-2372bfc067d0 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:37:36.970880 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.fernet_utils [None req-6a57bc67-6166-491d-b2c9-2372bfc067d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:37:36.978235 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f020da-0801-44c2-b782-26f97f8f9386 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:36.984824 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:37.017606 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c72971c-8ec3-4720-a0f7-851b08512138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:37.019842 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c72971c-8ec3-4720-a0f7-851b08512138 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:37.019842 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c72971c-8ec3-4720-a0f7-851b08512138 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:37.019842 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c72971c-8ec3-4720-a0f7-851b08512138 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:37.020097 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c72971c-8ec3-4720-a0f7-851b08512138 None admin] RBAC: Adding query filter params (role.id=b894e00b17e34f0cbde0ab3d760e2433, scope.project.id=0c950d0347b741b68c581d47c8968d0d, user.id=90ad82926efe47569ea5db9e4f598327) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:37:37.020304 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c72971c-8ec3-4720-a0f7-851b08512138 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:37.025888 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c72971c-8ec3-4720-a0f7-851b08512138 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:37.028996 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 132/270] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:37:36 2022] GET /identity/v3/role_assignments?user.id=90ad82926efe47569ea5db9e4f598327&scope.project.id=0c950d0347b741b68c581d47c8968d0d&role.id=b894e00b17e34f0cbde0ab3d760e2433 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:37.104995 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 139/271] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:37:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:37:37.141339 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c72971c-8ec3-4720-a0f7-851b08512138 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:37.208371 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:37.270869 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c72971c-8ec3-4720-a0f7-851b08512138 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:37.308396 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0043a8-824c-462d-acb7-94e344b221fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:37.309281 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0043a8-824c-462d-acb7-94e344b221fa None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:37.309572 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0043a8-824c-462d-acb7-94e344b221fa None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:37.309875 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0043a8-824c-462d-acb7-94e344b221fa None admin] PATH_INFO: `/v3/roles/reader` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:37.322216 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0043a8-824c-462d-acb7-94e344b221fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:37:37.330348 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-fc0043a8-824c-462d-acb7-94e344b221fa None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:37:37.330348 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:37:37.330348 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:37:37.330348 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:37:37.330348 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:37:37.330348 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:37:37.330348 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:37:37.330348 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:37:37.330348 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:37:37.330348 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:37:37.330348 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:37:37.330348 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:37:37.330348 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:37:37.330348 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:37:37.330348 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:37:37.330348 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:37:37.331370 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:37:37.331370 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:37:37.331370 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:37:37.331370 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:37:37.331370 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:37:37.331370 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:37:37.331370 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:37:37.331370 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:37:37.331370 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:37:37.331370 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:37:37.331370 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:37:37.331370 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:37:37.331370 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:37:37.331370 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:37:37.331370 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:37:37.331370 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:37:37.332359 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:37:37.332359 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:37:37.332359 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:37:37.332359 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:37:37.332359 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:37:37.332359 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:37:37.332359 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:37:37.332359 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:37:37.332359 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:37:37.332359 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:37:37.332359 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:37:37.332359 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:37:37.332359 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 133/272] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:37:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:37:37.343030 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a57bc67-6166-491d-b2c9-2372bfc067d0 None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:37.473000 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a57bc67-6166-491d-b2c9-2372bfc067d0 None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:37.519003 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7829dc-95f3-440c-8ec4-6c3e99b3919e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:37.519680 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7829dc-95f3-440c-8ec4-6c3e99b3919e None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:37.519959 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7829dc-95f3-440c-8ec4-6c3e99b3919e None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:37.520225 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7829dc-95f3-440c-8ec4-6c3e99b3919e None admin] PATH_INFO: `/v3/roles` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:37.520980 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7829dc-95f3-440c-8ec4-6c3e99b3919e None admin] RBAC: Adding query filter params (name=reader) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:37:37.521282 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7829dc-95f3-440c-8ec4-6c3e99b3919e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:37.525112 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7829dc-95f3-440c-8ec4-6c3e99b3919e None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:37.534503 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 140/273] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:37:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:37.540522 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0043a8-824c-462d-acb7-94e344b221fa None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:37.567619 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cbd7f1-41d1-4add-b4e5-cf79665f8537 None None] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:37.567964 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cbd7f1-41d1-4add-b4e5-cf79665f8537 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:37.568190 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cbd7f1-41d1-4add-b4e5-cf79665f8537 None None] PATH_INFO: `/` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:37.569920 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 141/274] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:37:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:37:37.575807 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fb0190-1fc2-4e27-b034-2e442ae44641 None None] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:37.576151 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fb0190-1fc2-4e27-b034-2e442ae44641 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:37.576375 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fb0190-1fc2-4e27-b034-2e442ae44641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:37.595100 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.auth.core [None req-21fb0190-1fc2-4e27-b034-2e442ae44641 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:37:37.596340 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.fernet_utils [None req-21fb0190-1fc2-4e27-b034-2e442ae44641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:37:37.598824 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:37.676006 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 142/275] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:37:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:37:37.678876 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0043a8-824c-462d-acb7-94e344b221fa None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:37.689071 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:37.703165 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fb0190-1fc2-4e27-b034-2e442ae44641 None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:37.717335 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40850a5c-3738-4e82-9651-d5130e1d9b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:37.718259 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40850a5c-3738-4e82-9651-d5130e1d9b4e None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:37.718659 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40850a5c-3738-4e82-9651-d5130e1d9b4e None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:37.718860 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40850a5c-3738-4e82-9651-d5130e1d9b4e None admin] PATH_INFO: `/v3/users/331dd46a8ec64f4598751c2eeacba71e` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:37.720231 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40850a5c-3738-4e82-9651-d5130e1d9b4e None admin] RBAC: Authorizing `identity:get_user(user_id=331dd46a8ec64f4598751c2eeacba71e)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:37.721903 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40850a5c-3738-4e82-9651-d5130e1d9b4e None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:37.724192 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 134/276] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:37:37 2022] GET /identity/v3/users/331dd46a8ec64f4598751c2eeacba71e => generated 297 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:37:37.730824 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40850a5c-3738-4e82-9651-d5130e1d9b4e None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:37.829343 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fb0190-1fc2-4e27-b034-2e442ae44641 None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:37.851312 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40850a5c-3738-4e82-9651-d5130e1d9b4e None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:37.866212 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5e30b9-0b6a-490e-b8d2-a6e3ebba3253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:37.867026 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5e30b9-0b6a-490e-b8d2-a6e3ebba3253 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:37.867886 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5e30b9-0b6a-490e-b8d2-a6e3ebba3253 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:37.868173 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5e30b9-0b6a-490e-b8d2-a6e3ebba3253 None admin] PATH_INFO: `/v3/roles/member` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:37.875971 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:37.880042 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5e30b9-0b6a-490e-b8d2-a6e3ebba3253 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:37:37.889240 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-cf5e30b9-0b6a-490e-b8d2-a6e3ebba3253 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:37:37.889240 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:37:37.889240 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:37:37.889240 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:37:37.889240 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:37:37.889240 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:37:37.889240 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:37:37.889240 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:37:37.889240 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:37:37.889240 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:37:37.889240 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:37:37.889240 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:37:37.889240 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:37:37.889240 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:37:37.889240 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:37:37.889240 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:37:37.890288 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:37:37.890288 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:37:37.890288 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:37:37.890288 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:37:37.890288 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:37:37.890288 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:37:37.890288 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:37:37.890288 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:37:37.890288 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:37:37.890288 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:37:37.890288 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:37:37.890288 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:37:37.890288 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:37:37.890288 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:37:37.890288 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:37:37.890288 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:37:37.891211 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:37:37.891211 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:37:37.891211 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:37:37.891211 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:37:37.891211 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:37:37.891211 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:37:37.891211 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:37:37.891211 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:37:37.891211 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:37:37.891211 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:37:37.891211 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:37:37.891211 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:37:37.891211 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 143/277] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:37:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:37:37.891958 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918b08c6-3448-43b5-b156-8746f4c8445b None None] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:37.892676 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918b08c6-3448-43b5-b156-8746f4c8445b None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:37.892676 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918b08c6-3448-43b5-b156-8746f4c8445b None None] PATH_INFO: `/` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:37.893806 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 144/278] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:37:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:37:37.897242 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3236362-cd05-462d-861c-19e8502d1fd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:37.898168 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3236362-cd05-462d-861c-19e8502d1fd2 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:37.898485 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3236362-cd05-462d-861c-19e8502d1fd2 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:37.898772 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3236362-cd05-462d-861c-19e8502d1fd2 None admin] PATH_INFO: `/v3/projects/0c950d0347b741b68c581d47c8968d0d` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:37.898879 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918b08c6-3448-43b5-b156-8746f4c8445b None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:37.900090 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3236362-cd05-462d-861c-19e8502d1fd2 None admin] RBAC: Authorizing `identity:get_project(project_id=0c950d0347b741b68c581d47c8968d0d)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:37.901565 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3236362-cd05-462d-861c-19e8502d1fd2 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:37.908260 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 135/279] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:37:37 2022] GET /identity/v3/projects/0c950d0347b741b68c581d47c8968d0d => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:37.909399 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a28297-6422-4d00-b996-9852ed096d31 None None] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:37.909694 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a28297-6422-4d00-b996-9852ed096d31 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:37.910017 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a28297-6422-4d00-b996-9852ed096d31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:37.926965 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.auth.core [None req-e7a28297-6422-4d00-b996-9852ed096d31 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:37:37.928167 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-e7a28297-6422-4d00-b996-9852ed096d31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:37:37.972770 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:38.004495 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 136/280] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:37:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:37:38.005314 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a28297-6422-4d00-b996-9852ed096d31 None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:38.025233 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918b08c6-3448-43b5-b156-8746f4c8445b None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:38.063215 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb94795-2225-4ddf-8127-e5f0322e9557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:38.063905 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb94795-2225-4ddf-8127-e5f0322e9557 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:38.064192 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb94795-2225-4ddf-8127-e5f0322e9557 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:38.064472 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb94795-2225-4ddf-8127-e5f0322e9557 None admin] PATH_INFO: `/v3/roles` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:38.065229 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb94795-2225-4ddf-8127-e5f0322e9557 None admin] RBAC: Adding query filter params (name=member) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:37:38.065540 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb94795-2225-4ddf-8127-e5f0322e9557 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:38.066976 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb94795-2225-4ddf-8127-e5f0322e9557 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:38.076638 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 145/281] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:37:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:38.077537 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb94795-2225-4ddf-8127-e5f0322e9557 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:38.126813 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a28297-6422-4d00-b996-9852ed096d31 None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:38.146292 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:38.175522 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916525d4-33cc-4007-9980-bef5002979c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:38.176448 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916525d4-33cc-4007-9980-bef5002979c8 None admin] REQUEST_METHOD: `PUT` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:38.177017 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916525d4-33cc-4007-9980-bef5002979c8 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:38.177017 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916525d4-33cc-4007-9980-bef5002979c8 None admin] PATH_INFO: `/v3/projects/0c950d0347b741b68c581d47c8968d0d/users/331dd46a8ec64f4598751c2eeacba71e/roles/6129e931488746afb2145fa463c08395` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:38.179238 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916525d4-33cc-4007-9980-bef5002979c8 None admin] RBAC: Authorizing `identity:create_grant(project_id=0c950d0347b741b68c581d47c8968d0d, user_id=331dd46a8ec64f4598751c2eeacba71e, role_id=6129e931488746afb2145fa463c08395)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:38.181017 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916525d4-33cc-4007-9980-bef5002979c8 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:38.196297 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb94795-2225-4ddf-8127-e5f0322e9557 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:38.197893 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 137/282] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 14:37:38 2022] PUT /identity/v3/projects/0c950d0347b741b68c581d47c8968d0d/users/331dd46a8ec64f4598751c2eeacba71e/roles/6129e931488746afb2145fa463c08395 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:37:38.203050 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916525d4-33cc-4007-9980-bef5002979c8 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:38.243275 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e27f03e-dd4e-4047-86cb-7b3b17dda566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:38.244149 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e27f03e-dd4e-4047-86cb-7b3b17dda566 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:38.244597 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e27f03e-dd4e-4047-86cb-7b3b17dda566 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:38.244885 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e27f03e-dd4e-4047-86cb-7b3b17dda566 None admin] PATH_INFO: `/v3/domains/default` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:38.246512 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e27f03e-dd4e-4047-86cb-7b3b17dda566 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:38.248140 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e27f03e-dd4e-4047-86cb-7b3b17dda566 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:38.254340 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 146/283] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:37:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:38.255040 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:38.257282 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e27f03e-dd4e-4047-86cb-7b3b17dda566 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:38.339464 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916525d4-33cc-4007-9980-bef5002979c8 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:38.378096 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e27f03e-dd4e-4047-86cb-7b3b17dda566 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:38.380379 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c2897e-7763-4dee-a55c-b6978d114262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:38.381286 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c2897e-7763-4dee-a55c-b6978d114262 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:38.381572 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c2897e-7763-4dee-a55c-b6978d114262 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:38.381872 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c2897e-7763-4dee-a55c-b6978d114262 None admin] PATH_INFO: `/v3/users/b75a3e8a3fb64327b5a213817e370659` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:38.400945 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c2897e-7763-4dee-a55c-b6978d114262 None admin] RBAC: Authorizing `identity:get_user(user_id=b75a3e8a3fb64327b5a213817e370659)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:38.402476 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c2897e-7763-4dee-a55c-b6978d114262 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:38.404324 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 138/284] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:37:38 2022] GET /identity/v3/users/b75a3e8a3fb64327b5a213817e370659 => generated 293 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:38.405602 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f356d3a5-4ee4-4ab2-baa7-fb6bbbf66487 None None] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:38.406758 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f356d3a5-4ee4-4ab2-baa7-fb6bbbf66487 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:38.407052 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f356d3a5-4ee4-4ab2-baa7-fb6bbbf66487 None None] PATH_INFO: `/` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:38.408349 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 139/285] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:37:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:37:38.411530 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f356d3a5-4ee4-4ab2-baa7-fb6bbbf66487 None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:38.417876 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:38.428732 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b771d6-aa51-4db7-8b30-f141b314479b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:38.429418 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b771d6-aa51-4db7-8b30-f141b314479b None admin] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:38.430248 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b771d6-aa51-4db7-8b30-f141b314479b None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:38.430540 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b771d6-aa51-4db7-8b30-f141b314479b None admin] PATH_INFO: `/v3/users` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:38.431451 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b771d6-aa51-4db7-8b30-f141b314479b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:38.432937 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b771d6-aa51-4db7-8b30-f141b314479b None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:38.455844 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 147/286] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 19 14:37:38 2022] POST /identity/v3/users => generated 293 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 14:37:38.455844 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c683f48b-cff4-4c14-9087-321f3c43a027 None None] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:38.455844 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c683f48b-cff4-4c14-9087-321f3c43a027 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:38.455844 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c683f48b-cff4-4c14-9087-321f3c43a027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:38.482374 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.auth.core [None req-c683f48b-cff4-4c14-9087-321f3c43a027 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:37:38.485313 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.fernet_utils [None req-c683f48b-cff4-4c14-9087-321f3c43a027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:37:38.543487 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:38.564760 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f356d3a5-4ee4-4ab2-baa7-fb6bbbf66487 None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:38.588250 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 148/287] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:37:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:37:38.617168 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3b4651-340a-4797-a7a0-6a6a358ae250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:38.617999 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c683f48b-cff4-4c14-9087-321f3c43a027 None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:38.619131 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3b4651-340a-4797-a7a0-6a6a358ae250 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:38.619643 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3b4651-340a-4797-a7a0-6a6a358ae250 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:38.620079 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3b4651-340a-4797-a7a0-6a6a358ae250 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:38.621170 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3b4651-340a-4797-a7a0-6a6a358ae250 None admin] RBAC: Adding query filter params (role.id=7ebce5564f40456e94a3616961ed73ff, user.id=b75a3e8a3fb64327b5a213817e370659, scope.system=all) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:37:38.621723 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3b4651-340a-4797-a7a0-6a6a358ae250 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:38.623502 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3b4651-340a-4797-a7a0-6a6a358ae250 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:38.630433 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 140/288] 10.222.0.57 () {56 vars in 1308 bytes} [Thu May 19 14:37:38 2022] GET /identity/v3/role_assignments?user.id=b75a3e8a3fb64327b5a213817e370659&scope.system=all&role.id=7ebce5564f40456e94a3616961ed73ff => generated 226 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:37:38.749102 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c683f48b-cff4-4c14-9087-321f3c43a027 None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:38.766541 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:38.793036 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d16d92-2d2c-47fc-95c9-d38beb28dd32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:38.793933 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d16d92-2d2c-47fc-95c9-d38beb28dd32 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:38.794174 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d16d92-2d2c-47fc-95c9-d38beb28dd32 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:38.794450 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d16d92-2d2c-47fc-95c9-d38beb28dd32 None admin] PATH_INFO: `/v3/roles/member` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:38.808970 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d16d92-2d2c-47fc-95c9-d38beb28dd32 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:37:38.817229 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-04d16d92-2d2c-47fc-95c9-d38beb28dd32 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:37:38.817229 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:37:38.817229 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:37:38.817229 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:37:38.817229 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:37:38.817229 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:37:38.817229 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:37:38.817229 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:37:38.817229 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:37:38.817229 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:37:38.817229 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:37:38.817229 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:37:38.817229 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:37:38.817229 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:37:38.817229 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:37:38.817229 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:37:38.818329 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:37:38.818329 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:37:38.818329 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:37:38.818329 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:37:38.818329 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:37:38.818329 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:37:38.818329 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:37:38.818329 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:37:38.818329 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:37:38.818329 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:37:38.818329 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:37:38.818329 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:37:38.818329 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:37:38.818329 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:37:38.818329 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:37:38.818329 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:37:38.819236 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:37:38.819236 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:37:38.819236 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:37:38.819236 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:37:38.819236 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:37:38.819236 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:37:38.819236 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:37:38.819236 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:37:38.819236 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:37:38.819236 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:37:38.819236 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:37:38.819236 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:37:38.819236 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 149/289] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:37:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:37:38.825161 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3b4651-340a-4797-a7a0-6a6a358ae250 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:38.947629 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3b4651-340a-4797-a7a0-6a6a358ae250 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:38.986232 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b907805-97d4-4e3a-8e49-0790e09ba5d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:38.987351 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b907805-97d4-4e3a-8e49-0790e09ba5d2 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:38.987839 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b907805-97d4-4e3a-8e49-0790e09ba5d2 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:38.988318 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b907805-97d4-4e3a-8e49-0790e09ba5d2 None admin] PATH_INFO: `/v3/roles` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:38.989314 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b907805-97d4-4e3a-8e49-0790e09ba5d2 None admin] RBAC: Adding query filter params (name=member) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:37:38.989821 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b907805-97d4-4e3a-8e49-0790e09ba5d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:38.991476 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b907805-97d4-4e3a-8e49-0790e09ba5d2 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:39.001337 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 141/290] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:37:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:39.006534 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d16d92-2d2c-47fc-95c9-d38beb28dd32 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:39.142977 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d16d92-2d2c-47fc-95c9-d38beb28dd32 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:39.191825 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905e3c65-57c3-4fa9-b3c9-4863e52c1f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:39.193080 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905e3c65-57c3-4fa9-b3c9-4863e52c1f37 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:39.193572 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905e3c65-57c3-4fa9-b3c9-4863e52c1f37 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:39.194106 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905e3c65-57c3-4fa9-b3c9-4863e52c1f37 None admin] PATH_INFO: `/v3/users/bd1bb65a5946439a9f11bcbd467ddf68` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:39.196128 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905e3c65-57c3-4fa9-b3c9-4863e52c1f37 None admin] RBAC: Authorizing `identity:get_user(user_id=bd1bb65a5946439a9f11bcbd467ddf68)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:39.197782 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905e3c65-57c3-4fa9-b3c9-4863e52c1f37 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:39.200074 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 150/291] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:37:39 2022] GET /identity/v3/users/bd1bb65a5946439a9f11bcbd467ddf68 => generated 297 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:37:39.207174 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b907805-97d4-4e3a-8e49-0790e09ba5d2 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:39.234672 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:39.340891 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b907805-97d4-4e3a-8e49-0790e09ba5d2 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:39.381924 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3728ef3c-e9ff-4068-88f5-9b828e603d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:39.383637 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3728ef3c-e9ff-4068-88f5-9b828e603d95 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:39.384174 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3728ef3c-e9ff-4068-88f5-9b828e603d95 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:39.384655 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3728ef3c-e9ff-4068-88f5-9b828e603d95 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:39.473316 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3728ef3c-e9ff-4068-88f5-9b828e603d95 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:39.475266 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3728ef3c-e9ff-4068-88f5-9b828e603d95 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:39.502950 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:39.613447 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 142/292] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:37:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:37:39.619764 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905e3c65-57c3-4fa9-b3c9-4863e52c1f37 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:39.703392 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:39.784841 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905e3c65-57c3-4fa9-b3c9-4863e52c1f37 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:39.830661 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c552350b-7ea4-44f4-b5ff-18598881b7be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:39.831538 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c552350b-7ea4-44f4-b5ff-18598881b7be None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:39.831825 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c552350b-7ea4-44f4-b5ff-18598881b7be None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:39.832108 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c552350b-7ea4-44f4-b5ff-18598881b7be None admin] PATH_INFO: `/v3/projects/0c950d0347b741b68c581d47c8968d0d` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:39.833451 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c552350b-7ea4-44f4-b5ff-18598881b7be None admin] RBAC: Authorizing `identity:get_project(project_id=0c950d0347b741b68c581d47c8968d0d)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:39.835199 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c552350b-7ea4-44f4-b5ff-18598881b7be None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:39.837408 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 151/293] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:37:39 2022] GET /identity/v3/projects/0c950d0347b741b68c581d47c8968d0d => generated 302 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:39.843004 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3728ef3c-e9ff-4068-88f5-9b828e603d95 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:39.993981 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3728ef3c-e9ff-4068-88f5-9b828e603d95 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:40.014300 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:40.031709 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18e8ec3-be1b-4ee4-8804-bdebd0d3157f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:40.032361 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18e8ec3-be1b-4ee4-8804-bdebd0d3157f None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:40.032646 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18e8ec3-be1b-4ee4-8804-bdebd0d3157f None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:40.032919 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18e8ec3-be1b-4ee4-8804-bdebd0d3157f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:40.033764 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18e8ec3-be1b-4ee4-8804-bdebd0d3157f None admin] RBAC: Adding query filter params (role.id=7ebce5564f40456e94a3616961ed73ff, scope.project.id=0c950d0347b741b68c581d47c8968d0d, user.id=bd1bb65a5946439a9f11bcbd467ddf68) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:37:40.034106 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18e8ec3-be1b-4ee4-8804-bdebd0d3157f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:40.035612 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18e8ec3-be1b-4ee4-8804-bdebd0d3157f None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:40.042327 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 143/294] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:37:39 2022] GET /identity/v3/role_assignments?user.id=bd1bb65a5946439a9f11bcbd467ddf68&scope.project.id=0c950d0347b741b68c581d47c8968d0d&role.id=7ebce5564f40456e94a3616961ed73ff => generated 608 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:40.732371 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffa4614-8589-497a-887a-a42a39ebd2f6 None None] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:40.732909 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffa4614-8589-497a-887a-a42a39ebd2f6 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:40.732909 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffa4614-8589-497a-887a-a42a39ebd2f6 None None] PATH_INFO: `/` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:40.734612 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 152/295] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:37:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:37:40.740206 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f870a7-277d-4504-aaed-dbe144f58d7a None None] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:40.740494 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f870a7-277d-4504-aaed-dbe144f58d7a None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:40.740772 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f870a7-277d-4504-aaed-dbe144f58d7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:40.762949 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.auth.core [None req-27f870a7-277d-4504-aaed-dbe144f58d7a None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:37:40.764152 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-27f870a7-277d-4504-aaed-dbe144f58d7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:37:40.843655 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 144/296] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:37:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:37:40.867545 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffa4614-8589-497a-887a-a42a39ebd2f6 None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:40.974874 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffa4614-8589-497a-887a-a42a39ebd2f6 None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:40.986886 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:41.015900 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3aa270-b50d-4a84-905f-3006078e1531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:41.017193 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3aa270-b50d-4a84-905f-3006078e1531 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:41.017707 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3aa270-b50d-4a84-905f-3006078e1531 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:41.018230 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3aa270-b50d-4a84-905f-3006078e1531 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:41.030367 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3aa270-b50d-4a84-905f-3006078e1531 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:37:41.038444 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-9b3aa270-b50d-4a84-905f-3006078e1531 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:37:41.038444 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:37:41.038444 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:37:41.038444 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:37:41.038444 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:37:41.038444 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:37:41.038444 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:37:41.038444 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:37:41.038444 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:37:41.038444 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:37:41.038444 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:37:41.038444 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:37:41.038444 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:37:41.038444 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:37:41.038444 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:37:41.038444 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:37:41.039587 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:37:41.039587 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:37:41.039587 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:37:41.039587 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:37:41.039587 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:37:41.039587 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:37:41.039587 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:37:41.039587 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:37:41.039587 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:37:41.039587 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:37:41.039587 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:37:41.039587 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:37:41.039587 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:37:41.039587 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:37:41.039587 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:37:41.039587 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:37:41.040577 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:37:41.040577 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:37:41.040577 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:37:41.040577 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:37:41.040577 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:37:41.040577 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:37:41.040577 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:37:41.040577 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:37:41.040577 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:37:41.040577 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:37:41.040577 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:37:41.040577 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:37:41.043427 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 153/297] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:37:40 2022] GET /identity/v3/roles/reader => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:37:41.050988 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f870a7-277d-4504-aaed-dbe144f58d7a None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:41.160114 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f870a7-277d-4504-aaed-dbe144f58d7a None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:41.181463 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:41.206202 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eef446-7f9a-4dba-99ed-69e75fb00aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:41.207253 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eef446-7f9a-4dba-99ed-69e75fb00aaa None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:41.207742 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eef446-7f9a-4dba-99ed-69e75fb00aaa None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:41.208257 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eef446-7f9a-4dba-99ed-69e75fb00aaa None admin] PATH_INFO: `/v3/roles` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:41.209211 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7eef446-7f9a-4dba-99ed-69e75fb00aaa None admin] RBAC: Adding query filter params (name=reader) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:37:41.209715 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7eef446-7f9a-4dba-99ed-69e75fb00aaa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:41.212220 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7eef446-7f9a-4dba-99ed-69e75fb00aaa None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:41.221709 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 145/298] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:37:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:41.227076 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3aa270-b50d-4a84-905f-3006078e1531 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:41.284769 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3bbaaf-7713-4209-99fd-9d88e36bed1f None None] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:41.285405 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3bbaaf-7713-4209-99fd-9d88e36bed1f None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:41.285922 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3bbaaf-7713-4209-99fd-9d88e36bed1f None None] PATH_INFO: `/` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:41.287597 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 146/299] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:37:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:37:41.297423 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11822e03-04cf-44d8-8714-fd1989cf674d None None] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:41.297762 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11822e03-04cf-44d8-8714-fd1989cf674d None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:41.298028 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11822e03-04cf-44d8-8714-fd1989cf674d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:41.316450 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.auth.core [None req-11822e03-04cf-44d8-8714-fd1989cf674d None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:37:41.317312 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-11822e03-04cf-44d8-8714-fd1989cf674d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:37:41.351871 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3aa270-b50d-4a84-905f-3006078e1531 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:41.392676 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9a788b-14a3-4c20-bca8-e37940357a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:41.394015 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9a788b-14a3-4c20-bca8-e37940357a6d None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:41.394531 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9a788b-14a3-4c20-bca8-e37940357a6d None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:41.395013 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9a788b-14a3-4c20-bca8-e37940357a6d None admin] PATH_INFO: `/v3/users/331dd46a8ec64f4598751c2eeacba71e` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:41.396572 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9a788b-14a3-4c20-bca8-e37940357a6d None admin] RBAC: Authorizing `identity:get_user(user_id=331dd46a8ec64f4598751c2eeacba71e)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:41.398150 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 147/300] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:37:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:37:41.399110 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9a788b-14a3-4c20-bca8-e37940357a6d None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:41.401390 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 154/301] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:37:41 2022] GET /identity/v3/users/331dd46a8ec64f4598751c2eeacba71e => generated 297 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:41.407008 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11822e03-04cf-44d8-8714-fd1989cf674d None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:41.418926 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9a788b-14a3-4c20-bca8-e37940357a6d None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:41.477215 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:41.520040 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11822e03-04cf-44d8-8714-fd1989cf674d None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:41.524063 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:41.541572 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9a788b-14a3-4c20-bca8-e37940357a6d None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:41.558648 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124b998b-8948-48ad-8bf8-67bb020d4588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:41.559302 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124b998b-8948-48ad-8bf8-67bb020d4588 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:41.559581 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124b998b-8948-48ad-8bf8-67bb020d4588 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:41.559845 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124b998b-8948-48ad-8bf8-67bb020d4588 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:41.580466 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bc0323-35d4-4a5a-8fe2-1be1be062618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:41.581360 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bc0323-35d4-4a5a-8fe2-1be1be062618 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:41.581643 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bc0323-35d4-4a5a-8fe2-1be1be062618 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:41.581951 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bc0323-35d4-4a5a-8fe2-1be1be062618 None admin] PATH_INFO: `/v3/roles/member` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:41.593996 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bc0323-35d4-4a5a-8fe2-1be1be062618 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:37:41.602221 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-05bc0323-35d4-4a5a-8fe2-1be1be062618 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:37:41.602221 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:37:41.602221 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:37:41.602221 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:37:41.602221 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:37:41.602221 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:37:41.602221 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:37:41.602221 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:37:41.602221 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:37:41.602221 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:37:41.602221 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:37:41.602221 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:37:41.602221 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:37:41.602221 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:37:41.602221 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:37:41.602221 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:37:41.604175 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:37:41.604175 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:37:41.604175 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:37:41.604175 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:37:41.604175 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:37:41.604175 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:37:41.604175 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:37:41.604175 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:37:41.604175 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:37:41.604175 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:37:41.604175 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:37:41.604175 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:37:41.604175 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:37:41.604175 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:37:41.604175 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:37:41.604175 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:37:41.605329 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:37:41.605329 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:37:41.605329 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:37:41.605329 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:37:41.605329 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:37:41.605329 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:37:41.605329 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:37:41.605329 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:37:41.605329 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:37:41.605329 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:37:41.605329 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:37:41.605329 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:37:41.605329 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 155/302] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:37:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:37:41.606288 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5382d1-7549-40d8-ace1-64e61ad36e6d None None] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:41.606288 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5382d1-7549-40d8-ace1-64e61ad36e6d None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:41.606288 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5382d1-7549-40d8-ace1-64e61ad36e6d None None] PATH_INFO: `/` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:41.609967 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 156/303] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:37:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:37:41.611403 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d34115-05a6-4cc4-9f4c-4e8a1d68940f None None] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:41.611762 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d34115-05a6-4cc4-9f4c-4e8a1d68940f None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:41.612057 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d34115-05a6-4cc4-9f4c-4e8a1d68940f None None] PATH_INFO: `/` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:41.613335 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 157/304] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:37:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:37:41.614063 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d34115-05a6-4cc4-9f4c-4e8a1d68940f None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:41.637607 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124b998b-8948-48ad-8bf8-67bb020d4588 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:41.639133 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124b998b-8948-48ad-8bf8-67bb020d4588 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:41.715976 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:41.720675 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d34115-05a6-4cc4-9f4c-4e8a1d68940f None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:41.738187 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 148/305] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:37:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 14:37:41.739559 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966a6172-d482-4cf2-8df3-873331261cb2 None None] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:41.739843 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966a6172-d482-4cf2-8df3-873331261cb2 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:41.740119 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966a6172-d482-4cf2-8df3-873331261cb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:41.746106 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:41.756684 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.auth.core [None req-966a6172-d482-4cf2-8df3-873331261cb2 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:37:41.758469 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-966a6172-d482-4cf2-8df3-873331261cb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:37:41.761882 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3e7778-232b-495d-8587-25ecf2ba6da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:41.762558 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3e7778-232b-495d-8587-25ecf2ba6da3 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:41.762839 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3e7778-232b-495d-8587-25ecf2ba6da3 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:41.763115 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3e7778-232b-495d-8587-25ecf2ba6da3 None admin] PATH_INFO: `/v3/roles` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:41.763861 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3e7778-232b-495d-8587-25ecf2ba6da3 None admin] RBAC: Adding query filter params (name=member) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:37:41.764165 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3e7778-232b-495d-8587-25ecf2ba6da3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:41.765542 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3e7778-232b-495d-8587-25ecf2ba6da3 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:41.775209 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 158/306] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:37:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:41.776528 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c17eae3-8661-43f0-85eb-67d7ac414f35 None None] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:41.776823 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c17eae3-8661-43f0-85eb-67d7ac414f35 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:41.777098 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c17eae3-8661-43f0-85eb-67d7ac414f35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:41.793249 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.auth.core [None req-4c17eae3-8661-43f0-85eb-67d7ac414f35 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:37:41.794396 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.fernet_utils [None req-4c17eae3-8661-43f0-85eb-67d7ac414f35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:37:41.830068 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 149/307] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:37:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:37:41.830922 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966a6172-d482-4cf2-8df3-873331261cb2 None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:41.864127 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 159/308] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:37:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:37:41.864841 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c17eae3-8661-43f0-85eb-67d7ac414f35 None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:41.933095 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966a6172-d482-4cf2-8df3-873331261cb2 None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:41.958964 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:41.968701 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73943d59-f99d-40bb-a90d-d9e9b9881457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:41.969478 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c17eae3-8661-43f0-85eb-67d7ac414f35 None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:41.969599 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73943d59-f99d-40bb-a90d-d9e9b9881457 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:41.969836 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73943d59-f99d-40bb-a90d-d9e9b9881457 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:41.970137 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73943d59-f99d-40bb-a90d-d9e9b9881457 None admin] PATH_INFO: `/v3/projects/0c950d0347b741b68c581d47c8968d0d` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:41.971336 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73943d59-f99d-40bb-a90d-d9e9b9881457 None admin] RBAC: Authorizing `identity:get_project(project_id=0c950d0347b741b68c581d47c8968d0d)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:41.972804 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73943d59-f99d-40bb-a90d-d9e9b9881457 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:41.974570 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 150/309] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:37:41 2022] GET /identity/v3/projects/0c950d0347b741b68c581d47c8968d0d => generated 302 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:41.975338 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73943d59-f99d-40bb-a90d-d9e9b9881457 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:41.981709 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:42.004270 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079f9e35-cf81-47c2-90d0-4da15aac2688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:42.005153 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079f9e35-cf81-47c2-90d0-4da15aac2688 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:42.005433 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079f9e35-cf81-47c2-90d0-4da15aac2688 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:42.005707 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079f9e35-cf81-47c2-90d0-4da15aac2688 None admin] PATH_INFO: `/v3/users/b75a3e8a3fb64327b5a213817e370659` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:42.006941 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079f9e35-cf81-47c2-90d0-4da15aac2688 None admin] RBAC: Authorizing `identity:get_user(user_id=b75a3e8a3fb64327b5a213817e370659)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:42.008945 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079f9e35-cf81-47c2-90d0-4da15aac2688 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:42.010627 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 160/310] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:37:41 2022] GET /identity/v3/users/b75a3e8a3fb64327b5a213817e370659 => generated 293 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:42.011389 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079f9e35-cf81-47c2-90d0-4da15aac2688 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:42.081530 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73943d59-f99d-40bb-a90d-d9e9b9881457 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:42.111817 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079f9e35-cf81-47c2-90d0-4da15aac2688 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:42.115817 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726993a0-eb9b-4672-92a7-1a91a5198b6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:42.116604 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726993a0-eb9b-4672-92a7-1a91a5198b6c None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:42.116910 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726993a0-eb9b-4672-92a7-1a91a5198b6c None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:42.118177 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726993a0-eb9b-4672-92a7-1a91a5198b6c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:42.128282 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726993a0-eb9b-4672-92a7-1a91a5198b6c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:37:42.137257 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-726993a0-eb9b-4672-92a7-1a91a5198b6c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:37:42.137257 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:37:42.137257 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:37:42.137257 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:37:42.137257 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:37:42.137257 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:37:42.137257 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:37:42.137257 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:37:42.137257 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:37:42.137257 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:37:42.137257 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:37:42.137257 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:37:42.137257 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:37:42.137257 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:37:42.137257 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:37:42.137257 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:37:42.138309 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:37:42.138309 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:37:42.138309 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:37:42.138309 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:37:42.138309 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:37:42.138309 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:37:42.138309 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:37:42.138309 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:37:42.138309 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:37:42.138309 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:37:42.138309 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:37:42.138309 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:37:42.138309 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:37:42.138309 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:37:42.138309 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:37:42.138309 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:37:42.139244 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:37:42.139244 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:37:42.139244 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:37:42.139244 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:37:42.139244 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:37:42.139244 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:37:42.139244 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:37:42.139244 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:37:42.139244 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:37:42.139244 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:37:42.139244 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:37:42.139244 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:37:42.139244 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 151/311] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:37:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:37:42.139244 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726993a0-eb9b-4672-92a7-1a91a5198b6c None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:42.151334 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8d8b69-b963-47bb-b3ca-cc90e9813a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:42.152133 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8d8b69-b963-47bb-b3ca-cc90e9813a55 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:42.152410 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8d8b69-b963-47bb-b3ca-cc90e9813a55 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:42.152683 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8d8b69-b963-47bb-b3ca-cc90e9813a55 None admin] PATH_INFO: `/v3/domains/default` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:42.153880 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8d8b69-b963-47bb-b3ca-cc90e9813a55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:42.155347 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8d8b69-b963-47bb-b3ca-cc90e9813a55 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:42.157149 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 161/312] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:37:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:42.158131 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8d8b69-b963-47bb-b3ca-cc90e9813a55 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:42.189410 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:42.205035 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:42.245747 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726993a0-eb9b-4672-92a7-1a91a5198b6c None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:42.263675 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8d8b69-b963-47bb-b3ca-cc90e9813a55 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:42.280724 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89571a34-353e-4c2b-951b-a2c115889433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:42.281380 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89571a34-353e-4c2b-951b-a2c115889433 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:42.281666 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89571a34-353e-4c2b-951b-a2c115889433 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:42.282000 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89571a34-353e-4c2b-951b-a2c115889433 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:42.282845 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89571a34-353e-4c2b-951b-a2c115889433 None admin] RBAC: Adding query filter params (role.id=6129e931488746afb2145fa463c08395, scope.project.id=0c950d0347b741b68c581d47c8968d0d, user.id=331dd46a8ec64f4598751c2eeacba71e) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:37:42.283183 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89571a34-353e-4c2b-951b-a2c115889433 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:42.284620 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89571a34-353e-4c2b-951b-a2c115889433 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:42.290658 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 152/313] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:37:42 2022] GET /identity/v3/role_assignments?user.id=331dd46a8ec64f4598751c2eeacba71e&scope.project.id=0c950d0347b741b68c581d47c8968d0d&role.id=6129e931488746afb2145fa463c08395 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:42.291458 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89571a34-353e-4c2b-951b-a2c115889433 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:42.298599 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285ee6e2-0ab1-45d5-961e-df8b3c9625b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:42.299487 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285ee6e2-0ab1-45d5-961e-df8b3c9625b8 None admin] REQUEST_METHOD: `PUT` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:42.299768 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285ee6e2-0ab1-45d5-961e-df8b3c9625b8 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:42.300044 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285ee6e2-0ab1-45d5-961e-df8b3c9625b8 None admin] PATH_INFO: `/v3/system/users/b75a3e8a3fb64327b5a213817e370659/roles/7ebce5564f40456e94a3616961ed73ff` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:42.301638 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285ee6e2-0ab1-45d5-961e-df8b3c9625b8 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=b75a3e8a3fb64327b5a213817e370659, role_id=7ebce5564f40456e94a3616961ed73ff)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:42.303204 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285ee6e2-0ab1-45d5-961e-df8b3c9625b8 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:42.319237 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 162/314] 10.222.0.57 () {58 vars in 1334 bytes} [Thu May 19 14:37:42 2022] PUT /identity/v3/system/users/b75a3e8a3fb64327b5a213817e370659/roles/7ebce5564f40456e94a3616961ed73ff => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:37:42.321070 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285ee6e2-0ab1-45d5-961e-df8b3c9625b8 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:42.431777 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89571a34-353e-4c2b-951b-a2c115889433 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:42.465541 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:42.477048 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb986c6f-bea8-460e-bbd6-a267d3382542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:42.477925 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb986c6f-bea8-460e-bbd6-a267d3382542 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:42.478312 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb986c6f-bea8-460e-bbd6-a267d3382542 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:42.478713 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb986c6f-bea8-460e-bbd6-a267d3382542 None admin] PATH_INFO: `/v3/roles` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:42.479601 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb986c6f-bea8-460e-bbd6-a267d3382542 None admin] RBAC: Adding query filter params (name=reader) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:37:42.480044 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb986c6f-bea8-460e-bbd6-a267d3382542 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:42.480520 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:42.482660 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb986c6f-bea8-460e-bbd6-a267d3382542 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:42.495931 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 153/315] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:37:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:42.497511 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285ee6e2-0ab1-45d5-961e-df8b3c9625b8 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:42.502477 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb986c6f-bea8-460e-bbd6-a267d3382542 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:42.563003 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317fc831-ff44-4085-ad22-6f749a7d82bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:42.563955 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317fc831-ff44-4085-ad22-6f749a7d82bc None admin] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:42.564435 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317fc831-ff44-4085-ad22-6f749a7d82bc None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:42.564871 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317fc831-ff44-4085-ad22-6f749a7d82bc None admin] PATH_INFO: `/v3/groups` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:42.565790 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317fc831-ff44-4085-ad22-6f749a7d82bc None admin] RBAC: Authorizing `identity:create_group()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:42.569137 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317fc831-ff44-4085-ad22-6f749a7d82bc None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:42.601330 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 163/316] 10.222.0.57 () {60 vars in 1134 bytes} [Thu May 19 14:37:42 2022] POST /identity/v3/groups => generated 229 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:37:42.645203 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb986c6f-bea8-460e-bbd6-a267d3382542 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:42.683609 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c7e87b-5b97-4ff8-9f59-b0a17473ccf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:42.684418 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c7e87b-5b97-4ff8-9f59-b0a17473ccf3 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:42.684701 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c7e87b-5b97-4ff8-9f59-b0a17473ccf3 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:42.684976 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c7e87b-5b97-4ff8-9f59-b0a17473ccf3 None admin] PATH_INFO: `/v3/users/420839393bbd467e9559d478ad6bdfb3` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:42.700468 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c7e87b-5b97-4ff8-9f59-b0a17473ccf3 None admin] RBAC: Authorizing `identity:get_user(user_id=420839393bbd467e9559d478ad6bdfb3)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:42.702016 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c7e87b-5b97-4ff8-9f59-b0a17473ccf3 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:42.703811 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 154/317] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:37:42 2022] GET /identity/v3/users/420839393bbd467e9559d478ad6bdfb3 => generated 293 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:37:42.709814 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317fc831-ff44-4085-ad22-6f749a7d82bc None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:42.826361 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317fc831-ff44-4085-ad22-6f749a7d82bc None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:42.868014 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd6a5c5-c0d0-4176-9337-2abbfe331d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:42.868712 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd6a5c5-c0d0-4176-9337-2abbfe331d75 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:42.868993 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd6a5c5-c0d0-4176-9337-2abbfe331d75 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:42.869252 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd6a5c5-c0d0-4176-9337-2abbfe331d75 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:42.870175 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd6a5c5-c0d0-4176-9337-2abbfe331d75 None admin] RBAC: Adding query filter params (role.id=6129e931488746afb2145fa463c08395, user.id=420839393bbd467e9559d478ad6bdfb3, scope.system=all) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:37:42.870487 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd6a5c5-c0d0-4176-9337-2abbfe331d75 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:42.871952 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd6a5c5-c0d0-4176-9337-2abbfe331d75 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:42.877581 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 164/318] 10.222.0.57 () {56 vars in 1308 bytes} [Thu May 19 14:37:42 2022] GET /identity/v3/role_assignments?user.id=420839393bbd467e9559d478ad6bdfb3&scope.system=all&role.id=6129e931488746afb2145fa463c08395 => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:44.665455 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754399a2-a862-44a6-858c-4672e9b3d793 None None] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:44.666018 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754399a2-a862-44a6-858c-4672e9b3d793 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:44.666127 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754399a2-a862-44a6-858c-4672e9b3d793 None None] PATH_INFO: `/` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:44.667465 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 155/319] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:37:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:37:44.672786 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e50eb2c-62cc-4b62-974b-5decdaacb0b4 None None] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:44.673412 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e50eb2c-62cc-4b62-974b-5decdaacb0b4 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:44.673412 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e50eb2c-62cc-4b62-974b-5decdaacb0b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:44.692304 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.auth.core [None req-1e50eb2c-62cc-4b62-974b-5decdaacb0b4 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:37:44.693557 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.fernet_utils [None req-1e50eb2c-62cc-4b62-974b-5decdaacb0b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:37:44.706298 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:44.773892 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 165/320] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:37:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:37:44.797048 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754399a2-a862-44a6-858c-4672e9b3d793 None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:44.848985 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:44.883872 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5398a7-5743-485d-85bf-27bf48c223cf None None] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:44.884162 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5398a7-5743-485d-85bf-27bf48c223cf None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:44.884627 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5398a7-5743-485d-85bf-27bf48c223cf None None] PATH_INFO: `/` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:44.886048 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 166/321] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:37:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:37:44.890984 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7124ed-7d98-49c0-8c84-8c491043f3fc None None] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:44.891345 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7124ed-7d98-49c0-8c84-8c491043f3fc None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:44.892146 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7124ed-7d98-49c0-8c84-8c491043f3fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:44.908514 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.auth.core [None req-3b7124ed-7d98-49c0-8c84-8c491043f3fc None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:37:44.909807 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.fernet_utils [None req-3b7124ed-7d98-49c0-8c84-8c491043f3fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:37:44.911222 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754399a2-a862-44a6-858c-4672e9b3d793 None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:44.951163 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84813621-8b42-4d0e-8d7c-aba197afce2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:44.951898 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84813621-8b42-4d0e-8d7c-aba197afce2c None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:44.952182 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84813621-8b42-4d0e-8d7c-aba197afce2c None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:44.952458 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84813621-8b42-4d0e-8d7c-aba197afce2c None admin] PATH_INFO: `/v3/roles/member` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:44.965272 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84813621-8b42-4d0e-8d7c-aba197afce2c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:37:44.973490 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-84813621-8b42-4d0e-8d7c-aba197afce2c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:37:44.973490 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:37:44.973490 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:37:44.973490 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:37:44.973490 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:37:44.973490 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:37:44.973490 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:37:44.973490 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:37:44.973490 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:37:44.973490 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:37:44.973490 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:37:44.973490 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:37:44.973490 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:37:44.973490 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:37:44.973490 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:37:44.973490 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:37:44.975297 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:37:44.975297 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:37:44.975297 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:37:44.975297 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:37:44.975297 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:37:44.975297 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:37:44.975297 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:37:44.975297 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:37:44.975297 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:37:44.975297 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:37:44.975297 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:37:44.975297 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:37:44.975297 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:37:44.975297 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:37:44.975297 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:37:44.975297 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:37:44.976220 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:37:44.976220 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:37:44.976220 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:37:44.976220 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:37:44.976220 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:37:44.976220 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:37:44.976220 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:37:44.976220 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:37:44.976220 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:37:44.976220 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:37:44.976220 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:37:44.976220 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:37:44.976220 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 156/322] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:37:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:37:44.980911 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84813621-8b42-4d0e-8d7c-aba197afce2c None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:44.994649 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 167/323] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:37:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:37:45.016791 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7124ed-7d98-49c0-8c84-8c491043f3fc None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:45.096163 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84813621-8b42-4d0e-8d7c-aba197afce2c None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:45.098756 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:45.106709 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:45.125770 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7124ed-7d98-49c0-8c84-8c491043f3fc None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:45.139749 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62eaccd7-f4c8-4a23-b390-210273de3629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:45.140421 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eaccd7-f4c8-4a23-b390-210273de3629 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:45.140702 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eaccd7-f4c8-4a23-b390-210273de3629 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:45.141050 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eaccd7-f4c8-4a23-b390-210273de3629 None admin] PATH_INFO: `/v3/roles` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:45.141870 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62eaccd7-f4c8-4a23-b390-210273de3629 None admin] RBAC: Adding query filter params (name=member) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:37:45.142177 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62eaccd7-f4c8-4a23-b390-210273de3629 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:45.143551 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62eaccd7-f4c8-4a23-b390-210273de3629 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:45.152319 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 157/324] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:37:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:37:45.153566 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909d9788-8621-4675-a56d-1ebf38abdd90 None None] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:45.153870 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909d9788-8621-4675-a56d-1ebf38abdd90 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:45.154155 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909d9788-8621-4675-a56d-1ebf38abdd90 None None] PATH_INFO: `/` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:45.157993 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 158/325] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:37:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:37:45.162549 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909d9788-8621-4675-a56d-1ebf38abdd90 None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:45.166525 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443c5a10-b579-49bd-95e9-693fdbfa23ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:45.167533 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443c5a10-b579-49bd-95e9-693fdbfa23ae None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:45.167630 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443c5a10-b579-49bd-95e9-693fdbfa23ae None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:45.167874 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443c5a10-b579-49bd-95e9-693fdbfa23ae None admin] PATH_INFO: `/v3/domains/default` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:45.169075 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443c5a10-b579-49bd-95e9-693fdbfa23ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:45.170591 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443c5a10-b579-49bd-95e9-693fdbfa23ae None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:45.172270 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 168/326] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:37:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:37:45.173506 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d72ca7a-2996-4ade-b30e-578ad0c18900 None None] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:45.173802 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d72ca7a-2996-4ade-b30e-578ad0c18900 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:45.174105 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d72ca7a-2996-4ade-b30e-578ad0c18900 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:45.190372 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.auth.core [None req-1d72ca7a-2996-4ade-b30e-578ad0c18900 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:37:45.191599 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.fernet_utils [None req-1d72ca7a-2996-4ade-b30e-578ad0c18900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:37:45.259211 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 169/327] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:37:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:37:45.260295 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d72ca7a-2996-4ade-b30e-578ad0c18900 None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:45.268915 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909d9788-8621-4675-a56d-1ebf38abdd90 None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:45.306820 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be51fd47-ac5e-47c0-82ea-f8b020d0d1ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:45.307690 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be51fd47-ac5e-47c0-82ea-f8b020d0d1ba None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:45.307965 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be51fd47-ac5e-47c0-82ea-f8b020d0d1ba None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:45.308249 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be51fd47-ac5e-47c0-82ea-f8b020d0d1ba None admin] PATH_INFO: `/v3/users/b75a3e8a3fb64327b5a213817e370659` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:45.309439 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be51fd47-ac5e-47c0-82ea-f8b020d0d1ba None admin] RBAC: Authorizing `identity:get_user(user_id=b75a3e8a3fb64327b5a213817e370659)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:45.310907 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be51fd47-ac5e-47c0-82ea-f8b020d0d1ba None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:45.312570 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 159/328] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:37:45 2022] GET /identity/v3/users/b75a3e8a3fb64327b5a213817e370659 => generated 293 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:45.313337 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be51fd47-ac5e-47c0-82ea-f8b020d0d1ba None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:45.335670 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:45.349803 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:45.367670 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d72ca7a-2996-4ade-b30e-578ad0c18900 None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:45.402403 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e61ed01-0bc8-44aa-8cbd-7d671423f46e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:45.403074 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e61ed01-0bc8-44aa-8cbd-7d671423f46e None admin] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:45.403360 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e61ed01-0bc8-44aa-8cbd-7d671423f46e None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:45.403638 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e61ed01-0bc8-44aa-8cbd-7d671423f46e None admin] PATH_INFO: `/v3/groups` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:45.404324 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e61ed01-0bc8-44aa-8cbd-7d671423f46e None admin] RBAC: Authorizing `identity:create_group()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:45.405775 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e61ed01-0bc8-44aa-8cbd-7d671423f46e None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:45.415168 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be51fd47-ac5e-47c0-82ea-f8b020d0d1ba None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:45.419932 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 170/329] 10.222.0.57 () {60 vars in 1134 bytes} [Thu May 19 14:37:45 2022] POST /identity/v3/groups => generated 226 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:37:45.420740 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e61ed01-0bc8-44aa-8cbd-7d671423f46e None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:45.452785 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a137b4-442d-43a4-b41d-641ba6fb0460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:45.454371 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a137b4-442d-43a4-b41d-641ba6fb0460 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:45.454613 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a137b4-442d-43a4-b41d-641ba6fb0460 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:45.454897 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a137b4-442d-43a4-b41d-641ba6fb0460 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:45.471278 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a137b4-442d-43a4-b41d-641ba6fb0460 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:37:45.482930 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-71a137b4-442d-43a4-b41d-641ba6fb0460 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:37:45.482930 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:37:45.482930 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:37:45.482930 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:37:45.482930 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:37:45.482930 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:37:45.482930 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:37:45.482930 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:37:45.482930 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:37:45.482930 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:37:45.482930 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:37:45.482930 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:37:45.482930 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:37:45.482930 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:37:45.482930 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:37:45.482930 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:37:45.484049 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:37:45.484049 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:37:45.484049 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:37:45.484049 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:37:45.484049 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:37:45.484049 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:37:45.484049 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:37:45.484049 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:37:45.484049 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:37:45.484049 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:37:45.484049 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:37:45.484049 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:37:45.484049 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:37:45.484049 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:37:45.484049 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:37:45.484049 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:37:45.484972 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:37:45.484972 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:37:45.484972 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:37:45.484972 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:37:45.484972 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:37:45.484972 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:37:45.484972 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:37:45.484972 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:37:45.484972 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:37:45.484972 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:37:45.484972 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:37:45.484972 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:37:45.485756 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 160/330] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:37:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:37:45.491067 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a137b4-442d-43a4-b41d-641ba6fb0460 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:45.539697 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e61ed01-0bc8-44aa-8cbd-7d671423f46e None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:45.575072 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:45.581949 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0e9fd0-5113-412d-8cbd-fa5f9dfd88fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:45.583138 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0e9fd0-5113-412d-8cbd-fa5f9dfd88fc None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:45.583857 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0e9fd0-5113-412d-8cbd-fa5f9dfd88fc None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:45.584211 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0e9fd0-5113-412d-8cbd-fa5f9dfd88fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:45.585510 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0e9fd0-5113-412d-8cbd-fa5f9dfd88fc None admin] RBAC: Adding query filter params (role.id=7ebce5564f40456e94a3616961ed73ff, user.id=b75a3e8a3fb64327b5a213817e370659, scope.system=all) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:37:45.586743 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0e9fd0-5113-412d-8cbd-fa5f9dfd88fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:45.589468 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0e9fd0-5113-412d-8cbd-fa5f9dfd88fc None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:45.603227 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 171/331] 10.222.0.57 () {56 vars in 1308 bytes} [Thu May 19 14:37:45 2022] GET /identity/v3/role_assignments?user.id=b75a3e8a3fb64327b5a213817e370659&scope.system=all&role.id=7ebce5564f40456e94a3616961ed73ff => generated 510 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:45.603916 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:45.611000 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a137b4-442d-43a4-b41d-641ba6fb0460 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:45.654243 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e7136e-56b3-484e-ab87-d7763cfe5ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:45.654928 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e7136e-56b3-484e-ab87-d7763cfe5ada None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:45.655216 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e7136e-56b3-484e-ab87-d7763cfe5ada None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:45.655497 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e7136e-56b3-484e-ab87-d7763cfe5ada None admin] PATH_INFO: `/v3/roles` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:45.656272 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e7136e-56b3-484e-ab87-d7763cfe5ada None admin] RBAC: Adding query filter params (name=reader) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:37:45.656588 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e7136e-56b3-484e-ab87-d7763cfe5ada None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:45.658078 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e7136e-56b3-484e-ab87-d7763cfe5ada None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:45.668238 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 161/332] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:37:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:37:45.674399 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0e9fd0-5113-412d-8cbd-fa5f9dfd88fc None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:45.796622 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0e9fd0-5113-412d-8cbd-fa5f9dfd88fc None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:45.835507 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def7155d-3891-4763-90a0-ecbc13c691fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:45.837126 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def7155d-3891-4763-90a0-ecbc13c691fd None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:45.837409 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def7155d-3891-4763-90a0-ecbc13c691fd None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:45.837682 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def7155d-3891-4763-90a0-ecbc13c691fd None admin] PATH_INFO: `/v3/users/420839393bbd467e9559d478ad6bdfb3` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:45.838965 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def7155d-3891-4763-90a0-ecbc13c691fd None admin] RBAC: Authorizing `identity:get_user(user_id=420839393bbd467e9559d478ad6bdfb3)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:45.840448 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def7155d-3891-4763-90a0-ecbc13c691fd None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:45.842120 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 172/333] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:37:45 2022] GET /identity/v3/users/420839393bbd467e9559d478ad6bdfb3 => generated 293 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:45.847833 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e7136e-56b3-484e-ab87-d7763cfe5ada None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:45.969994 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e7136e-56b3-484e-ab87-d7763cfe5ada None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:46.009496 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006b4bca-446d-4ee8-916b-9d02e558ec5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:46.010398 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006b4bca-446d-4ee8-916b-9d02e558ec5f None admin] REQUEST_METHOD: `PUT` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:46.010681 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006b4bca-446d-4ee8-916b-9d02e558ec5f None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:46.010956 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006b4bca-446d-4ee8-916b-9d02e558ec5f None admin] PATH_INFO: `/v3/system/users/420839393bbd467e9559d478ad6bdfb3/roles/6129e931488746afb2145fa463c08395` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:46.012474 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006b4bca-446d-4ee8-916b-9d02e558ec5f None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=420839393bbd467e9559d478ad6bdfb3, role_id=6129e931488746afb2145fa463c08395)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:46.013872 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006b4bca-446d-4ee8-916b-9d02e558ec5f None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:46.024380 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 162/334] 10.222.0.57 () {58 vars in 1334 bytes} [Thu May 19 14:37:45 2022] PUT /identity/v3/system/users/420839393bbd467e9559d478ad6bdfb3/roles/6129e931488746afb2145fa463c08395 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:37:48.467765 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ff8817-537e-4f37-905e-2104da4dee88 None None] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:48.468287 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ff8817-537e-4f37-905e-2104da4dee88 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:48.468287 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ff8817-537e-4f37-905e-2104da4dee88 None None] PATH_INFO: `/` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:48.469758 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 173/335] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:37:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:37:48.475371 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10abcd4f-730c-49b8-8d1f-0ed8d9f1456b None None] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:48.475825 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10abcd4f-730c-49b8-8d1f-0ed8d9f1456b None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:48.475916 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10abcd4f-730c-49b8-8d1f-0ed8d9f1456b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:48.499698 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.auth.core [None req-10abcd4f-730c-49b8-8d1f-0ed8d9f1456b None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:37:48.501058 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-10abcd4f-730c-49b8-8d1f-0ed8d9f1456b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:37:48.524165 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:48.593407 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 163/336] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:37:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:37:48.644021 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ff8817-537e-4f37-905e-2104da4dee88 None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:48.725553 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:48.785237 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ff8817-537e-4f37-905e-2104da4dee88 None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:48.842571 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a29be35-493a-4c26-bdb1-ceeebdf35e1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:48.843842 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a29be35-493a-4c26-bdb1-ceeebdf35e1c None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:48.844367 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a29be35-493a-4c26-bdb1-ceeebdf35e1c None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:48.844900 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a29be35-493a-4c26-bdb1-ceeebdf35e1c None admin] PATH_INFO: `/v3/roles/member` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:48.856754 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a29be35-493a-4c26-bdb1-ceeebdf35e1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:37:48.864918 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-2a29be35-493a-4c26-bdb1-ceeebdf35e1c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:37:48.864918 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:37:48.864918 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:37:48.864918 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:37:48.864918 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:37:48.864918 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:37:48.864918 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:37:48.864918 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:37:48.864918 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:37:48.864918 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:37:48.864918 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:37:48.864918 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:37:48.864918 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:37:48.864918 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:37:48.864918 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:37:48.864918 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:37:48.866122 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:37:48.866122 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:37:48.866122 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:37:48.866122 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:37:48.866122 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:37:48.866122 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:37:48.866122 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:37:48.866122 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:37:48.866122 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:37:48.866122 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:37:48.866122 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:37:48.866122 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:37:48.866122 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:37:48.866122 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:37:48.866122 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:37:48.866122 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:37:48.867657 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:37:48.867657 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:37:48.867657 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:37:48.867657 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:37:48.867657 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:37:48.867657 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:37:48.867657 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:37:48.867657 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:37:48.867657 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:37:48.867657 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:37:48.867657 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:37:48.867657 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:37:48.870749 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 174/337] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:37:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:37:48.876339 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10abcd4f-730c-49b8-8d1f-0ed8d9f1456b None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:48.988519 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1b3f5b-9521-49c3-9289-c4aea5a73b34 None None] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:48.989166 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1b3f5b-9521-49c3-9289-c4aea5a73b34 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:48.989718 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1b3f5b-9521-49c3-9289-c4aea5a73b34 None None] PATH_INFO: `/` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:48.991641 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 175/338] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:37:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:37:48.999598 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d96bd8-2b73-4585-964e-44a7c66a8577 None None] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:49.000266 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d96bd8-2b73-4585-964e-44a7c66a8577 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:49.000769 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d96bd8-2b73-4585-964e-44a7c66a8577 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:49.028951 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.auth.core [None req-d2d96bd8-2b73-4585-964e-44a7c66a8577 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:37:49.030270 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.fernet_utils [None req-d2d96bd8-2b73-4585-964e-44a7c66a8577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:37:49.049132 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10abcd4f-730c-49b8-8d1f-0ed8d9f1456b None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:49.098549 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2572d2e-be85-4c03-b978-7bae8a76c752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:49.099634 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2572d2e-be85-4c03-b978-7bae8a76c752 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:49.100136 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2572d2e-be85-4c03-b978-7bae8a76c752 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:49.100664 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2572d2e-be85-4c03-b978-7bae8a76c752 None admin] PATH_INFO: `/v3/roles` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:49.102480 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2572d2e-be85-4c03-b978-7bae8a76c752 None admin] RBAC: Adding query filter params (name=member) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:37:49.103022 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2572d2e-be85-4c03-b978-7bae8a76c752 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:49.104663 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2572d2e-be85-4c03-b978-7bae8a76c752 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:49.114653 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 164/339] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:37:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:49.122924 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2572d2e-be85-4c03-b978-7bae8a76c752 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:49.149238 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:49.180186 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 176/340] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:37:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) May 19 14:37:49.214018 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d96bd8-2b73-4585-964e-44a7c66a8577 None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:49.330292 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:49.332257 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2572d2e-be85-4c03-b978-7bae8a76c752 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:49.358845 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d96bd8-2b73-4585-964e-44a7c66a8577 None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:49.396501 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ef1bdf-5e59-4f9c-a0d3-ac859a168e96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:49.397327 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ef1bdf-5e59-4f9c-a0d3-ac859a168e96 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:49.397609 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ef1bdf-5e59-4f9c-a0d3-ac859a168e96 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:49.397908 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ef1bdf-5e59-4f9c-a0d3-ac859a168e96 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:49.410032 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ef1bdf-5e59-4f9c-a0d3-ac859a168e96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:37:49.416536 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4be029-e653-4e76-8858-e682322e014a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:49.417215 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4be029-e653-4e76-8858-e682322e014a None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:49.417550 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4be029-e653-4e76-8858-e682322e014a None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:49.417863 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4be029-e653-4e76-8858-e682322e014a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:49.421219 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-88ef1bdf-5e59-4f9c-a0d3-ac859a168e96 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:37:49.421219 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:37:49.421219 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:37:49.421219 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:37:49.421219 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:37:49.421219 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:37:49.421219 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:37:49.421219 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:37:49.421219 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:37:49.421219 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:37:49.421219 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:37:49.421219 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:37:49.421219 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:37:49.421219 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:37:49.421219 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:37:49.421219 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:37:49.422247 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:37:49.422247 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:37:49.422247 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:37:49.422247 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:37:49.422247 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:37:49.422247 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:37:49.422247 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:37:49.422247 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:37:49.422247 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:37:49.422247 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:37:49.422247 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:37:49.422247 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:37:49.422247 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:37:49.422247 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:37:49.422247 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:37:49.422247 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:37:49.423253 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:37:49.423253 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:37:49.423253 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:37:49.423253 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:37:49.423253 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:37:49.423253 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:37:49.423253 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:37:49.423253 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:37:49.423253 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:37:49.423253 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:37:49.423253 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:37:49.423253 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:37:49.426315 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 177/341] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 19 14:37:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:37:49.427926 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bea7137-85c9-49b3-a931-5acdf032debb None None] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:49.428418 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bea7137-85c9-49b3-a931-5acdf032debb None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:49.428900 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bea7137-85c9-49b3-a931-5acdf032debb None None] PATH_INFO: `/` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:49.435350 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 178/342] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:37:49 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:37:49.438152 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bea7137-85c9-49b3-a931-5acdf032debb None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:49.505571 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4be029-e653-4e76-8858-e682322e014a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:49.507301 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4be029-e653-4e76-8858-e682322e014a None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:49.523806 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:49.569040 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bea7137-85c9-49b3-a931-5acdf032debb None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:49.579592 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:49.607199 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7b683f-0ab4-46ec-a07c-ae781ada29f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:49.608232 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7b683f-0ab4-46ec-a07c-ae781ada29f3 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:49.608720 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7b683f-0ab4-46ec-a07c-ae781ada29f3 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:49.609219 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7b683f-0ab4-46ec-a07c-ae781ada29f3 None admin] PATH_INFO: `/v3/roles` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:49.610346 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7b683f-0ab4-46ec-a07c-ae781ada29f3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:37:49.610877 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7b683f-0ab4-46ec-a07c-ae781ada29f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:49.612506 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7b683f-0ab4-46ec-a07c-ae781ada29f3 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:49.624139 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 179/343] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:37:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:37:49.625944 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c1be18-cefe-4957-a0ff-6aa6a5d344e4 None None] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:49.626450 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c1be18-cefe-4957-a0ff-6aa6a5d344e4 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:49.626946 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c1be18-cefe-4957-a0ff-6aa6a5d344e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:49.648168 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.auth.core [None req-76c1be18-cefe-4957-a0ff-6aa6a5d344e4 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:37:49.650937 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.fernet_utils [None req-76c1be18-cefe-4957-a0ff-6aa6a5d344e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:37:49.694791 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 165/344] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:37:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 572 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 14:37:49.695890 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4be029-e653-4e76-8858-e682322e014a None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:49.724820 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 180/345] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:37:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:37:49.727094 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f4263d-87cb-4cc0-8346-3355a2ff53f0 None None] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:49.727865 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f4263d-87cb-4cc0-8346-3355a2ff53f0 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:49.728618 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f4263d-87cb-4cc0-8346-3355a2ff53f0 None None] PATH_INFO: `/` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:49.730232 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 181/346] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:37:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:37:49.731164 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f4263d-87cb-4cc0-8346-3355a2ff53f0 None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:49.844594 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:49.848783 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f4263d-87cb-4cc0-8346-3355a2ff53f0 None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:49.860604 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4be029-e653-4e76-8858-e682322e014a None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:49.881368 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:49.892914 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb75519-2310-4a57-823f-5dd6e4fdd780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:49.894163 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb75519-2310-4a57-823f-5dd6e4fdd780 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:49.894670 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb75519-2310-4a57-823f-5dd6e4fdd780 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:49.895300 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb75519-2310-4a57-823f-5dd6e4fdd780 None admin] PATH_INFO: `/v3/projects/ff0814efdfab4314800f7e0e2109a949` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:49.896712 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb75519-2310-4a57-823f-5dd6e4fdd780 None admin] RBAC: Authorizing `identity:get_project(project_id=ff0814efdfab4314800f7e0e2109a949)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:49.899528 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb75519-2310-4a57-823f-5dd6e4fdd780 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:49.901403 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2a3fa9-7757-4a44-9c70-06f548411de5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:49.902172 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2a3fa9-7757-4a44-9c70-06f548411de5 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:49.902467 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2a3fa9-7757-4a44-9c70-06f548411de5 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:49.902808 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2a3fa9-7757-4a44-9c70-06f548411de5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:49.903025 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 182/347] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:37:49 2022] GET /identity/v3/projects/ff0814efdfab4314800f7e0e2109a949 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:49.904516 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13b9efc-b1f5-4285-8846-e8a7fee90470 None None] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:49.905020 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13b9efc-b1f5-4285-8846-e8a7fee90470 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:49.905511 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13b9efc-b1f5-4285-8846-e8a7fee90470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:49.923505 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.auth.core [None req-e13b9efc-b1f5-4285-8846-e8a7fee90470 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:37:49.925051 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.fernet_utils [None req-e13b9efc-b1f5-4285-8846-e8a7fee90470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:37:49.986410 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2a3fa9-7757-4a44-9c70-06f548411de5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:49.988022 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2a3fa9-7757-4a44-9c70-06f548411de5 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:49.999637 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 183/348] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:37:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:37:50.000825 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13b9efc-b1f5-4285-8846-e8a7fee90470 None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:50.111891 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 166/349] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:37:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:37:50.113161 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2a3fa9-7757-4a44-9c70-06f548411de5 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:50.122781 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:50.126597 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13b9efc-b1f5-4285-8846-e8a7fee90470 None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:50.135424 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:50.200218 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc78511-9532-4fab-bd35-843b40ac0da2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:50.201373 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc78511-9532-4fab-bd35-843b40ac0da2 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:50.201654 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc78511-9532-4fab-bd35-843b40ac0da2 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:50.201964 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc78511-9532-4fab-bd35-843b40ac0da2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:50.220386 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc78511-9532-4fab-bd35-843b40ac0da2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:37:50.225825 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-3dc78511-9532-4fab-bd35-843b40ac0da2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:37:50.225825 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:37:50.225825 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:37:50.225825 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:37:50.225825 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:37:50.225825 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:37:50.225825 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:37:50.225825 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:37:50.225825 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:37:50.225825 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:37:50.225825 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:37:50.225825 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:37:50.225825 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:37:50.225825 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:37:50.225825 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:37:50.225825 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:37:50.227335 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:37:50.227335 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:37:50.227335 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:37:50.227335 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:37:50.227335 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:37:50.227335 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:37:50.227335 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:37:50.227335 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:37:50.227335 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:37:50.227335 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:37:50.227335 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:37:50.227335 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:37:50.227335 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:37:50.227335 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:37:50.227335 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:37:50.227335 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:37:50.228245 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:37:50.228245 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:37:50.228245 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:37:50.228245 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:37:50.228245 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:37:50.228245 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:37:50.228245 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:37:50.228245 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:37:50.228245 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:37:50.228245 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:37:50.228245 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:37:50.228245 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:37:50.229538 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 184/350] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:37:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:37:50.234373 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ffb74b-af76-429d-b0e3-2824fbf64937 None None] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:50.234373 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ffb74b-af76-429d-b0e3-2824fbf64937 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:50.234373 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ffb74b-af76-429d-b0e3-2824fbf64937 None None] PATH_INFO: `/` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:50.234373 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 185/351] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:37:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:37:50.234373 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ffb74b-af76-429d-b0e3-2824fbf64937 None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:50.263293 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2a3fa9-7757-4a44-9c70-06f548411de5 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:50.307123 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3870ce7-05a9-499e-a7fe-2d85e88f3c10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:50.307494 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3870ce7-05a9-499e-a7fe-2d85e88f3c10 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:50.307766 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3870ce7-05a9-499e-a7fe-2d85e88f3c10 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:50.308038 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3870ce7-05a9-499e-a7fe-2d85e88f3c10 None admin] PATH_INFO: `/v3/groups/8b0f774e11544ebba71829c939937be9` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:50.317229 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3870ce7-05a9-499e-a7fe-2d85e88f3c10 None admin] RBAC: Authorizing `identity:get_group(group_id=8b0f774e11544ebba71829c939937be9)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:50.318737 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3870ce7-05a9-499e-a7fe-2d85e88f3c10 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:50.321550 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 167/352] 10.222.0.57 () {56 vars in 1180 bytes} [Thu May 19 14:37:50 2022] GET /identity/v3/groups/8b0f774e11544ebba71829c939937be9 => generated 226 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:50.325873 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3870ce7-05a9-499e-a7fe-2d85e88f3c10 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:50.376402 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ffb74b-af76-429d-b0e3-2824fbf64937 None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:50.418152 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:50.421498 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ecd4ca-e662-455b-95d6-be7deff00fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:50.422743 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ecd4ca-e662-455b-95d6-be7deff00fab None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:50.423032 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ecd4ca-e662-455b-95d6-be7deff00fab None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:50.423591 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ecd4ca-e662-455b-95d6-be7deff00fab None admin] PATH_INFO: `/v3/roles/reader` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:50.437877 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ecd4ca-e662-455b-95d6-be7deff00fab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:37:50.443657 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3870ce7-05a9-499e-a7fe-2d85e88f3c10 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:50.446958 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-47ecd4ca-e662-455b-95d6-be7deff00fab None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:37:50.446958 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:37:50.446958 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:37:50.446958 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:37:50.446958 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:37:50.446958 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:37:50.446958 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:37:50.446958 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:37:50.446958 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:37:50.446958 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:37:50.446958 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:37:50.446958 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:37:50.446958 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:37:50.446958 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:37:50.446958 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:37:50.446958 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:37:50.447984 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:37:50.447984 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:37:50.447984 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:37:50.447984 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:37:50.447984 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:37:50.447984 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:37:50.447984 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:37:50.447984 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:37:50.447984 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:37:50.447984 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:37:50.447984 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:37:50.447984 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:37:50.447984 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:37:50.447984 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:37:50.447984 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:37:50.447984 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:37:50.449291 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:37:50.449291 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:37:50.449291 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:37:50.449291 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:37:50.449291 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:37:50.449291 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:37:50.449291 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:37:50.449291 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:37:50.449291 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:37:50.449291 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:37:50.449291 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:37:50.449291 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:37:50.449291 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 186/353] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:37:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:37:50.449291 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ecd4ca-e662-455b-95d6-be7deff00fab None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:50.471161 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:50.489435 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e8fcd5-0cc2-4412-9ccb-9cebf6472744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:50.490400 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e8fcd5-0cc2-4412-9ccb-9cebf6472744 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:50.491807 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e8fcd5-0cc2-4412-9ccb-9cebf6472744 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:50.491807 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e8fcd5-0cc2-4412-9ccb-9cebf6472744 None admin] PATH_INFO: `/v3/projects/ff0814efdfab4314800f7e0e2109a949` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:50.492829 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e8fcd5-0cc2-4412-9ccb-9cebf6472744 None admin] RBAC: Authorizing `identity:get_project(project_id=ff0814efdfab4314800f7e0e2109a949)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:50.496091 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e8fcd5-0cc2-4412-9ccb-9cebf6472744 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:50.496308 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 168/354] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:37:50 2022] GET /identity/v3/projects/ff0814efdfab4314800f7e0e2109a949 => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:50.498619 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c3ade1-b8d8-463d-8120-25a3ea5a0843 None None] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:50.498619 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c3ade1-b8d8-463d-8120-25a3ea5a0843 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:50.498619 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c3ade1-b8d8-463d-8120-25a3ea5a0843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:50.517770 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.auth.core [None req-30c3ade1-b8d8-463d-8120-25a3ea5a0843 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:37:50.519354 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-30c3ade1-b8d8-463d-8120-25a3ea5a0843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:37:50.566398 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ecd4ca-e662-455b-95d6-be7deff00fab None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:50.605664 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 169/355] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:37:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:37:50.606340 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c3ade1-b8d8-463d-8120-25a3ea5a0843 None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:50.612360 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c804060-237b-404f-9056-31747c15e4a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:50.612856 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c804060-237b-404f-9056-31747c15e4a5 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:50.613873 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c804060-237b-404f-9056-31747c15e4a5 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:50.613873 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c804060-237b-404f-9056-31747c15e4a5 None admin] PATH_INFO: `/v3/roles` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:50.614382 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c804060-237b-404f-9056-31747c15e4a5 None admin] RBAC: Adding query filter params (name=member) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:37:50.614569 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c804060-237b-404f-9056-31747c15e4a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:50.616035 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c804060-237b-404f-9056-31747c15e4a5 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:50.627840 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 187/356] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:37:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:50.629193 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089b4e28-fe1c-4ee8-b6b6-3f569aba5a0e None None] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:50.631319 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089b4e28-fe1c-4ee8-b6b6-3f569aba5a0e None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:50.631319 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089b4e28-fe1c-4ee8-b6b6-3f569aba5a0e None None] PATH_INFO: `/` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:50.631319 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 188/357] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:37:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:37:50.631813 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089b4e28-fe1c-4ee8-b6b6-3f569aba5a0e None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:50.705311 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:50.732580 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c3ade1-b8d8-463d-8120-25a3ea5a0843 None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:50.734525 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:50.762133 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089b4e28-fe1c-4ee8-b6b6-3f569aba5a0e None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:50.774671 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20d70a2-4297-4ed3-b6a4-ecd950bbfb5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:50.775336 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20d70a2-4297-4ed3-b6a4-ecd950bbfb5f None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:50.775624 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20d70a2-4297-4ed3-b6a4-ecd950bbfb5f None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:50.775896 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20d70a2-4297-4ed3-b6a4-ecd950bbfb5f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:50.776738 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20d70a2-4297-4ed3-b6a4-ecd950bbfb5f None admin] RBAC: Adding query filter params (group.id=8b0f774e11544ebba71829c939937be9, role.id=7ebce5564f40456e94a3616961ed73ff, scope.project.id=ff0814efdfab4314800f7e0e2109a949) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:37:50.777057 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20d70a2-4297-4ed3-b6a4-ecd950bbfb5f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:50.778530 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20d70a2-4297-4ed3-b6a4-ecd950bbfb5f None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:50.784596 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 170/358] 10.222.0.57 () {56 vars in 1376 bytes} [Thu May 19 14:37:50 2022] GET /identity/v3/role_assignments?group.id=8b0f774e11544ebba71829c939937be9&scope.project.id=ff0814efdfab4314800f7e0e2109a949&role.id=7ebce5564f40456e94a3616961ed73ff => generated 260 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:50.785433 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20d70a2-4297-4ed3-b6a4-ecd950bbfb5f None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:50.811860 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ca5265-5961-43c7-aebd-bbf08fb94e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:50.812557 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ca5265-5961-43c7-aebd-bbf08fb94e87 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:50.812840 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ca5265-5961-43c7-aebd-bbf08fb94e87 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:50.813943 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ca5265-5961-43c7-aebd-bbf08fb94e87 None admin] PATH_INFO: `/v3/roles` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:50.817864 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ca5265-5961-43c7-aebd-bbf08fb94e87 None admin] RBAC: Adding query filter params (name=reader) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:37:50.817864 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ca5265-5961-43c7-aebd-bbf08fb94e87 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:50.817864 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ca5265-5961-43c7-aebd-bbf08fb94e87 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:50.830622 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 189/359] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:37:50 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:50.831406 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ca5265-5961-43c7-aebd-bbf08fb94e87 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:50.932633 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20d70a2-4297-4ed3-b6a4-ecd950bbfb5f None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:50.983047 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ca5265-5961-43c7-aebd-bbf08fb94e87 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:50.986588 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345925f1-3def-457b-a1e7-35a387305439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:50.987464 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345925f1-3def-457b-a1e7-35a387305439 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:50.987750 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345925f1-3def-457b-a1e7-35a387305439 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:50.988030 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345925f1-3def-457b-a1e7-35a387305439 None admin] PATH_INFO: `/v3/groups/8b0f774e11544ebba71829c939937be9` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:50.991731 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345925f1-3def-457b-a1e7-35a387305439 None admin] RBAC: Authorizing `identity:get_group(group_id=8b0f774e11544ebba71829c939937be9)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:50.991731 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345925f1-3def-457b-a1e7-35a387305439 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:50.994732 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 171/360] 10.222.0.57 () {56 vars in 1180 bytes} [Thu May 19 14:37:50 2022] GET /identity/v3/groups/8b0f774e11544ebba71829c939937be9 => generated 226 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:50.995687 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345925f1-3def-457b-a1e7-35a387305439 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:51.008622 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:51.032975 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6117b75f-8ac1-47b4-902e-0c50e924c6f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:51.033799 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6117b75f-8ac1-47b4-902e-0c50e924c6f9 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:51.034114 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6117b75f-8ac1-47b4-902e-0c50e924c6f9 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:51.034394 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6117b75f-8ac1-47b4-902e-0c50e924c6f9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:51.049354 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6117b75f-8ac1-47b4-902e-0c50e924c6f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:37:51.058570 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-6117b75f-8ac1-47b4-902e-0c50e924c6f9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:37:51.058570 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:37:51.058570 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:37:51.058570 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:37:51.058570 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:37:51.058570 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:37:51.058570 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:37:51.058570 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:37:51.058570 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:37:51.058570 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:37:51.058570 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:37:51.058570 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:37:51.058570 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:37:51.058570 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:37:51.058570 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:37:51.058570 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:37:51.059978 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:37:51.059978 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:37:51.059978 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:37:51.059978 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:37:51.059978 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:37:51.059978 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:37:51.059978 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:37:51.059978 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:37:51.059978 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:37:51.059978 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:37:51.059978 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:37:51.059978 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:37:51.059978 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:37:51.059978 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:37:51.059978 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:37:51.059978 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:37:51.060961 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:37:51.060961 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:37:51.060961 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:37:51.060961 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:37:51.060961 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:37:51.060961 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:37:51.060961 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:37:51.060961 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:37:51.060961 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:37:51.060961 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:37:51.060961 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:37:51.060961 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:37:51.063564 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 190/361] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 19 14:37:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:37:51.065328 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9c4be4-f50d-44de-a1a2-314436878070 None None] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:51.065732 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9c4be4-f50d-44de-a1a2-314436878070 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:51.066128 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9c4be4-f50d-44de-a1a2-314436878070 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:51.075697 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:51.088510 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.auth.core [None req-cc9c4be4-f50d-44de-a1a2-314436878070 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:37:51.090121 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.fernet_utils [None req-cc9c4be4-f50d-44de-a1a2-314436878070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:37:51.120862 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345925f1-3def-457b-a1e7-35a387305439 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:51.169572 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ade91f-cfe5-4e0a-b2f8-30a7dfd01a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:51.173216 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ade91f-cfe5-4e0a-b2f8-30a7dfd01a71 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:51.173216 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ade91f-cfe5-4e0a-b2f8-30a7dfd01a71 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:51.173216 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ade91f-cfe5-4e0a-b2f8-30a7dfd01a71 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:51.182950 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 191/362] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:37:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:37:51.182950 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9c4be4-f50d-44de-a1a2-314436878070 None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:51.261940 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ade91f-cfe5-4e0a-b2f8-30a7dfd01a71 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:51.264031 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ade91f-cfe5-4e0a-b2f8-30a7dfd01a71 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:51.276158 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:51.321703 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9c4be4-f50d-44de-a1a2-314436878070 None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:51.365005 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:51.373722 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5816efe5-d9e2-4fc2-b67a-9536e78a011d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:51.374967 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5816efe5-d9e2-4fc2-b67a-9536e78a011d None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:51.375478 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5816efe5-d9e2-4fc2-b67a-9536e78a011d None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:51.375834 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5816efe5-d9e2-4fc2-b67a-9536e78a011d None admin] PATH_INFO: `/v3/users/420839393bbd467e9559d478ad6bdfb3` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:51.377097 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5816efe5-d9e2-4fc2-b67a-9536e78a011d None admin] RBAC: Authorizing `identity:get_user(user_id=420839393bbd467e9559d478ad6bdfb3)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:51.378821 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5816efe5-d9e2-4fc2-b67a-9536e78a011d None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:51.380410 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 192/363] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:37:51 2022] GET /identity/v3/users/420839393bbd467e9559d478ad6bdfb3 => generated 293 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:51.386007 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5816efe5-d9e2-4fc2-b67a-9536e78a011d None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:51.412439 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 172/364] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:37:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:37:51.415424 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ade91f-cfe5-4e0a-b2f8-30a7dfd01a71 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:51.513806 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5816efe5-d9e2-4fc2-b67a-9536e78a011d None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:51.543462 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ade91f-cfe5-4e0a-b2f8-30a7dfd01a71 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:51.552956 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:51.558903 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0415d6f-5490-43b6-9bd4-008c66d4fef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:51.559968 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0415d6f-5490-43b6-9bd4-008c66d4fef2 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:51.560247 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0415d6f-5490-43b6-9bd4-008c66d4fef2 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:51.560524 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0415d6f-5490-43b6-9bd4-008c66d4fef2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:51.561499 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0415d6f-5490-43b6-9bd4-008c66d4fef2 None admin] RBAC: Adding query filter params (group.id=8b0f774e11544ebba71829c939937be9, role.id=b894e00b17e34f0cbde0ab3d760e2433, scope.project.id=ff0814efdfab4314800f7e0e2109a949) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:37:51.562005 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0415d6f-5490-43b6-9bd4-008c66d4fef2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:51.564829 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0415d6f-5490-43b6-9bd4-008c66d4fef2 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:51.574417 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 193/365] 10.222.0.57 () {56 vars in 1376 bytes} [Thu May 19 14:37:51 2022] GET /identity/v3/role_assignments?group.id=8b0f774e11544ebba71829c939937be9&scope.project.id=ff0814efdfab4314800f7e0e2109a949&role.id=b894e00b17e34f0cbde0ab3d760e2433 => generated 260 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:51.575286 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0415d6f-5490-43b6-9bd4-008c66d4fef2 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:51.592426 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61951633-dd96-4774-9661-59dc96408d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:51.594337 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61951633-dd96-4774-9661-59dc96408d46 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:51.595013 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61951633-dd96-4774-9661-59dc96408d46 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:51.595649 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61951633-dd96-4774-9661-59dc96408d46 None admin] PATH_INFO: `/v3/roles` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:51.597311 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61951633-dd96-4774-9661-59dc96408d46 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:37:51.597918 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61951633-dd96-4774-9661-59dc96408d46 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:51.599586 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61951633-dd96-4774-9661-59dc96408d46 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:51.613972 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 173/366] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:37:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:51.615201 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61951633-dd96-4774-9661-59dc96408d46 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:51.669040 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:51.706925 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0415d6f-5490-43b6-9bd4-008c66d4fef2 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:51.755795 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74e3ec5-d70a-45a4-aa26-37b2db133672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:51.755795 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74e3ec5-d70a-45a4-aa26-37b2db133672 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:51.755795 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74e3ec5-d70a-45a4-aa26-37b2db133672 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:51.755795 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74e3ec5-d70a-45a4-aa26-37b2db133672 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:51.771902 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74e3ec5-d70a-45a4-aa26-37b2db133672 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:37:51.783146 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-c74e3ec5-d70a-45a4-aa26-37b2db133672 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:37:51.783146 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:37:51.783146 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:37:51.783146 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:37:51.783146 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:37:51.783146 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:37:51.783146 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:37:51.783146 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:37:51.783146 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:37:51.783146 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:37:51.783146 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:37:51.783146 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:37:51.783146 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:37:51.783146 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:37:51.783146 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:37:51.783146 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:37:51.784527 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:37:51.784527 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:37:51.784527 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:37:51.784527 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:37:51.784527 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:37:51.784527 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:37:51.784527 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:37:51.784527 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:37:51.784527 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:37:51.784527 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:37:51.784527 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:37:51.784527 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:37:51.784527 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:37:51.784527 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:37:51.784527 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:37:51.784527 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:37:51.785758 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:37:51.785758 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:37:51.785758 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:37:51.785758 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:37:51.785758 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:37:51.785758 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:37:51.785758 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:37:51.785758 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:37:51.785758 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:37:51.785758 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:37:51.785758 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:37:51.785758 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:37:51.787667 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61951633-dd96-4774-9661-59dc96408d46 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:51.789449 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 194/367] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:37:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:37:51.790731 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74e3ec5-d70a-45a4-aa26-37b2db133672 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:51.849647 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c5f235-a60f-4502-ab01-5bdcdd67feff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:51.849647 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c5f235-a60f-4502-ab01-5bdcdd67feff None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:51.849647 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c5f235-a60f-4502-ab01-5bdcdd67feff None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:51.849647 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c5f235-a60f-4502-ab01-5bdcdd67feff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:51.850305 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c5f235-a60f-4502-ab01-5bdcdd67feff None admin] RBAC: Adding query filter params (role.id=6129e931488746afb2145fa463c08395, user.id=420839393bbd467e9559d478ad6bdfb3, scope.system=all) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:37:51.850555 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c5f235-a60f-4502-ab01-5bdcdd67feff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:51.852565 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c5f235-a60f-4502-ab01-5bdcdd67feff None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:51.872299 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 174/368] 10.222.0.57 () {56 vars in 1308 bytes} [Thu May 19 14:37:51 2022] GET /identity/v3/role_assignments?user.id=420839393bbd467e9559d478ad6bdfb3&scope.system=all&role.id=6129e931488746afb2145fa463c08395 => generated 510 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:51.878151 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c5f235-a60f-4502-ab01-5bdcdd67feff None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:51.918233 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:51.981012 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74e3ec5-d70a-45a4-aa26-37b2db133672 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:51.999699 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c5f235-a60f-4502-ab01-5bdcdd67feff None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:52.053575 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1322cb73-d0fc-4709-a557-405de1074cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:52.053783 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:52.055751 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1322cb73-d0fc-4709-a557-405de1074cda None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:52.056608 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1322cb73-d0fc-4709-a557-405de1074cda None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:52.058431 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1322cb73-d0fc-4709-a557-405de1074cda None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:52.079463 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbd7c25-5601-442f-baf7-6e0be3716a48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:52.083678 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbd7c25-5601-442f-baf7-6e0be3716a48 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:52.084326 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbd7c25-5601-442f-baf7-6e0be3716a48 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:52.085094 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbd7c25-5601-442f-baf7-6e0be3716a48 None admin] PATH_INFO: `/v3/projects/0c950d0347b741b68c581d47c8968d0d` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:52.089647 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbd7c25-5601-442f-baf7-6e0be3716a48 None admin] RBAC: Authorizing `identity:get_project(project_id=0c950d0347b741b68c581d47c8968d0d)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:52.093094 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbd7c25-5601-442f-baf7-6e0be3716a48 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:52.097159 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 195/369] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:37:51 2022] GET /identity/v3/projects/0c950d0347b741b68c581d47c8968d0d => generated 302 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:52.101120 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbd7c25-5601-442f-baf7-6e0be3716a48 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:52.198995 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1322cb73-d0fc-4709-a557-405de1074cda None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:52.200493 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1322cb73-d0fc-4709-a557-405de1074cda None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:52.234667 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:52.245482 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbd7c25-5601-442f-baf7-6e0be3716a48 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:52.292784 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d2924c-e1fd-4284-8840-5f6c4984820d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:52.293752 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d2924c-e1fd-4284-8840-5f6c4984820d None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:52.294087 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d2924c-e1fd-4284-8840-5f6c4984820d None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:52.294369 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d2924c-e1fd-4284-8840-5f6c4984820d None admin] PATH_INFO: `/v3/roles` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:52.295186 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d2924c-e1fd-4284-8840-5f6c4984820d None admin] RBAC: Adding query filter params (name=admin) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:37:52.295523 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d2924c-e1fd-4284-8840-5f6c4984820d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:52.297830 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d2924c-e1fd-4284-8840-5f6c4984820d None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:52.312266 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 196/370] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:37:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:52.313053 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d2924c-e1fd-4284-8840-5f6c4984820d None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:52.348447 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 175/371] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:37:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:37:52.349425 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1322cb73-d0fc-4709-a557-405de1074cda None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:52.395556 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:52.458685 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d2924c-e1fd-4284-8840-5f6c4984820d None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:52.488129 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1322cb73-d0fc-4709-a557-405de1074cda None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:52.503212 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2247e2dc-69af-47bd-8b32-48db5a57e419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:52.504191 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2247e2dc-69af-47bd-8b32-48db5a57e419 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:52.504486 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2247e2dc-69af-47bd-8b32-48db5a57e419 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:52.504768 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2247e2dc-69af-47bd-8b32-48db5a57e419 None admin] PATH_INFO: `/v3/groups/8b0f774e11544ebba71829c939937be9` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:52.506147 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2247e2dc-69af-47bd-8b32-48db5a57e419 None admin] RBAC: Authorizing `identity:get_group(group_id=8b0f774e11544ebba71829c939937be9)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:52.508318 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2247e2dc-69af-47bd-8b32-48db5a57e419 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:52.510714 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 197/372] 10.222.0.57 () {56 vars in 1180 bytes} [Thu May 19 14:37:52 2022] GET /identity/v3/groups/8b0f774e11544ebba71829c939937be9 => generated 226 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:52.511503 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2247e2dc-69af-47bd-8b32-48db5a57e419 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:52.521259 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:52.538277 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e8cefb-9cb7-4056-a2ff-656aeb11a44b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:52.539470 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e8cefb-9cb7-4056-a2ff-656aeb11a44b None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:52.539742 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e8cefb-9cb7-4056-a2ff-656aeb11a44b None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:52.540017 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e8cefb-9cb7-4056-a2ff-656aeb11a44b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:52.640555 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e8cefb-9cb7-4056-a2ff-656aeb11a44b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:52.642270 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2247e2dc-69af-47bd-8b32-48db5a57e419 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:52.643157 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e8cefb-9cb7-4056-a2ff-656aeb11a44b None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:52.669555 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:52.691830 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2e6cea-a911-4375-97ef-ebd8dad2bacd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:52.693260 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2e6cea-a911-4375-97ef-ebd8dad2bacd None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:52.694647 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2e6cea-a911-4375-97ef-ebd8dad2bacd None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:52.695190 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2e6cea-a911-4375-97ef-ebd8dad2bacd None admin] PATH_INFO: `/v3/projects/0c950d0347b741b68c581d47c8968d0d` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:52.696681 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2e6cea-a911-4375-97ef-ebd8dad2bacd None admin] RBAC: Authorizing `identity:get_project(project_id=0c950d0347b741b68c581d47c8968d0d)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:52.698348 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2e6cea-a911-4375-97ef-ebd8dad2bacd None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:52.700856 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 198/373] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:37:52 2022] GET /identity/v3/projects/0c950d0347b741b68c581d47c8968d0d => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:52.701800 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2e6cea-a911-4375-97ef-ebd8dad2bacd None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:52.785008 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 176/374] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:37:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:37:52.786199 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e8cefb-9cb7-4056-a2ff-656aeb11a44b None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:52.817543 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:52.853345 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2e6cea-a911-4375-97ef-ebd8dad2bacd None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:52.896831 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dc795d-057c-459c-8864-5949c9690b35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:52.897598 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dc795d-057c-459c-8864-5949c9690b35 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:52.898838 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dc795d-057c-459c-8864-5949c9690b35 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:52.898838 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dc795d-057c-459c-8864-5949c9690b35 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:52.899852 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dc795d-057c-459c-8864-5949c9690b35 None admin] RBAC: Adding query filter params (group.id=8b0f774e11544ebba71829c939937be9, role.id=7ebce5564f40456e94a3616961ed73ff, scope.project.id=0c950d0347b741b68c581d47c8968d0d) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:37:52.900329 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dc795d-057c-459c-8864-5949c9690b35 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:52.901818 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dc795d-057c-459c-8864-5949c9690b35 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:52.908081 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 199/375] 10.222.0.57 () {56 vars in 1376 bytes} [Thu May 19 14:37:52 2022] GET /identity/v3/role_assignments?group.id=8b0f774e11544ebba71829c939937be9&scope.project.id=0c950d0347b741b68c581d47c8968d0d&role.id=7ebce5564f40456e94a3616961ed73ff => generated 260 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:52.908860 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dc795d-057c-459c-8864-5949c9690b35 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:52.915910 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e8cefb-9cb7-4056-a2ff-656aeb11a44b None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:52.961616 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1255b5d-cbd0-4b9f-a6f4-303dc04f5f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:52.962937 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1255b5d-cbd0-4b9f-a6f4-303dc04f5f49 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:52.963330 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1255b5d-cbd0-4b9f-a6f4-303dc04f5f49 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:52.963712 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1255b5d-cbd0-4b9f-a6f4-303dc04f5f49 None admin] PATH_INFO: `/v3/groups/8b0f774e11544ebba71829c939937be9` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:52.965212 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1255b5d-cbd0-4b9f-a6f4-303dc04f5f49 None admin] RBAC: Authorizing `identity:get_group(group_id=8b0f774e11544ebba71829c939937be9)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:52.966845 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1255b5d-cbd0-4b9f-a6f4-303dc04f5f49 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:52.968893 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 177/376] 10.222.0.57 () {56 vars in 1180 bytes} [Thu May 19 14:37:52 2022] GET /identity/v3/groups/8b0f774e11544ebba71829c939937be9 => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:52.976141 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1255b5d-cbd0-4b9f-a6f4-303dc04f5f49 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:52.989662 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:53.053279 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dc795d-057c-459c-8864-5949c9690b35 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:53.092976 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9592b60-fd70-4369-ad1b-009fd615972f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:53.094269 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9592b60-fd70-4369-ad1b-009fd615972f None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:53.094741 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9592b60-fd70-4369-ad1b-009fd615972f None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:53.095193 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9592b60-fd70-4369-ad1b-009fd615972f None admin] PATH_INFO: `/v3/projects/c3cba789bd6f4fdaaa058dac80cb6fbe` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:53.096731 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9592b60-fd70-4369-ad1b-009fd615972f None admin] RBAC: Authorizing `identity:get_project(project_id=c3cba789bd6f4fdaaa058dac80cb6fbe)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:53.098407 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9592b60-fd70-4369-ad1b-009fd615972f None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:53.100504 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 200/377] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:37:52 2022] GET /identity/v3/projects/c3cba789bd6f4fdaaa058dac80cb6fbe => generated 344 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:53.106951 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9592b60-fd70-4369-ad1b-009fd615972f None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:53.141598 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:53.152082 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1255b5d-cbd0-4b9f-a6f4-303dc04f5f49 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:53.220838 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796e3f06-f845-40fe-ae95-82b724a7f165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:53.221075 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796e3f06-f845-40fe-ae95-82b724a7f165 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:53.221161 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796e3f06-f845-40fe-ae95-82b724a7f165 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:53.221430 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796e3f06-f845-40fe-ae95-82b724a7f165 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:53.222596 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796e3f06-f845-40fe-ae95-82b724a7f165 None admin] RBAC: Adding query filter params (group.id=8b0f774e11544ebba71829c939937be9, role.id=b894e00b17e34f0cbde0ab3d760e2433, scope.project.id=0c950d0347b741b68c581d47c8968d0d) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:37:53.223039 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796e3f06-f845-40fe-ae95-82b724a7f165 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:53.224733 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796e3f06-f845-40fe-ae95-82b724a7f165 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:53.231169 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 178/378] 10.222.0.57 () {56 vars in 1376 bytes} [Thu May 19 14:37:52 2022] GET /identity/v3/role_assignments?group.id=8b0f774e11544ebba71829c939937be9&scope.project.id=0c950d0347b741b68c581d47c8968d0d&role.id=b894e00b17e34f0cbde0ab3d760e2433 => generated 260 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:53.240081 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9592b60-fd70-4369-ad1b-009fd615972f None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:53.268270 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:53.282481 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b07d4b8-4e40-4482-92fe-c4320acafad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:53.283659 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b07d4b8-4e40-4482-92fe-c4320acafad0 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:53.284105 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b07d4b8-4e40-4482-92fe-c4320acafad0 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:53.284541 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b07d4b8-4e40-4482-92fe-c4320acafad0 None admin] PATH_INFO: `/v3/groups/f684214fae064c5282cd3d9f859d81e4` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:53.292134 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b07d4b8-4e40-4482-92fe-c4320acafad0 None admin] RBAC: Authorizing `identity:get_group(group_id=f684214fae064c5282cd3d9f859d81e4)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:53.293885 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b07d4b8-4e40-4482-92fe-c4320acafad0 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:53.296195 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 201/379] 10.222.0.57 () {56 vars in 1180 bytes} [Thu May 19 14:37:53 2022] GET /identity/v3/groups/f684214fae064c5282cd3d9f859d81e4 => generated 229 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:53.301961 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796e3f06-f845-40fe-ae95-82b724a7f165 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:53.374608 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92df513-e1b8-4b37-8b2c-7904a10df261 None None] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:53.375285 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92df513-e1b8-4b37-8b2c-7904a10df261 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:53.375774 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92df513-e1b8-4b37-8b2c-7904a10df261 None None] PATH_INFO: `/` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:53.377679 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 202/380] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:37:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:37:53.383915 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5929d464-4201-4f48-af1f-8bf888be81bd None None] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:53.384568 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5929d464-4201-4f48-af1f-8bf888be81bd None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:53.385158 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5929d464-4201-4f48-af1f-8bf888be81bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:53.406992 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.auth.core [None req-5929d464-4201-4f48-af1f-8bf888be81bd None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:37:53.411481 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.fernet_utils [None req-5929d464-4201-4f48-af1f-8bf888be81bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:37:53.485682 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796e3f06-f845-40fe-ae95-82b724a7f165 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:53.500390 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 203/381] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:37:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:37:53.545945 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efe4301-8175-4980-8c68-9c47020941c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:53.546621 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efe4301-8175-4980-8c68-9c47020941c7 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:53.546912 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efe4301-8175-4980-8c68-9c47020941c7 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:53.547195 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efe4301-8175-4980-8c68-9c47020941c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:53.547592 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5929d464-4201-4f48-af1f-8bf888be81bd None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:53.548049 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efe4301-8175-4980-8c68-9c47020941c7 None admin] RBAC: Adding query filter params (group.id=f684214fae064c5282cd3d9f859d81e4, role.id=c2935b1a25ca46fcba419fe3d1715330, scope.project.id=c3cba789bd6f4fdaaa058dac80cb6fbe) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:37:53.548369 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efe4301-8175-4980-8c68-9c47020941c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:53.549830 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efe4301-8175-4980-8c68-9c47020941c7 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:53.563291 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 179/382] 10.222.0.57 () {56 vars in 1376 bytes} [Thu May 19 14:37:53 2022] GET /identity/v3/role_assignments?group.id=f684214fae064c5282cd3d9f859d81e4&scope.project.id=c3cba789bd6f4fdaaa058dac80cb6fbe&role.id=c2935b1a25ca46fcba419fe3d1715330 => generated 260 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:53.680203 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5929d464-4201-4f48-af1f-8bf888be81bd None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:53.700743 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:53.721392 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f22dfcd-0693-4b48-818b-3d58b366d268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:53.722231 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f22dfcd-0693-4b48-818b-3d58b366d268 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:53.722521 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f22dfcd-0693-4b48-818b-3d58b366d268 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:53.722801 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f22dfcd-0693-4b48-818b-3d58b366d268 None admin] PATH_INFO: `/v3/roles/member` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:53.734357 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f22dfcd-0693-4b48-818b-3d58b366d268 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:37:53.743285 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-2f22dfcd-0693-4b48-818b-3d58b366d268 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:37:53.743285 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:37:53.743285 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:37:53.743285 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:37:53.743285 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:37:53.743285 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:37:53.743285 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:37:53.743285 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:37:53.743285 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:37:53.743285 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:37:53.743285 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:37:53.743285 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:37:53.743285 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:37:53.743285 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:37:53.743285 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:37:53.743285 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:37:53.744369 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:37:53.744369 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:37:53.744369 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:37:53.744369 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:37:53.744369 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:37:53.744369 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:37:53.744369 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:37:53.744369 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:37:53.744369 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:37:53.744369 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:37:53.744369 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:37:53.744369 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:37:53.744369 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:37:53.744369 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:37:53.744369 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:37:53.744369 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:37:53.745277 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:37:53.745277 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:37:53.745277 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:37:53.745277 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:37:53.745277 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:37:53.745277 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:37:53.745277 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:37:53.745277 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:37:53.745277 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:37:53.745277 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:37:53.745277 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:37:53.745277 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:37:53.745277 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 204/383] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:37:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:37:53.752165 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efe4301-8175-4980-8c68-9c47020941c7 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:53.793533 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:53.894808 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efe4301-8175-4980-8c68-9c47020941c7 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:53.948643 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854b03c5-c800-4a18-bd6f-6b412c933f38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:53.949246 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854b03c5-c800-4a18-bd6f-6b412c933f38 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:53.949532 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854b03c5-c800-4a18-bd6f-6b412c933f38 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:53.949808 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854b03c5-c800-4a18-bd6f-6b412c933f38 None admin] PATH_INFO: `/v3/roles` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:53.950605 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854b03c5-c800-4a18-bd6f-6b412c933f38 None admin] RBAC: Adding query filter params (name=member) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:37:53.950917 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854b03c5-c800-4a18-bd6f-6b412c933f38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:53.952320 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854b03c5-c800-4a18-bd6f-6b412c933f38 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:53.966392 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 180/384] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:37:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:53.972243 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f22dfcd-0693-4b48-818b-3d58b366d268 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:54.105752 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f22dfcd-0693-4b48-818b-3d58b366d268 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:54.142050 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb7f1e0-09b7-4da3-8e02-8350767c5a34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:54.142919 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb7f1e0-09b7-4da3-8e02-8350767c5a34 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:54.143206 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb7f1e0-09b7-4da3-8e02-8350767c5a34 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:54.143489 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb7f1e0-09b7-4da3-8e02-8350767c5a34 None admin] PATH_INFO: `/v3/groups/8b0f774e11544ebba71829c939937be9` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:54.144689 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb7f1e0-09b7-4da3-8e02-8350767c5a34 None admin] RBAC: Authorizing `identity:get_group(group_id=8b0f774e11544ebba71829c939937be9)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:54.146259 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb7f1e0-09b7-4da3-8e02-8350767c5a34 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:54.148810 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 205/385] 10.222.0.57 () {56 vars in 1180 bytes} [Thu May 19 14:37:53 2022] GET /identity/v3/groups/8b0f774e11544ebba71829c939937be9 => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:54.154440 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854b03c5-c800-4a18-bd6f-6b412c933f38 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:54.268741 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:54.287253 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854b03c5-c800-4a18-bd6f-6b412c933f38 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:54.325448 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed259fec-31d9-414f-a3c3-0e831a334bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:54.326354 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed259fec-31d9-414f-a3c3-0e831a334bec None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:54.326641 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed259fec-31d9-414f-a3c3-0e831a334bec None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:54.326918 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed259fec-31d9-414f-a3c3-0e831a334bec None admin] PATH_INFO: `/v3/projects/ff0814efdfab4314800f7e0e2109a949` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:54.328132 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed259fec-31d9-414f-a3c3-0e831a334bec None admin] RBAC: Authorizing `identity:get_project(project_id=ff0814efdfab4314800f7e0e2109a949)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:54.329577 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed259fec-31d9-414f-a3c3-0e831a334bec None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:54.331923 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 181/386] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:37:54 2022] GET /identity/v3/projects/ff0814efdfab4314800f7e0e2109a949 => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:54.338485 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb7f1e0-09b7-4da3-8e02-8350767c5a34 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:54.388594 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:54.447463 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb7f1e0-09b7-4da3-8e02-8350767c5a34 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:54.475851 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60514ad7-9cc3-42c5-8043-f2ba6eb315bc None None] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:54.476189 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60514ad7-9cc3-42c5-8043-f2ba6eb315bc None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:54.476483 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60514ad7-9cc3-42c5-8043-f2ba6eb315bc None None] PATH_INFO: `/` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:54.478035 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 182/387] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:37:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:37:54.483151 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838f4de5-5fd8-43ed-9555-890318b56f85 None None] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:54.483438 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838f4de5-5fd8-43ed-9555-890318b56f85 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:54.483841 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838f4de5-5fd8-43ed-9555-890318b56f85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:54.496189 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61db6b66-1377-4633-8643-2eafa77dbbd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:54.497083 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61db6b66-1377-4633-8643-2eafa77dbbd6 None admin] REQUEST_METHOD: `PUT` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:54.497371 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61db6b66-1377-4633-8643-2eafa77dbbd6 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:54.497651 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61db6b66-1377-4633-8643-2eafa77dbbd6 None admin] PATH_INFO: `/v3/projects/ff0814efdfab4314800f7e0e2109a949/groups/8b0f774e11544ebba71829c939937be9/roles/7ebce5564f40456e94a3616961ed73ff` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:54.505476 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61db6b66-1377-4633-8643-2eafa77dbbd6 None admin] RBAC: Authorizing `identity:create_grant(project_id=ff0814efdfab4314800f7e0e2109a949, group_id=8b0f774e11544ebba71829c939937be9, role_id=7ebce5564f40456e94a3616961ed73ff)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:54.506969 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61db6b66-1377-4633-8643-2eafa77dbbd6 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:54.514031 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.auth.core [None req-838f4de5-5fd8-43ed-9555-890318b56f85 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:37:54.516412 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-838f4de5-5fd8-43ed-9555-890318b56f85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:37:54.532826 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 206/388] 10.222.0.57 () {58 vars in 1442 bytes} [Thu May 19 14:37:54 2022] PUT /identity/v3/projects/ff0814efdfab4314800f7e0e2109a949/groups/8b0f774e11544ebba71829c939937be9/roles/7ebce5564f40456e94a3616961ed73ff => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:37:54.631808 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 183/389] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:37:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:37:54.671422 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61db6b66-1377-4633-8643-2eafa77dbbd6 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:54.798250 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61db6b66-1377-4633-8643-2eafa77dbbd6 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:54.807176 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:54.853712 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e7ef7a-bf7b-40d2-935c-401c7c47a37f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:54.854578 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e7ef7a-bf7b-40d2-935c-401c7c47a37f None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:54.854867 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e7ef7a-bf7b-40d2-935c-401c7c47a37f None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:54.855149 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e7ef7a-bf7b-40d2-935c-401c7c47a37f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:54.869400 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e7ef7a-bf7b-40d2-935c-401c7c47a37f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:37:54.878227 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-e9e7ef7a-bf7b-40d2-935c-401c7c47a37f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:37:54.878227 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:37:54.878227 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:37:54.878227 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:37:54.878227 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:37:54.878227 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:37:54.878227 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:37:54.878227 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:37:54.878227 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:37:54.878227 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:37:54.878227 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:37:54.878227 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:37:54.878227 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:37:54.878227 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:37:54.878227 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:37:54.878227 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:37:54.879253 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:37:54.879253 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:37:54.879253 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:37:54.879253 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:37:54.879253 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:37:54.879253 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:37:54.879253 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:37:54.879253 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:37:54.879253 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:37:54.879253 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:37:54.879253 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:37:54.879253 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:37:54.879253 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:37:54.879253 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:37:54.879253 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:37:54.879253 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:37:54.880182 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:37:54.880182 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:37:54.880182 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:37:54.880182 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:37:54.880182 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:37:54.880182 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:37:54.880182 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:37:54.880182 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:37:54.880182 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:37:54.880182 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:37:54.880182 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:37:54.880182 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:37:54.883083 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 207/390] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 19 14:37:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:37:54.889893 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838f4de5-5fd8-43ed-9555-890318b56f85 None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:54.998376 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:55.016789 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838f4de5-5fd8-43ed-9555-890318b56f85 None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:55.065879 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a20ec07-703d-4f38-815e-5e9341b3d9d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:55.066534 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a20ec07-703d-4f38-815e-5e9341b3d9d2 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:55.066820 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a20ec07-703d-4f38-815e-5e9341b3d9d2 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:55.067099 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a20ec07-703d-4f38-815e-5e9341b3d9d2 None admin] PATH_INFO: `/v3/roles` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:55.067904 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a20ec07-703d-4f38-815e-5e9341b3d9d2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:37:55.068217 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a20ec07-703d-4f38-815e-5e9341b3d9d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:55.069605 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a20ec07-703d-4f38-815e-5e9341b3d9d2 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:55.084673 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 184/391] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:37:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:55.091407 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e7ef7a-bf7b-40d2-935c-401c7c47a37f None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:55.238892 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e7ef7a-bf7b-40d2-935c-401c7c47a37f None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:55.275398 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bde76dc-2106-40ce-af31-266afda949a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:55.276270 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bde76dc-2106-40ce-af31-266afda949a5 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:55.276554 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bde76dc-2106-40ce-af31-266afda949a5 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:55.276833 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bde76dc-2106-40ce-af31-266afda949a5 None admin] PATH_INFO: `/v3/groups/8b0f774e11544ebba71829c939937be9` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:55.278052 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bde76dc-2106-40ce-af31-266afda949a5 None admin] RBAC: Authorizing `identity:get_group(group_id=8b0f774e11544ebba71829c939937be9)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:55.279510 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bde76dc-2106-40ce-af31-266afda949a5 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:55.281455 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 208/392] 10.222.0.57 () {56 vars in 1180 bytes} [Thu May 19 14:37:55 2022] GET /identity/v3/groups/8b0f774e11544ebba71829c939937be9 => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:55.287069 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a20ec07-703d-4f38-815e-5e9341b3d9d2 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:55.432067 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a20ec07-703d-4f38-815e-5e9341b3d9d2 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:55.473638 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350e0f58-c6aa-4047-bd3a-db27ee5c9c33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:55.474539 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350e0f58-c6aa-4047-bd3a-db27ee5c9c33 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:55.474826 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350e0f58-c6aa-4047-bd3a-db27ee5c9c33 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:55.475114 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350e0f58-c6aa-4047-bd3a-db27ee5c9c33 None admin] PATH_INFO: `/v3/projects/ff0814efdfab4314800f7e0e2109a949` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:55.476356 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350e0f58-c6aa-4047-bd3a-db27ee5c9c33 None admin] RBAC: Authorizing `identity:get_project(project_id=ff0814efdfab4314800f7e0e2109a949)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:55.477810 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350e0f58-c6aa-4047-bd3a-db27ee5c9c33 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:55.479882 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 185/393] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:37:55 2022] GET /identity/v3/projects/ff0814efdfab4314800f7e0e2109a949 => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:55.485365 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bde76dc-2106-40ce-af31-266afda949a5 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:55.528788 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:55.624559 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bde76dc-2106-40ce-af31-266afda949a5 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:55.675779 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eebcdf9-ad7f-4850-a479-3f028cb0e48a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:55.677058 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eebcdf9-ad7f-4850-a479-3f028cb0e48a None admin] REQUEST_METHOD: `PUT` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:55.677563 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eebcdf9-ad7f-4850-a479-3f028cb0e48a None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:55.678073 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eebcdf9-ad7f-4850-a479-3f028cb0e48a None admin] PATH_INFO: `/v3/projects/ff0814efdfab4314800f7e0e2109a949/groups/8b0f774e11544ebba71829c939937be9/roles/b894e00b17e34f0cbde0ab3d760e2433` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:55.683734 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eebcdf9-ad7f-4850-a479-3f028cb0e48a None admin] RBAC: Authorizing `identity:create_grant(project_id=ff0814efdfab4314800f7e0e2109a949, group_id=8b0f774e11544ebba71829c939937be9, role_id=b894e00b17e34f0cbde0ab3d760e2433)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:55.685423 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eebcdf9-ad7f-4850-a479-3f028cb0e48a None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:55.704447 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 209/394] 10.222.0.57 () {58 vars in 1442 bytes} [Thu May 19 14:37:55 2022] PUT /identity/v3/projects/ff0814efdfab4314800f7e0e2109a949/groups/8b0f774e11544ebba71829c939937be9/roles/b894e00b17e34f0cbde0ab3d760e2433 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:37:55.936153 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddce1ac0-9b25-45c6-9cd2-6d8c3314d02a None None] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:55.936713 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddce1ac0-9b25-45c6-9cd2-6d8c3314d02a None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:55.936713 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddce1ac0-9b25-45c6-9cd2-6d8c3314d02a None None] PATH_INFO: `/` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:55.938524 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 186/395] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:37:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:37:55.944707 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fb7e72-6193-4cdc-9340-12607a20d426 None None] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:55.946059 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fb7e72-6193-4cdc-9340-12607a20d426 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:55.946059 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fb7e72-6193-4cdc-9340-12607a20d426 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:55.971282 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.auth.core [None req-97fb7e72-6193-4cdc-9340-12607a20d426 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:37:55.972470 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.fernet_utils [None req-97fb7e72-6193-4cdc-9340-12607a20d426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:37:56.056167 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 210/396] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:37:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:37:56.080851 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddce1ac0-9b25-45c6-9cd2-6d8c3314d02a None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:56.111051 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:56.131308 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78268062-0a3f-4d9b-8b03-2c8f9307621d None None] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:56.131597 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78268062-0a3f-4d9b-8b03-2c8f9307621d None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:56.131882 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78268062-0a3f-4d9b-8b03-2c8f9307621d None None] PATH_INFO: `/` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:56.136247 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 211/397] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:37:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:37:56.147494 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3295e59-d7ea-4ae2-839c-bec1fe474907 None None] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:56.147767 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3295e59-d7ea-4ae2-839c-bec1fe474907 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:56.148051 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3295e59-d7ea-4ae2-839c-bec1fe474907 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:56.187501 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.auth.core [None req-c3295e59-d7ea-4ae2-839c-bec1fe474907 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:37:56.190372 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.fernet_utils [None req-c3295e59-d7ea-4ae2-839c-bec1fe474907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:37:56.223818 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:56.250788 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddce1ac0-9b25-45c6-9cd2-6d8c3314d02a None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:56.316579 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eb3fd1-00e8-4499-be2d-59e473e55226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:56.317398 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eb3fd1-00e8-4499-be2d-59e473e55226 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:56.317681 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eb3fd1-00e8-4499-be2d-59e473e55226 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:56.317981 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eb3fd1-00e8-4499-be2d-59e473e55226 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:56.328183 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 212/398] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:37:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:37:56.341402 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29eb3fd1-00e8-4499-be2d-59e473e55226 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:37:56.354312 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-29eb3fd1-00e8-4499-be2d-59e473e55226 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:37:56.354312 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:37:56.354312 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:37:56.354312 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:37:56.354312 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:37:56.354312 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:37:56.354312 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:37:56.354312 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:37:56.354312 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:37:56.354312 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:37:56.354312 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:37:56.354312 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:37:56.354312 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:37:56.354312 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:37:56.354312 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:37:56.354312 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:37:56.355368 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:37:56.355368 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:37:56.355368 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:37:56.355368 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:37:56.355368 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:37:56.355368 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:37:56.355368 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:37:56.355368 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:37:56.355368 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:37:56.355368 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:37:56.355368 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:37:56.355368 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:37:56.355368 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:37:56.355368 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:37:56.355368 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:37:56.355368 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:37:56.356261 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:37:56.356261 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:37:56.356261 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:37:56.356261 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:37:56.356261 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:37:56.356261 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:37:56.356261 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:37:56.356261 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:37:56.356261 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:37:56.356261 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:37:56.356261 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:37:56.356261 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:37:56.358416 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 187/399] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 19 14:37:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:37:56.361599 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3295e59-d7ea-4ae2-839c-bec1fe474907 None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:56.392132 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eb3fd1-00e8-4499-be2d-59e473e55226 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:56.495832 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:56.538344 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eb3fd1-00e8-4499-be2d-59e473e55226 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:56.578311 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3295e59-d7ea-4ae2-839c-bec1fe474907 None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:56.578799 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79903503-2d94-4d0d-b85f-48ac658963cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:56.579659 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79903503-2d94-4d0d-b85f-48ac658963cd None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:56.579948 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79903503-2d94-4d0d-b85f-48ac658963cd None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:56.580224 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79903503-2d94-4d0d-b85f-48ac658963cd None admin] PATH_INFO: `/v3/roles/member` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:56.599826 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79903503-2d94-4d0d-b85f-48ac658963cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:37:56.610812 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-79903503-2d94-4d0d-b85f-48ac658963cd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:37:56.610812 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:37:56.610812 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:37:56.610812 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:37:56.610812 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:37:56.610812 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:37:56.610812 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:37:56.610812 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:37:56.610812 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:37:56.610812 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:37:56.610812 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:37:56.610812 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:37:56.610812 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:37:56.610812 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:37:56.610812 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:37:56.610812 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:37:56.611806 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:37:56.611806 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:37:56.611806 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:37:56.611806 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:37:56.611806 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:37:56.611806 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:37:56.611806 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:37:56.611806 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:37:56.611806 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:37:56.611806 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:37:56.611806 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:37:56.611806 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:37:56.611806 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:37:56.611806 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:37:56.611806 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:37:56.611806 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:37:56.612749 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:37:56.612749 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:37:56.612749 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:37:56.612749 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:37:56.612749 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:37:56.612749 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:37:56.612749 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:37:56.612749 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:37:56.612749 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:37:56.612749 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:37:56.612749 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:37:56.612749 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:37:56.612749 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 188/400] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:37:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:37:56.625495 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79903503-2d94-4d0d-b85f-48ac658963cd None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:56.646754 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1016392-addf-403a-bbd4-1abca2561293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:56.647438 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1016392-addf-403a-bbd4-1abca2561293 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:56.647727 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1016392-addf-403a-bbd4-1abca2561293 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:56.648006 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1016392-addf-403a-bbd4-1abca2561293 None admin] PATH_INFO: `/v3/roles` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:56.648970 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1016392-addf-403a-bbd4-1abca2561293 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:37:56.649282 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1016392-addf-403a-bbd4-1abca2561293 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:56.650715 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1016392-addf-403a-bbd4-1abca2561293 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:56.659540 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 213/401] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:37:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:56.660846 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c5e7ac-3550-415f-93c6-cd7e7244a309 None None] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:56.661138 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c5e7ac-3550-415f-93c6-cd7e7244a309 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:56.661421 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c5e7ac-3550-415f-93c6-cd7e7244a309 None None] PATH_INFO: `/` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:56.662945 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 214/402] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:37:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:37:56.667789 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c5e7ac-3550-415f-93c6-cd7e7244a309 None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:56.687807 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:56.737984 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79903503-2d94-4d0d-b85f-48ac658963cd None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:56.785698 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d498e504-43fc-465a-b618-688a7dfb071e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:56.786321 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d498e504-43fc-465a-b618-688a7dfb071e None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:56.786607 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d498e504-43fc-465a-b618-688a7dfb071e None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:56.786885 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d498e504-43fc-465a-b618-688a7dfb071e None admin] PATH_INFO: `/v3/roles` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:56.787940 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d498e504-43fc-465a-b618-688a7dfb071e None admin] RBAC: Adding query filter params (name=member) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:37:56.788254 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d498e504-43fc-465a-b618-688a7dfb071e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:56.790862 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d498e504-43fc-465a-b618-688a7dfb071e None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:56.792741 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:56.805932 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 189/403] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:37:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:56.807682 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9e0786-039b-4dfb-a07b-21f20b2b3058 None None] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:56.808189 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9e0786-039b-4dfb-a07b-21f20b2b3058 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:56.808674 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9e0786-039b-4dfb-a07b-21f20b2b3058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:56.817913 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c5e7ac-3550-415f-93c6-cd7e7244a309 None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:56.827403 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.auth.core [None req-ee9e0786-039b-4dfb-a07b-21f20b2b3058 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:37:56.828912 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-ee9e0786-039b-4dfb-a07b-21f20b2b3058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:37:56.856509 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcb9285-9466-4808-bd31-810c90f9a725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:56.857380 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcb9285-9466-4808-bd31-810c90f9a725 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:56.857717 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcb9285-9466-4808-bd31-810c90f9a725 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:56.858029 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcb9285-9466-4808-bd31-810c90f9a725 None admin] PATH_INFO: `/v3/groups/8b0f774e11544ebba71829c939937be9` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:56.859278 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcb9285-9466-4808-bd31-810c90f9a725 None admin] RBAC: Authorizing `identity:get_group(group_id=8b0f774e11544ebba71829c939937be9)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:56.860779 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcb9285-9466-4808-bd31-810c90f9a725 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:56.862821 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 215/404] 10.222.0.57 () {56 vars in 1180 bytes} [Thu May 19 14:37:56 2022] GET /identity/v3/groups/8b0f774e11544ebba71829c939937be9 => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:56.863741 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcb9285-9466-4808-bd31-810c90f9a725 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:56.906674 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 190/405] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:37:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:37:56.907880 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9e0786-039b-4dfb-a07b-21f20b2b3058 None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:56.954238 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:56.974897 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcb9285-9466-4808-bd31-810c90f9a725 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:57.014558 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb97d9e-d451-479c-b2f1-73f63e50a521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:57.015434 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb97d9e-d451-479c-b2f1-73f63e50a521 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:57.015722 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb97d9e-d451-479c-b2f1-73f63e50a521 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:57.016001 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb97d9e-d451-479c-b2f1-73f63e50a521 None admin] PATH_INFO: `/v3/groups/8b0f774e11544ebba71829c939937be9` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:57.017291 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb97d9e-d451-479c-b2f1-73f63e50a521 None admin] RBAC: Authorizing `identity:get_group(group_id=8b0f774e11544ebba71829c939937be9)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:57.018877 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb97d9e-d451-479c-b2f1-73f63e50a521 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:57.020811 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 216/406] 10.222.0.57 () {56 vars in 1180 bytes} [Thu May 19 14:37:56 2022] GET /identity/v3/groups/8b0f774e11544ebba71829c939937be9 => generated 226 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:57.021504 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb97d9e-d451-479c-b2f1-73f63e50a521 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:57.024556 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9e0786-039b-4dfb-a07b-21f20b2b3058 None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:57.051715 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:57.061168 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a443634-c3d6-4f8b-90d5-53e1eed469a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:57.062190 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a443634-c3d6-4f8b-90d5-53e1eed469a3 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:57.062537 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a443634-c3d6-4f8b-90d5-53e1eed469a3 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:57.063047 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a443634-c3d6-4f8b-90d5-53e1eed469a3 None admin] PATH_INFO: `/v3/projects/0c950d0347b741b68c581d47c8968d0d` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:57.064833 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a443634-c3d6-4f8b-90d5-53e1eed469a3 None admin] RBAC: Authorizing `identity:get_project(project_id=0c950d0347b741b68c581d47c8968d0d)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:57.066528 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a443634-c3d6-4f8b-90d5-53e1eed469a3 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:57.068642 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 191/407] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:37:56 2022] GET /identity/v3/projects/0c950d0347b741b68c581d47c8968d0d => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:57.069489 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a443634-c3d6-4f8b-90d5-53e1eed469a3 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:57.145085 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb97d9e-d451-479c-b2f1-73f63e50a521 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:57.184212 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a443634-c3d6-4f8b-90d5-53e1eed469a3 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:57.189975 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19f3d42-4699-48c0-b984-6ef6ed536a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:57.190859 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19f3d42-4699-48c0-b984-6ef6ed536a20 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:57.191143 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19f3d42-4699-48c0-b984-6ef6ed536a20 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:57.191420 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19f3d42-4699-48c0-b984-6ef6ed536a20 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:57.203955 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19f3d42-4699-48c0-b984-6ef6ed536a20 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:37:57.212873 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-b19f3d42-4699-48c0-b984-6ef6ed536a20 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:37:57.212873 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:37:57.212873 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:37:57.212873 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:37:57.212873 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:37:57.212873 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:37:57.212873 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:37:57.212873 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:37:57.212873 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:37:57.212873 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:37:57.212873 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:37:57.212873 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:37:57.212873 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:37:57.212873 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:37:57.212873 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:37:57.212873 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:37:57.213901 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:37:57.213901 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:37:57.213901 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:37:57.213901 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:37:57.213901 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:37:57.213901 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:37:57.213901 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:37:57.213901 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:37:57.213901 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:37:57.213901 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:37:57.213901 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:37:57.213901 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:37:57.213901 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:37:57.213901 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:37:57.213901 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:37:57.213901 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:37:57.214838 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:37:57.214838 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:37:57.214838 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:37:57.214838 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:37:57.214838 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:37:57.214838 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:37:57.214838 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:37:57.214838 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:37:57.214838 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:37:57.214838 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:37:57.214838 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:37:57.214838 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:37:57.214838 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 217/408] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:37:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:37:57.215584 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19f3d42-4699-48c0-b984-6ef6ed536a20 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:57.226547 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5562d93-8dc5-4f5d-9339-ade157486fca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:57.227833 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5562d93-8dc5-4f5d-9339-ade157486fca None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:57.228286 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5562d93-8dc5-4f5d-9339-ade157486fca None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:57.228779 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5562d93-8dc5-4f5d-9339-ade157486fca None admin] PATH_INFO: `/v3/projects/0c950d0347b741b68c581d47c8968d0d` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:57.229991 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:57.230456 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5562d93-8dc5-4f5d-9339-ade157486fca None admin] RBAC: Authorizing `identity:get_project(project_id=0c950d0347b741b68c581d47c8968d0d)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:57.232125 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5562d93-8dc5-4f5d-9339-ade157486fca None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:57.234133 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 192/409] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:37:57 2022] GET /identity/v3/projects/0c950d0347b741b68c581d47c8968d0d => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:57.235167 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5562d93-8dc5-4f5d-9339-ade157486fca None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:57.302256 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:57.340060 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19f3d42-4699-48c0-b984-6ef6ed536a20 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:57.345394 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5562d93-8dc5-4f5d-9339-ade157486fca None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:57.381314 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9c4626-c84b-4d93-b3ab-b10037474dfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:57.382268 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9c4626-c84b-4d93-b3ab-b10037474dfb None admin] REQUEST_METHOD: `PUT` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:57.382659 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9c4626-c84b-4d93-b3ab-b10037474dfb None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:57.383436 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9c4626-c84b-4d93-b3ab-b10037474dfb None admin] PATH_INFO: `/v3/projects/0c950d0347b741b68c581d47c8968d0d/groups/8b0f774e11544ebba71829c939937be9/roles/b894e00b17e34f0cbde0ab3d760e2433` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:57.386021 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9c4626-c84b-4d93-b3ab-b10037474dfb None admin] RBAC: Authorizing `identity:create_grant(project_id=0c950d0347b741b68c581d47c8968d0d, group_id=8b0f774e11544ebba71829c939937be9, role_id=b894e00b17e34f0cbde0ab3d760e2433)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:57.386986 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89b6c3d-eb39-40a0-930a-126ea38ef54a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:57.387506 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9c4626-c84b-4d93-b3ab-b10037474dfb None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:57.388138 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89b6c3d-eb39-40a0-930a-126ea38ef54a None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:57.388639 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89b6c3d-eb39-40a0-930a-126ea38ef54a None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:57.389139 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89b6c3d-eb39-40a0-930a-126ea38ef54a None admin] PATH_INFO: `/v3/roles` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:57.390150 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89b6c3d-eb39-40a0-930a-126ea38ef54a None admin] RBAC: Adding query filter params (name=admin) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:37:57.390701 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89b6c3d-eb39-40a0-930a-126ea38ef54a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:57.392580 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89b6c3d-eb39-40a0-930a-126ea38ef54a None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:57.402182 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 193/410] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:37:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:57.403208 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89b6c3d-eb39-40a0-930a-126ea38ef54a None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:57.407475 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 218/411] 10.222.0.57 () {58 vars in 1442 bytes} [Thu May 19 14:37:57 2022] PUT /identity/v3/projects/0c950d0347b741b68c581d47c8968d0d/groups/8b0f774e11544ebba71829c939937be9/roles/b894e00b17e34f0cbde0ab3d760e2433 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:37:57.408650 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9c4626-c84b-4d93-b3ab-b10037474dfb None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:57.497650 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:57.538297 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9c4626-c84b-4d93-b3ab-b10037474dfb None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:57.579442 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde3214e-4f0c-4518-a1fa-9bb183fc4f9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:57.580670 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde3214e-4f0c-4518-a1fa-9bb183fc4f9d None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:57.581163 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde3214e-4f0c-4518-a1fa-9bb183fc4f9d None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:57.581605 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde3214e-4f0c-4518-a1fa-9bb183fc4f9d None admin] PATH_INFO: `/v3/groups/f684214fae064c5282cd3d9f859d81e4` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:57.583121 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde3214e-4f0c-4518-a1fa-9bb183fc4f9d None admin] RBAC: Authorizing `identity:get_group(group_id=f684214fae064c5282cd3d9f859d81e4)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:57.585099 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde3214e-4f0c-4518-a1fa-9bb183fc4f9d None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:57.587374 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 219/412] 10.222.0.57 () {56 vars in 1180 bytes} [Thu May 19 14:37:57 2022] GET /identity/v3/groups/f684214fae064c5282cd3d9f859d81e4 => generated 229 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:57.597561 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde3214e-4f0c-4518-a1fa-9bb183fc4f9d None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:57.631872 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89b6c3d-eb39-40a0-930a-126ea38ef54a None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:57.658985 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:57.671755 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f921d7-bba5-4b2a-8367-8d605df0896a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:57.672644 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f921d7-bba5-4b2a-8367-8d605df0896a None admin] REQUEST_METHOD: `PUT` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:57.672927 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f921d7-bba5-4b2a-8367-8d605df0896a None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:57.673203 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f921d7-bba5-4b2a-8367-8d605df0896a None admin] PATH_INFO: `/v3/projects/0c950d0347b741b68c581d47c8968d0d/groups/8b0f774e11544ebba71829c939937be9/roles/7ebce5564f40456e94a3616961ed73ff` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:57.675349 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f921d7-bba5-4b2a-8367-8d605df0896a None admin] RBAC: Authorizing `identity:create_grant(project_id=0c950d0347b741b68c581d47c8968d0d, group_id=8b0f774e11544ebba71829c939937be9, role_id=7ebce5564f40456e94a3616961ed73ff)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:57.676790 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f921d7-bba5-4b2a-8367-8d605df0896a None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:57.692039 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 194/413] 10.222.0.57 () {58 vars in 1442 bytes} [Thu May 19 14:37:57 2022] PUT /identity/v3/projects/0c950d0347b741b68c581d47c8968d0d/groups/8b0f774e11544ebba71829c939937be9/roles/7ebce5564f40456e94a3616961ed73ff => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:37:57.693394 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954b01f6-21d2-4e80-b182-63ebb5c0f6d2 None None] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:57.693686 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954b01f6-21d2-4e80-b182-63ebb5c0f6d2 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:57.697899 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954b01f6-21d2-4e80-b182-63ebb5c0f6d2 None None] PATH_INFO: `/` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:57.697899 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 195/414] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:37:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:37:57.703008 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d900a3f3-5d85-4a3a-8a9c-db5477af692b None None] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:57.703344 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d900a3f3-5d85-4a3a-8a9c-db5477af692b None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:57.703838 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d900a3f3-5d85-4a3a-8a9c-db5477af692b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:57.739714 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.auth.core [None req-d900a3f3-5d85-4a3a-8a9c-db5477af692b None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:37:57.744909 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-d900a3f3-5d85-4a3a-8a9c-db5477af692b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:37:57.763716 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde3214e-4f0c-4518-a1fa-9bb183fc4f9d None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:57.807334 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfd3288-b895-4e2a-ac85-358e1f330931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:57.808221 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfd3288-b895-4e2a-ac85-358e1f330931 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:57.808502 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfd3288-b895-4e2a-ac85-358e1f330931 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:57.808776 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfd3288-b895-4e2a-ac85-358e1f330931 None admin] PATH_INFO: `/v3/projects/c3cba789bd6f4fdaaa058dac80cb6fbe` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:57.810038 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfd3288-b895-4e2a-ac85-358e1f330931 None admin] RBAC: Authorizing `identity:get_project(project_id=c3cba789bd6f4fdaaa058dac80cb6fbe)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:57.812291 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfd3288-b895-4e2a-ac85-358e1f330931 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:57.815393 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 220/415] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:37:57 2022] GET /identity/v3/projects/c3cba789bd6f4fdaaa058dac80cb6fbe => generated 344 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:37:57.821542 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfd3288-b895-4e2a-ac85-358e1f330931 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:57.846486 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:57.871412 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 196/416] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:37:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:37:57.926827 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d900a3f3-5d85-4a3a-8a9c-db5477af692b None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:57.983830 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfd3288-b895-4e2a-ac85-358e1f330931 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:58.023831 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0fa25a-dbc0-480e-acf6-80a67d8645b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:58.025349 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0fa25a-dbc0-480e-acf6-80a67d8645b3 None admin] REQUEST_METHOD: `PUT` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:58.025904 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0fa25a-dbc0-480e-acf6-80a67d8645b3 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:58.026405 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0fa25a-dbc0-480e-acf6-80a67d8645b3 None admin] PATH_INFO: `/v3/projects/c3cba789bd6f4fdaaa058dac80cb6fbe/groups/f684214fae064c5282cd3d9f859d81e4/roles/c2935b1a25ca46fcba419fe3d1715330` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:58.028789 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0fa25a-dbc0-480e-acf6-80a67d8645b3 None admin] RBAC: Authorizing `identity:create_grant(project_id=c3cba789bd6f4fdaaa058dac80cb6fbe, group_id=f684214fae064c5282cd3d9f859d81e4, role_id=c2935b1a25ca46fcba419fe3d1715330)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:58.030558 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0fa25a-dbc0-480e-acf6-80a67d8645b3 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:58.043152 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d900a3f3-5d85-4a3a-8a9c-db5477af692b None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:58.052896 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 221/417] 10.222.0.57 () {58 vars in 1442 bytes} [Thu May 19 14:37:57 2022] PUT /identity/v3/projects/c3cba789bd6f4fdaaa058dac80cb6fbe/groups/f684214fae064c5282cd3d9f859d81e4/roles/c2935b1a25ca46fcba419fe3d1715330 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 14:37:58.064114 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:58.090263 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21503f32-34f1-4856-aab1-b809c83b7c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:58.091079 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21503f32-34f1-4856-aab1-b809c83b7c97 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:58.091359 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21503f32-34f1-4856-aab1-b809c83b7c97 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:58.091631 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21503f32-34f1-4856-aab1-b809c83b7c97 None admin] PATH_INFO: `/v3/roles/member` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:58.104558 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21503f32-34f1-4856-aab1-b809c83b7c97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:37:58.113650 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-21503f32-34f1-4856-aab1-b809c83b7c97 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:37:58.113650 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:37:58.113650 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:37:58.113650 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:37:58.113650 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:37:58.113650 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:37:58.113650 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:37:58.113650 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:37:58.113650 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:37:58.113650 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:37:58.113650 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:37:58.113650 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:37:58.113650 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:37:58.113650 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:37:58.113650 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:37:58.113650 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:37:58.114765 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:37:58.114765 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:37:58.114765 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:37:58.114765 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:37:58.114765 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:37:58.114765 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:37:58.114765 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:37:58.114765 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:37:58.114765 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:37:58.114765 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:37:58.114765 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:37:58.114765 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:37:58.114765 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:37:58.114765 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:37:58.114765 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:37:58.114765 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:37:58.115678 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:37:58.115678 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:37:58.115678 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:37:58.115678 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:37:58.115678 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:37:58.115678 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:37:58.115678 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:37:58.115678 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:37:58.115678 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:37:58.115678 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:37:58.115678 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:37:58.115678 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:37:58.119120 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 197/418] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:37:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:37:58.123405 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0fa25a-dbc0-480e-acf6-80a67d8645b3 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:58.227445 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:58.280633 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0fa25a-dbc0-480e-acf6-80a67d8645b3 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:58.326878 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c5d681-8f99-4c6b-8fb1-2a1da24db1a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:58.327571 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c5d681-8f99-4c6b-8fb1-2a1da24db1a5 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:58.327854 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c5d681-8f99-4c6b-8fb1-2a1da24db1a5 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:58.328131 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c5d681-8f99-4c6b-8fb1-2a1da24db1a5 None admin] PATH_INFO: `/v3/roles` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:58.328893 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c5d681-8f99-4c6b-8fb1-2a1da24db1a5 None admin] RBAC: Adding query filter params (name=member) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:37:58.329197 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c5d681-8f99-4c6b-8fb1-2a1da24db1a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:58.330643 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c5d681-8f99-4c6b-8fb1-2a1da24db1a5 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:58.346384 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 222/419] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:37:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:58.352372 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21503f32-34f1-4856-aab1-b809c83b7c97 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:58.466069 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21503f32-34f1-4856-aab1-b809c83b7c97 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:58.477756 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f4de5e-fb43-46b5-b942-56596879d4cd None None] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:58.478113 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f4de5e-fb43-46b5-b942-56596879d4cd None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:58.478412 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f4de5e-fb43-46b5-b942-56596879d4cd None None] PATH_INFO: `/` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:58.480040 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 223/420] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:37:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:37:58.485333 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606ee5ce-a16f-4764-ad83-4dc8de9a4bd7 None None] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:58.485629 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606ee5ce-a16f-4764-ad83-4dc8de9a4bd7 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:58.485945 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606ee5ce-a16f-4764-ad83-4dc8de9a4bd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:58.506678 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.auth.core [None req-606ee5ce-a16f-4764-ad83-4dc8de9a4bd7 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:37:58.508559 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.fernet_utils [None req-606ee5ce-a16f-4764-ad83-4dc8de9a4bd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:37:58.509955 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5932251-0b75-4591-a1c8-b6b2d99f3d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:58.510718 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5932251-0b75-4591-a1c8-b6b2d99f3d65 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:58.511006 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5932251-0b75-4591-a1c8-b6b2d99f3d65 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:58.511303 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5932251-0b75-4591-a1c8-b6b2d99f3d65 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:58.560224 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:58.582267 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 224/421] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:37:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:37:58.590988 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5932251-0b75-4591-a1c8-b6b2d99f3d65 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:58.593562 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5932251-0b75-4591-a1c8-b6b2d99f3d65 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:58.606718 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606ee5ce-a16f-4764-ad83-4dc8de9a4bd7 None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:58.668328 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:58.716285 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606ee5ce-a16f-4764-ad83-4dc8de9a4bd7 None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:58.716626 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 198/422] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:37:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:37:58.720319 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5932251-0b75-4591-a1c8-b6b2d99f3d65 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:58.753672 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d3bbff-df43-4832-a288-3485b5b9bf68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:58.754520 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d3bbff-df43-4832-a288-3485b5b9bf68 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:58.754806 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d3bbff-df43-4832-a288-3485b5b9bf68 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:58.755088 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d3bbff-df43-4832-a288-3485b5b9bf68 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:58.766374 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d3bbff-df43-4832-a288-3485b5b9bf68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:37:58.775935 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-a0d3bbff-df43-4832-a288-3485b5b9bf68 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:37:58.775935 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:37:58.775935 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:37:58.775935 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:37:58.775935 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:37:58.775935 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:37:58.775935 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:37:58.775935 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:37:58.775935 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:37:58.775935 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:37:58.775935 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:37:58.775935 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:37:58.775935 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:37:58.775935 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:37:58.775935 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:37:58.775935 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:37:58.776989 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:37:58.776989 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:37:58.776989 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:37:58.776989 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:37:58.776989 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:37:58.776989 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:37:58.776989 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:37:58.776989 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:37:58.776989 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:37:58.776989 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:37:58.776989 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:37:58.776989 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:37:58.776989 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:37:58.776989 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:37:58.776989 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:37:58.776989 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:37:58.778165 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:37:58.778165 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:37:58.778165 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:37:58.778165 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:37:58.778165 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:37:58.778165 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:37:58.778165 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:37:58.778165 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:37:58.778165 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:37:58.778165 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:37:58.778165 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:37:58.778165 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:37:58.778165 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 225/423] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 19 14:37:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:37:58.783949 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d3bbff-df43-4832-a288-3485b5b9bf68 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:58.814159 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:58.843019 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5932251-0b75-4591-a1c8-b6b2d99f3d65 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:58.879372 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9bbb15-cbbb-4449-ad0c-9b1873f6d8c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:58.880602 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9bbb15-cbbb-4449-ad0c-9b1873f6d8c8 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:58.881083 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9bbb15-cbbb-4449-ad0c-9b1873f6d8c8 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:58.881568 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9bbb15-cbbb-4449-ad0c-9b1873f6d8c8 None admin] PATH_INFO: `/v3/projects/ff0814efdfab4314800f7e0e2109a949` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:58.883591 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9bbb15-cbbb-4449-ad0c-9b1873f6d8c8 None admin] RBAC: Authorizing `identity:get_project(project_id=ff0814efdfab4314800f7e0e2109a949)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:58.885503 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9bbb15-cbbb-4449-ad0c-9b1873f6d8c8 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:58.891605 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 199/424] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:37:58 2022] GET /identity/v3/projects/ff0814efdfab4314800f7e0e2109a949 => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:58.893998 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9bbb15-cbbb-4449-ad0c-9b1873f6d8c8 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:58.904462 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d3bbff-df43-4832-a288-3485b5b9bf68 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:58.927545 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:58.948580 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc7c95e-9af8-437f-b6f7-ac1f42a96157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:58.949262 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc7c95e-9af8-437f-b6f7-ac1f42a96157 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:58.949543 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc7c95e-9af8-437f-b6f7-ac1f42a96157 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:58.949818 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc7c95e-9af8-437f-b6f7-ac1f42a96157 None admin] PATH_INFO: `/v3/roles` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:58.950601 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc7c95e-9af8-437f-b6f7-ac1f42a96157 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:37:58.950909 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc7c95e-9af8-437f-b6f7-ac1f42a96157 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:58.952296 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc7c95e-9af8-437f-b6f7-ac1f42a96157 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:58.962730 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 226/425] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:37:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:58.969109 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc7c95e-9af8-437f-b6f7-ac1f42a96157 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:59.092597 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9bbb15-cbbb-4449-ad0c-9b1873f6d8c8 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:59.097800 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc7c95e-9af8-437f-b6f7-ac1f42a96157 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:59.138673 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1103048e-db7f-4c8f-b23e-46529faed0e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:59.139330 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1103048e-db7f-4c8f-b23e-46529faed0e2 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:59.139611 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1103048e-db7f-4c8f-b23e-46529faed0e2 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:59.139886 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1103048e-db7f-4c8f-b23e-46529faed0e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:59.186638 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71779473-6109-4f0d-81fb-989b3648c11d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:59.187654 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71779473-6109-4f0d-81fb-989b3648c11d None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:59.187951 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71779473-6109-4f0d-81fb-989b3648c11d None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:59.188234 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71779473-6109-4f0d-81fb-989b3648c11d None admin] PATH_INFO: `/v3/groups/8b0f774e11544ebba71829c939937be9` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:59.189571 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71779473-6109-4f0d-81fb-989b3648c11d None admin] RBAC: Authorizing `identity:get_group(group_id=8b0f774e11544ebba71829c939937be9)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:59.191171 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71779473-6109-4f0d-81fb-989b3648c11d None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:59.193273 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 200/426] 10.222.0.57 () {56 vars in 1180 bytes} [Thu May 19 14:37:58 2022] GET /identity/v3/groups/8b0f774e11544ebba71829c939937be9 => generated 226 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:37:59.202577 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:59.215970 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71779473-6109-4f0d-81fb-989b3648c11d None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:59.225207 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1103048e-db7f-4c8f-b23e-46529faed0e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:59.227662 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1103048e-db7f-4c8f-b23e-46529faed0e2 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:59.240657 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:59.351131 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 227/427] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:37:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 14:37:59.357134 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1103048e-db7f-4c8f-b23e-46529faed0e2 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:59.414452 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71779473-6109-4f0d-81fb-989b3648c11d None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:59.481723 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b3df45-b722-4328-aa2a-91fe344cd2d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:59.482330 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b3df45-b722-4328-aa2a-91fe344cd2d3 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:59.482617 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b3df45-b722-4328-aa2a-91fe344cd2d3 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:59.482896 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b3df45-b722-4328-aa2a-91fe344cd2d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:59.483739 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b3df45-b722-4328-aa2a-91fe344cd2d3 None admin] RBAC: Adding query filter params (group.id=8b0f774e11544ebba71829c939937be9, role.id=7ebce5564f40456e94a3616961ed73ff, scope.project.id=ff0814efdfab4314800f7e0e2109a949) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:37:59.484053 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b3df45-b722-4328-aa2a-91fe344cd2d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:59.485760 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b3df45-b722-4328-aa2a-91fe344cd2d3 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:59.496688 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:59.508541 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 201/428] 10.222.0.57 () {56 vars in 1376 bytes} [Thu May 19 14:37:59 2022] GET /identity/v3/role_assignments?group.id=8b0f774e11544ebba71829c939937be9&scope.project.id=ff0814efdfab4314800f7e0e2109a949&role.id=7ebce5564f40456e94a3616961ed73ff => generated 611 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:59.516874 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1103048e-db7f-4c8f-b23e-46529faed0e2 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:59.556409 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ce2ba0-37a6-46fa-9e1f-374b87ef7af0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:59.557295 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ce2ba0-37a6-46fa-9e1f-374b87ef7af0 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:59.557573 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ce2ba0-37a6-46fa-9e1f-374b87ef7af0 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:59.557861 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ce2ba0-37a6-46fa-9e1f-374b87ef7af0 None admin] PATH_INFO: `/v3/projects/ff0814efdfab4314800f7e0e2109a949` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:59.559055 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ce2ba0-37a6-46fa-9e1f-374b87ef7af0 None admin] RBAC: Authorizing `identity:get_project(project_id=ff0814efdfab4314800f7e0e2109a949)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:59.560565 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ce2ba0-37a6-46fa-9e1f-374b87ef7af0 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:59.562680 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 228/429] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:37:59 2022] GET /identity/v3/projects/ff0814efdfab4314800f7e0e2109a949 => generated 298 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:37:59.568112 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b3df45-b722-4328-aa2a-91fe344cd2d3 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:59.691235 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:59.733206 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b3df45-b722-4328-aa2a-91fe344cd2d3 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:59.779483 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb7f946-d3fb-4bbd-92ce-5332db7479dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:59.780355 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb7f946-d3fb-4bbd-92ce-5332db7479dd None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:59.780640 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb7f946-d3fb-4bbd-92ce-5332db7479dd None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:59.780913 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb7f946-d3fb-4bbd-92ce-5332db7479dd None admin] PATH_INFO: `/v3/groups/8b0f774e11544ebba71829c939937be9` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:59.786672 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb7f946-d3fb-4bbd-92ce-5332db7479dd None admin] RBAC: Authorizing `identity:get_group(group_id=8b0f774e11544ebba71829c939937be9)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:59.788149 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb7f946-d3fb-4bbd-92ce-5332db7479dd None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:59.790779 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 202/430] 10.222.0.57 () {56 vars in 1180 bytes} [Thu May 19 14:37:59 2022] GET /identity/v3/groups/8b0f774e11544ebba71829c939937be9 => generated 226 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:59.796444 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ce2ba0-37a6-46fa-9e1f-374b87ef7af0 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:59.943425 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ce2ba0-37a6-46fa-9e1f-374b87ef7af0 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:59.983977 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52fcb51-899e-44ef-b851-665a3d100956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:59.985021 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52fcb51-899e-44ef-b851-665a3d100956 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:59.985546 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52fcb51-899e-44ef-b851-665a3d100956 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:59.986076 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52fcb51-899e-44ef-b851-665a3d100956 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:59.987134 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52fcb51-899e-44ef-b851-665a3d100956 None admin] RBAC: Adding query filter params (group.id=8b0f774e11544ebba71829c939937be9, role.id=b894e00b17e34f0cbde0ab3d760e2433, scope.project.id=ff0814efdfab4314800f7e0e2109a949) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:37:59.987682 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52fcb51-899e-44ef-b851-665a3d100956 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:59.989335 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52fcb51-899e-44ef-b851-665a3d100956 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:59.996582 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 229/431] 10.222.0.57 () {56 vars in 1376 bytes} [Thu May 19 14:37:59 2022] GET /identity/v3/role_assignments?group.id=8b0f774e11544ebba71829c939937be9&scope.project.id=ff0814efdfab4314800f7e0e2109a949&role.id=b894e00b17e34f0cbde0ab3d760e2433 => generated 611 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:00.370509 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed53139-344b-4f2a-b8fe-1be7a85b466f None None] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:00.371507 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed53139-344b-4f2a-b8fe-1be7a85b466f None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:00.372051 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed53139-344b-4f2a-b8fe-1be7a85b466f None None] PATH_INFO: `/` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:00.373874 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 203/432] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:38:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:38:00.379733 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3a6e6b-8df5-47e6-9224-e2ffdcf762c0 None None] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:00.380652 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3a6e6b-8df5-47e6-9224-e2ffdcf762c0 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:00.381155 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3a6e6b-8df5-47e6-9224-e2ffdcf762c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:00.399857 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.auth.core [None req-ba3a6e6b-8df5-47e6-9224-e2ffdcf762c0 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:38:00.401068 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.fernet_utils [None req-ba3a6e6b-8df5-47e6-9224-e2ffdcf762c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:38:00.411514 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:00.427280 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836311c8-9942-4a45-8d32-8473720b9e7d None None] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:00.427601 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836311c8-9942-4a45-8d32-8473720b9e7d None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:00.427856 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836311c8-9942-4a45-8d32-8473720b9e7d None None] PATH_INFO: `/` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:00.430267 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 204/433] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:38:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:38:00.436931 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dde913-d02c-4e45-9709-58d54a60d535 None None] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:00.437226 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dde913-d02c-4e45-9709-58d54a60d535 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:00.437507 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dde913-d02c-4e45-9709-58d54a60d535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:00.462969 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.auth.core [None req-b7dde913-d02c-4e45-9709-58d54a60d535 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:38:00.463358 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-b7dde913-d02c-4e45-9709-58d54a60d535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:38:00.490876 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 230/434] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:38:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:38:00.515330 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3a6e6b-8df5-47e6-9224-e2ffdcf762c0 None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:00.547209 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 205/435] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:38:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:38:00.571566 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dde913-d02c-4e45-9709-58d54a60d535 None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:00.651612 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3a6e6b-8df5-47e6-9224-e2ffdcf762c0 None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:00.666474 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:00.699791 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d65eeb4-aeac-4cdd-9b02-20278fa31a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:00.700730 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d65eeb4-aeac-4cdd-9b02-20278fa31a29 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:00.701135 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d65eeb4-aeac-4cdd-9b02-20278fa31a29 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:00.701418 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d65eeb4-aeac-4cdd-9b02-20278fa31a29 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:00.713936 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dde913-d02c-4e45-9709-58d54a60d535 None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:00.716954 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d65eeb4-aeac-4cdd-9b02-20278fa31a29 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:38:00.725870 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-7d65eeb4-aeac-4cdd-9b02-20278fa31a29 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:38:00.725870 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:00.725870 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:00.725870 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:00.725870 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:00.725870 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:00.725870 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:00.725870 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:00.725870 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:00.725870 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:00.725870 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:00.725870 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:00.725870 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:38:00.725870 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:38:00.725870 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:38:00.725870 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:38:00.727422 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:38:00.727422 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:38:00.727422 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:38:00.727422 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:38:00.727422 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:00.727422 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:00.727422 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:00.727422 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:00.727422 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:00.727422 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:00.727422 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:00.727422 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:38:00.727422 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:00.727422 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:00.727422 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:00.727422 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:00.728325 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:00.728325 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:00.728325 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:00.728325 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:00.728325 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:38:00.728325 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:38:00.728325 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:38:00.728325 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:38:00.728325 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:38:00.728325 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:38:00.728325 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:38:00.728325 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:38:00.728325 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 231/436] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 19 14:38:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:38:00.735636 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d65eeb4-aeac-4cdd-9b02-20278fa31a29 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:00.738566 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:00.755759 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3147c4-69e2-4185-a2cc-162efdeb1f2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:00.756635 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3147c4-69e2-4185-a2cc-162efdeb1f2a None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:00.756952 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3147c4-69e2-4185-a2cc-162efdeb1f2a None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:00.757235 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3147c4-69e2-4185-a2cc-162efdeb1f2a None admin] PATH_INFO: `/v3/roles/member` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:00.771319 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3147c4-69e2-4185-a2cc-162efdeb1f2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:38:00.782258 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-fa3147c4-69e2-4185-a2cc-162efdeb1f2a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:38:00.782258 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:00.782258 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:00.782258 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:00.782258 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:00.782258 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:00.782258 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:00.782258 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:00.782258 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:00.782258 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:00.782258 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:00.782258 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:00.782258 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:38:00.782258 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:38:00.782258 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:38:00.782258 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:38:00.783359 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:38:00.783359 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:38:00.783359 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:38:00.783359 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:38:00.783359 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:00.783359 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:00.783359 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:00.783359 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:00.783359 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:00.783359 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:00.783359 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:00.783359 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:38:00.783359 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:00.783359 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:00.783359 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:00.783359 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:00.784360 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:00.784360 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:00.784360 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:00.784360 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:00.784360 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:38:00.784360 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:38:00.784360 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:38:00.784360 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:38:00.784360 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:38:00.784360 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:38:00.784360 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:38:00.784360 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:38:00.784360 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 206/437] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:38:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:00.789684 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3147c4-69e2-4185-a2cc-162efdeb1f2a None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:00.856039 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d65eeb4-aeac-4cdd-9b02-20278fa31a29 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:00.895322 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f7617f-d6ee-4f62-9b50-d2aa190b57fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:00.895996 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f7617f-d6ee-4f62-9b50-d2aa190b57fa None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:00.896286 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f7617f-d6ee-4f62-9b50-d2aa190b57fa None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:00.896564 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f7617f-d6ee-4f62-9b50-d2aa190b57fa None admin] PATH_INFO: `/v3/roles` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:00.897319 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f7617f-d6ee-4f62-9b50-d2aa190b57fa None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:00.897632 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f7617f-d6ee-4f62-9b50-d2aa190b57fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:00.899355 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f7617f-d6ee-4f62-9b50-d2aa190b57fa None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:00.907011 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3147c4-69e2-4185-a2cc-162efdeb1f2a None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:00.908015 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 232/438] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:38:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:38:00.913411 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f7617f-d6ee-4f62-9b50-d2aa190b57fa None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:00.940845 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:00.949083 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dd330a-e6cc-4dde-99c3-fb74c470d277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:00.950429 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dd330a-e6cc-4dde-99c3-fb74c470d277 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:00.950561 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dd330a-e6cc-4dde-99c3-fb74c470d277 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:00.950805 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dd330a-e6cc-4dde-99c3-fb74c470d277 None admin] PATH_INFO: `/v3/roles` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:00.951655 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dd330a-e6cc-4dde-99c3-fb74c470d277 None admin] RBAC: Adding query filter params (name=member) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:00.951973 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dd330a-e6cc-4dde-99c3-fb74c470d277 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:00.953381 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dd330a-e6cc-4dde-99c3-fb74c470d277 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:00.962719 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 207/439] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:38:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:38:00.964005 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd5ec3b-07bd-4de8-b573-bd3402c05873 None None] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:00.964298 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd5ec3b-07bd-4de8-b573-bd3402c05873 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:00.964577 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd5ec3b-07bd-4de8-b573-bd3402c05873 None None] PATH_INFO: `/` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:00.966027 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 208/440] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:38:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:38:00.968374 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd5ec3b-07bd-4de8-b573-bd3402c05873 None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:00.984988 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:01.027104 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f7617f-d6ee-4f62-9b50-d2aa190b57fa None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:01.061703 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c639160f-a344-4154-884c-c8eaa099d543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:01.062389 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c639160f-a344-4154-884c-c8eaa099d543 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:01.062676 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c639160f-a344-4154-884c-c8eaa099d543 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:01.062957 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c639160f-a344-4154-884c-c8eaa099d543 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:01.073112 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd5ec3b-07bd-4de8-b573-bd3402c05873 None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:01.107311 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd56c7b5-89d9-4ca6-8c57-ef48c879a34b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:01.107950 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd56c7b5-89d9-4ca6-8c57-ef48c879a34b None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:01.108231 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd56c7b5-89d9-4ca6-8c57-ef48c879a34b None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:01.108504 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd56c7b5-89d9-4ca6-8c57-ef48c879a34b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:01.132001 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c639160f-a344-4154-884c-c8eaa099d543 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:01.134391 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c639160f-a344-4154-884c-c8eaa099d543 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:01.171303 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:01.180867 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd56c7b5-89d9-4ca6-8c57-ef48c879a34b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:01.182360 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd56c7b5-89d9-4ca6-8c57-ef48c879a34b None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:01.197990 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:01.238005 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 233/441] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:38:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 14:38:01.239588 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ca94d3-1f28-482c-a3aa-485b2021c3b9 None None] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:01.239888 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ca94d3-1f28-482c-a3aa-485b2021c3b9 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:01.240168 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ca94d3-1f28-482c-a3aa-485b2021c3b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:01.256725 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.auth.core [None req-c1ca94d3-1f28-482c-a3aa-485b2021c3b9 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:38:01.258724 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.fernet_utils [None req-c1ca94d3-1f28-482c-a3aa-485b2021c3b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:38:01.285614 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 209/442] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:38:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 14:38:01.285614 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd56c7b5-89d9-4ca6-8c57-ef48c879a34b None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:01.327128 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 234/443] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:38:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:38:01.327979 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ca94d3-1f28-482c-a3aa-485b2021c3b9 None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:01.384454 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:01.391450 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd56c7b5-89d9-4ca6-8c57-ef48c879a34b None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:01.425720 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327fa054-2a31-4aeb-abc9-f7dc260ae39d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:01.426530 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327fa054-2a31-4aeb-abc9-f7dc260ae39d None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:01.426814 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327fa054-2a31-4aeb-abc9-f7dc260ae39d None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:01.427092 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327fa054-2a31-4aeb-abc9-f7dc260ae39d None admin] PATH_INFO: `/v3/projects/0c950d0347b741b68c581d47c8968d0d` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:01.427485 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:01.428752 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327fa054-2a31-4aeb-abc9-f7dc260ae39d None admin] RBAC: Authorizing `identity:get_project(project_id=0c950d0347b741b68c581d47c8968d0d)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:01.430938 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327fa054-2a31-4aeb-abc9-f7dc260ae39d None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:01.432841 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 210/444] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:38:01 2022] GET /identity/v3/projects/0c950d0347b741b68c581d47c8968d0d => generated 302 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:01.433445 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327fa054-2a31-4aeb-abc9-f7dc260ae39d None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:01.435899 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ca94d3-1f28-482c-a3aa-485b2021c3b9 None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:01.470320 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38827351-23ec-4909-a660-8232a4a9ed79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:01.471176 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38827351-23ec-4909-a660-8232a4a9ed79 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:01.471460 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38827351-23ec-4909-a660-8232a4a9ed79 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:01.471738 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38827351-23ec-4909-a660-8232a4a9ed79 None admin] PATH_INFO: `/v3/projects/0c950d0347b741b68c581d47c8968d0d` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:01.472932 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38827351-23ec-4909-a660-8232a4a9ed79 None admin] RBAC: Authorizing `identity:get_project(project_id=0c950d0347b741b68c581d47c8968d0d)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:01.474439 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38827351-23ec-4909-a660-8232a4a9ed79 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:01.476797 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 235/445] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:38:01 2022] GET /identity/v3/projects/0c950d0347b741b68c581d47c8968d0d => generated 302 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:01.477582 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38827351-23ec-4909-a660-8232a4a9ed79 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:01.533337 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327fa054-2a31-4aeb-abc9-f7dc260ae39d None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:01.568011 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d4dd4b-d9db-44c1-9d3f-91d79898f57a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:01.568796 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d4dd4b-d9db-44c1-9d3f-91d79898f57a None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:01.569075 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d4dd4b-d9db-44c1-9d3f-91d79898f57a None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:01.569346 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d4dd4b-d9db-44c1-9d3f-91d79898f57a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:01.577427 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38827351-23ec-4909-a660-8232a4a9ed79 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:01.579635 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d4dd4b-d9db-44c1-9d3f-91d79898f57a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:38:01.587467 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-d8d4dd4b-d9db-44c1-9d3f-91d79898f57a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:38:01.587467 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:01.587467 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:01.587467 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:01.587467 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:01.587467 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:01.587467 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:01.587467 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:01.587467 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:01.587467 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:01.587467 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:01.587467 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:01.587467 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:38:01.587467 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:38:01.587467 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:38:01.587467 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:38:01.588686 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:38:01.588686 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:38:01.588686 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:38:01.588686 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:38:01.588686 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:01.588686 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:01.588686 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:01.588686 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:01.588686 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:01.588686 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:01.588686 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:01.588686 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:38:01.588686 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:01.588686 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:01.588686 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:01.588686 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:01.590257 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:01.590257 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:01.590257 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:01.590257 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:01.590257 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:38:01.590257 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:38:01.590257 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:38:01.590257 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:38:01.590257 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:38:01.590257 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:38:01.590257 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:38:01.590257 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:38:01.590257 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 211/446] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:38:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:01.590257 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d4dd4b-d9db-44c1-9d3f-91d79898f57a None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:01.615650 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:01.616461 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90a379d-a5d8-4f4e-9aab-50935d97800c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:01.618026 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90a379d-a5d8-4f4e-9aab-50935d97800c None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:01.618642 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90a379d-a5d8-4f4e-9aab-50935d97800c None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:01.618761 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90a379d-a5d8-4f4e-9aab-50935d97800c None admin] PATH_INFO: `/v3/groups/8b0f774e11544ebba71829c939937be9` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:01.619984 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90a379d-a5d8-4f4e-9aab-50935d97800c None admin] RBAC: Authorizing `identity:get_group(group_id=8b0f774e11544ebba71829c939937be9)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:01.621442 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90a379d-a5d8-4f4e-9aab-50935d97800c None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:01.623128 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 236/447] 10.222.0.57 () {56 vars in 1180 bytes} [Thu May 19 14:38:01 2022] GET /identity/v3/groups/8b0f774e11544ebba71829c939937be9 => generated 226 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:01.623903 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90a379d-a5d8-4f4e-9aab-50935d97800c None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:01.654161 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:01.694784 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d4dd4b-d9db-44c1-9d3f-91d79898f57a None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:01.726290 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90a379d-a5d8-4f4e-9aab-50935d97800c None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:01.729430 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c4146b-8378-4fc5-8f9d-b519b16a7525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:01.730325 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c4146b-8378-4fc5-8f9d-b519b16a7525 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:01.730612 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c4146b-8378-4fc5-8f9d-b519b16a7525 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:01.730889 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c4146b-8378-4fc5-8f9d-b519b16a7525 None admin] PATH_INFO: `/v3/groups/8b0f774e11544ebba71829c939937be9` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:01.732083 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c4146b-8378-4fc5-8f9d-b519b16a7525 None admin] RBAC: Authorizing `identity:get_group(group_id=8b0f774e11544ebba71829c939937be9)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:01.733532 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c4146b-8378-4fc5-8f9d-b519b16a7525 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:01.735220 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 212/448] 10.222.0.57 () {56 vars in 1180 bytes} [Thu May 19 14:38:01 2022] GET /identity/v3/groups/8b0f774e11544ebba71829c939937be9 => generated 226 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:01.735991 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c4146b-8378-4fc5-8f9d-b519b16a7525 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:01.766614 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc3dddc-dae6-4c95-9bae-41fd68600af5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:01.767361 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc3dddc-dae6-4c95-9bae-41fd68600af5 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:01.767649 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc3dddc-dae6-4c95-9bae-41fd68600af5 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:01.767922 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc3dddc-dae6-4c95-9bae-41fd68600af5 None admin] PATH_INFO: `/v3/roles` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:01.768661 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc3dddc-dae6-4c95-9bae-41fd68600af5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:01.768966 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc3dddc-dae6-4c95-9bae-41fd68600af5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:01.770375 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc3dddc-dae6-4c95-9bae-41fd68600af5 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:01.780179 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 237/449] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:38:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:01.781027 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc3dddc-dae6-4c95-9bae-41fd68600af5 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:01.830597 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:01.841466 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c4146b-8378-4fc5-8f9d-b519b16a7525 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:01.874655 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:01.877722 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6f2d91-f372-4a31-9cab-45858d4ca621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:01.878425 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6f2d91-f372-4a31-9cab-45858d4ca621 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:01.878717 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6f2d91-f372-4a31-9cab-45858d4ca621 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:01.879003 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6f2d91-f372-4a31-9cab-45858d4ca621 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:01.879848 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6f2d91-f372-4a31-9cab-45858d4ca621 None admin] RBAC: Adding query filter params (group.id=8b0f774e11544ebba71829c939937be9, role.id=b894e00b17e34f0cbde0ab3d760e2433, scope.project.id=0c950d0347b741b68c581d47c8968d0d) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:01.880252 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6f2d91-f372-4a31-9cab-45858d4ca621 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:01.881633 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6f2d91-f372-4a31-9cab-45858d4ca621 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:01.884780 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc3dddc-dae6-4c95-9bae-41fd68600af5 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:01.888141 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 213/450] 10.222.0.57 () {56 vars in 1376 bytes} [Thu May 19 14:38:01 2022] GET /identity/v3/role_assignments?group.id=8b0f774e11544ebba71829c939937be9&scope.project.id=0c950d0347b741b68c581d47c8968d0d&role.id=b894e00b17e34f0cbde0ab3d760e2433 => generated 611 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:01.888940 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6f2d91-f372-4a31-9cab-45858d4ca621 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:01.923325 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c9d853-1dd9-4509-98fd-a5577d3241e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:01.924184 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c9d853-1dd9-4509-98fd-a5577d3241e8 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:01.924473 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c9d853-1dd9-4509-98fd-a5577d3241e8 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:01.925153 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c9d853-1dd9-4509-98fd-a5577d3241e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:01.926059 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c9d853-1dd9-4509-98fd-a5577d3241e8 None admin] RBAC: Adding query filter params (group.id=8b0f774e11544ebba71829c939937be9, role.id=7ebce5564f40456e94a3616961ed73ff, scope.project.id=0c950d0347b741b68c581d47c8968d0d) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:01.926385 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c9d853-1dd9-4509-98fd-a5577d3241e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:01.927914 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c9d853-1dd9-4509-98fd-a5577d3241e8 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:01.946095 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 238/451] 10.222.0.57 () {56 vars in 1376 bytes} [Thu May 19 14:38:01 2022] GET /identity/v3/role_assignments?group.id=8b0f774e11544ebba71829c939937be9&scope.project.id=0c950d0347b741b68c581d47c8968d0d&role.id=7ebce5564f40456e94a3616961ed73ff => generated 611 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:02.030983 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6f2d91-f372-4a31-9cab-45858d4ca621 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:02.073806 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798b4bbe-75ae-40a2-85a2-9d0779226fad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:02.074512 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798b4bbe-75ae-40a2-85a2-9d0779226fad None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:02.074798 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798b4bbe-75ae-40a2-85a2-9d0779226fad None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:02.075072 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798b4bbe-75ae-40a2-85a2-9d0779226fad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:02.105437 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:02.158272 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798b4bbe-75ae-40a2-85a2-9d0779226fad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:02.159756 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798b4bbe-75ae-40a2-85a2-9d0779226fad None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:02.265340 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 214/452] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:38:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:38:02.270599 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c9d853-1dd9-4509-98fd-a5577d3241e8 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:02.376975 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c9d853-1dd9-4509-98fd-a5577d3241e8 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:02.411800 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1daf3af-3a03-4223-855b-3b9a28b52dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:02.412671 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1daf3af-3a03-4223-855b-3b9a28b52dad None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:02.412955 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1daf3af-3a03-4223-855b-3b9a28b52dad None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:02.413231 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1daf3af-3a03-4223-855b-3b9a28b52dad None admin] PATH_INFO: `/v3/projects/c3cba789bd6f4fdaaa058dac80cb6fbe` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:02.414587 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1daf3af-3a03-4223-855b-3b9a28b52dad None admin] RBAC: Authorizing `identity:get_project(project_id=c3cba789bd6f4fdaaa058dac80cb6fbe)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:02.416046 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1daf3af-3a03-4223-855b-3b9a28b52dad None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:02.417763 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 239/453] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:38:02 2022] GET /identity/v3/projects/c3cba789bd6f4fdaaa058dac80cb6fbe => generated 344 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:38:02.423497 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798b4bbe-75ae-40a2-85a2-9d0779226fad None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:02.481198 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:02.526422 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798b4bbe-75ae-40a2-85a2-9d0779226fad None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:02.559533 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00328f8-a4b1-412e-938b-978b42566999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:02.560382 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00328f8-a4b1-412e-938b-978b42566999 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:02.560671 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00328f8-a4b1-412e-938b-978b42566999 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:02.560950 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00328f8-a4b1-412e-938b-978b42566999 None admin] PATH_INFO: `/v3/groups/f684214fae064c5282cd3d9f859d81e4` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:02.562147 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00328f8-a4b1-412e-938b-978b42566999 None admin] RBAC: Authorizing `identity:get_group(group_id=f684214fae064c5282cd3d9f859d81e4)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:02.563609 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00328f8-a4b1-412e-938b-978b42566999 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:02.565235 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 215/454] 10.222.0.57 () {56 vars in 1180 bytes} [Thu May 19 14:38:02 2022] GET /identity/v3/groups/f684214fae064c5282cd3d9f859d81e4 => generated 229 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:02.571846 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1daf3af-3a03-4223-855b-3b9a28b52dad None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:02.597416 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:02.670448 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1daf3af-3a03-4223-855b-3b9a28b52dad None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:02.703925 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f86d49-c32a-4dd6-8072-9ceeafe3da6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:02.704570 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f86d49-c32a-4dd6-8072-9ceeafe3da6f None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:02.704854 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f86d49-c32a-4dd6-8072-9ceeafe3da6f None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:02.705132 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f86d49-c32a-4dd6-8072-9ceeafe3da6f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:02.705980 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f86d49-c32a-4dd6-8072-9ceeafe3da6f None admin] RBAC: Adding query filter params (group.id=f684214fae064c5282cd3d9f859d81e4, role.id=c2935b1a25ca46fcba419fe3d1715330, scope.project.id=c3cba789bd6f4fdaaa058dac80cb6fbe) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:02.706294 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f86d49-c32a-4dd6-8072-9ceeafe3da6f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:02.707745 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f86d49-c32a-4dd6-8072-9ceeafe3da6f None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:02.713913 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 240/455] 10.222.0.57 () {56 vars in 1376 bytes} [Thu May 19 14:38:02 2022] GET /identity/v3/role_assignments?group.id=f684214fae064c5282cd3d9f859d81e4&scope.project.id=c3cba789bd6f4fdaaa058dac80cb6fbe&role.id=c2935b1a25ca46fcba419fe3d1715330 => generated 611 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:05.408782 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8085bf14-600a-4ad8-9cd1-c51519e5b611 None None] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:05.409708 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8085bf14-600a-4ad8-9cd1-c51519e5b611 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:05.410202 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8085bf14-600a-4ad8-9cd1-c51519e5b611 None None] PATH_INFO: `/` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:05.412045 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 216/456] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:38:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:38:05.428259 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ed7fb7-53fb-4d0c-b5a1-267339538258 None None] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:05.429875 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ed7fb7-53fb-4d0c-b5a1-267339538258 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:05.430385 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ed7fb7-53fb-4d0c-b5a1-267339538258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:05.456736 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.auth.core [None req-c1ed7fb7-53fb-4d0c-b5a1-267339538258 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:38:05.458347 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.fernet_utils [None req-c1ed7fb7-53fb-4d0c-b5a1-267339538258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:38:05.549807 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 241/457] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:38:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:38:05.595295 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8085bf14-600a-4ad8-9cd1-c51519e5b611 None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:05.620966 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a450e3f-ca88-4a8f-b87b-76a900596404 None None] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:05.621294 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a450e3f-ca88-4a8f-b87b-76a900596404 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:05.621549 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a450e3f-ca88-4a8f-b87b-76a900596404 None None] PATH_INFO: `/` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:05.623245 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 242/458] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:38:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:38:05.629658 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d17b68-38c0-47f5-bf53-8a5c6b4a6678 None None] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:05.630279 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d17b68-38c0-47f5-bf53-8a5c6b4a6678 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:05.630771 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d17b68-38c0-47f5-bf53-8a5c6b4a6678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:05.652135 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.auth.core [None req-02d17b68-38c0-47f5-bf53-8a5c6b4a6678 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:38:05.653681 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.fernet_utils [None req-02d17b68-38c0-47f5-bf53-8a5c6b4a6678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:38:05.676800 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:05.784105 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 243/459] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:38:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:38:05.787564 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8085bf14-600a-4ad8-9cd1-c51519e5b611 None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:05.813985 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:05.827717 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6633eb7-b7eb-4e0b-9315-b5dff268b1fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:05.829078 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d17b68-38c0-47f5-bf53-8a5c6b4a6678 None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:05.832938 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6633eb7-b7eb-4e0b-9315-b5dff268b1fe None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:05.838316 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6633eb7-b7eb-4e0b-9315-b5dff268b1fe None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:05.838837 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6633eb7-b7eb-4e0b-9315-b5dff268b1fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:05.864139 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6633eb7-b7eb-4e0b-9315-b5dff268b1fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:05.865927 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6633eb7-b7eb-4e0b-9315-b5dff268b1fe None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:05.895661 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-e6633eb7-b7eb-4e0b-9315-b5dff268b1fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:05.895661 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:05.895661 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:05.895661 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:05.895661 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:05.895661 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:05.895661 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:05.895661 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:05.895661 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:05.895661 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:05.895661 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:05.895661 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:05.895661 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:38:05.895661 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:38:05.895661 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:38:05.896889 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:38:05.896889 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:05.896889 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:05.896889 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:05.896889 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:05.896889 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:05.896889 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:05.896889 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:05.896889 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:38:05.896889 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:05.896889 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:05.896889 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:05.896889 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:05.896889 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:05.896889 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:05.896889 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:05.896889 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:05.898328 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:38:05.898328 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:38:05.898328 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:05.898328 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:38:05.908244 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 217/460] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:38:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:05.914220 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6633eb7-b7eb-4e0b-9315-b5dff268b1fe None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:05.984571 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d17b68-38c0-47f5-bf53-8a5c6b4a6678 None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:06.028590 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe3cdd2-13cc-4996-8264-0052f6b2ca44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:06.029558 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe3cdd2-13cc-4996-8264-0052f6b2ca44 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:06.030635 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe3cdd2-13cc-4996-8264-0052f6b2ca44 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:06.031024 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe3cdd2-13cc-4996-8264-0052f6b2ca44 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:06.046305 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe3cdd2-13cc-4996-8264-0052f6b2ca44 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:06.046489 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6633eb7-b7eb-4e0b-9315-b5dff268b1fe None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:06.047921 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe3cdd2-13cc-4996-8264-0052f6b2ca44 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:06.063348 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:06.079383 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-bbe3cdd2-13cc-4996-8264-0052f6b2ca44 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:06.079383 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:06.079383 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:06.079383 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:06.079383 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:06.079383 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:06.079383 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:06.079383 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:06.079383 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:06.079383 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:06.079383 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:06.079383 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:06.079383 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:38:06.079383 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:38:06.079383 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:38:06.080413 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:38:06.080413 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:06.080413 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:06.080413 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:06.080413 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:06.080413 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:06.080413 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:06.080413 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:06.080413 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:38:06.080413 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:06.080413 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:06.080413 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:06.080413 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:06.080413 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:06.080413 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:06.080413 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:06.080413 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:06.081365 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:38:06.081365 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:38:06.081365 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:06.081365 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:38:06.081614 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 244/461] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:38:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:06.083787 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfa3825-03ca-4218-bedb-bf5cef558f8a None None] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:06.084136 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfa3825-03ca-4218-bedb-bf5cef558f8a None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:06.084494 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfa3825-03ca-4218-bedb-bf5cef558f8a None None] PATH_INFO: `/` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:06.087553 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 245/462] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:38:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:38:06.087994 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b01993-ab29-483d-9bd2-01d3841aefd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:06.088441 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfa3825-03ca-4218-bedb-bf5cef558f8a None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:06.088664 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b01993-ab29-483d-9bd2-01d3841aefd1 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:06.088956 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b01993-ab29-483d-9bd2-01d3841aefd1 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:06.089236 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b01993-ab29-483d-9bd2-01d3841aefd1 None admin] PATH_INFO: `/v3/domains` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:06.090118 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b01993-ab29-483d-9bd2-01d3841aefd1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:06.090433 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b01993-ab29-483d-9bd2-01d3841aefd1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:06.091950 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b01993-ab29-483d-9bd2-01d3841aefd1 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:06.111882 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 218/463] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:38:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:06.111882 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a23e58-e325-4e55-80f1-b48e5d9f976d None None] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:06.111882 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a23e58-e325-4e55-80f1-b48e5d9f976d None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:06.111882 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a23e58-e325-4e55-80f1-b48e5d9f976d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:06.130852 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.auth.core [None req-b5a23e58-e325-4e55-80f1-b48e5d9f976d None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:38:06.133080 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-b5a23e58-e325-4e55-80f1-b48e5d9f976d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:38:06.171446 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:06.222878 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 219/464] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:38:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:38:06.223777 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a23e58-e325-4e55-80f1-b48e5d9f976d None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:06.235476 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfa3825-03ca-4218-bedb-bf5cef558f8a None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:06.275002 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46aaa3b1-bbf5-41dd-9f11-f87f373d93b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:06.275679 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aaa3b1-bbf5-41dd-9f11-f87f373d93b7 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:06.275975 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aaa3b1-bbf5-41dd-9f11-f87f373d93b7 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:06.276253 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aaa3b1-bbf5-41dd-9f11-f87f373d93b7 None admin] PATH_INFO: `/v3/domains` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:06.276984 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46aaa3b1-bbf5-41dd-9f11-f87f373d93b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:06.277296 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46aaa3b1-bbf5-41dd-9f11-f87f373d93b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:06.278714 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46aaa3b1-bbf5-41dd-9f11-f87f373d93b7 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:06.295591 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 246/465] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:38:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:06.295591 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e33458-cf6b-4e86-aaca-f2f295e82ce8 None None] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:06.295591 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e33458-cf6b-4e86-aaca-f2f295e82ce8 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:06.295591 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e33458-cf6b-4e86-aaca-f2f295e82ce8 None None] PATH_INFO: `/` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:06.295591 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 247/466] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:38:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:38:06.295591 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e33458-cf6b-4e86-aaca-f2f295e82ce8 None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:06.337683 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a23e58-e325-4e55-80f1-b48e5d9f976d None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:06.362831 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:06.376319 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d44e9d-860a-4cb4-bfcf-f6a92f676bc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:06.376995 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d44e9d-860a-4cb4-bfcf-f6a92f676bc1 None admin] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:06.377280 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d44e9d-860a-4cb4-bfcf-f6a92f676bc1 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:06.377558 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d44e9d-860a-4cb4-bfcf-f6a92f676bc1 None admin] PATH_INFO: `/v3/users` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:06.378311 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d44e9d-860a-4cb4-bfcf-f6a92f676bc1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:06.379758 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d44e9d-860a-4cb4-bfcf-f6a92f676bc1 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:06.403207 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 220/467] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 19 14:38:06 2022] POST /identity/v3/users => generated 246 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:38:06.409587 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e33458-cf6b-4e86-aaca-f2f295e82ce8 None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:06.415136 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d44e9d-860a-4cb4-bfcf-f6a92f676bc1 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:06.451575 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:06.453794 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f924b768-319e-473d-823c-a7a146937c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:06.455425 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f924b768-319e-473d-823c-a7a146937c43 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:06.456281 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f924b768-319e-473d-823c-a7a146937c43 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:06.456794 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f924b768-319e-473d-823c-a7a146937c43 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:06.474836 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f924b768-319e-473d-823c-a7a146937c43 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:06.476981 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f924b768-319e-473d-823c-a7a146937c43 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:06.499408 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-f924b768-319e-473d-823c-a7a146937c43 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:06.499408 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:06.499408 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:06.499408 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:06.499408 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:06.499408 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:06.499408 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:06.499408 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:06.499408 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:06.499408 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:06.499408 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:06.499408 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:06.499408 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:38:06.499408 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:38:06.499408 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:38:06.500533 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:38:06.500533 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:06.500533 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:06.500533 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:06.500533 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:06.500533 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:06.500533 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:06.500533 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:06.500533 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:38:06.500533 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:06.500533 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:06.500533 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:06.500533 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:06.500533 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:06.500533 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:06.500533 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:06.500533 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:06.501565 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:38:06.501565 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:38:06.501565 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:06.501565 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:38:06.509473 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 248/468] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:38:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:06.512074 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9076a0f9-490c-45a8-8ee7-61d46039970e None None] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:06.512625 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9076a0f9-490c-45a8-8ee7-61d46039970e None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:06.513161 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9076a0f9-490c-45a8-8ee7-61d46039970e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:06.533455 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.auth.core [None req-9076a0f9-490c-45a8-8ee7-61d46039970e None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:38:06.535390 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.fernet_utils [None req-9076a0f9-490c-45a8-8ee7-61d46039970e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:38:06.557816 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d44e9d-860a-4cb4-bfcf-f6a92f676bc1 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:06.605368 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a96003-d459-41e2-869f-1e13b8bd00f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:06.606060 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a96003-d459-41e2-869f-1e13b8bd00f6 None admin] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:06.606332 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a96003-d459-41e2-869f-1e13b8bd00f6 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:06.607797 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a96003-d459-41e2-869f-1e13b8bd00f6 None admin] PATH_INFO: `/v3/users` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:06.608553 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a96003-d459-41e2-869f-1e13b8bd00f6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:06.610359 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a96003-d459-41e2-869f-1e13b8bd00f6 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:06.634579 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 221/469] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 19 14:38:06 2022] POST /identity/v3/users => generated 248 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:38:06.635262 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a96003-d459-41e2-869f-1e13b8bd00f6 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:06.641027 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 249/470] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:38:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:38:06.668188 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9076a0f9-490c-45a8-8ee7-61d46039970e None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:06.722344 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:06.741539 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:06.776251 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a96003-d459-41e2-869f-1e13b8bd00f6 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:06.818010 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9076a0f9-490c-45a8-8ee7-61d46039970e None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:06.830541 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c593e92-ce3f-4a4d-8eba-6ecf59408c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:06.831341 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c593e92-ce3f-4a4d-8eba-6ecf59408c53 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:06.831886 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c593e92-ce3f-4a4d-8eba-6ecf59408c53 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:06.831886 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c593e92-ce3f-4a4d-8eba-6ecf59408c53 None admin] PATH_INFO: `/v3/domains` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:06.833202 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c593e92-ce3f-4a4d-8eba-6ecf59408c53 None admin] RBAC: Adding query filter params (name=Default) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:06.833600 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c593e92-ce3f-4a4d-8eba-6ecf59408c53 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:06.835530 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c593e92-ce3f-4a4d-8eba-6ecf59408c53 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:06.850465 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 222/471] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:38:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:06.856764 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c593e92-ce3f-4a4d-8eba-6ecf59408c53 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:06.874081 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac795b6-5699-4e9b-bb32-5432e3fec3fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:06.874898 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac795b6-5699-4e9b-bb32-5432e3fec3fe None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:06.875182 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac795b6-5699-4e9b-bb32-5432e3fec3fe None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:06.875526 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac795b6-5699-4e9b-bb32-5432e3fec3fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:06.891679 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac795b6-5699-4e9b-bb32-5432e3fec3fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:06.893273 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac795b6-5699-4e9b-bb32-5432e3fec3fe None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:06.916703 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-9ac795b6-5699-4e9b-bb32-5432e3fec3fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:06.916703 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:06.916703 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:06.916703 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:06.916703 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:06.916703 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:06.916703 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:06.916703 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:06.916703 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:06.916703 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:06.916703 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:06.916703 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:06.916703 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:38:06.916703 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:38:06.916703 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:38:06.917775 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:38:06.917775 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:06.917775 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:06.917775 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:06.917775 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:06.917775 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:06.917775 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:06.917775 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:06.917775 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:38:06.917775 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:06.917775 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:06.917775 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:06.917775 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:06.917775 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:06.917775 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:06.917775 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:06.917775 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:06.918765 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:38:06.918765 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:38:06.918765 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:06.918765 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:38:06.918765 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 250/472] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:38:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:38:06.924299 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac795b6-5699-4e9b-bb32-5432e3fec3fe None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:07.003500 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c593e92-ce3f-4a4d-8eba-6ecf59408c53 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:07.037131 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:07.047920 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c5a941-4106-4be6-8db6-56ab07fa2cd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:07.048340 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c5a941-4106-4be6-8db6-56ab07fa2cd3 None admin] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:07.048626 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c5a941-4106-4be6-8db6-56ab07fa2cd3 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:07.048902 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c5a941-4106-4be6-8db6-56ab07fa2cd3 None admin] PATH_INFO: `/v3/users` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:07.049636 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c5a941-4106-4be6-8db6-56ab07fa2cd3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:07.050437 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac795b6-5699-4e9b-bb32-5432e3fec3fe None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:07.051195 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c5a941-4106-4be6-8db6-56ab07fa2cd3 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:07.061364 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:07.094335 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 223/473] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 19 14:38:06 2022] POST /identity/v3/users => generated 248 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:38:07.105749 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debedfb9-f625-42e6-9e4a-fe0394a28c1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:07.106539 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debedfb9-f625-42e6-9e4a-fe0394a28c1e None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:07.106892 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debedfb9-f625-42e6-9e4a-fe0394a28c1e None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:07.107237 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debedfb9-f625-42e6-9e4a-fe0394a28c1e None admin] PATH_INFO: `/v3/domains` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:07.108021 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debedfb9-f625-42e6-9e4a-fe0394a28c1e None admin] RBAC: Adding query filter params (name=Default) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:07.108342 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debedfb9-f625-42e6-9e4a-fe0394a28c1e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:07.109982 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debedfb9-f625-42e6-9e4a-fe0394a28c1e None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:07.118739 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd690c07-81e2-4bb9-9d23-9876db8bf326 None None] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:07.119027 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd690c07-81e2-4bb9-9d23-9876db8bf326 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:07.119310 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd690c07-81e2-4bb9-9d23-9876db8bf326 None None] PATH_INFO: `/` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:07.122934 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 224/474] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:38:07 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:38:07.127949 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd1f63e-953f-4c15-9a72-fb522e6d99d4 None None] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:07.128258 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd1f63e-953f-4c15-9a72-fb522e6d99d4 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:07.128520 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd1f63e-953f-4c15-9a72-fb522e6d99d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:07.130366 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 251/475] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:38:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:38:07.132342 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debedfb9-f625-42e6-9e4a-fe0394a28c1e None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:07.162718 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.auth.core [None req-2dd1f63e-953f-4c15-9a72-fb522e6d99d4 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:38:07.165132 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-2dd1f63e-953f-4c15-9a72-fb522e6d99d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:38:07.259898 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 225/476] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:38:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:38:07.300399 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd1f63e-953f-4c15-9a72-fb522e6d99d4 None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:07.346080 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debedfb9-f625-42e6-9e4a-fe0394a28c1e None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:07.387508 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf8b75e-507e-4c3c-b9cf-c67e4586720a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:07.388197 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf8b75e-507e-4c3c-b9cf-c67e4586720a None admin] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:07.388479 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf8b75e-507e-4c3c-b9cf-c67e4586720a None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:07.388756 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf8b75e-507e-4c3c-b9cf-c67e4586720a None admin] PATH_INFO: `/v3/users` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:07.389464 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf8b75e-507e-4c3c-b9cf-c67e4586720a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:07.391048 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf8b75e-507e-4c3c-b9cf-c67e4586720a None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:07.414621 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 252/477] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 19 14:38:07 2022] POST /identity/v3/users => generated 249 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:38:07.433357 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd1f63e-953f-4c15-9a72-fb522e6d99d4 None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:07.463274 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:07.477100 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f338e924-e213-4dd6-a72d-79e91677497e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:07.478234 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f338e924-e213-4dd6-a72d-79e91677497e None admin] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:07.478783 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f338e924-e213-4dd6-a72d-79e91677497e None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:07.479271 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f338e924-e213-4dd6-a72d-79e91677497e None admin] PATH_INFO: `/v3/roles` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:07.480259 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f338e924-e213-4dd6-a72d-79e91677497e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:07.481958 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f338e924-e213-4dd6-a72d-79e91677497e None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:07.492872 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.sql.core [None req-f338e924-e213-4dd6-a72d-79e91677497e None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 19 14:38:07.492872 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 19 14:38:07.492872 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [parameters: {'id': '64f014ced843434f86911393a0ff2f6e', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 19 14:38:07.492872 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=218624) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 19 14:38:07.513508 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-f338e924-e213-4dd6-a72d-79e91677497e None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 19 14:38:07.513508 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:07.513508 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 14:38:07.513508 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 14:38:07.513508 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 14:38:07.513508 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 14:38:07.513508 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 14:38:07.513508 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application result = self._query(query) May 19 14:38:07.513508 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 14:38:07.513508 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application conn.query(q) May 19 14:38:07.513508 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 14:38:07.513508 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 14:38:07.513508 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 14:38:07.513508 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application result.read() May 19 14:38:07.513508 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 14:38:07.514824 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 14:38:07.514824 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 14:38:07.514824 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 14:38:07.514824 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 14:38:07.514824 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 14:38:07.514824 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 14:38:07.514824 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 14:38:07.514824 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 19 14:38:07.514824 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:38:07.514824 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 19 14:38:07.514824 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:38:07.514824 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:07.514824 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 19 14:38:07.514824 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 19 14:38:07.514824 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role May 19 14:38:07.514824 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return ref.to_dict() May 19 14:38:07.514824 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 14:38:07.514824 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application next(self.gen) May 19 14:38:07.515950 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 19 14:38:07.515950 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application yield resource May 19 14:38:07.515950 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 14:38:07.515950 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application next(self.gen) May 19 14:38:07.515950 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 19 14:38:07.515950 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application self.session.rollback() May 19 14:38:07.515950 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 19 14:38:07.515950 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application self.force_reraise() May 19 14:38:07.515950 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 19 14:38:07.515950 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise self.value May 19 14:38:07.515950 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 19 14:38:07.515950 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 19 14:38:07.515950 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 19 14:38:07.515950 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application session.commit() May 19 14:38:07.515950 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 19 14:38:07.515950 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 19 14:38:07.515950 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 19 14:38:07.516957 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application self._prepare_impl() May 19 14:38:07.516957 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 19 14:38:07.516957 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application self.session.flush() May 19 14:38:07.516957 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 19 14:38:07.516957 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application self._flush(objects) May 19 14:38:07.516957 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 19 14:38:07.516957 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 19 14:38:07.516957 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 19 14:38:07.516957 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application compat.raise_( May 19 14:38:07.516957 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 14:38:07.516957 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception May 19 14:38:07.516957 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 19 14:38:07.516957 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application flush_context.execute() May 19 14:38:07.516957 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 19 14:38:07.516957 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rec.execute(self) May 19 14:38:07.516957 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 19 14:38:07.516957 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 19 14:38:07.518798 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 19 14:38:07.518798 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application _emit_insert_statements( May 19 14:38:07.518798 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 19 14:38:07.518798 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application c = connection._execute_20( May 19 14:38:07.518798 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 19 14:38:07.518798 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 19 14:38:07.518798 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 19 14:38:07.518798 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 19 14:38:07.518798 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 19 14:38:07.518798 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application ret = self._execute_context( May 19 14:38:07.518798 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 19 14:38:07.518798 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 19 14:38:07.518798 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 19 14:38:07.518798 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 19 14:38:07.518798 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 14:38:07.518798 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception May 19 14:38:07.520933 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 14:38:07.520933 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 14:38:07.520933 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 14:38:07.520933 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 14:38:07.520933 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 14:38:07.520933 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application result = self._query(query) May 19 14:38:07.520933 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 14:38:07.520933 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application conn.query(q) May 19 14:38:07.520933 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 14:38:07.520933 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 14:38:07.520933 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 14:38:07.520933 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application result.read() May 19 14:38:07.520933 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 14:38:07.520933 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 14:38:07.520933 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 14:38:07.520933 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 14:38:07.523618 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 14:38:07.523618 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 14:38:07.523618 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 14:38:07.523618 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 14:38:07.523618 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 19 14:38:07.523618 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 19 14:38:07.523618 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [parameters: {'id': '64f014ced843434f86911393a0ff2f6e', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 19 14:38:07.523618 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 19 14:38:07.523618 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:38:07.523618 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:38:07.523618 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:38:07.523618 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:07.523618 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:07.523618 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:07.523618 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:07.523618 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:07.526040 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:07.526040 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:07.526040 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:07.526040 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:07.526040 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:07.526040 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:07.526040 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post May 19 14:38:07.526040 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( May 19 14:38:07.526040 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:07.526040 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:07.526040 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role May 19 14:38:07.526040 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) May 19 14:38:07.526040 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 19 14:38:07.526040 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 19 14:38:07.526040 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 19 14:38:07.526040 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:38:07.528147 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 226/478] 10.222.0.57 () {60 vars in 1131 bytes} [Thu May 19 14:38:07 2022] POST /identity/v3/roles => generated 144 bytes in 220 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:07.528291 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf8b75e-507e-4c3c-b9cf-c67e4586720a None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:07.540983 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:07.644027 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf8b75e-507e-4c3c-b9cf-c67e4586720a None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:07.678637 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c5c00e-624f-4fc0-bcfd-41a92412dba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:07.679977 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c5c00e-624f-4fc0-bcfd-41a92412dba9 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:07.680254 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c5c00e-624f-4fc0-bcfd-41a92412dba9 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:07.680526 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c5c00e-624f-4fc0-bcfd-41a92412dba9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:07.691356 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c5c00e-624f-4fc0-bcfd-41a92412dba9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:38:07.700339 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-66c5c00e-624f-4fc0-bcfd-41a92412dba9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:38:07.700339 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:07.700339 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:07.700339 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:07.700339 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:07.700339 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:07.700339 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:07.700339 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:07.700339 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:07.700339 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:07.700339 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:07.700339 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:07.700339 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:38:07.700339 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:38:07.700339 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:38:07.700339 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:38:07.701461 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:38:07.701461 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:38:07.701461 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:38:07.701461 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:38:07.701461 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:07.701461 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:07.701461 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:07.701461 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:07.701461 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:07.701461 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:07.701461 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:07.701461 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:38:07.701461 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:07.701461 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:07.701461 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:07.701461 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:07.702494 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:07.702494 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:07.702494 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:07.702494 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:07.702494 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:38:07.702494 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:38:07.702494 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:38:07.702494 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:38:07.702494 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:38:07.702494 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:38:07.702494 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:38:07.702494 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:38:07.705900 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 253/479] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 19 14:38:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:07.707580 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f338e924-e213-4dd6-a72d-79e91677497e None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:07.821798 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f338e924-e213-4dd6-a72d-79e91677497e None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:07.864053 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0817d0a1-05c0-43f5-9f4d-2e75bdeefb07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:07.864737 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0817d0a1-05c0-43f5-9f4d-2e75bdeefb07 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:07.865079 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0817d0a1-05c0-43f5-9f4d-2e75bdeefb07 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:07.865361 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0817d0a1-05c0-43f5-9f4d-2e75bdeefb07 None admin] PATH_INFO: `/v3/roles` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:07.866202 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0817d0a1-05c0-43f5-9f4d-2e75bdeefb07 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:07.866525 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0817d0a1-05c0-43f5-9f4d-2e75bdeefb07 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:07.868478 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0817d0a1-05c0-43f5-9f4d-2e75bdeefb07 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:07.877515 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 227/480] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:38:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:09.348498 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61da7799-8b8e-4161-910a-81f24289433c None None] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:09.349032 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61da7799-8b8e-4161-910a-81f24289433c None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:09.349032 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61da7799-8b8e-4161-910a-81f24289433c None None] PATH_INFO: `/` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:09.350798 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 254/481] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:38:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:38:09.357038 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cfcc74-dea6-4bc8-abc5-c91696528411 None None] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:09.357518 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cfcc74-dea6-4bc8-abc5-c91696528411 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:09.357615 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cfcc74-dea6-4bc8-abc5-c91696528411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:09.374546 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.auth.core [None req-a1cfcc74-dea6-4bc8-abc5-c91696528411 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:38:09.375746 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-a1cfcc74-dea6-4bc8-abc5-c91696528411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:38:09.430937 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:09.461955 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 228/482] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:38:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:38:09.482836 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61da7799-8b8e-4161-910a-81f24289433c None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:09.579185 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:09.614385 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61da7799-8b8e-4161-910a-81f24289433c None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:09.662647 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b48448-d455-4cae-b226-c2d88bcf9f2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:09.663485 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b48448-d455-4cae-b226-c2d88bcf9f2d None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:09.663784 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b48448-d455-4cae-b226-c2d88bcf9f2d None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:09.664217 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b48448-d455-4cae-b226-c2d88bcf9f2d None admin] PATH_INFO: `/v3/roles/service` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:09.686840 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b48448-d455-4cae-b226-c2d88bcf9f2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:38:09.695216 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-40b48448-d455-4cae-b226-c2d88bcf9f2d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:38:09.695216 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:09.695216 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:09.695216 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:09.695216 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:09.695216 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:09.695216 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:09.695216 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:09.695216 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:09.695216 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:09.695216 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:09.695216 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:09.695216 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:38:09.695216 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:38:09.695216 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:38:09.695216 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:38:09.696579 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:38:09.696579 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:38:09.696579 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:38:09.696579 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:38:09.696579 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:09.696579 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:09.696579 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:09.696579 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:09.696579 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:09.696579 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:09.696579 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:09.696579 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:38:09.696579 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:09.696579 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:09.696579 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:09.696579 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:09.697499 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:09.697499 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:09.697499 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:09.697499 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:09.697499 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:38:09.697499 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:38:09.697499 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:38:09.697499 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:38:09.697499 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:38:09.697499 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:38:09.697499 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:38:09.697499 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:38:09.698263 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 255/483] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 14:38:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:09.701433 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bc4cee-be60-48e8-9bf9-443ce079e648 None None] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:09.701726 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bc4cee-be60-48e8-9bf9-443ce079e648 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:09.702032 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bc4cee-be60-48e8-9bf9-443ce079e648 None None] PATH_INFO: `/` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:09.703516 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 229/484] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:38:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:38:09.706279 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b48448-d455-4cae-b226-c2d88bcf9f2d None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:09.708973 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589412b0-36c6-4178-a386-2e190b6eb028 None None] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:09.709505 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589412b0-36c6-4178-a386-2e190b6eb028 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:09.710271 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589412b0-36c6-4178-a386-2e190b6eb028 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:09.744141 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.auth.core [None req-589412b0-36c6-4178-a386-2e190b6eb028 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:38:09.745741 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-589412b0-36c6-4178-a386-2e190b6eb028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:38:09.838842 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 230/485] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:38:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:38:09.855357 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b48448-d455-4cae-b226-c2d88bcf9f2d None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:09.863458 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589412b0-36c6-4178-a386-2e190b6eb028 None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:09.891969 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:09.902329 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c946926-27a0-4bb8-8c07-3420802a50e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:09.903149 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c946926-27a0-4bb8-8c07-3420802a50e9 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:09.903363 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c946926-27a0-4bb8-8c07-3420802a50e9 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:09.903649 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c946926-27a0-4bb8-8c07-3420802a50e9 None admin] PATH_INFO: `/v3/roles` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:09.904495 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c946926-27a0-4bb8-8c07-3420802a50e9 None admin] RBAC: Adding query filter params (name=service) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:09.904810 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c946926-27a0-4bb8-8c07-3420802a50e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:09.906279 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c946926-27a0-4bb8-8c07-3420802a50e9 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:09.925278 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 256/486] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 19 14:38:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:09.937892 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c946926-27a0-4bb8-8c07-3420802a50e9 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:10.051811 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589412b0-36c6-4178-a386-2e190b6eb028 None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:10.062624 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c946926-27a0-4bb8-8c07-3420802a50e9 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:10.067067 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:10.109214 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2df539-594c-4386-9ec9-2dfad08c9f43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:10.110055 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2df539-594c-4386-9ec9-2dfad08c9f43 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:10.110344 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2df539-594c-4386-9ec9-2dfad08c9f43 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:10.110628 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2df539-594c-4386-9ec9-2dfad08c9f43 None admin] PATH_INFO: `/v3/roles/service` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:10.129041 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4e17b2-fb75-4aa1-8492-fbcce680c0f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:10.129867 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4e17b2-fb75-4aa1-8492-fbcce680c0f6 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:10.130152 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4e17b2-fb75-4aa1-8492-fbcce680c0f6 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:10.130436 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4e17b2-fb75-4aa1-8492-fbcce680c0f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:10.135299 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2df539-594c-4386-9ec9-2dfad08c9f43 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:38:10.148985 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-cc2df539-594c-4386-9ec9-2dfad08c9f43 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:38:10.148985 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:10.148985 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:10.148985 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:10.148985 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:10.148985 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:10.148985 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:10.148985 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:10.148985 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:10.148985 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:10.148985 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:10.148985 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:10.148985 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:38:10.148985 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:38:10.148985 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:38:10.148985 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:38:10.150257 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:38:10.150257 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:38:10.150257 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:38:10.150257 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:38:10.150257 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:10.150257 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:10.150257 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:10.150257 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:10.150257 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:10.150257 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:10.150257 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:10.150257 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:38:10.150257 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:10.150257 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:10.150257 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:10.150257 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:10.151599 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:10.151599 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:10.151599 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:10.151599 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:10.151599 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:38:10.151599 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:38:10.151599 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:38:10.151599 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:38:10.151599 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:38:10.151599 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:38:10.151599 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:38:10.151599 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:38:10.151599 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 231/487] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 14:38:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:38:10.154912 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4e17b2-fb75-4aa1-8492-fbcce680c0f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:10.156374 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4e17b2-fb75-4aa1-8492-fbcce680c0f6 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:10.156565 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2df539-594c-4386-9ec9-2dfad08c9f43 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:10.193103 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-0d4e17b2-fb75-4aa1-8492-fbcce680c0f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:10.193103 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:10.193103 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:10.193103 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:10.193103 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:10.193103 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:10.193103 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:10.193103 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:10.193103 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:10.193103 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:10.193103 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:10.193103 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:10.193103 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:38:10.193103 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:38:10.193103 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:38:10.194144 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:38:10.194144 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:10.194144 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:10.194144 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:10.194144 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:10.194144 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:10.194144 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:10.194144 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:10.194144 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:38:10.194144 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:10.194144 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:10.194144 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:10.194144 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:10.194144 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:10.194144 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:10.194144 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:10.194144 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:10.195371 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:38:10.195371 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:38:10.195371 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:10.195371 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:38:10.202967 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 257/488] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:38:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:38:10.204585 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dbbe73-ed26-426d-94ff-a35993491003 None None] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:10.205106 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dbbe73-ed26-426d-94ff-a35993491003 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:10.205616 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dbbe73-ed26-426d-94ff-a35993491003 None None] PATH_INFO: `/` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:10.207219 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 258/489] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:38:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:38:10.208341 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dbbe73-ed26-426d-94ff-a35993491003 None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:10.276639 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:10.294189 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2df539-594c-4386-9ec9-2dfad08c9f43 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:10.337926 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac81b449-a7d9-445f-90f6-38209ebebcac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:10.339119 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac81b449-a7d9-445f-90f6-38209ebebcac None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:10.340114 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac81b449-a7d9-445f-90f6-38209ebebcac None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:10.340893 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac81b449-a7d9-445f-90f6-38209ebebcac None admin] PATH_INFO: `/v3/roles` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:10.342074 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac81b449-a7d9-445f-90f6-38209ebebcac None admin] RBAC: Adding query filter params (name=service) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:10.342914 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac81b449-a7d9-445f-90f6-38209ebebcac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:10.344805 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac81b449-a7d9-445f-90f6-38209ebebcac None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:10.346336 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dbbe73-ed26-426d-94ff-a35993491003 None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:10.356060 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 232/490] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 19 14:38:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:38:10.357668 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f3cb55-7420-4608-ad4a-31cfd5d3734e None None] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:10.358745 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f3cb55-7420-4608-ad4a-31cfd5d3734e None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:10.359233 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f3cb55-7420-4608-ad4a-31cfd5d3734e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:10.382880 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.auth.core [None req-61f3cb55-7420-4608-ad4a-31cfd5d3734e None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:38:10.385086 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-61f3cb55-7420-4608-ad4a-31cfd5d3734e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:38:10.392926 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12eaf91-14ad-4878-94e3-ce7cdc3d7846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:10.393597 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12eaf91-14ad-4878-94e3-ce7cdc3d7846 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:10.393908 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12eaf91-14ad-4878-94e3-ce7cdc3d7846 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:10.394192 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12eaf91-14ad-4878-94e3-ce7cdc3d7846 None admin] PATH_INFO: `/v3/domains` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:10.394921 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12eaf91-14ad-4878-94e3-ce7cdc3d7846 None admin] RBAC: Adding query filter params (name=Default) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:10.395227 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12eaf91-14ad-4878-94e3-ce7cdc3d7846 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:10.397153 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12eaf91-14ad-4878-94e3-ce7cdc3d7846 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:10.418273 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 259/491] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:38:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:10.418273 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12eaf91-14ad-4878-94e3-ce7cdc3d7846 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:10.422516 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:10.471482 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 233/492] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:38:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:38:10.472260 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f3cb55-7420-4608-ad4a-31cfd5d3734e None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:10.539576 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12eaf91-14ad-4878-94e3-ce7cdc3d7846 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:10.542135 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:10.577944 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f918194-dcd2-4424-8333-0a29be99496c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:10.578961 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f918194-dcd2-4424-8333-0a29be99496c None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:10.579242 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f918194-dcd2-4424-8333-0a29be99496c None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:10.579513 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f918194-dcd2-4424-8333-0a29be99496c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:10.582399 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f3cb55-7420-4608-ad4a-31cfd5d3734e None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:10.590585 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f918194-dcd2-4424-8333-0a29be99496c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:10.592237 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f918194-dcd2-4424-8333-0a29be99496c None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:10.610944 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-4f918194-dcd2-4424-8333-0a29be99496c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:10.610944 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:10.610944 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:10.610944 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:10.610944 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:10.610944 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:10.610944 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:10.610944 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:10.610944 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:10.610944 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:10.610944 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:10.610944 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:10.610944 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:38:10.610944 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:38:10.610944 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:38:10.611962 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:38:10.611962 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:10.611962 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:10.611962 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:10.611962 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:10.611962 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:10.611962 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:10.611962 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:10.611962 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:38:10.611962 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:10.611962 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:10.611962 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:10.611962 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:10.611962 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:10.611962 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:10.611962 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:10.611962 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:10.612932 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:38:10.612932 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:38:10.612932 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:10.612932 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:38:10.612932 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 260/493] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:38:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:10.614007 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94d99db-52dc-4337-8114-b6d4ed5f34f0 None None] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:10.614007 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94d99db-52dc-4337-8114-b6d4ed5f34f0 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:10.614195 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94d99db-52dc-4337-8114-b6d4ed5f34f0 None None] PATH_INFO: `/` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:10.617982 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 261/494] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:38:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:38:10.617982 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94d99db-52dc-4337-8114-b6d4ed5f34f0 None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:10.628122 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075ef3e9-fb36-4cf2-8e97-70c8eb4df4fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:10.628929 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075ef3e9-fb36-4cf2-8e97-70c8eb4df4fb None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:10.629214 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075ef3e9-fb36-4cf2-8e97-70c8eb4df4fb None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:10.629489 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075ef3e9-fb36-4cf2-8e97-70c8eb4df4fb None admin] PATH_INFO: `/v3/users/nova` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:10.638013 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075ef3e9-fb36-4cf2-8e97-70c8eb4df4fb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:10.639478 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075ef3e9-fb36-4cf2-8e97-70c8eb4df4fb None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:10.664087 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-075ef3e9-fb36-4cf2-8e97-70c8eb4df4fb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 14:38:10.664087 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:10.664087 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:10.664087 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:10.664087 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:10.664087 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:10.664087 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:10.664087 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:10.664087 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:10.664087 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:10.664087 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:10.664087 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:10.664087 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:38:10.664087 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:38:10.664087 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:38:10.664087 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:38:10.665249 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:10.665249 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:10.665249 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:38:10.665249 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:38:10.665249 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:38:10.665249 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:38:10.665249 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:10.665249 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:10.665249 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:10.665249 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:10.665249 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:10.665249 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:38:10.665249 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:10.665249 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:10.665249 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:10.665249 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:10.665249 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:10.665249 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:10.671156 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:10.671156 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:10.671156 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:38:10.671156 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:38:10.671156 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:38:10.671156 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:38:10.671156 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:38:10.671156 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:38:10.671156 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 14:38:10.671156 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:38:10.671156 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 234/495] 10.222.0.57 () {56 vars in 1093 bytes} [Thu May 19 14:38:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:10.671156 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075ef3e9-fb36-4cf2-8e97-70c8eb4df4fb None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:10.689025 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:10.736062 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94d99db-52dc-4337-8114-b6d4ed5f34f0 None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:10.772697 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8086a0b5-4c39-4fac-8288-823f861fdbe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:10.774216 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8086a0b5-4c39-4fac-8288-823f861fdbe6 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:10.774513 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8086a0b5-4c39-4fac-8288-823f861fdbe6 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:10.774796 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8086a0b5-4c39-4fac-8288-823f861fdbe6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:10.777972 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075ef3e9-fb36-4cf2-8e97-70c8eb4df4fb None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:10.786239 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8086a0b5-4c39-4fac-8288-823f861fdbe6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:38:10.794693 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-8086a0b5-4c39-4fac-8288-823f861fdbe6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:38:10.794693 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:10.794693 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:10.794693 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:10.794693 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:10.794693 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:10.794693 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:10.794693 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:10.794693 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:10.794693 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:10.794693 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:10.794693 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:10.794693 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:38:10.794693 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:38:10.794693 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:38:10.794693 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:38:10.796620 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:38:10.796620 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:38:10.796620 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:38:10.796620 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:38:10.796620 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:10.796620 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:10.796620 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:10.796620 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:10.796620 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:10.796620 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:10.796620 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:10.796620 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:38:10.796620 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:10.796620 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:10.796620 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:10.796620 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:10.798578 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:10.798578 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:10.798578 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:10.798578 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:10.798578 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:38:10.798578 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:38:10.798578 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:38:10.798578 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:38:10.798578 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:38:10.798578 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:38:10.798578 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:38:10.798578 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:38:10.798578 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 262/496] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 14:38:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:10.798578 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87cba97-a209-4bec-b684-1cb3b7da5a1e None None] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:10.800516 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87cba97-a209-4bec-b684-1cb3b7da5a1e None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:10.800516 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87cba97-a209-4bec-b684-1cb3b7da5a1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:10.817860 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.auth.core [None req-e87cba97-a209-4bec-b684-1cb3b7da5a1e None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:38:10.819180 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.fernet_utils [None req-e87cba97-a209-4bec-b684-1cb3b7da5a1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:38:10.828787 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef0e00c-264d-49fb-9d19-82ae68699531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:10.829477 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef0e00c-264d-49fb-9d19-82ae68699531 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:10.829760 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef0e00c-264d-49fb-9d19-82ae68699531 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:10.830074 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef0e00c-264d-49fb-9d19-82ae68699531 None admin] PATH_INFO: `/v3/domains` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:10.830840 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef0e00c-264d-49fb-9d19-82ae68699531 None admin] RBAC: Adding query filter params (name=Default) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:10.831154 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef0e00c-264d-49fb-9d19-82ae68699531 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:10.832264 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:10.832612 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef0e00c-264d-49fb-9d19-82ae68699531 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:10.851794 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 235/497] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:38:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:10.851794 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef0e00c-264d-49fb-9d19-82ae68699531 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:10.901944 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 263/498] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:38:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:38:10.903032 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87cba97-a209-4bec-b684-1cb3b7da5a1e None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:10.946998 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:10.970653 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef0e00c-264d-49fb-9d19-82ae68699531 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:11.008044 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ea4daa-122f-4021-92d4-246a9c0bedb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:11.008753 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ea4daa-122f-4021-92d4-246a9c0bedb7 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:11.009033 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ea4daa-122f-4021-92d4-246a9c0bedb7 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:11.009306 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ea4daa-122f-4021-92d4-246a9c0bedb7 None admin] PATH_INFO: `/v3/users` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:11.010188 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ea4daa-122f-4021-92d4-246a9c0bedb7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:11.010496 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ea4daa-122f-4021-92d4-246a9c0bedb7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:11.011938 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ea4daa-122f-4021-92d4-246a9c0bedb7 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:11.020029 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87cba97-a209-4bec-b684-1cb3b7da5a1e None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:11.050071 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 236/499] 10.222.0.57 () {56 vars in 1133 bytes} [Thu May 19 14:38:10 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:11.050818 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ea4daa-122f-4021-92d4-246a9c0bedb7 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:11.058830 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29971dc5-a6e6-460f-afbf-6a88dc560604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:11.059517 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29971dc5-a6e6-460f-afbf-6a88dc560604 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:11.059804 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29971dc5-a6e6-460f-afbf-6a88dc560604 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:11.060082 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29971dc5-a6e6-460f-afbf-6a88dc560604 None admin] PATH_INFO: `/v3/roles` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:11.060836 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29971dc5-a6e6-460f-afbf-6a88dc560604 None admin] RBAC: Adding query filter params (name=service) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:11.061141 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29971dc5-a6e6-460f-afbf-6a88dc560604 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:11.062575 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29971dc5-a6e6-460f-afbf-6a88dc560604 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:11.071910 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 264/500] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 19 14:38:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:11.072743 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29971dc5-a6e6-460f-afbf-6a88dc560604 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:11.083606 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:11.157712 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ea4daa-122f-4021-92d4-246a9c0bedb7 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:11.177535 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29971dc5-a6e6-460f-afbf-6a88dc560604 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:11.192345 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50685e1e-7c6e-44cf-a44e-4f5172a5db0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:11.193156 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50685e1e-7c6e-44cf-a44e-4f5172a5db0c None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:11.193437 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50685e1e-7c6e-44cf-a44e-4f5172a5db0c None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:11.193709 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50685e1e-7c6e-44cf-a44e-4f5172a5db0c None admin] PATH_INFO: `/v3/users/glance` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:11.201452 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50685e1e-7c6e-44cf-a44e-4f5172a5db0c None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:11.202988 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50685e1e-7c6e-44cf-a44e-4f5172a5db0c None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:11.204834 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:11.213359 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959e63c7-e2a0-4088-9a3d-ea4182205643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:11.214202 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959e63c7-e2a0-4088-9a3d-ea4182205643 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:11.214490 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959e63c7-e2a0-4088-9a3d-ea4182205643 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:11.214770 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959e63c7-e2a0-4088-9a3d-ea4182205643 None admin] PATH_INFO: `/v3/roles/service` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:11.220121 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-50685e1e-7c6e-44cf-a44e-4f5172a5db0c None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 19 14:38:11.220121 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:11.220121 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:11.220121 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:11.220121 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:11.220121 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:11.220121 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:11.220121 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:11.220121 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:11.220121 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:11.220121 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:11.220121 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:11.220121 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:38:11.220121 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:38:11.220121 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:38:11.220121 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:38:11.221140 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:11.221140 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:11.221140 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:38:11.221140 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:38:11.221140 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:38:11.221140 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:38:11.221140 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:11.221140 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:11.221140 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:11.221140 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:11.221140 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:11.221140 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:38:11.221140 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:11.221140 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:11.221140 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:11.221140 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:11.221140 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:11.221140 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:11.222224 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:11.222224 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:11.222224 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:38:11.222224 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:38:11.222224 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:38:11.222224 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:38:11.222224 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:38:11.222224 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:38:11.222224 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 19 14:38:11.222224 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:38:11.222224 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 237/501] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:38:11 2022] GET /identity/v3/users/glance => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:11.223152 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f2eabd-5698-448b-8e80-0fad0d54cde4 None None] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:11.223152 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f2eabd-5698-448b-8e80-0fad0d54cde4 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:11.223152 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f2eabd-5698-448b-8e80-0fad0d54cde4 None None] PATH_INFO: `/` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:11.224318 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 238/502] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:38:11 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:38:11.225007 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f2eabd-5698-448b-8e80-0fad0d54cde4 None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:11.230228 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959e63c7-e2a0-4088-9a3d-ea4182205643 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:38:11.238124 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-959e63c7-e2a0-4088-9a3d-ea4182205643 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:38:11.238124 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:11.238124 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:11.238124 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:11.238124 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:11.238124 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:11.238124 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:11.238124 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:11.238124 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:11.238124 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:11.238124 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:11.238124 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:11.238124 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:38:11.238124 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:38:11.238124 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:38:11.238124 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:38:11.239333 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:38:11.239333 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:38:11.239333 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:38:11.239333 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:38:11.239333 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:11.239333 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:11.239333 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:11.239333 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:11.239333 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:11.239333 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:11.239333 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:11.239333 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:38:11.239333 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:11.239333 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:11.239333 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:11.239333 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:11.240291 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:11.240291 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:11.240291 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:11.240291 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:11.240291 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:38:11.240291 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:38:11.240291 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:38:11.240291 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:38:11.240291 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:38:11.240291 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:38:11.240291 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:38:11.240291 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:38:11.240291 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 265/503] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 14:38:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:11.241046 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959e63c7-e2a0-4088-9a3d-ea4182205643 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:11.306576 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:11.338249 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f2eabd-5698-448b-8e80-0fad0d54cde4 None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:11.342325 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959e63c7-e2a0-4088-9a3d-ea4182205643 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:11.373444 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6adb2f-1221-4303-9706-14584f03c4d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:11.374130 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6adb2f-1221-4303-9706-14584f03c4d0 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:11.374416 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6adb2f-1221-4303-9706-14584f03c4d0 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:11.374693 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6adb2f-1221-4303-9706-14584f03c4d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:11.378725 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23794565-2fcf-4fd2-8fb0-139ad2f1cb92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:11.379532 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23794565-2fcf-4fd2-8fb0-139ad2f1cb92 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:11.379813 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23794565-2fcf-4fd2-8fb0-139ad2f1cb92 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:11.380085 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23794565-2fcf-4fd2-8fb0-139ad2f1cb92 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:11.390734 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23794565-2fcf-4fd2-8fb0-139ad2f1cb92 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:11.392238 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23794565-2fcf-4fd2-8fb0-139ad2f1cb92 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:11.409669 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-23794565-2fcf-4fd2-8fb0-139ad2f1cb92 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:11.409669 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:11.409669 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:11.409669 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:11.409669 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:11.409669 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:11.409669 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:11.409669 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:11.409669 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:11.409669 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:11.409669 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:11.409669 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:11.409669 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:38:11.409669 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:38:11.409669 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:38:11.411152 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:38:11.411152 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:11.411152 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:11.411152 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:11.411152 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:11.411152 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:11.411152 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:11.411152 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:11.411152 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:38:11.411152 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:11.411152 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:11.411152 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:11.411152 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:11.411152 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:11.411152 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:11.411152 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:11.411152 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:11.412163 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:38:11.412163 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:38:11.412163 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:11.412163 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:38:11.412163 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 266/504] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:38:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:11.412163 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23794565-2fcf-4fd2-8fb0-139ad2f1cb92 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:11.444434 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:11.449604 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6adb2f-1221-4303-9706-14584f03c4d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:11.451685 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6adb2f-1221-4303-9706-14584f03c4d0 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:11.512210 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23794565-2fcf-4fd2-8fb0-139ad2f1cb92 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:11.532458 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:11.547972 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69090456-5423-422e-9f58-a6ca39e9205a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:11.548642 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69090456-5423-422e-9f58-a6ca39e9205a None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:11.548933 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69090456-5423-422e-9f58-a6ca39e9205a None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:11.549191 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69090456-5423-422e-9f58-a6ca39e9205a None admin] PATH_INFO: `/v3/users` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:11.550053 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69090456-5423-422e-9f58-a6ca39e9205a None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:11.550452 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69090456-5423-422e-9f58-a6ca39e9205a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:11.551787 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69090456-5423-422e-9f58-a6ca39e9205a None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:11.553174 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 239/505] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:38:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:38:11.555042 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e33268-c9c4-4e10-a753-8b3001f80c04 None None] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:11.555168 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e33268-c9c4-4e10-a753-8b3001f80c04 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:11.555620 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e33268-c9c4-4e10-a753-8b3001f80c04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:11.573697 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.auth.core [None req-00e33268-c9c4-4e10-a753-8b3001f80c04 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:38:11.574866 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-00e33268-c9c4-4e10-a753-8b3001f80c04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:38:11.588710 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 267/506] 10.222.0.57 () {56 vars in 1137 bytes} [Thu May 19 14:38:11 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:11.589534 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69090456-5423-422e-9f58-a6ca39e9205a None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:11.647666 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 240/507] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:38:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:38:11.649038 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e33268-c9c4-4e10-a753-8b3001f80c04 None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:11.688682 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69090456-5423-422e-9f58-a6ca39e9205a None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:11.689004 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:11.724290 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecad8a8-74c7-49e2-925f-cf1ce9beb843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:11.724962 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecad8a8-74c7-49e2-925f-cf1ce9beb843 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:11.725248 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecad8a8-74c7-49e2-925f-cf1ce9beb843 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:11.725526 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecad8a8-74c7-49e2-925f-cf1ce9beb843 None admin] PATH_INFO: `/v3/roles` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:11.726391 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecad8a8-74c7-49e2-925f-cf1ce9beb843 None admin] RBAC: Adding query filter params (name=service) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:11.726879 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecad8a8-74c7-49e2-925f-cf1ce9beb843 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:11.728286 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecad8a8-74c7-49e2-925f-cf1ce9beb843 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:11.736301 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 268/508] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 19 14:38:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:11.737104 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecad8a8-74c7-49e2-925f-cf1ce9beb843 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:11.763705 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e33268-c9c4-4e10-a753-8b3001f80c04 None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:11.778908 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:11.799547 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734a6c88-3226-4843-b12b-9dd659bdb963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:11.800207 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734a6c88-3226-4843-b12b-9dd659bdb963 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:11.800492 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734a6c88-3226-4843-b12b-9dd659bdb963 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:11.800768 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734a6c88-3226-4843-b12b-9dd659bdb963 None admin] PATH_INFO: `/v3/domains` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:11.801492 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734a6c88-3226-4843-b12b-9dd659bdb963 None admin] RBAC: Adding query filter params (name=Default) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:11.801800 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734a6c88-3226-4843-b12b-9dd659bdb963 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:11.803245 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734a6c88-3226-4843-b12b-9dd659bdb963 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:11.815167 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 241/509] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:38:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:11.815980 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734a6c88-3226-4843-b12b-9dd659bdb963 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:11.842791 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecad8a8-74c7-49e2-925f-cf1ce9beb843 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:11.877555 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1653b1e5-b69c-477f-a2fe-f8acb19827ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:11.878391 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1653b1e5-b69c-477f-a2fe-f8acb19827ec None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:11.878675 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1653b1e5-b69c-477f-a2fe-f8acb19827ec None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:11.878954 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1653b1e5-b69c-477f-a2fe-f8acb19827ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:11.889795 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1653b1e5-b69c-477f-a2fe-f8acb19827ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:11.891312 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1653b1e5-b69c-477f-a2fe-f8acb19827ec None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:11.904041 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:11.908922 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-1653b1e5-b69c-477f-a2fe-f8acb19827ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:11.908922 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:11.908922 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:11.908922 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:11.908922 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:11.908922 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:11.908922 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:11.908922 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:11.908922 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:11.908922 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:11.908922 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:11.908922 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:11.908922 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:38:11.908922 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:38:11.908922 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:38:11.910059 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:38:11.910059 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:11.910059 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:11.910059 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:11.910059 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:11.910059 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:11.910059 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:11.910059 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:11.910059 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:38:11.910059 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:11.910059 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:11.910059 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:11.910059 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:11.910059 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:11.910059 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:11.910059 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:11.910059 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:11.911030 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:38:11.911030 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:38:11.911030 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:11.911030 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:38:11.911030 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 269/510] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:38:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:11.911030 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1653b1e5-b69c-477f-a2fe-f8acb19827ec None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:11.921320 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734a6c88-3226-4843-b12b-9dd659bdb963 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:11.957399 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72f724d-1106-4b9d-aa26-d321a0383e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:11.958084 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72f724d-1106-4b9d-aa26-d321a0383e02 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:11.958375 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72f724d-1106-4b9d-aa26-d321a0383e02 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:11.958656 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72f724d-1106-4b9d-aa26-d321a0383e02 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:12.005179 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:12.015210 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1653b1e5-b69c-477f-a2fe-f8acb19827ec None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:12.039723 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72f724d-1106-4b9d-aa26-d321a0383e02 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:12.041211 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72f724d-1106-4b9d-aa26-d321a0383e02 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:12.050134 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7007a6b-5069-4d8e-9a87-a2a71629b64a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:12.050936 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7007a6b-5069-4d8e-9a87-a2a71629b64a None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:12.051218 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7007a6b-5069-4d8e-9a87-a2a71629b64a None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:12.051494 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7007a6b-5069-4d8e-9a87-a2a71629b64a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:12.063038 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7007a6b-5069-4d8e-9a87-a2a71629b64a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:12.064496 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7007a6b-5069-4d8e-9a87-a2a71629b64a None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:12.082665 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-a7007a6b-5069-4d8e-9a87-a2a71629b64a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:12.082665 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:12.082665 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:12.082665 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:12.082665 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:12.082665 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:12.082665 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:12.082665 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:12.082665 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:12.082665 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:12.082665 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:12.082665 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:12.082665 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:38:12.082665 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:38:12.082665 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:38:12.083818 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:38:12.083818 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:12.083818 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:12.083818 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:12.083818 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:12.083818 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:12.083818 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:12.083818 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:12.083818 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:38:12.083818 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:12.083818 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:12.083818 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:12.083818 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:12.083818 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:12.083818 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:12.083818 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:12.083818 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:12.085150 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:38:12.085150 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:38:12.085150 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:12.085150 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:38:12.085150 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 270/511] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:38:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:12.085150 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7007a6b-5069-4d8e-9a87-a2a71629b64a None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:12.134617 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:12.147958 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 242/512] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:38:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:38:12.149018 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72f724d-1106-4b9d-aa26-d321a0383e02 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:12.191623 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7007a6b-5069-4d8e-9a87-a2a71629b64a None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:12.226663 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e757be53-2821-43d6-9784-1499303dcc81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:12.227454 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e757be53-2821-43d6-9784-1499303dcc81 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:12.227731 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e757be53-2821-43d6-9784-1499303dcc81 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:12.228001 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e757be53-2821-43d6-9784-1499303dcc81 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:12.238412 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e757be53-2821-43d6-9784-1499303dcc81 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:12.239871 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e757be53-2821-43d6-9784-1499303dcc81 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:12.243167 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:12.251097 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72f724d-1106-4b9d-aa26-d321a0383e02 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:12.259805 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-e757be53-2821-43d6-9784-1499303dcc81 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:12.259805 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:12.259805 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:12.259805 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:12.259805 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:12.259805 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:12.259805 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:12.259805 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:12.259805 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:12.259805 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:12.259805 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:12.259805 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:12.259805 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:38:12.259805 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:38:12.259805 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:38:12.260814 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:38:12.260814 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:12.260814 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:12.260814 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:12.260814 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:12.260814 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:12.260814 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:12.260814 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:12.260814 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:38:12.260814 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:12.260814 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:12.260814 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:12.260814 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:12.260814 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:12.260814 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:12.260814 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:12.260814 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:12.261800 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:38:12.261800 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:38:12.261800 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:12.261800 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:38:12.261800 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 271/513] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:38:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:12.261800 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e757be53-2821-43d6-9784-1499303dcc81 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:12.288151 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25482825-cb1e-4966-bbde-f2773c2af994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:12.290052 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25482825-cb1e-4966-bbde-f2773c2af994 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:12.290338 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25482825-cb1e-4966-bbde-f2773c2af994 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:12.290611 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25482825-cb1e-4966-bbde-f2773c2af994 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:12.297651 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25482825-cb1e-4966-bbde-f2773c2af994 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:12.299118 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25482825-cb1e-4966-bbde-f2773c2af994 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:12.314371 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-25482825-cb1e-4966-bbde-f2773c2af994 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 14:38:12.314371 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:12.314371 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:12.314371 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:12.314371 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:12.314371 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:12.314371 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:12.314371 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:12.314371 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:12.314371 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:12.314371 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:12.314371 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:12.314371 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:38:12.314371 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:38:12.314371 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:38:12.314371 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:38:12.315594 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:12.315594 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:12.315594 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:38:12.315594 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:38:12.315594 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:38:12.315594 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:38:12.315594 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:12.315594 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:12.315594 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:12.315594 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:12.315594 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:12.315594 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:38:12.315594 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:12.315594 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:12.315594 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:12.315594 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:12.315594 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:12.315594 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:12.316663 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:12.316663 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:12.316663 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:38:12.316663 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:38:12.316663 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:38:12.316663 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:38:12.316663 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:38:12.316663 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:38:12.316663 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 14:38:12.316663 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:38:12.316663 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 243/514] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:38:12 2022] GET /identity/v3/users/cinder => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:12.317323 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25482825-cb1e-4966-bbde-f2773c2af994 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:12.368206 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:12.372421 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e757be53-2821-43d6-9784-1499303dcc81 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:12.406147 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511c985a-30ad-4334-adfa-50125ab697be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:12.406738 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511c985a-30ad-4334-adfa-50125ab697be None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:12.407024 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511c985a-30ad-4334-adfa-50125ab697be None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:12.407290 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511c985a-30ad-4334-adfa-50125ab697be None admin] PATH_INFO: `/v3/domains` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:12.408014 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511c985a-30ad-4334-adfa-50125ab697be None admin] RBAC: Adding query filter params (name=Default) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:12.408319 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511c985a-30ad-4334-adfa-50125ab697be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:12.409702 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511c985a-30ad-4334-adfa-50125ab697be None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:12.419589 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25482825-cb1e-4966-bbde-f2773c2af994 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:12.421294 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 272/515] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:38:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:12.422132 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511c985a-30ad-4334-adfa-50125ab697be None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:12.454752 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d43216-9f8d-4db5-a1a5-916022fdfdb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:12.455409 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d43216-9f8d-4db5-a1a5-916022fdfdb9 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:12.455693 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d43216-9f8d-4db5-a1a5-916022fdfdb9 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:12.455968 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d43216-9f8d-4db5-a1a5-916022fdfdb9 None admin] PATH_INFO: `/v3/domains` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:12.456687 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d43216-9f8d-4db5-a1a5-916022fdfdb9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:12.456992 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d43216-9f8d-4db5-a1a5-916022fdfdb9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:12.458415 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d43216-9f8d-4db5-a1a5-916022fdfdb9 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:12.470224 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 244/516] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:38:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:12.471027 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d43216-9f8d-4db5-a1a5-916022fdfdb9 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:12.482551 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:12.526997 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511c985a-30ad-4334-adfa-50125ab697be None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:12.561279 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b088981-36df-4313-a9e1-8821f292bb5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:12.562151 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b088981-36df-4313-a9e1-8821f292bb5d None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:12.562450 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b088981-36df-4313-a9e1-8821f292bb5d None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:12.562733 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b088981-36df-4313-a9e1-8821f292bb5d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:12.571055 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d43216-9f8d-4db5-a1a5-916022fdfdb9 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:12.573328 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b088981-36df-4313-a9e1-8821f292bb5d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:12.574889 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b088981-36df-4313-a9e1-8821f292bb5d None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:12.579813 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:12.592947 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-7b088981-36df-4313-a9e1-8821f292bb5d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:12.592947 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:12.592947 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:12.592947 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:12.592947 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:12.592947 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:12.592947 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:12.592947 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:12.592947 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:12.592947 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:12.592947 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:12.592947 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:12.592947 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:38:12.592947 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:38:12.592947 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:38:12.593990 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:38:12.593990 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:12.593990 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:12.593990 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:12.593990 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:12.593990 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:12.593990 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:12.593990 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:12.593990 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:38:12.593990 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:12.593990 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:12.593990 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:12.593990 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:12.593990 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:12.593990 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:12.593990 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:12.593990 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:12.594970 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:38:12.594970 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:38:12.594970 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:12.594970 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:38:12.594970 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 273/517] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:38:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:12.595286 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b088981-36df-4313-a9e1-8821f292bb5d None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:12.611581 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23161c5b-f929-4695-924b-3e6f96730dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:12.612250 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23161c5b-f929-4695-924b-3e6f96730dbd None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:12.612540 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23161c5b-f929-4695-924b-3e6f96730dbd None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:12.612819 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23161c5b-f929-4695-924b-3e6f96730dbd None admin] PATH_INFO: `/v3/domains` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:12.613547 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23161c5b-f929-4695-924b-3e6f96730dbd None admin] RBAC: Adding query filter params (name=Default) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:12.613879 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23161c5b-f929-4695-924b-3e6f96730dbd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:12.615259 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23161c5b-f929-4695-924b-3e6f96730dbd None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:12.627551 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 245/518] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:38:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:12.628367 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23161c5b-f929-4695-924b-3e6f96730dbd None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:12.700982 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b088981-36df-4313-a9e1-8821f292bb5d None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:12.710562 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:12.730108 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23161c5b-f929-4695-924b-3e6f96730dbd None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:12.738593 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29618ad-10af-4982-9911-41c9e17ffb45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:12.739270 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29618ad-10af-4982-9911-41c9e17ffb45 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:12.739557 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29618ad-10af-4982-9911-41c9e17ffb45 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:12.739837 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29618ad-10af-4982-9911-41c9e17ffb45 None admin] PATH_INFO: `/v3/users` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:12.740677 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29618ad-10af-4982-9911-41c9e17ffb45 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:12.740988 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29618ad-10af-4982-9911-41c9e17ffb45 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:12.743222 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29618ad-10af-4982-9911-41c9e17ffb45 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:12.760244 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 274/519] 10.222.0.57 () {56 vars in 1137 bytes} [Thu May 19 14:38:12 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:12.761032 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29618ad-10af-4982-9911-41c9e17ffb45 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:12.765248 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96bb503-062a-4437-a832-a37f1457ea47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:12.766242 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96bb503-062a-4437-a832-a37f1457ea47 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:12.766524 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96bb503-062a-4437-a832-a37f1457ea47 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:12.766801 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96bb503-062a-4437-a832-a37f1457ea47 None admin] PATH_INFO: `/v3/projects/service` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:12.773538 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96bb503-062a-4437-a832-a37f1457ea47 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:12.775083 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96bb503-062a-4437-a832-a37f1457ea47 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:12.790852 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-c96bb503-062a-4437-a832-a37f1457ea47 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:38:12.790852 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:12.790852 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:12.790852 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:12.790852 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:12.790852 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:12.790852 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:12.790852 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:12.790852 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:12.790852 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:12.790852 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:12.790852 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:12.790852 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:38:12.790852 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:38:12.790852 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:38:12.792013 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:38:12.792013 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:12.792013 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:12.792013 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:12.792013 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:12.792013 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:12.792013 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:12.792013 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:12.792013 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:38:12.792013 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:12.792013 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:12.792013 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:12.792013 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:12.792013 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:12.792013 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:12.792013 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:12.792013 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:12.793013 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:38:12.793013 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:38:12.793013 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:38:12.793013 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:38:12.793013 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:38:12.793013 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:38:12.793013 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:38:12.793013 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:38:12.793013 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 246/520] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:38:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:12.793013 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96bb503-062a-4437-a832-a37f1457ea47 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:12.822329 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:12.878325 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29618ad-10af-4982-9911-41c9e17ffb45 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:12.896267 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96bb503-062a-4437-a832-a37f1457ea47 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:12.914280 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc23b3fc-ba18-44c6-b1bc-85d570d5f11d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:12.914958 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc23b3fc-ba18-44c6-b1bc-85d570d5f11d None admin] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:12.915241 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc23b3fc-ba18-44c6-b1bc-85d570d5f11d None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:12.915516 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc23b3fc-ba18-44c6-b1bc-85d570d5f11d None admin] PATH_INFO: `/v3/users` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:12.916221 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc23b3fc-ba18-44c6-b1bc-85d570d5f11d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:12.917672 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc23b3fc-ba18-44c6-b1bc-85d570d5f11d None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:12.932116 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edd735f-ec92-4cc2-86db-d39732cb8344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:12.932786 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edd735f-ec92-4cc2-86db-d39732cb8344 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:12.933072 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edd735f-ec92-4cc2-86db-d39732cb8344 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:12.933360 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edd735f-ec92-4cc2-86db-d39732cb8344 None admin] PATH_INFO: `/v3/domains` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:12.934380 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edd735f-ec92-4cc2-86db-d39732cb8344 None admin] RBAC: Adding query filter params (name=Default) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:12.934705 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edd735f-ec92-4cc2-86db-d39732cb8344 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:12.936157 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edd735f-ec92-4cc2-86db-d39732cb8344 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:12.940651 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 275/521] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 19 14:38:12 2022] POST /identity/v3/users => generated 247 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:38:12.940651 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc23b3fc-ba18-44c6-b1bc-85d570d5f11d None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:12.949379 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 247/522] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:38:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:12.955122 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edd735f-ec92-4cc2-86db-d39732cb8344 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:12.984131 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:13.063885 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:13.075555 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc23b3fc-ba18-44c6-b1bc-85d570d5f11d None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:13.079359 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edd735f-ec92-4cc2-86db-d39732cb8344 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:13.117064 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b16391f-0fff-4326-b54e-f573c8a304a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:13.117907 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b16391f-0fff-4326-b54e-f573c8a304a7 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:13.118211 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b16391f-0fff-4326-b54e-f573c8a304a7 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:13.118478 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b16391f-0fff-4326-b54e-f573c8a304a7 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:13.120520 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9eaf2f-6ccf-461f-8e37-ab619fa476b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:13.121356 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9eaf2f-6ccf-461f-8e37-ab619fa476b0 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:13.121644 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9eaf2f-6ccf-461f-8e37-ab619fa476b0 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:13.121947 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9eaf2f-6ccf-461f-8e37-ab619fa476b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:13.127377 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b16391f-0fff-4326-b54e-f573c8a304a7 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:13.128863 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b16391f-0fff-4326-b54e-f573c8a304a7 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:13.145770 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-5b16391f-0fff-4326-b54e-f573c8a304a7 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 19 14:38:13.145770 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:13.145770 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:13.145770 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:13.145770 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:13.145770 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:13.145770 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:13.145770 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:13.145770 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:13.145770 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:13.145770 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:13.145770 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:13.145770 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:38:13.145770 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:38:13.145770 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:38:13.145770 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:38:13.147046 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:13.147046 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:13.147046 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:38:13.147046 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:38:13.147046 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:38:13.147046 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:38:13.147046 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:13.147046 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:13.147046 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:13.147046 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:13.147046 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:13.147046 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:38:13.147046 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:13.147046 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:13.147046 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:13.147046 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:13.147046 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:13.147046 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:13.148801 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:13.148801 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:13.148801 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:38:13.148801 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:38:13.148801 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:38:13.148801 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:38:13.148801 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:38:13.148801 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:38:13.148801 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 19 14:38:13.148801 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:38:13.148801 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 276/523] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 14:38:12 2022] GET /identity/v3/users/neutron => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:13.148801 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b16391f-0fff-4326-b54e-f573c8a304a7 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:13.207143 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9eaf2f-6ccf-461f-8e37-ab619fa476b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:13.209114 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9eaf2f-6ccf-461f-8e37-ab619fa476b0 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:13.246161 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:13.265456 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b16391f-0fff-4326-b54e-f573c8a304a7 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:13.301751 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8002c99d-dd8a-4493-9b91-262624bd4593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:13.302450 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8002c99d-dd8a-4493-9b91-262624bd4593 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:13.302735 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8002c99d-dd8a-4493-9b91-262624bd4593 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:13.303011 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8002c99d-dd8a-4493-9b91-262624bd4593 None admin] PATH_INFO: `/v3/projects` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:13.303799 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8002c99d-dd8a-4493-9b91-262624bd4593 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:13.304216 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8002c99d-dd8a-4493-9b91-262624bd4593 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:13.305781 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8002c99d-dd8a-4493-9b91-262624bd4593 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:13.308206 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:13.329879 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 248/524] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:38:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:38:13.329879 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9eaf2f-6ccf-461f-8e37-ab619fa476b0 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:13.330161 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 277/525] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:38:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:13.330161 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8002c99d-dd8a-4493-9b91-262624bd4593 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:13.438476 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9eaf2f-6ccf-461f-8e37-ab619fa476b0 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:13.442980 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8002c99d-dd8a-4493-9b91-262624bd4593 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:13.475600 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c694e718-4224-4b8e-b97b-a561c35018a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:13.477104 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c694e718-4224-4b8e-b97b-a561c35018a3 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:13.477397 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c694e718-4224-4b8e-b97b-a561c35018a3 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:13.477728 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c694e718-4224-4b8e-b97b-a561c35018a3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:13.484118 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d1dc2a-c86a-4779-a893-9c4db6989eab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:13.484800 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d1dc2a-c86a-4779-a893-9c4db6989eab None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:13.485089 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d1dc2a-c86a-4779-a893-9c4db6989eab None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:13.485224 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c694e718-4224-4b8e-b97b-a561c35018a3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:13.485390 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d1dc2a-c86a-4779-a893-9c4db6989eab None admin] PATH_INFO: `/v3/users` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:13.486316 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d1dc2a-c86a-4779-a893-9c4db6989eab None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:13.486633 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d1dc2a-c86a-4779-a893-9c4db6989eab None admin] RBAC: Authorizing `identity:list_users()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:13.486733 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c694e718-4224-4b8e-b97b-a561c35018a3 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:13.488083 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d1dc2a-c86a-4779-a893-9c4db6989eab None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:13.501483 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-c694e718-4224-4b8e-b97b-a561c35018a3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:38:13.501483 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:13.501483 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:13.501483 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:13.501483 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:13.501483 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:13.501483 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:13.501483 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:13.501483 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:13.501483 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:13.501483 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:13.501483 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:13.501483 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:38:13.501483 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:38:13.501483 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:38:13.503319 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:38:13.503319 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:13.503319 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:13.503319 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:13.503319 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:13.503319 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:13.503319 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:13.503319 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:13.503319 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:38:13.503319 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:13.503319 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:13.503319 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:13.503319 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:13.503319 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:13.503319 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:13.503319 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:13.503319 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:13.504314 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:38:13.504314 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:38:13.504314 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:38:13.504314 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:38:13.504314 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:38:13.504314 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:38:13.504314 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:38:13.504314 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:38:13.504314 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 249/526] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:38:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:13.504314 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c694e718-4224-4b8e-b97b-a561c35018a3 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:13.506723 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:13.510647 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 278/527] 10.222.0.57 () {56 vars in 1139 bytes} [Thu May 19 14:38:13 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:13.511384 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d1dc2a-c86a-4779-a893-9c4db6989eab None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:13.571979 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:13.616359 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d1dc2a-c86a-4779-a893-9c4db6989eab None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:13.639428 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c694e718-4224-4b8e-b97b-a561c35018a3 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:13.654458 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7f13ac-7e1d-412e-b396-7807ecb6c9b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:13.655118 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7f13ac-7e1d-412e-b396-7807ecb6c9b0 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:13.655405 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7f13ac-7e1d-412e-b396-7807ecb6c9b0 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:13.655681 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7f13ac-7e1d-412e-b396-7807ecb6c9b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:13.656518 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7f13ac-7e1d-412e-b396-7807ecb6c9b0 None admin] RBAC: Adding query filter params (role.id=ffb8da90af53491980816087ee621ccb, scope.project.id=42f35b96b9d4499abfb5e5f7cca49adc, user.id=a9591727b2f54244baf90b11a8e2277f) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:13.656831 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7f13ac-7e1d-412e-b396-7807ecb6c9b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:13.658342 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7f13ac-7e1d-412e-b396-7807ecb6c9b0 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:13.665071 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 279/528] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:38:13 2022] GET /identity/v3/role_assignments?user.id=a9591727b2f54244baf90b11a8e2277f&scope.project.id=42f35b96b9d4499abfb5e5f7cca49adc&role.id=ffb8da90af53491980816087ee621ccb => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:13.670860 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7f13ac-7e1d-412e-b396-7807ecb6c9b0 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:13.683071 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b12cea0-3ddb-45a5-badf-28a1c0c32f3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:13.683071 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b12cea0-3ddb-45a5-badf-28a1c0c32f3e None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:13.683071 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b12cea0-3ddb-45a5-badf-28a1c0c32f3e None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:13.683457 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b12cea0-3ddb-45a5-badf-28a1c0c32f3e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:13.699038 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b12cea0-3ddb-45a5-badf-28a1c0c32f3e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:13.700634 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b12cea0-3ddb-45a5-badf-28a1c0c32f3e None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:13.721059 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-5b12cea0-3ddb-45a5-badf-28a1c0c32f3e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:13.721059 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:13.721059 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:13.721059 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:13.721059 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:13.721059 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:13.721059 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:13.721059 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:13.721059 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:13.721059 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:13.721059 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:13.721059 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:13.721059 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:38:13.721059 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:38:13.721059 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:38:13.722135 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:38:13.722135 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:13.722135 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:13.722135 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:13.722135 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:13.722135 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:13.722135 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:13.722135 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:13.722135 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:38:13.722135 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:13.722135 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:13.722135 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:13.722135 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:13.722135 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:13.722135 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:13.722135 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:13.722135 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:13.723534 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:38:13.723534 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:38:13.723534 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:13.723534 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:38:13.723534 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 250/529] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:38:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:13.724614 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b12cea0-3ddb-45a5-badf-28a1c0c32f3e None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:13.783282 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:13.831201 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7f13ac-7e1d-412e-b396-7807ecb6c9b0 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:13.857919 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:13.863389 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b12cea0-3ddb-45a5-badf-28a1c0c32f3e None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:13.885389 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a166a4e1-57dd-4e8c-9a5d-ffdf9c39ee20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:13.886383 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a166a4e1-57dd-4e8c-9a5d-ffdf9c39ee20 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:13.886678 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a166a4e1-57dd-4e8c-9a5d-ffdf9c39ee20 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:13.886962 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a166a4e1-57dd-4e8c-9a5d-ffdf9c39ee20 None admin] PATH_INFO: `/v3/projects` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:13.888287 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a166a4e1-57dd-4e8c-9a5d-ffdf9c39ee20 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:13.888608 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a166a4e1-57dd-4e8c-9a5d-ffdf9c39ee20 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:13.890410 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a166a4e1-57dd-4e8c-9a5d-ffdf9c39ee20 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:13.908289 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 280/530] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:38:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:13.909410 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a166a4e1-57dd-4e8c-9a5d-ffdf9c39ee20 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:13.914532 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db39aeaa-ea3f-4179-9f59-907708d6d6f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:13.915183 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db39aeaa-ea3f-4179-9f59-907708d6d6f7 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:13.915475 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db39aeaa-ea3f-4179-9f59-907708d6d6f7 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:13.915740 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db39aeaa-ea3f-4179-9f59-907708d6d6f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:14.014765 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db39aeaa-ea3f-4179-9f59-907708d6d6f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:14.016984 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db39aeaa-ea3f-4179-9f59-907708d6d6f7 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:14.046611 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a166a4e1-57dd-4e8c-9a5d-ffdf9c39ee20 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:14.079539 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:14.097564 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe6756d-1d73-41aa-8458-905f4f9c60b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:14.098863 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe6756d-1d73-41aa-8458-905f4f9c60b4 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:14.099389 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe6756d-1d73-41aa-8458-905f4f9c60b4 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:14.099878 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe6756d-1d73-41aa-8458-905f4f9c60b4 None admin] PATH_INFO: `/v3/domains` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:14.101087 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe6756d-1d73-41aa-8458-905f4f9c60b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:14.101649 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe6756d-1d73-41aa-8458-905f4f9c60b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:14.104054 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe6756d-1d73-41aa-8458-905f4f9c60b4 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:14.120051 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 281/531] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:38:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:14.121394 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe6756d-1d73-41aa-8458-905f4f9c60b4 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:14.137989 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:14.149292 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 251/532] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:38:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:38:14.150355 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db39aeaa-ea3f-4179-9f59-907708d6d6f7 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:14.254950 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe6756d-1d73-41aa-8458-905f4f9c60b4 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:14.265330 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db39aeaa-ea3f-4179-9f59-907708d6d6f7 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:14.293236 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd51f95-399f-4b03-9c53-87d3cf456c5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:14.293995 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd51f95-399f-4b03-9c53-87d3cf456c5f None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:14.294289 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd51f95-399f-4b03-9c53-87d3cf456c5f None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:14.294624 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd51f95-399f-4b03-9c53-87d3cf456c5f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:14.295470 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd51f95-399f-4b03-9c53-87d3cf456c5f None admin] RBAC: Adding query filter params (role.id=ffb8da90af53491980816087ee621ccb, scope.project.id=42f35b96b9d4499abfb5e5f7cca49adc, user.id=ca35fa8e73ce489081eb7ea2ca5c5e09) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:14.295791 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd51f95-399f-4b03-9c53-87d3cf456c5f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:14.297302 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd51f95-399f-4b03-9c53-87d3cf456c5f None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:14.301606 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a33f51-3385-49d9-8901-a6397bf4eaa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:14.302449 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a33f51-3385-49d9-8901-a6397bf4eaa8 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:14.302740 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a33f51-3385-49d9-8901-a6397bf4eaa8 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:14.303020 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a33f51-3385-49d9-8901-a6397bf4eaa8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:14.303440 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 282/533] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:38:14 2022] GET /identity/v3/role_assignments?user.id=ca35fa8e73ce489081eb7ea2ca5c5e09&scope.project.id=42f35b96b9d4499abfb5e5f7cca49adc&role.id=ffb8da90af53491980816087ee621ccb => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:14.304295 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd51f95-399f-4b03-9c53-87d3cf456c5f None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:14.327685 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a33f51-3385-49d9-8901-a6397bf4eaa8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:14.329223 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a33f51-3385-49d9-8901-a6397bf4eaa8 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:14.353592 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-45a33f51-3385-49d9-8901-a6397bf4eaa8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:38:14.353592 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:14.353592 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:14.353592 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:14.353592 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:14.353592 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:14.353592 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:14.353592 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:14.353592 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:14.353592 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:14.353592 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:14.353592 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:14.353592 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:38:14.353592 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:38:14.353592 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:38:14.354651 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:38:14.354651 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:14.354651 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:14.354651 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:14.354651 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:14.354651 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:14.354651 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:14.354651 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:14.354651 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:38:14.354651 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:14.354651 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:14.354651 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:14.354651 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:14.354651 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:14.354651 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:14.354651 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:14.354651 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:14.355634 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:38:14.355634 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:38:14.355634 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:38:14.355634 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:38:14.355634 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:38:14.355634 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:38:14.355634 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:38:14.355634 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:38:14.368434 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 252/534] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:38:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:14.373398 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a33f51-3385-49d9-8901-a6397bf4eaa8 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:14.427254 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:14.510252 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:14.541386 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd51f95-399f-4b03-9c53-87d3cf456c5f None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:14.566340 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a33f51-3385-49d9-8901-a6397bf4eaa8 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:14.604395 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c0b9cb-52c4-410e-9602-f8118f8b7ae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:14.605065 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c0b9cb-52c4-410e-9602-f8118f8b7ae1 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:14.605355 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c0b9cb-52c4-410e-9602-f8118f8b7ae1 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:14.605635 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c0b9cb-52c4-410e-9602-f8118f8b7ae1 None admin] PATH_INFO: `/v3/projects` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:14.606577 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c0b9cb-52c4-410e-9602-f8118f8b7ae1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:14.606898 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c0b9cb-52c4-410e-9602-f8118f8b7ae1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:14.608462 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c0b9cb-52c4-410e-9602-f8118f8b7ae1 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:14.627465 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b685c73c-582c-4a9d-bcf1-4c0ca1a84b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:14.628112 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 253/535] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:38:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:14.628962 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b685c73c-582c-4a9d-bcf1-4c0ca1a84b0c None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:14.629249 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b685c73c-582c-4a9d-bcf1-4c0ca1a84b0c None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:14.629520 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b685c73c-582c-4a9d-bcf1-4c0ca1a84b0c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:14.639028 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c0b9cb-52c4-410e-9602-f8118f8b7ae1 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:14.641475 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b685c73c-582c-4a9d-bcf1-4c0ca1a84b0c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:14.643251 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b685c73c-582c-4a9d-bcf1-4c0ca1a84b0c None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:14.661915 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-b685c73c-582c-4a9d-bcf1-4c0ca1a84b0c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:14.661915 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:14.661915 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:14.661915 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:14.661915 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:14.661915 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:14.661915 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:14.661915 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:14.661915 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:14.661915 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:14.661915 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:14.661915 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:14.661915 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:38:14.661915 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:38:14.661915 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:38:14.662905 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:38:14.662905 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:14.662905 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:14.662905 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:14.662905 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:14.662905 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:14.662905 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:14.662905 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:14.662905 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:38:14.662905 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:14.662905 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:14.662905 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:14.662905 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:14.662905 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:14.662905 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:14.662905 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:14.662905 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:14.664762 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:38:14.664762 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:38:14.664762 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:14.664762 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:38:14.666945 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 283/536] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:38:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 363 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:14.674000 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b685c73c-582c-4a9d-bcf1-4c0ca1a84b0c None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:14.766737 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c0b9cb-52c4-410e-9602-f8118f8b7ae1 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:14.791191 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:14.820220 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80988c03-dd63-4d77-a0bc-a4deaa71f759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:14.821225 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80988c03-dd63-4d77-a0bc-a4deaa71f759 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:14.821739 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80988c03-dd63-4d77-a0bc-a4deaa71f759 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:14.822255 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80988c03-dd63-4d77-a0bc-a4deaa71f759 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:14.823378 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80988c03-dd63-4d77-a0bc-a4deaa71f759 None admin] RBAC: Adding query filter params (role.id=ffb8da90af53491980816087ee621ccb, scope.project.id=42f35b96b9d4499abfb5e5f7cca49adc, user.id=aa7df5964bed4638a9bf19d37971ec20) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:14.823961 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80988c03-dd63-4d77-a0bc-a4deaa71f759 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:14.825744 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80988c03-dd63-4d77-a0bc-a4deaa71f759 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:14.838269 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 254/537] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:38:14 2022] GET /identity/v3/role_assignments?user.id=aa7df5964bed4638a9bf19d37971ec20&scope.project.id=42f35b96b9d4499abfb5e5f7cca49adc&role.id=ffb8da90af53491980816087ee621ccb => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:14.858280 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:14.861486 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b685c73c-582c-4a9d-bcf1-4c0ca1a84b0c None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:14.907220 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a9ef99-16c0-476a-bede-d5c87682c936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:14.907916 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a9ef99-16c0-476a-bede-d5c87682c936 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:14.908201 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a9ef99-16c0-476a-bede-d5c87682c936 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:14.908480 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a9ef99-16c0-476a-bede-d5c87682c936 None admin] PATH_INFO: `/v3/domains` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:14.909233 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a9ef99-16c0-476a-bede-d5c87682c936 None admin] RBAC: Adding query filter params (name=Default) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:14.909546 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a9ef99-16c0-476a-bede-d5c87682c936 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:14.911042 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a9ef99-16c0-476a-bede-d5c87682c936 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:14.926015 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 284/538] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:38:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:14.931657 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80988c03-dd63-4d77-a0bc-a4deaa71f759 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:15.098513 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80988c03-dd63-4d77-a0bc-a4deaa71f759 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:15.136689 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dfcbb5-94e3-4707-a0c0-3bc17dd50895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:15.137486 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dfcbb5-94e3-4707-a0c0-3bc17dd50895 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:15.137775 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dfcbb5-94e3-4707-a0c0-3bc17dd50895 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:15.138081 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dfcbb5-94e3-4707-a0c0-3bc17dd50895 None admin] PATH_INFO: `/v3/projects/service` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:15.145244 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dfcbb5-94e3-4707-a0c0-3bc17dd50895 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:15.146733 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dfcbb5-94e3-4707-a0c0-3bc17dd50895 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:15.162027 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-b3dfcbb5-94e3-4707-a0c0-3bc17dd50895 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:38:15.162027 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:15.162027 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:15.162027 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:15.162027 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:15.162027 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:15.162027 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:15.162027 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:15.162027 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:15.162027 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:15.162027 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:15.162027 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:15.162027 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:38:15.162027 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:38:15.162027 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:38:15.163034 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:38:15.163034 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:15.163034 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:15.163034 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:15.163034 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:15.163034 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:15.163034 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:15.163034 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:15.163034 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:38:15.163034 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:15.163034 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:15.163034 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:15.163034 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:15.163034 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:15.163034 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:15.163034 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:15.163034 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:15.164854 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:38:15.164854 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:38:15.164854 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:38:15.164854 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:38:15.164854 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:38:15.164854 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:38:15.164854 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:38:15.164854 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:38:15.164854 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 255/539] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:38:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:15.169043 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a9ef99-16c0-476a-bede-d5c87682c936 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:15.282446 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a9ef99-16c0-476a-bede-d5c87682c936 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:15.321155 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96225920-bdd4-4898-8e54-01ae1d857168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:15.321830 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96225920-bdd4-4898-8e54-01ae1d857168 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:15.322145 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96225920-bdd4-4898-8e54-01ae1d857168 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:15.322423 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96225920-bdd4-4898-8e54-01ae1d857168 None admin] PATH_INFO: `/v3/projects` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:15.323212 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96225920-bdd4-4898-8e54-01ae1d857168 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:15.323527 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96225920-bdd4-4898-8e54-01ae1d857168 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:15.324980 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96225920-bdd4-4898-8e54-01ae1d857168 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:15.338493 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 285/540] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:38:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:15.346951 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dfcbb5-94e3-4707-a0c0-3bc17dd50895 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:15.397304 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:15.435166 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28082c98-28d1-448a-a48e-e68a9da7b422 None None] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:15.435818 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28082c98-28d1-448a-a48e-e68a9da7b422 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:15.436305 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28082c98-28d1-448a-a48e-e68a9da7b422 None None] PATH_INFO: `/` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:15.438204 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 286/541] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:38:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:38:15.443943 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddc5081-545d-416d-b94e-6e43310e4146 None None] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:15.444515 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddc5081-545d-416d-b94e-6e43310e4146 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:15.445045 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddc5081-545d-416d-b94e-6e43310e4146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:15.466007 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.auth.core [None req-cddc5081-545d-416d-b94e-6e43310e4146 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:38:15.467603 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.fernet_utils [None req-cddc5081-545d-416d-b94e-6e43310e4146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:38:15.474655 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dfcbb5-94e3-4707-a0c0-3bc17dd50895 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:15.487777 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:15.549075 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 287/542] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:38:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:38:15.576123 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddc5081-545d-416d-b94e-6e43310e4146 None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:15.591143 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9113888-ed2c-4a62-89dc-a96d081294c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:15.591850 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9113888-ed2c-4a62-89dc-a96d081294c3 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:15.592104 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9113888-ed2c-4a62-89dc-a96d081294c3 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:15.592381 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9113888-ed2c-4a62-89dc-a96d081294c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:15.593234 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9113888-ed2c-4a62-89dc-a96d081294c3 None admin] RBAC: Adding query filter params (role.id=ffb8da90af53491980816087ee621ccb, scope.project.id=42f35b96b9d4499abfb5e5f7cca49adc, user.id=c7cd031eea90473aaaa0f1375f04dc66) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:15.593574 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9113888-ed2c-4a62-89dc-a96d081294c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:15.595064 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9113888-ed2c-4a62-89dc-a96d081294c3 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:15.612163 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 256/543] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:38:15 2022] GET /identity/v3/role_assignments?user.id=c7cd031eea90473aaaa0f1375f04dc66&scope.project.id=42f35b96b9d4499abfb5e5f7cca49adc&role.id=ffb8da90af53491980816087ee621ccb => generated 259 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:15.691167 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddc5081-545d-416d-b94e-6e43310e4146 None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:15.737678 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae448ed6-4354-43ab-abc9-50f5784e6d9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:15.738540 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae448ed6-4354-43ab-abc9-50f5784e6d9e None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:15.738820 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae448ed6-4354-43ab-abc9-50f5784e6d9e None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:15.739092 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae448ed6-4354-43ab-abc9-50f5784e6d9e None admin] PATH_INFO: `/v3/roles/service` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:15.757855 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae448ed6-4354-43ab-abc9-50f5784e6d9e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:38:15.774428 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-ae448ed6-4354-43ab-abc9-50f5784e6d9e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:38:15.774428 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:15.774428 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:15.774428 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:15.774428 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:15.774428 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:15.774428 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:15.774428 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:15.774428 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:15.774428 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:15.774428 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:15.774428 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:15.774428 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:38:15.774428 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:38:15.774428 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:38:15.774428 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:38:15.775530 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:38:15.775530 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:38:15.775530 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:38:15.775530 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:38:15.775530 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:15.775530 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:15.775530 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:15.775530 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:15.775530 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:15.775530 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:15.775530 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:15.775530 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:38:15.775530 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:15.775530 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:15.775530 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:15.775530 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:15.776434 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:15.776434 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:15.776434 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:15.776434 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:15.776434 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:38:15.776434 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:38:15.776434 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:38:15.776434 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:38:15.776434 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:38:15.776434 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:38:15.776434 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:38:15.776434 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:38:15.776434 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 288/544] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 14:38:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:38:15.784125 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9113888-ed2c-4a62-89dc-a96d081294c3 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:15.956004 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9113888-ed2c-4a62-89dc-a96d081294c3 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:15.969929 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:16.012100 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1891d2c6-0656-4856-a7d6-952b582c21d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:16.012799 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1891d2c6-0656-4856-a7d6-952b582c21d0 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:16.013086 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1891d2c6-0656-4856-a7d6-952b582c21d0 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:16.013368 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1891d2c6-0656-4856-a7d6-952b582c21d0 None admin] PATH_INFO: `/v3/roles` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:16.014193 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1891d2c6-0656-4856-a7d6-952b582c21d0 None admin] RBAC: Adding query filter params (name=service) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:16.014510 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1891d2c6-0656-4856-a7d6-952b582c21d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:16.015958 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1891d2c6-0656-4856-a7d6-952b582c21d0 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:16.030954 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 257/545] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 19 14:38:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:16.036557 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae448ed6-4354-43ab-abc9-50f5784e6d9e None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:16.059335 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:16.213016 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae448ed6-4354-43ab-abc9-50f5784e6d9e None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:16.263292 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17751c19-7fe1-42d7-b1d3-563116473c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:16.264357 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17751c19-7fe1-42d7-b1d3-563116473c9c None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:16.264802 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17751c19-7fe1-42d7-b1d3-563116473c9c None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:16.265239 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17751c19-7fe1-42d7-b1d3-563116473c9c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:16.283030 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17751c19-7fe1-42d7-b1d3-563116473c9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:16.284777 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17751c19-7fe1-42d7-b1d3-563116473c9c None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:16.291486 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512e00e5-a6bf-4d9b-a634-ea88273519cd None None] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:16.291783 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512e00e5-a6bf-4d9b-a634-ea88273519cd None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:16.292073 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512e00e5-a6bf-4d9b-a634-ea88273519cd None None] PATH_INFO: `/` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:16.293750 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 258/546] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:38:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:38:16.299186 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65740214-ec76-4d52-a61a-03ddf8c3edf4 None None] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:16.299476 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65740214-ec76-4d52-a61a-03ddf8c3edf4 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:16.299755 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65740214-ec76-4d52-a61a-03ddf8c3edf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:16.305288 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-17751c19-7fe1-42d7-b1d3-563116473c9c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:16.305288 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:16.305288 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:16.305288 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:16.305288 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:16.305288 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:16.305288 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:16.305288 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:16.305288 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:16.305288 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:16.305288 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:16.305288 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:16.305288 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:38:16.305288 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:38:16.305288 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:38:16.306471 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:38:16.306471 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:16.306471 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:16.306471 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:16.306471 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:16.306471 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:16.306471 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:16.306471 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:16.306471 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:38:16.306471 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:16.306471 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:16.306471 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:16.306471 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:16.306471 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:16.306471 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:16.306471 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:16.306471 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:16.307666 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:38:16.307666 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:38:16.307666 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:16.307666 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:38:16.309625 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 289/547] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:38:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:16.314852 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17751c19-7fe1-42d7-b1d3-563116473c9c None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:16.341737 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.auth.core [None req-65740214-ec76-4d52-a61a-03ddf8c3edf4 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:38:16.342966 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-65740214-ec76-4d52-a61a-03ddf8c3edf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:38:16.407383 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:16.473282 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17751c19-7fe1-42d7-b1d3-563116473c9c None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:16.487691 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 259/548] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:38:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:38:16.520518 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65740214-ec76-4d52-a61a-03ddf8c3edf4 None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:16.549907 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3355b1d-3974-45c3-8504-ce9cc6b5819d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:16.552331 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3355b1d-3974-45c3-8504-ce9cc6b5819d None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:16.552331 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3355b1d-3974-45c3-8504-ce9cc6b5819d None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:16.552331 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3355b1d-3974-45c3-8504-ce9cc6b5819d None admin] PATH_INFO: `/v3/domains` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:16.552999 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3355b1d-3974-45c3-8504-ce9cc6b5819d None admin] RBAC: Adding query filter params (name=Default) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:16.553312 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3355b1d-3974-45c3-8504-ce9cc6b5819d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:16.555484 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3355b1d-3974-45c3-8504-ce9cc6b5819d None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:16.582979 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 290/549] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:38:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:16.597004 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3355b1d-3974-45c3-8504-ce9cc6b5819d None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:16.659822 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:16.668566 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65740214-ec76-4d52-a61a-03ddf8c3edf4 None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:16.705037 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf605db-9d9d-488d-93a9-44b75360f965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:16.706277 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf605db-9d9d-488d-93a9-44b75360f965 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:16.706789 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf605db-9d9d-488d-93a9-44b75360f965 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:16.707285 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf605db-9d9d-488d-93a9-44b75360f965 None admin] PATH_INFO: `/v3/roles/service` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:16.719431 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf605db-9d9d-488d-93a9-44b75360f965 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:38:16.728585 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-ddf605db-9d9d-488d-93a9-44b75360f965 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:38:16.728585 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:16.728585 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:16.728585 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:16.728585 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:16.728585 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:16.728585 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:16.728585 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:16.728585 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:16.728585 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:16.728585 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:16.728585 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:16.728585 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:38:16.728585 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:38:16.728585 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:38:16.728585 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:38:16.729669 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:38:16.729669 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:38:16.729669 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:38:16.729669 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:38:16.729669 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:16.729669 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:16.729669 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:16.729669 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:16.729669 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:16.729669 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:16.729669 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:16.729669 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:38:16.729669 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:16.729669 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:16.729669 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:16.729669 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:16.730693 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:16.730693 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:16.730693 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:16.730693 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:16.730693 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:38:16.730693 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:38:16.730693 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:38:16.730693 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:38:16.730693 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:38:16.730693 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:38:16.730693 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:38:16.730693 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:38:16.733592 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 260/550] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 14:38:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:38:16.738900 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf605db-9d9d-488d-93a9-44b75360f965 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:16.803039 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3355b1d-3974-45c3-8504-ce9cc6b5819d None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:16.853048 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:16.854548 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf605db-9d9d-488d-93a9-44b75360f965 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:16.880309 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2ca6ad-64cf-40f4-b58b-db56d5912e43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:16.881152 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2ca6ad-64cf-40f4-b58b-db56d5912e43 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:16.881442 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2ca6ad-64cf-40f4-b58b-db56d5912e43 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:16.881716 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2ca6ad-64cf-40f4-b58b-db56d5912e43 None admin] PATH_INFO: `/v3/users/swift` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:16.895660 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041244d2-6e30-4ab8-96ff-753b7ae11187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:16.896656 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041244d2-6e30-4ab8-96ff-753b7ae11187 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:16.897162 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041244d2-6e30-4ab8-96ff-753b7ae11187 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:16.897948 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041244d2-6e30-4ab8-96ff-753b7ae11187 None admin] PATH_INFO: `/v3/roles` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:16.898941 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041244d2-6e30-4ab8-96ff-753b7ae11187 None admin] RBAC: Adding query filter params (name=service) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:16.899532 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041244d2-6e30-4ab8-96ff-753b7ae11187 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:16.901138 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041244d2-6e30-4ab8-96ff-753b7ae11187 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:16.907135 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2ca6ad-64cf-40f4-b58b-db56d5912e43 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:16.908596 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2ca6ad-64cf-40f4-b58b-db56d5912e43 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:16.911053 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 261/551] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 19 14:38:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:38:16.917073 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041244d2-6e30-4ab8-96ff-753b7ae11187 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:16.942238 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-dd2ca6ad-64cf-40f4-b58b-db56d5912e43 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 14:38:16.942238 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:16.942238 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:16.942238 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:16.942238 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:16.942238 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:16.942238 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:16.942238 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:16.942238 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:16.942238 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:16.942238 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:16.942238 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:16.942238 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:38:16.942238 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:38:16.942238 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:38:16.942238 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:38:16.943365 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:16.943365 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:16.943365 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:38:16.943365 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:38:16.943365 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:38:16.943365 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:38:16.943365 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:16.943365 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:16.943365 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:16.943365 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:16.943365 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:16.943365 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:38:16.943365 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:16.943365 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:16.943365 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:16.943365 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:16.943365 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:16.943365 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:16.944403 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:16.944403 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:16.944403 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:38:16.944403 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:38:16.944403 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:38:16.944403 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:38:16.944403 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:38:16.944403 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:38:16.944403 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 14:38:16.944403 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:38:16.949824 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 291/552] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:38:16 2022] GET /identity/v3/users/swift => generated 83 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:38:16.956481 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2ca6ad-64cf-40f4-b58b-db56d5912e43 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:16.986319 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:17.119850 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2ca6ad-64cf-40f4-b58b-db56d5912e43 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:17.163990 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041244d2-6e30-4ab8-96ff-753b7ae11187 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:17.192076 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88e570f-8a37-4559-835b-91db59737d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:17.192768 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88e570f-8a37-4559-835b-91db59737d28 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:17.193053 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88e570f-8a37-4559-835b-91db59737d28 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:17.193631 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88e570f-8a37-4559-835b-91db59737d28 None admin] PATH_INFO: `/v3/users` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:17.195544 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88e570f-8a37-4559-835b-91db59737d28 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:17.195862 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88e570f-8a37-4559-835b-91db59737d28 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:17.198191 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88e570f-8a37-4559-835b-91db59737d28 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:17.208187 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1cab60-7034-4744-bd38-8ce5354fb7af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:17.209013 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1cab60-7034-4744-bd38-8ce5354fb7af None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:17.209304 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1cab60-7034-4744-bd38-8ce5354fb7af None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:17.209580 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1cab60-7034-4744-bd38-8ce5354fb7af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:17.230808 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1cab60-7034-4744-bd38-8ce5354fb7af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:17.232095 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 292/553] 10.222.0.57 () {56 vars in 1135 bytes} [Thu May 19 14:38:16 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:38:17.232299 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1cab60-7034-4744-bd38-8ce5354fb7af None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:17.233457 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a63fe4-c1d0-488b-a791-a9b01812823d None None] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:17.233751 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a63fe4-c1d0-488b-a791-a9b01812823d None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:17.234081 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a63fe4-c1d0-488b-a791-a9b01812823d None None] PATH_INFO: `/` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:17.235626 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 293/554] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:38:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:38:17.238156 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a63fe4-c1d0-488b-a791-a9b01812823d None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:17.266671 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-eb1cab60-7034-4744-bd38-8ce5354fb7af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:17.266671 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:17.266671 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:17.266671 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:17.266671 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:17.266671 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:17.266671 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:17.266671 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:17.266671 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:17.266671 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:17.266671 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:17.266671 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:17.266671 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:38:17.266671 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:38:17.266671 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:38:17.267750 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:38:17.267750 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:17.267750 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:17.267750 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:17.267750 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:17.267750 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:17.267750 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:17.267750 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:17.267750 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:38:17.267750 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:17.267750 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:17.267750 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:17.267750 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:17.267750 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:17.267750 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:17.267750 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:17.267750 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:17.268755 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:38:17.268755 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:38:17.268755 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:17.268755 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:38:17.271617 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 262/555] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:38:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 355 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:17.273869 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84104a14-f233-4a3c-8fe6-c469721b4669 None None] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:17.274331 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84104a14-f233-4a3c-8fe6-c469721b4669 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:17.274763 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84104a14-f233-4a3c-8fe6-c469721b4669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:17.293018 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:17.300348 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.auth.core [None req-84104a14-f233-4a3c-8fe6-c469721b4669 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:38:17.302100 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-84104a14-f233-4a3c-8fe6-c469721b4669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:38:17.364209 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a63fe4-c1d0-488b-a791-a9b01812823d None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:17.400997 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a1c8d2-31a6-41a5-a825-72c1a3cd7ff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:17.407431 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a1c8d2-31a6-41a5-a825-72c1a3cd7ff1 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:17.407431 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a1c8d2-31a6-41a5-a825-72c1a3cd7ff1 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:17.407431 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a1c8d2-31a6-41a5-a825-72c1a3cd7ff1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:17.407757 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 263/556] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:38:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:38:17.407757 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84104a14-f233-4a3c-8fe6-c469721b4669 None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:17.414829 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:17.503091 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a1c8d2-31a6-41a5-a825-72c1a3cd7ff1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:17.506002 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a1c8d2-31a6-41a5-a825-72c1a3cd7ff1 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:17.545470 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84104a14-f233-4a3c-8fe6-c469721b4669 None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:17.559589 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:17.595284 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8896d0-60e0-49af-92bf-afb529fbf480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:17.595972 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8896d0-60e0-49af-92bf-afb529fbf480 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:17.596260 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8896d0-60e0-49af-92bf-afb529fbf480 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:17.596530 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8896d0-60e0-49af-92bf-afb529fbf480 None admin] PATH_INFO: `/v3/domains` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:17.597326 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8896d0-60e0-49af-92bf-afb529fbf480 None admin] RBAC: Adding query filter params (name=Default) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:17.597650 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8896d0-60e0-49af-92bf-afb529fbf480 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:17.599184 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8896d0-60e0-49af-92bf-afb529fbf480 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:17.617945 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 264/557] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:38:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:17.619170 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8896d0-60e0-49af-92bf-afb529fbf480 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:17.639358 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 294/558] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:38:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 14:38:17.640494 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a1c8d2-31a6-41a5-a825-72c1a3cd7ff1 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:17.691626 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:17.745424 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8896d0-60e0-49af-92bf-afb529fbf480 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:17.768157 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a1c8d2-31a6-41a5-a825-72c1a3cd7ff1 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:17.784680 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2074be14-bf7f-47ba-ab08-92e019fe8987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:17.785658 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2074be14-bf7f-47ba-ab08-92e019fe8987 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:17.785982 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2074be14-bf7f-47ba-ab08-92e019fe8987 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:17.786349 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2074be14-bf7f-47ba-ab08-92e019fe8987 None admin] PATH_INFO: `/v3/roles/service` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:17.799083 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2074be14-bf7f-47ba-ab08-92e019fe8987 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:38:17.805936 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606590c9-912b-482f-9479-f757c4d8d06b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:17.806732 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606590c9-912b-482f-9479-f757c4d8d06b None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:17.807015 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606590c9-912b-482f-9479-f757c4d8d06b None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:17.807288 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606590c9-912b-482f-9479-f757c4d8d06b None admin] PATH_INFO: `/v3/users/nova` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:17.807592 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-2074be14-bf7f-47ba-ab08-92e019fe8987 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:38:17.807592 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:17.807592 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:17.807592 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:17.807592 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:17.807592 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:17.807592 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:17.807592 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:17.807592 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:17.807592 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:17.807592 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:17.807592 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:17.807592 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:38:17.807592 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:38:17.807592 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:38:17.807592 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:38:17.808556 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:38:17.808556 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:38:17.808556 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:38:17.808556 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:38:17.808556 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:17.808556 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:17.808556 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:17.808556 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:17.808556 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:17.808556 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:17.808556 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:17.808556 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:38:17.808556 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:17.808556 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:17.808556 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:17.808556 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:17.809497 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:17.809497 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:17.809497 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:17.809497 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:17.809497 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:38:17.809497 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:38:17.809497 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:38:17.809497 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:38:17.809497 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:38:17.809497 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:38:17.809497 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:38:17.809497 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:38:17.809497 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 265/559] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 14:38:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:17.810330 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2074be14-bf7f-47ba-ab08-92e019fe8987 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:17.821508 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606590c9-912b-482f-9479-f757c4d8d06b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:17.823015 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606590c9-912b-482f-9479-f757c4d8d06b None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:17.826178 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:17.839683 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-606590c9-912b-482f-9479-f757c4d8d06b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 14:38:17.839683 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:17.839683 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:17.839683 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:17.839683 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:17.839683 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:17.839683 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:17.839683 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:17.839683 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:17.839683 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:17.839683 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:17.839683 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:17.839683 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:38:17.839683 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:38:17.839683 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:38:17.839683 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:38:17.840950 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:17.840950 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:17.840950 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:38:17.840950 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:38:17.840950 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:38:17.840950 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:38:17.840950 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:17.840950 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:17.840950 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:17.840950 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:17.840950 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:17.840950 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:38:17.840950 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:17.840950 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:17.840950 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:17.840950 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:17.840950 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:17.840950 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:17.842028 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:17.842028 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:17.842028 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:38:17.842028 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:38:17.842028 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:38:17.842028 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:38:17.842028 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:38:17.842028 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:38:17.842028 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 14:38:17.842028 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:38:17.849042 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 295/560] 10.222.0.57 () {56 vars in 1093 bytes} [Thu May 19 14:38:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:17.849042 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28a2d0e-19ca-47b1-a438-530ae8e837a7 None None] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:17.849042 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28a2d0e-19ca-47b1-a438-530ae8e837a7 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:17.849042 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28a2d0e-19ca-47b1-a438-530ae8e837a7 None None] PATH_INFO: `/` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:17.849042 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 296/561] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:38:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:38:17.849042 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28a2d0e-19ca-47b1-a438-530ae8e837a7 None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:17.932437 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2074be14-bf7f-47ba-ab08-92e019fe8987 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:17.958475 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28a2d0e-19ca-47b1-a438-530ae8e837a7 None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:17.960966 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:17.974795 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd41dbd-daa0-4c7b-95f7-599d3a1c8e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:17.975746 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd41dbd-daa0-4c7b-95f7-599d3a1c8e11 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:17.977159 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd41dbd-daa0-4c7b-95f7-599d3a1c8e11 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:17.977471 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd41dbd-daa0-4c7b-95f7-599d3a1c8e11 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:17.988529 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd41dbd-daa0-4c7b-95f7-599d3a1c8e11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:17.990010 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd41dbd-daa0-4c7b-95f7-599d3a1c8e11 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:18.000091 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9b425a-6672-4d1c-b429-8a888e494a0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:18.001307 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9b425a-6672-4d1c-b429-8a888e494a0a None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:18.001805 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9b425a-6672-4d1c-b429-8a888e494a0a None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:18.002314 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9b425a-6672-4d1c-b429-8a888e494a0a None admin] PATH_INFO: `/v3/roles` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:18.003259 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9b425a-6672-4d1c-b429-8a888e494a0a None admin] RBAC: Adding query filter params (name=service) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:18.003772 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9b425a-6672-4d1c-b429-8a888e494a0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:18.005345 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9b425a-6672-4d1c-b429-8a888e494a0a None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:18.009204 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-ffd41dbd-daa0-4c7b-95f7-599d3a1c8e11 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:18.009204 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:18.009204 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:18.009204 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:18.009204 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:18.009204 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:18.009204 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:18.009204 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:18.009204 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:18.009204 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:18.009204 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:18.009204 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:18.009204 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:38:18.009204 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:38:18.009204 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:38:18.010313 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:38:18.010313 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:18.010313 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:18.010313 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:18.010313 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:18.010313 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:18.010313 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:18.010313 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:18.010313 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:38:18.010313 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:18.010313 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:18.010313 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:18.010313 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:18.010313 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:18.010313 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:18.010313 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:18.010313 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:18.011321 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:38:18.011321 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:38:18.011321 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:18.011321 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:38:18.014066 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 266/562] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:38:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:18.015146 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd41dbd-daa0-4c7b-95f7-599d3a1c8e11 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:18.015955 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 297/563] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 19 14:38:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:18.020624 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600f72f7-b95b-42f8-9fda-b53d59d3b811 None None] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:18.020624 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600f72f7-b95b-42f8-9fda-b53d59d3b811 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:18.020624 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600f72f7-b95b-42f8-9fda-b53d59d3b811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:18.036359 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.auth.core [None req-600f72f7-b95b-42f8-9fda-b53d59d3b811 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:38:18.037582 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.fernet_utils [None req-600f72f7-b95b-42f8-9fda-b53d59d3b811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:38:18.102660 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:18.119147 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 298/564] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:38:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:38:18.120239 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600f72f7-b95b-42f8-9fda-b53d59d3b811 None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:18.126891 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd41dbd-daa0-4c7b-95f7-599d3a1c8e11 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:18.169202 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9511842f-541e-4f54-b398-f3c9e197334d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:18.169811 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9511842f-541e-4f54-b398-f3c9e197334d None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:18.170197 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9511842f-541e-4f54-b398-f3c9e197334d None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:18.170472 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9511842f-541e-4f54-b398-f3c9e197334d None admin] PATH_INFO: `/v3/users` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:18.171309 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9511842f-541e-4f54-b398-f3c9e197334d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:18.171618 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9511842f-541e-4f54-b398-f3c9e197334d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:18.173079 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9511842f-541e-4f54-b398-f3c9e197334d None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:18.192654 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 267/565] 10.222.0.57 () {56 vars in 1133 bytes} [Thu May 19 14:38:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:18.193818 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9511842f-541e-4f54-b398-f3c9e197334d None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:18.234025 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:18.258884 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600f72f7-b95b-42f8-9fda-b53d59d3b811 None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:18.296400 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d5b955-4a7a-48ae-b0d2-aed4a4f94fea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:18.297031 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d5b955-4a7a-48ae-b0d2-aed4a4f94fea None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:18.297315 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d5b955-4a7a-48ae-b0d2-aed4a4f94fea None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:18.297586 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d5b955-4a7a-48ae-b0d2-aed4a4f94fea None admin] PATH_INFO: `/v3/domains` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:18.298347 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d5b955-4a7a-48ae-b0d2-aed4a4f94fea None admin] RBAC: Adding query filter params (name=Default) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:18.298650 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d5b955-4a7a-48ae-b0d2-aed4a4f94fea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:18.300044 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d5b955-4a7a-48ae-b0d2-aed4a4f94fea None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:18.309534 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9511842f-541e-4f54-b398-f3c9e197334d None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:18.311913 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 299/566] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:38:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:18.312713 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d5b955-4a7a-48ae-b0d2-aed4a4f94fea None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:18.345828 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8408e6-9f1a-4907-bffd-965727722fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:18.346650 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8408e6-9f1a-4907-bffd-965727722fa3 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:18.346936 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8408e6-9f1a-4907-bffd-965727722fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:18.347217 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8408e6-9f1a-4907-bffd-965727722fa3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:18.358108 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8408e6-9f1a-4907-bffd-965727722fa3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:18.359581 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8408e6-9f1a-4907-bffd-965727722fa3 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:18.366058 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:18.377613 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-3d8408e6-9f1a-4907-bffd-965727722fa3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:18.377613 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:18.377613 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:18.377613 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:18.377613 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:18.377613 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:18.377613 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:18.377613 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:18.377613 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:18.377613 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:18.377613 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:18.377613 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:18.377613 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:38:18.377613 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:38:18.377613 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:38:18.378830 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:38:18.378830 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:18.378830 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:18.378830 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:18.378830 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:18.378830 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:18.378830 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:18.378830 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:18.378830 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:38:18.378830 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:18.378830 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:18.378830 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:18.378830 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:18.378830 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:18.378830 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:18.378830 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:18.378830 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:18.379814 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:38:18.379814 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:38:18.379814 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:18.379814 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:38:18.379814 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 268/567] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:38:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:18.379814 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8408e6-9f1a-4907-bffd-965727722fa3 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:18.415110 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d5b955-4a7a-48ae-b0d2-aed4a4f94fea None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:18.450049 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dac28be-e5c1-4882-b6cf-0a0be5b71091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:18.450848 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dac28be-e5c1-4882-b6cf-0a0be5b71091 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:18.451133 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dac28be-e5c1-4882-b6cf-0a0be5b71091 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:18.451408 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dac28be-e5c1-4882-b6cf-0a0be5b71091 None admin] PATH_INFO: `/v3/roles/service` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:18.463110 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dac28be-e5c1-4882-b6cf-0a0be5b71091 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:38:18.465634 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:18.471020 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-8dac28be-e5c1-4882-b6cf-0a0be5b71091 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:38:18.471020 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:18.471020 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:18.471020 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:18.471020 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:18.471020 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:18.471020 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:18.471020 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:18.471020 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:18.471020 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:18.471020 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:18.471020 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:18.471020 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:38:18.471020 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:38:18.471020 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:38:18.471020 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:38:18.472245 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:38:18.472245 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:38:18.472245 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:38:18.472245 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:38:18.472245 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:18.472245 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:18.472245 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:18.472245 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:18.472245 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:18.472245 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:18.472245 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:18.472245 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:38:18.472245 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:18.472245 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:18.472245 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:18.472245 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:18.479113 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:18.479113 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:18.479113 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:18.479113 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:18.479113 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:38:18.479113 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:38:18.479113 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:38:18.479113 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:38:18.479113 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:38:18.479113 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:38:18.479113 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:38:18.479113 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:38:18.479113 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 300/568] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 14:38:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:18.479113 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d782f59f-3432-4116-bc41-ff2ff4f27091 None None] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:18.480642 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d782f59f-3432-4116-bc41-ff2ff4f27091 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:18.480642 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d782f59f-3432-4116-bc41-ff2ff4f27091 None None] PATH_INFO: `/` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:18.480642 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 301/569] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:38:18 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:38:18.480642 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d782f59f-3432-4116-bc41-ff2ff4f27091 None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:18.491288 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8408e6-9f1a-4907-bffd-965727722fa3 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:18.526494 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a1fda4-2859-4e9e-b1e6-c8fcc1be3848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:18.527287 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a1fda4-2859-4e9e-b1e6-c8fcc1be3848 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:18.527572 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a1fda4-2859-4e9e-b1e6-c8fcc1be3848 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:18.527852 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a1fda4-2859-4e9e-b1e6-c8fcc1be3848 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:18.537974 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a1fda4-2859-4e9e-b1e6-c8fcc1be3848 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:18.539470 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a1fda4-2859-4e9e-b1e6-c8fcc1be3848 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:18.557397 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-14a1fda4-2859-4e9e-b1e6-c8fcc1be3848 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:18.557397 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:18.557397 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:18.557397 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:18.557397 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:18.557397 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:18.557397 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:18.557397 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:18.557397 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:18.557397 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:18.557397 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:18.557397 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:18.557397 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:38:18.557397 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:38:18.557397 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:38:18.558452 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:38:18.558452 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:18.558452 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:18.558452 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:18.558452 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:18.558452 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:18.558452 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:18.558452 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:18.558452 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:38:18.558452 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:18.558452 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:18.558452 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:18.558452 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:18.558452 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:18.558452 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:18.558452 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:18.558452 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:18.559446 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:38:18.559446 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:38:18.559446 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:18.559446 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:38:18.559446 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 269/570] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:38:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:18.559446 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a1fda4-2859-4e9e-b1e6-c8fcc1be3848 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:18.583361 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d782f59f-3432-4116-bc41-ff2ff4f27091 None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:18.597278 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:18.618599 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db537da8-dc11-47ad-aa43-580b9e06c107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:18.619393 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db537da8-dc11-47ad-aa43-580b9e06c107 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:18.619677 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db537da8-dc11-47ad-aa43-580b9e06c107 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:18.619969 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db537da8-dc11-47ad-aa43-580b9e06c107 None admin] PATH_INFO: `/v3/projects/service` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:18.626755 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db537da8-dc11-47ad-aa43-580b9e06c107 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:18.628209 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db537da8-dc11-47ad-aa43-580b9e06c107 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:18.643510 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-db537da8-dc11-47ad-aa43-580b9e06c107 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:38:18.643510 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:18.643510 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:18.643510 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:18.643510 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:18.643510 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:18.643510 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:18.643510 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:18.643510 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:18.643510 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:18.643510 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:18.643510 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:18.643510 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:38:18.643510 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:38:18.643510 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:38:18.644709 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:38:18.644709 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:18.644709 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:18.644709 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:18.644709 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:18.644709 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:18.644709 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:18.644709 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:18.644709 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:38:18.644709 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:18.644709 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:18.644709 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:18.644709 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:18.644709 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:18.644709 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:18.644709 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:18.644709 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:18.646061 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:38:18.646061 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:38:18.646061 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:38:18.646061 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:38:18.646061 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:38:18.646061 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:38:18.646061 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:38:18.646061 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:38:18.646061 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 302/571] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:38:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:18.646061 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db537da8-dc11-47ad-aa43-580b9e06c107 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:18.664930 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a1fda4-2859-4e9e-b1e6-c8fcc1be3848 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:18.699930 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31001297-43e9-40c1-a2ea-6d8fdea26703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:18.700593 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31001297-43e9-40c1-a2ea-6d8fdea26703 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:18.700861 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31001297-43e9-40c1-a2ea-6d8fdea26703 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:18.701138 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31001297-43e9-40c1-a2ea-6d8fdea26703 None admin] PATH_INFO: `/v3/domains` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:18.701891 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31001297-43e9-40c1-a2ea-6d8fdea26703 None admin] RBAC: Adding query filter params (name=Default) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:18.702204 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31001297-43e9-40c1-a2ea-6d8fdea26703 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:18.703581 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31001297-43e9-40c1-a2ea-6d8fdea26703 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:18.711186 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:18.716929 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 270/572] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:38:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:18.718298 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0ac5af-a7fd-44d0-a8ae-740c61a95177 None None] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:18.718581 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0ac5af-a7fd-44d0-a8ae-740c61a95177 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:18.718855 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0ac5af-a7fd-44d0-a8ae-740c61a95177 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:18.735222 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.auth.core [None req-2d0ac5af-a7fd-44d0-a8ae-740c61a95177 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:38:18.736404 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-2d0ac5af-a7fd-44d0-a8ae-740c61a95177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:38:18.751918 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db537da8-dc11-47ad-aa43-580b9e06c107 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:18.786682 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2611cce-2e4f-4f64-b677-543cdc528419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:18.787351 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2611cce-2e4f-4f64-b677-543cdc528419 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:18.787635 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2611cce-2e4f-4f64-b677-543cdc528419 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:18.788034 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2611cce-2e4f-4f64-b677-543cdc528419 None admin] PATH_INFO: `/v3/roles` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:18.789470 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2611cce-2e4f-4f64-b677-543cdc528419 None admin] RBAC: Adding query filter params (name=service) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:18.790049 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2611cce-2e4f-4f64-b677-543cdc528419 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:18.791262 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2611cce-2e4f-4f64-b677-543cdc528419 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:18.799353 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 303/573] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 19 14:38:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:18.800182 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2611cce-2e4f-4f64-b677-543cdc528419 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:18.810049 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 271/574] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:38:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:38:18.811164 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0ac5af-a7fd-44d0-a8ae-740c61a95177 None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:18.837798 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:18.905097 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2611cce-2e4f-4f64-b677-543cdc528419 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:18.918316 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0ac5af-a7fd-44d0-a8ae-740c61a95177 None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:18.939513 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9c5d9d-5e99-4f7d-acf3-3484856b9d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:18.940184 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9c5d9d-5e99-4f7d-acf3-3484856b9d3a None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:18.940468 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9c5d9d-5e99-4f7d-acf3-3484856b9d3a None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:18.940749 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9c5d9d-5e99-4f7d-acf3-3484856b9d3a None admin] PATH_INFO: `/v3/domains` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:18.941477 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9c5d9d-5e99-4f7d-acf3-3484856b9d3a None admin] RBAC: Adding query filter params (name=Default) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:18.941784 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9c5d9d-5e99-4f7d-acf3-3484856b9d3a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:18.943191 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9c5d9d-5e99-4f7d-acf3-3484856b9d3a None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:18.953281 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:18.956751 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 304/575] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:38:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:18.957601 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9c5d9d-5e99-4f7d-acf3-3484856b9d3a None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:18.963424 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7f258d-ea5b-45da-95cb-332b26c66dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:18.964173 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7f258d-ea5b-45da-95cb-332b26c66dca None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:18.964459 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7f258d-ea5b-45da-95cb-332b26c66dca None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:18.964734 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7f258d-ea5b-45da-95cb-332b26c66dca None admin] PATH_INFO: `/v3/projects` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:18.965776 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7f258d-ea5b-45da-95cb-332b26c66dca None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:18.966217 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7f258d-ea5b-45da-95cb-332b26c66dca None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:18.968204 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7f258d-ea5b-45da-95cb-332b26c66dca None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:18.982613 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 272/576] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:38:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:18.983470 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7f258d-ea5b-45da-95cb-332b26c66dca None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:19.058786 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:19.068201 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9c5d9d-5e99-4f7d-acf3-3484856b9d3a None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:19.091839 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7f258d-ea5b-45da-95cb-332b26c66dca None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:19.108355 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea94e5f-9f72-45b8-a335-c1bd909d0dd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:19.109214 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea94e5f-9f72-45b8-a335-c1bd909d0dd9 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:19.109505 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea94e5f-9f72-45b8-a335-c1bd909d0dd9 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:19.109779 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea94e5f-9f72-45b8-a335-c1bd909d0dd9 None admin] PATH_INFO: `/v3/users/glance` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:19.118850 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea94e5f-9f72-45b8-a335-c1bd909d0dd9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:19.120310 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea94e5f-9f72-45b8-a335-c1bd909d0dd9 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:19.132166 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8037419-5254-4000-b916-79c663b384be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:19.133889 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8037419-5254-4000-b916-79c663b384be None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:19.134182 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8037419-5254-4000-b916-79c663b384be None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:19.134460 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8037419-5254-4000-b916-79c663b384be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:19.140013 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-9ea94e5f-9f72-45b8-a335-c1bd909d0dd9 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 19 14:38:19.140013 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:19.140013 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:19.140013 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:19.140013 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:19.140013 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:19.140013 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:19.140013 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:19.140013 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:19.140013 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:19.140013 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:19.140013 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:19.140013 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:38:19.140013 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:38:19.140013 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:38:19.140013 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:38:19.141184 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:19.141184 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:19.141184 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:38:19.141184 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:38:19.141184 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:38:19.141184 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:38:19.141184 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:19.141184 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:19.141184 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:19.141184 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:19.141184 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:19.141184 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:38:19.141184 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:19.141184 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:19.141184 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:19.141184 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:19.141184 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:19.141184 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:19.142526 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:19.142526 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:19.142526 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:38:19.142526 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:38:19.142526 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:38:19.142526 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:38:19.142526 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:38:19.142526 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:38:19.142526 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 19 14:38:19.142526 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:38:19.142526 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 305/577] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:38:18 2022] GET /identity/v3/users/glance => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:19.142526 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea94e5f-9f72-45b8-a335-c1bd909d0dd9 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:19.149599 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8037419-5254-4000-b916-79c663b384be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:19.151124 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8037419-5254-4000-b916-79c663b384be None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:19.172233 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-c8037419-5254-4000-b916-79c663b384be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:19.172233 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:19.172233 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:19.172233 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:19.172233 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:19.172233 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:19.172233 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:19.172233 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:19.172233 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:19.172233 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:19.172233 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:19.172233 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:19.172233 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:38:19.172233 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:38:19.172233 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:38:19.173410 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:38:19.173410 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:19.173410 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:19.173410 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:19.173410 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:19.173410 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:19.173410 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:19.173410 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:19.173410 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:38:19.173410 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:19.173410 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:19.173410 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:19.173410 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:19.173410 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:19.173410 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:19.173410 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:19.173410 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:19.174417 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:38:19.174417 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:38:19.174417 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:19.174417 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:38:19.174417 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 273/578] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:38:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:19.174417 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8037419-5254-4000-b916-79c663b384be None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:19.221878 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:19.260343 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea94e5f-9f72-45b8-a335-c1bd909d0dd9 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:19.287888 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8037419-5254-4000-b916-79c663b384be None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:19.297495 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9d2733-8837-4de0-847e-622a5bf79b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:19.298348 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9d2733-8837-4de0-847e-622a5bf79b48 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:19.298637 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9d2733-8837-4de0-847e-622a5bf79b48 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:19.298917 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9d2733-8837-4de0-847e-622a5bf79b48 None admin] PATH_INFO: `/v3/roles/service` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:19.311136 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9d2733-8837-4de0-847e-622a5bf79b48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:38:19.319286 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-ca9d2733-8837-4de0-847e-622a5bf79b48 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:38:19.319286 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:19.319286 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:19.319286 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:19.319286 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:19.319286 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:19.319286 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:19.319286 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:19.319286 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:19.319286 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:19.319286 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:19.319286 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:19.319286 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:38:19.319286 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:38:19.319286 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:38:19.319286 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:38:19.321502 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:38:19.321502 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:38:19.321502 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:38:19.321502 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:38:19.321502 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:19.321502 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:19.321502 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:19.321502 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:19.321502 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:19.321502 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:19.321502 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:19.321502 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:38:19.321502 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:19.321502 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:19.321502 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:19.321502 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:19.323390 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:19.323390 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:19.323390 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:19.323390 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:19.323390 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:38:19.323390 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:38:19.323390 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:38:19.323390 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:38:19.323390 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:38:19.323390 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:38:19.323390 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:38:19.323390 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:38:19.323390 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 306/579] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 14:38:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:19.323390 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9d2733-8837-4de0-847e-622a5bf79b48 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:19.333686 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d914e119-34ab-43fb-84d3-b15738933424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:19.336172 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d914e119-34ab-43fb-84d3-b15738933424 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:19.336172 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d914e119-34ab-43fb-84d3-b15738933424 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:19.336172 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d914e119-34ab-43fb-84d3-b15738933424 None admin] PATH_INFO: `/v3/projects/service` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:19.343147 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d914e119-34ab-43fb-84d3-b15738933424 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:19.344875 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d914e119-34ab-43fb-84d3-b15738933424 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:19.345103 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:19.360389 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-d914e119-34ab-43fb-84d3-b15738933424 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:38:19.360389 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:19.360389 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:19.360389 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:19.360389 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:19.360389 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:19.360389 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:19.360389 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:19.360389 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:19.360389 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:19.360389 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:19.360389 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:19.360389 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:38:19.360389 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:38:19.360389 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:38:19.361633 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:38:19.361633 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:19.361633 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:19.361633 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:19.361633 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:19.361633 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:19.361633 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:19.361633 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:19.361633 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:38:19.361633 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:19.361633 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:19.361633 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:19.361633 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:19.361633 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:19.361633 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:19.361633 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:19.361633 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:19.363248 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:38:19.363248 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:38:19.363248 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:38:19.363248 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:38:19.363248 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:38:19.363248 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:38:19.363248 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:38:19.363248 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:38:19.363248 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 274/580] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:38:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:19.363248 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d914e119-34ab-43fb-84d3-b15738933424 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:19.444904 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9d2733-8837-4de0-847e-622a5bf79b48 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:19.469789 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:19.473411 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d914e119-34ab-43fb-84d3-b15738933424 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:19.480601 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4424af8-38d6-41f9-83a1-b0c320a8287e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:19.481272 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4424af8-38d6-41f9-83a1-b0c320a8287e None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:19.481558 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4424af8-38d6-41f9-83a1-b0c320a8287e None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:19.481863 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4424af8-38d6-41f9-83a1-b0c320a8287e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:19.482711 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4424af8-38d6-41f9-83a1-b0c320a8287e None admin] RBAC: Adding query filter params (role.id=ffb8da90af53491980816087ee621ccb, scope.project.id=42f35b96b9d4499abfb5e5f7cca49adc, user.id=4f8e2e0909074a85bb493c49ac3a6d4d) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:19.483028 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4424af8-38d6-41f9-83a1-b0c320a8287e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:19.484489 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4424af8-38d6-41f9-83a1-b0c320a8287e None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:19.490084 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 307/581] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:38:19 2022] GET /identity/v3/role_assignments?user.id=4f8e2e0909074a85bb493c49ac3a6d4d&scope.project.id=42f35b96b9d4499abfb5e5f7cca49adc&role.id=ffb8da90af53491980816087ee621ccb => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:19.490889 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4424af8-38d6-41f9-83a1-b0c320a8287e None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:19.512927 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701126db-94fb-41f7-9ce0-12ff6d9e3550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:19.513645 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701126db-94fb-41f7-9ce0-12ff6d9e3550 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:19.513956 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701126db-94fb-41f7-9ce0-12ff6d9e3550 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:19.514233 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701126db-94fb-41f7-9ce0-12ff6d9e3550 None admin] PATH_INFO: `/v3/users` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:19.515092 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701126db-94fb-41f7-9ce0-12ff6d9e3550 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:19.515409 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701126db-94fb-41f7-9ce0-12ff6d9e3550 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:19.516880 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701126db-94fb-41f7-9ce0-12ff6d9e3550 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:19.539633 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 275/582] 10.222.0.57 () {56 vars in 1137 bytes} [Thu May 19 14:38:19 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:19.540533 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701126db-94fb-41f7-9ce0-12ff6d9e3550 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:19.605224 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:19.631782 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4424af8-38d6-41f9-83a1-b0c320a8287e None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:19.666844 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701126db-94fb-41f7-9ce0-12ff6d9e3550 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:19.672088 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edd858c-1355-47d6-9980-8c91c068bf56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:19.672804 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edd858c-1355-47d6-9980-8c91c068bf56 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:19.673090 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edd858c-1355-47d6-9980-8c91c068bf56 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:19.673367 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edd858c-1355-47d6-9980-8c91c068bf56 None admin] PATH_INFO: `/v3/domains` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:19.674143 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edd858c-1355-47d6-9980-8c91c068bf56 None admin] RBAC: Adding query filter params (name=Default) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:19.674454 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edd858c-1355-47d6-9980-8c91c068bf56 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:19.675892 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edd858c-1355-47d6-9980-8c91c068bf56 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:19.689309 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 308/583] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:38:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:19.690252 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edd858c-1355-47d6-9980-8c91c068bf56 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:19.714862 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a753c785-cf95-4160-8a9c-36691b61fe84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:19.715555 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a753c785-cf95-4160-8a9c-36691b61fe84 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:19.715842 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a753c785-cf95-4160-8a9c-36691b61fe84 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:19.716118 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a753c785-cf95-4160-8a9c-36691b61fe84 None admin] PATH_INFO: `/v3/roles` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:19.717258 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a753c785-cf95-4160-8a9c-36691b61fe84 None admin] RBAC: Adding query filter params (name=service) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:19.717258 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a753c785-cf95-4160-8a9c-36691b61fe84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:19.719649 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a753c785-cf95-4160-8a9c-36691b61fe84 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:19.736393 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 276/584] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 19 14:38:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:19.737180 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a753c785-cf95-4160-8a9c-36691b61fe84 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:19.766256 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:19.823141 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edd858c-1355-47d6-9980-8c91c068bf56 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:19.859295 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a753c785-cf95-4160-8a9c-36691b61fe84 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:19.862020 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c139d607-53d5-4dcb-9eb0-0892837d20c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:19.862800 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c139d607-53d5-4dcb-9eb0-0892837d20c1 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:19.863177 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c139d607-53d5-4dcb-9eb0-0892837d20c1 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:19.863464 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c139d607-53d5-4dcb-9eb0-0892837d20c1 None admin] PATH_INFO: `/v3/projects` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:19.864266 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c139d607-53d5-4dcb-9eb0-0892837d20c1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:19.864579 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c139d607-53d5-4dcb-9eb0-0892837d20c1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:19.866135 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c139d607-53d5-4dcb-9eb0-0892837d20c1 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:19.874231 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:19.882888 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 309/585] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:38:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:19.883760 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c139d607-53d5-4dcb-9eb0-0892837d20c1 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:19.899629 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2063ac-ebec-45ca-8c4d-18da0222788d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:19.900596 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2063ac-ebec-45ca-8c4d-18da0222788d None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:19.900978 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2063ac-ebec-45ca-8c4d-18da0222788d None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:19.901483 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2063ac-ebec-45ca-8c4d-18da0222788d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:19.912717 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2063ac-ebec-45ca-8c4d-18da0222788d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:19.914635 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2063ac-ebec-45ca-8c4d-18da0222788d None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:19.933069 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-fc2063ac-ebec-45ca-8c4d-18da0222788d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:19.933069 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:19.933069 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:19.933069 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:19.933069 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:19.933069 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:19.933069 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:19.933069 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:19.933069 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:19.933069 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:19.933069 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:19.933069 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:19.933069 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:38:19.933069 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:38:19.933069 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:38:19.934356 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:38:19.934356 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:19.934356 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:19.934356 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:19.934356 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:19.934356 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:19.934356 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:19.934356 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:19.934356 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:38:19.934356 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:19.934356 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:19.934356 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:19.934356 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:19.934356 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:19.934356 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:19.934356 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:19.934356 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:19.935549 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:38:19.935549 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:38:19.935549 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:19.935549 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:38:19.936552 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 277/586] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:38:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:19.942044 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2063ac-ebec-45ca-8c4d-18da0222788d None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:20.006423 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c139d607-53d5-4dcb-9eb0-0892837d20c1 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:20.015663 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:20.044694 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2fe91b-f88d-4289-ace9-b94c3abcda6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:20.045504 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2fe91b-f88d-4289-ace9-b94c3abcda6f None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:20.045794 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2fe91b-f88d-4289-ace9-b94c3abcda6f None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:20.046107 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2fe91b-f88d-4289-ace9-b94c3abcda6f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:20.048315 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2063ac-ebec-45ca-8c4d-18da0222788d None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:20.055546 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2fe91b-f88d-4289-ace9-b94c3abcda6f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:20.057010 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2fe91b-f88d-4289-ace9-b94c3abcda6f None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:20.071729 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-af2fe91b-f88d-4289-ace9-b94c3abcda6f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 14:38:20.071729 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:20.071729 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:20.071729 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:20.071729 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:20.071729 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:20.071729 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:20.071729 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:20.071729 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:20.071729 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:20.071729 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:20.071729 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:20.071729 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:38:20.071729 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:38:20.071729 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:38:20.071729 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:38:20.072995 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:20.072995 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:20.072995 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:38:20.072995 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:38:20.072995 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:38:20.072995 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:38:20.072995 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:20.072995 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:20.072995 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:20.072995 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:20.072995 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:20.072995 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:38:20.072995 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:20.072995 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:20.072995 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:20.072995 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:20.072995 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:20.072995 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:20.074409 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:20.074409 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:20.074409 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:38:20.074409 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:38:20.074409 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:38:20.074409 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:38:20.074409 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:38:20.074409 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:38:20.074409 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 14:38:20.074409 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:38:20.074409 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 310/587] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:38:19 2022] GET /identity/v3/users/cinder => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:20.074409 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2fe91b-f88d-4289-ace9-b94c3abcda6f None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:20.091965 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3f7c8c-edbf-4c33-bbeb-38f8c0ecbe31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:20.092764 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3f7c8c-edbf-4c33-bbeb-38f8c0ecbe31 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:20.093045 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3f7c8c-edbf-4c33-bbeb-38f8c0ecbe31 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:20.093322 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3f7c8c-edbf-4c33-bbeb-38f8c0ecbe31 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:20.104870 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3f7c8c-edbf-4c33-bbeb-38f8c0ecbe31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:20.106357 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3f7c8c-edbf-4c33-bbeb-38f8c0ecbe31 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:20.125990 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-4e3f7c8c-edbf-4c33-bbeb-38f8c0ecbe31 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:20.125990 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:20.125990 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:20.125990 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:20.125990 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:20.125990 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:20.125990 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:20.125990 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:20.125990 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:20.125990 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:20.125990 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:20.125990 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:20.125990 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:38:20.125990 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:38:20.125990 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:38:20.127185 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:38:20.127185 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:20.127185 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:20.127185 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:20.127185 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:20.127185 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:20.127185 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:20.127185 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:20.127185 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:38:20.127185 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:20.127185 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:20.127185 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:20.127185 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:20.127185 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:20.127185 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:20.127185 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:20.127185 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:20.128722 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:38:20.128722 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:38:20.128722 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:20.128722 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:38:20.128722 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 278/588] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:38:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:20.132847 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3f7c8c-edbf-4c33-bbeb-38f8c0ecbe31 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:20.143696 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:20.200133 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2fe91b-f88d-4289-ace9-b94c3abcda6f None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:20.237413 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2647bf43-e40b-4649-ba77-4a9316cdce24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:20.238367 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2647bf43-e40b-4649-ba77-4a9316cdce24 None admin] REQUEST_METHOD: `PUT` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:20.238654 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2647bf43-e40b-4649-ba77-4a9316cdce24 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:20.238932 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2647bf43-e40b-4649-ba77-4a9316cdce24 None admin] PATH_INFO: `/v3/projects/42f35b96b9d4499abfb5e5f7cca49adc/users/a9591727b2f54244baf90b11a8e2277f/roles/ffb8da90af53491980816087ee621ccb` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:20.254540 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3f7c8c-edbf-4c33-bbeb-38f8c0ecbe31 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:20.256093 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2647bf43-e40b-4649-ba77-4a9316cdce24 None admin] RBAC: Authorizing `identity:create_grant(project_id=42f35b96b9d4499abfb5e5f7cca49adc, user_id=a9591727b2f54244baf90b11a8e2277f, role_id=ffb8da90af53491980816087ee621ccb)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:20.258406 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2647bf43-e40b-4649-ba77-4a9316cdce24 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:20.273680 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 311/589] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 14:38:20 2022] PUT /identity/v3/projects/42f35b96b9d4499abfb5e5f7cca49adc/users/a9591727b2f54244baf90b11a8e2277f/roles/ffb8da90af53491980816087ee621ccb => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:38:20.274784 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2647bf43-e40b-4649-ba77-4a9316cdce24 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:20.292317 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:20.307767 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95573676-1736-4d56-bdf8-29ec92f279da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:20.308722 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95573676-1736-4d56-bdf8-29ec92f279da None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:20.309322 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95573676-1736-4d56-bdf8-29ec92f279da None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:20.309607 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95573676-1736-4d56-bdf8-29ec92f279da None admin] PATH_INFO: `/v3/domains` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:20.310407 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95573676-1736-4d56-bdf8-29ec92f279da None admin] RBAC: Adding query filter params (name=Default) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:20.310815 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95573676-1736-4d56-bdf8-29ec92f279da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:20.312268 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95573676-1736-4d56-bdf8-29ec92f279da None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:20.334197 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 279/590] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:38:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:20.334197 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95573676-1736-4d56-bdf8-29ec92f279da None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:20.412437 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2647bf43-e40b-4649-ba77-4a9316cdce24 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:20.423160 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:20.455972 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24ab151-2d4e-45e0-8b40-51555b4ec519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:20.456668 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24ab151-2d4e-45e0-8b40-51555b4ec519 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:20.457008 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24ab151-2d4e-45e0-8b40-51555b4ec519 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:20.457294 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24ab151-2d4e-45e0-8b40-51555b4ec519 None admin] PATH_INFO: `/v3/users` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:20.458260 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24ab151-2d4e-45e0-8b40-51555b4ec519 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:20.458570 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24ab151-2d4e-45e0-8b40-51555b4ec519 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:20.460409 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24ab151-2d4e-45e0-8b40-51555b4ec519 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:20.485866 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 312/591] 10.222.0.57 () {56 vars in 1137 bytes} [Thu May 19 14:38:20 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:20.485866 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24ab151-2d4e-45e0-8b40-51555b4ec519 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:20.486163 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95573676-1736-4d56-bdf8-29ec92f279da None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:20.537804 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7b8478-8721-4b10-beee-867f013d870b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:20.540247 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7b8478-8721-4b10-beee-867f013d870b None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:20.541037 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7b8478-8721-4b10-beee-867f013d870b None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:20.541535 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7b8478-8721-4b10-beee-867f013d870b None admin] PATH_INFO: `/v3/domains` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:20.543155 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7b8478-8721-4b10-beee-867f013d870b None admin] RBAC: Adding query filter params (name=Default) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:20.543692 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7b8478-8721-4b10-beee-867f013d870b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:20.545350 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7b8478-8721-4b10-beee-867f013d870b None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:20.562501 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 280/592] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:38:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:20.569232 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7b8478-8721-4b10-beee-867f013d870b None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:20.604824 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24ab151-2d4e-45e0-8b40-51555b4ec519 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:20.622373 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:20.645350 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c38ad3-ac8f-440f-a209-e1dd30fa410d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:20.646901 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c38ad3-ac8f-440f-a209-e1dd30fa410d None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:20.647495 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c38ad3-ac8f-440f-a209-e1dd30fa410d None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:20.648747 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c38ad3-ac8f-440f-a209-e1dd30fa410d None admin] PATH_INFO: `/v3/projects/service` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:20.656612 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c38ad3-ac8f-440f-a209-e1dd30fa410d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:20.658574 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c38ad3-ac8f-440f-a209-e1dd30fa410d None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:20.677427 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-27c38ad3-ac8f-440f-a209-e1dd30fa410d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:38:20.677427 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:20.677427 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:20.677427 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:20.677427 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:20.677427 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:20.677427 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:20.677427 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:20.677427 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:20.677427 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:20.677427 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:20.677427 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:20.677427 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:38:20.677427 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:38:20.677427 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:38:20.678436 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:38:20.678436 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:20.678436 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:20.678436 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:20.678436 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:20.678436 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:20.678436 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:20.678436 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:20.678436 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:38:20.678436 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:20.678436 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:20.678436 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:20.678436 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:20.678436 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:20.678436 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:20.678436 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:20.678436 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:20.680159 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:38:20.680159 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:38:20.680159 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:38:20.680159 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:38:20.680159 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:38:20.680159 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:38:20.680159 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:38:20.680159 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:38:20.687476 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 313/593] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:38:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:20.688380 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7b8478-8721-4b10-beee-867f013d870b None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:20.688828 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c38ad3-ac8f-440f-a209-e1dd30fa410d None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:20.727980 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:20.731900 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f76e2e-a00b-408a-bf93-6502f1c67042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:20.733920 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f76e2e-a00b-408a-bf93-6502f1c67042 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:20.734375 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f76e2e-a00b-408a-bf93-6502f1c67042 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:20.734793 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f76e2e-a00b-408a-bf93-6502f1c67042 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:20.748720 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f76e2e-a00b-408a-bf93-6502f1c67042 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:20.750562 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f76e2e-a00b-408a-bf93-6502f1c67042 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:20.769852 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-53f76e2e-a00b-408a-bf93-6502f1c67042 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:20.769852 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:20.769852 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:20.769852 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:20.769852 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:20.769852 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:20.769852 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:20.769852 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:20.769852 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:20.769852 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:20.769852 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:20.769852 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:20.769852 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:38:20.769852 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:38:20.769852 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:38:20.770880 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:38:20.770880 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:20.770880 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:20.770880 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:20.770880 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:20.770880 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:20.770880 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:20.770880 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:20.770880 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:38:20.770880 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:20.770880 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:20.770880 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:20.770880 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:20.770880 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:20.770880 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:20.770880 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:20.770880 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:20.771852 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:38:20.771852 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:38:20.771852 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:20.771852 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:38:20.774117 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 281/594] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:38:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:20.775116 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f76e2e-a00b-408a-bf93-6502f1c67042 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:20.817987 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c38ad3-ac8f-440f-a209-e1dd30fa410d None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:20.868107 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85139b26-c168-4cd7-8d8d-6f0d931a457e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:20.868922 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85139b26-c168-4cd7-8d8d-6f0d931a457e None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:20.869208 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85139b26-c168-4cd7-8d8d-6f0d931a457e None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:20.869900 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85139b26-c168-4cd7-8d8d-6f0d931a457e None admin] PATH_INFO: `/v3/users/neutron` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:20.880665 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85139b26-c168-4cd7-8d8d-6f0d931a457e None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:20.882563 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85139b26-c168-4cd7-8d8d-6f0d931a457e None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:20.899164 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f76e2e-a00b-408a-bf93-6502f1c67042 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:20.902032 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:20.902975 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-85139b26-c168-4cd7-8d8d-6f0d931a457e None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 19 14:38:20.902975 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:20.902975 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:20.902975 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:20.902975 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:20.902975 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:20.902975 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:20.902975 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:20.902975 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:20.902975 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:20.902975 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:20.902975 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:20.902975 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:38:20.902975 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:38:20.902975 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:38:20.902975 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:38:20.904188 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:20.904188 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:20.904188 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:38:20.904188 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:38:20.904188 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:38:20.904188 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:38:20.904188 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:20.904188 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:20.904188 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:20.904188 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:20.904188 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:20.904188 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:38:20.904188 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:20.904188 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:20.904188 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:20.904188 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:20.904188 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:20.904188 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:20.905249 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:20.905249 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:20.905249 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:38:20.905249 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:38:20.905249 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:38:20.905249 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:38:20.905249 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:38:20.905249 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:38:20.905249 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 19 14:38:20.905249 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:38:20.909503 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 314/595] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 14:38:20 2022] GET /identity/v3/users/neutron => generated 85 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:20.910429 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85139b26-c168-4cd7-8d8d-6f0d931a457e None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:20.952241 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39880375-aac1-4b67-935a-cbda77dd2279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:20.953264 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39880375-aac1-4b67-935a-cbda77dd2279 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:20.953558 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39880375-aac1-4b67-935a-cbda77dd2279 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:20.953857 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39880375-aac1-4b67-935a-cbda77dd2279 None admin] PATH_INFO: `/v3/projects` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:20.954947 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39880375-aac1-4b67-935a-cbda77dd2279 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:20.955263 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39880375-aac1-4b67-935a-cbda77dd2279 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:20.956705 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39880375-aac1-4b67-935a-cbda77dd2279 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:20.977982 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 282/596] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:38:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:20.977982 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39880375-aac1-4b67-935a-cbda77dd2279 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:21.018836 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:21.036258 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85139b26-c168-4cd7-8d8d-6f0d931a457e None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:21.083992 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facd4938-5aba-4c0c-9ee7-c74e439489c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:21.084947 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facd4938-5aba-4c0c-9ee7-c74e439489c9 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:21.085880 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facd4938-5aba-4c0c-9ee7-c74e439489c9 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:21.086180 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facd4938-5aba-4c0c-9ee7-c74e439489c9 None admin] PATH_INFO: `/v3/domains` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:21.087188 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facd4938-5aba-4c0c-9ee7-c74e439489c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:21.087497 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facd4938-5aba-4c0c-9ee7-c74e439489c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:21.089773 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facd4938-5aba-4c0c-9ee7-c74e439489c9 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:21.097914 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39880375-aac1-4b67-935a-cbda77dd2279 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:21.106920 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 315/597] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:38:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:21.107905 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facd4938-5aba-4c0c-9ee7-c74e439489c9 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:21.150189 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e5e748-190a-4655-82e5-e00edec87cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:21.150921 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e5e748-190a-4655-82e5-e00edec87cda None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:21.151205 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e5e748-190a-4655-82e5-e00edec87cda None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:21.151481 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e5e748-190a-4655-82e5-e00edec87cda None admin] PATH_INFO: `/v3/users` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:21.152330 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e5e748-190a-4655-82e5-e00edec87cda None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:21.152638 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e5e748-190a-4655-82e5-e00edec87cda None admin] RBAC: Authorizing `identity:list_users()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:21.154145 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e5e748-190a-4655-82e5-e00edec87cda None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:21.185624 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 283/598] 10.222.0.57 () {56 vars in 1139 bytes} [Thu May 19 14:38:20 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:21.185624 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e5e748-190a-4655-82e5-e00edec87cda None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:21.205919 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:21.236514 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facd4938-5aba-4c0c-9ee7-c74e439489c9 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:21.276071 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0282160a-79db-4ce5-a456-d41c17e13ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:21.277423 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0282160a-79db-4ce5-a456-d41c17e13ae8 None admin] REQUEST_METHOD: `PUT` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:21.278003 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0282160a-79db-4ce5-a456-d41c17e13ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:21.278529 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0282160a-79db-4ce5-a456-d41c17e13ae8 None admin] PATH_INFO: `/v3/projects/42f35b96b9d4499abfb5e5f7cca49adc/users/ca35fa8e73ce489081eb7ea2ca5c5e09/roles/ffb8da90af53491980816087ee621ccb` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:21.296090 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0282160a-79db-4ce5-a456-d41c17e13ae8 None admin] RBAC: Authorizing `identity:create_grant(project_id=42f35b96b9d4499abfb5e5f7cca49adc, user_id=ca35fa8e73ce489081eb7ea2ca5c5e09, role_id=ffb8da90af53491980816087ee621ccb)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:21.298141 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0282160a-79db-4ce5-a456-d41c17e13ae8 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:21.302191 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:21.322026 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 316/599] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 14:38:21 2022] PUT /identity/v3/projects/42f35b96b9d4499abfb5e5f7cca49adc/users/ca35fa8e73ce489081eb7ea2ca5c5e09/roles/ffb8da90af53491980816087ee621ccb => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:38:21.322754 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e5e748-190a-4655-82e5-e00edec87cda None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:21.323661 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0282160a-79db-4ce5-a456-d41c17e13ae8 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:21.407140 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a08c74-7d84-492d-b837-a18c83d7d0c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:21.408438 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a08c74-7d84-492d-b837-a18c83d7d0c7 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:21.409811 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a08c74-7d84-492d-b837-a18c83d7d0c7 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:21.410977 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a08c74-7d84-492d-b837-a18c83d7d0c7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:21.424558 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a08c74-7d84-492d-b837-a18c83d7d0c7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:21.427559 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a08c74-7d84-492d-b837-a18c83d7d0c7 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:21.459278 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-c5a08c74-7d84-492d-b837-a18c83d7d0c7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:38:21.459278 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:21.459278 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:21.459278 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:21.459278 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:21.459278 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:21.459278 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:21.459278 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:21.459278 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:21.459278 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:21.459278 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:21.459278 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:21.459278 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:38:21.459278 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:38:21.459278 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:38:21.461108 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:38:21.461108 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:21.461108 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:21.461108 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:21.461108 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:21.461108 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:21.461108 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:21.461108 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:21.461108 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:38:21.461108 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:21.461108 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:21.461108 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:21.461108 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:21.461108 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:21.461108 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:21.461108 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:21.461108 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:21.462335 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:38:21.462335 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:38:21.462335 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:38:21.462335 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:38:21.462335 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:38:21.462335 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:38:21.462335 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:38:21.462335 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:38:21.467318 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 284/600] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:38:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:21.473250 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a08c74-7d84-492d-b837-a18c83d7d0c7 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:21.554727 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0282160a-79db-4ce5-a456-d41c17e13ae8 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:21.601804 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12e186a-735e-408a-bf35-91909a5d357b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:21.602526 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:21.602813 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12e186a-735e-408a-bf35-91909a5d357b None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:21.603094 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12e186a-735e-408a-bf35-91909a5d357b None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:21.603369 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12e186a-735e-408a-bf35-91909a5d357b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:21.617574 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12e186a-735e-408a-bf35-91909a5d357b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:21.619090 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12e186a-735e-408a-bf35-91909a5d357b None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:21.634935 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a08c74-7d84-492d-b837-a18c83d7d0c7 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:21.638412 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-a12e186a-735e-408a-bf35-91909a5d357b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:21.638412 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:21.638412 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:21.638412 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:21.638412 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:21.638412 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:21.638412 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:21.638412 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:21.638412 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:21.638412 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:21.638412 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:21.638412 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:21.638412 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:38:21.638412 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:38:21.638412 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:38:21.639478 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:38:21.639478 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:21.639478 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:21.639478 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:21.639478 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:21.639478 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:21.639478 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:21.639478 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:21.639478 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:38:21.639478 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:21.639478 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:21.639478 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:21.639478 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:21.639478 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:21.639478 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:21.639478 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:21.639478 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:21.640570 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:38:21.640570 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:38:21.640570 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:21.640570 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:38:21.643811 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 317/601] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:38:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:21.645073 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12e186a-735e-408a-bf35-91909a5d357b None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:21.679599 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28083057-aaf1-4678-97b8-d4e7602bc1f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:21.680697 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28083057-aaf1-4678-97b8-d4e7602bc1f7 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:21.681201 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28083057-aaf1-4678-97b8-d4e7602bc1f7 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:21.681746 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28083057-aaf1-4678-97b8-d4e7602bc1f7 None admin] PATH_INFO: `/v3/projects` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:21.683536 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28083057-aaf1-4678-97b8-d4e7602bc1f7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:21.684100 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28083057-aaf1-4678-97b8-d4e7602bc1f7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:21.685858 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28083057-aaf1-4678-97b8-d4e7602bc1f7 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:21.701182 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:21.705122 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 285/602] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:38:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:21.712696 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28083057-aaf1-4678-97b8-d4e7602bc1f7 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:21.804548 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12e186a-735e-408a-bf35-91909a5d357b None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:21.877466 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a93fb5-0add-4311-8cba-4394cbd1f6ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:21.879731 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a93fb5-0add-4311-8cba-4394cbd1f6ff None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:21.880524 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a93fb5-0add-4311-8cba-4394cbd1f6ff None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:21.883098 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a93fb5-0add-4311-8cba-4394cbd1f6ff None admin] PATH_INFO: `/v3/domains` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:21.884088 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a93fb5-0add-4311-8cba-4394cbd1f6ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:21.884617 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a93fb5-0add-4311-8cba-4394cbd1f6ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:21.886293 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a93fb5-0add-4311-8cba-4394cbd1f6ff None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:21.901713 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 318/603] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:38:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:21.903433 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7bc781-3c5e-4745-b8f2-ca527f6bd51a None None] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:21.903940 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7bc781-3c5e-4745-b8f2-ca527f6bd51a None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:21.904479 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7bc781-3c5e-4745-b8f2-ca527f6bd51a None None] PATH_INFO: `/` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:21.906353 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 319/604] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:38:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:38:21.907317 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7bc781-3c5e-4745-b8f2-ca527f6bd51a None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:21.920065 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28083057-aaf1-4678-97b8-d4e7602bc1f7 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:21.988048 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b963459-d339-46a1-aff3-a51e80d83473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:21.989651 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b963459-d339-46a1-aff3-a51e80d83473 None admin] REQUEST_METHOD: `PUT` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:21.990143 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b963459-d339-46a1-aff3-a51e80d83473 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:21.991065 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b963459-d339-46a1-aff3-a51e80d83473 None admin] PATH_INFO: `/v3/projects/42f35b96b9d4499abfb5e5f7cca49adc/users/aa7df5964bed4638a9bf19d37971ec20/roles/ffb8da90af53491980816087ee621ccb` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:21.997623 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:22.028485 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b963459-d339-46a1-aff3-a51e80d83473 None admin] RBAC: Authorizing `identity:create_grant(project_id=42f35b96b9d4499abfb5e5f7cca49adc, user_id=aa7df5964bed4638a9bf19d37971ec20, role_id=ffb8da90af53491980816087ee621ccb)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:22.032140 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b963459-d339-46a1-aff3-a51e80d83473 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:22.057601 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 286/605] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 14:38:21 2022] PUT /identity/v3/projects/42f35b96b9d4499abfb5e5f7cca49adc/users/aa7df5964bed4638a9bf19d37971ec20/roles/ffb8da90af53491980816087ee621ccb => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:38:22.060601 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496bffe2-1057-4e66-9d24-665db2a956c9 None None] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:22.061074 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496bffe2-1057-4e66-9d24-665db2a956c9 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:22.061535 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496bffe2-1057-4e66-9d24-665db2a956c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:22.085711 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7bc781-3c5e-4745-b8f2-ca527f6bd51a None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:22.087829 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.auth.core [None req-496bffe2-1057-4e66-9d24-665db2a956c9 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:38:22.089293 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-496bffe2-1057-4e66-9d24-665db2a956c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:38:22.098640 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:22.133495 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2113ab8-463f-4ba5-a702-e04c7285b5fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:22.135939 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2113ab8-463f-4ba5-a702-e04c7285b5fd None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:22.136814 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2113ab8-463f-4ba5-a702-e04c7285b5fd None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:22.137519 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2113ab8-463f-4ba5-a702-e04c7285b5fd None admin] PATH_INFO: `/v3/projects/service` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:22.152502 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2113ab8-463f-4ba5-a702-e04c7285b5fd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:22.155333 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2113ab8-463f-4ba5-a702-e04c7285b5fd None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:22.197380 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-c2113ab8-463f-4ba5-a702-e04c7285b5fd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:38:22.197380 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:22.197380 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:22.197380 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:22.197380 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:22.197380 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:22.197380 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:22.197380 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:22.197380 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:22.197380 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:22.197380 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:22.197380 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:22.197380 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:38:22.197380 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:38:22.197380 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:38:22.198664 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:38:22.198664 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:22.198664 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:22.198664 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:22.198664 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:22.198664 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:22.198664 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:22.198664 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:22.198664 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:38:22.198664 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:22.198664 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:22.198664 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:22.198664 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:22.198664 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:22.198664 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:22.198664 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:22.198664 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:22.199802 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:38:22.199802 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:38:22.199802 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:38:22.199802 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:38:22.199802 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:38:22.199802 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:38:22.199802 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:38:22.199802 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:38:22.201759 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 320/606] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:38:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:22.211470 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2113ab8-463f-4ba5-a702-e04c7285b5fd None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:22.255246 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 287/607] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:38:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:38:22.299007 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496bffe2-1057-4e66-9d24-665db2a956c9 None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:22.357711 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2113ab8-463f-4ba5-a702-e04c7285b5fd None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:22.400990 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8197fbca-b6ac-4db3-b491-58edf8ff2958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:22.401798 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8197fbca-b6ac-4db3-b491-58edf8ff2958 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:22.402114 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8197fbca-b6ac-4db3-b491-58edf8ff2958 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:22.402387 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8197fbca-b6ac-4db3-b491-58edf8ff2958 None admin] PATH_INFO: `/v3/projects` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:22.403177 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8197fbca-b6ac-4db3-b491-58edf8ff2958 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:22.403487 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8197fbca-b6ac-4db3-b491-58edf8ff2958 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:22.404927 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8197fbca-b6ac-4db3-b491-58edf8ff2958 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:22.418603 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 321/608] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:38:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:22.426860 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8197fbca-b6ac-4db3-b491-58edf8ff2958 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:22.472579 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496bffe2-1057-4e66-9d24-665db2a956c9 None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:22.490837 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:22.502200 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:22.543883 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70f57fc-7aea-45d3-aefb-8d1e2f407f84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:22.546030 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70f57fc-7aea-45d3-aefb-8d1e2f407f84 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:22.546320 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70f57fc-7aea-45d3-aefb-8d1e2f407f84 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:22.546595 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70f57fc-7aea-45d3-aefb-8d1e2f407f84 None admin] PATH_INFO: `/v3/roles/service` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:22.573079 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70f57fc-7aea-45d3-aefb-8d1e2f407f84 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:38:22.589248 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-f70f57fc-7aea-45d3-aefb-8d1e2f407f84 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:38:22.589248 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:22.589248 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:22.589248 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:22.589248 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:22.589248 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:22.589248 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:22.589248 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:22.589248 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:22.589248 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:22.589248 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:22.589248 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:22.589248 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:38:22.589248 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:38:22.589248 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:38:22.589248 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:38:22.590393 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:38:22.590393 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:38:22.590393 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:38:22.590393 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:38:22.590393 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:22.590393 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:22.590393 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:22.590393 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:22.590393 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:22.590393 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:22.590393 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:22.590393 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:38:22.590393 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:22.590393 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:22.590393 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:22.590393 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:22.591309 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:22.591309 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:22.591309 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:22.591309 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:22.591309 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:38:22.591309 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:38:22.591309 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:38:22.591309 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:38:22.591309 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:38:22.591309 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:38:22.591309 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:38:22.591309 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:38:22.592199 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 288/609] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 14:38:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:38:22.602994 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70f57fc-7aea-45d3-aefb-8d1e2f407f84 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:22.645225 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8197fbca-b6ac-4db3-b491-58edf8ff2958 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:22.711036 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cda0bd9-7d02-44e8-be5f-831ff86046bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:22.712006 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cda0bd9-7d02-44e8-be5f-831ff86046bc None admin] REQUEST_METHOD: `PUT` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:22.712292 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cda0bd9-7d02-44e8-be5f-831ff86046bc None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:22.712562 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cda0bd9-7d02-44e8-be5f-831ff86046bc None admin] PATH_INFO: `/v3/projects/42f35b96b9d4499abfb5e5f7cca49adc/users/c7cd031eea90473aaaa0f1375f04dc66/roles/ffb8da90af53491980816087ee621ccb` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:22.738522 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cda0bd9-7d02-44e8-be5f-831ff86046bc None admin] RBAC: Authorizing `identity:create_grant(project_id=42f35b96b9d4499abfb5e5f7cca49adc, user_id=c7cd031eea90473aaaa0f1375f04dc66, role_id=ffb8da90af53491980816087ee621ccb)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:22.739982 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cda0bd9-7d02-44e8-be5f-831ff86046bc None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:22.757619 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 322/610] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 14:38:22 2022] PUT /identity/v3/projects/42f35b96b9d4499abfb5e5f7cca49adc/users/c7cd031eea90473aaaa0f1375f04dc66/roles/ffb8da90af53491980816087ee621ccb => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:38:22.759232 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46e6f0c-bef1-4237-b497-69f7020a6414 None None] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:22.759571 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46e6f0c-bef1-4237-b497-69f7020a6414 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:22.759849 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46e6f0c-bef1-4237-b497-69f7020a6414 None None] PATH_INFO: `/` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:22.761511 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 323/611] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:38:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:38:22.767797 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7e5546-8394-4abf-80a6-722a3167c639 None None] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:22.768251 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7e5546-8394-4abf-80a6-722a3167c639 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:22.768629 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7e5546-8394-4abf-80a6-722a3167c639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:22.797267 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70f57fc-7aea-45d3-aefb-8d1e2f407f84 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:22.800588 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.auth.core [None req-dc7e5546-8394-4abf-80a6-722a3167c639 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:38:22.801817 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.fernet_utils [None req-dc7e5546-8394-4abf-80a6-722a3167c639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:38:22.870151 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713a8e42-7ec0-4c42-9a37-117fcb3d8cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:22.870846 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713a8e42-7ec0-4c42-9a37-117fcb3d8cf4 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:22.871128 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713a8e42-7ec0-4c42-9a37-117fcb3d8cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:22.871403 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713a8e42-7ec0-4c42-9a37-117fcb3d8cf4 None admin] PATH_INFO: `/v3/roles` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:22.872171 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713a8e42-7ec0-4c42-9a37-117fcb3d8cf4 None admin] RBAC: Adding query filter params (name=service) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:22.872478 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713a8e42-7ec0-4c42-9a37-117fcb3d8cf4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:22.873949 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713a8e42-7ec0-4c42-9a37-117fcb3d8cf4 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:22.900293 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 289/612] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 19 14:38:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:22.926618 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713a8e42-7ec0-4c42-9a37-117fcb3d8cf4 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:22.951892 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 324/613] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:38:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:38:22.976242 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7e5546-8394-4abf-80a6-722a3167c639 None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:22.988786 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:22.990545 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:23.071927 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713a8e42-7ec0-4c42-9a37-117fcb3d8cf4 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:23.098908 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7e5546-8394-4abf-80a6-722a3167c639 None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:23.110247 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d216f266-8397-49b7-bad3-46eaa3081818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:23.111116 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d216f266-8397-49b7-bad3-46eaa3081818 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:23.111402 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d216f266-8397-49b7-bad3-46eaa3081818 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:23.111671 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d216f266-8397-49b7-bad3-46eaa3081818 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:23.123037 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d216f266-8397-49b7-bad3-46eaa3081818 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:23.124660 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d216f266-8397-49b7-bad3-46eaa3081818 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:23.135504 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e850e456-1579-4298-913d-8cffa67abfe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:23.136311 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e850e456-1579-4298-913d-8cffa67abfe5 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:23.136592 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e850e456-1579-4298-913d-8cffa67abfe5 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:23.136864 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e850e456-1579-4298-913d-8cffa67abfe5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:23.143698 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-d216f266-8397-49b7-bad3-46eaa3081818 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:23.143698 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:23.143698 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:23.143698 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:23.143698 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:23.143698 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:23.143698 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:23.143698 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:23.143698 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:23.143698 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:23.143698 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:23.143698 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:23.143698 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:38:23.143698 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:38:23.143698 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:38:23.144911 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:38:23.144911 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:23.144911 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:23.144911 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:23.144911 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:23.144911 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:23.144911 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:23.144911 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:23.144911 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:38:23.144911 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:23.144911 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:23.144911 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:23.144911 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:23.144911 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:23.144911 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:23.144911 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:23.144911 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:23.146132 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:38:23.146132 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:38:23.146132 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:23.146132 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:38:23.146132 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 290/614] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:38:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:38:23.151600 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d216f266-8397-49b7-bad3-46eaa3081818 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:23.168685 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e850e456-1579-4298-913d-8cffa67abfe5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:38:23.184970 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-e850e456-1579-4298-913d-8cffa67abfe5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:38:23.184970 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:23.184970 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:23.184970 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:23.184970 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:23.184970 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:23.184970 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:23.184970 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:23.184970 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:23.184970 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:23.184970 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:23.184970 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:23.184970 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:38:23.184970 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:38:23.184970 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:38:23.184970 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:38:23.186103 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:38:23.186103 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:38:23.186103 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:38:23.186103 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:38:23.186103 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:23.186103 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:23.186103 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:23.186103 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:23.186103 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:23.186103 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:23.186103 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:23.186103 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:38:23.186103 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:23.186103 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:23.186103 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:23.186103 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:23.187160 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:23.187160 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:23.187160 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:23.187160 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:23.187160 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:38:23.187160 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:38:23.187160 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:38:23.187160 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:38:23.187160 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:38:23.187160 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:38:23.187160 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:38:23.187160 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:38:23.187160 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 325/615] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 14:38:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:38:23.193017 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e850e456-1579-4298-913d-8cffa67abfe5 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:23.272375 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d216f266-8397-49b7-bad3-46eaa3081818 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:23.298219 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:23.315272 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:23.332208 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2921b6b4-ed5c-44b0-afa8-dcc713a1a7c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:23.332799 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2921b6b4-ed5c-44b0-afa8-dcc713a1a7c1 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:23.333076 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2921b6b4-ed5c-44b0-afa8-dcc713a1a7c1 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:23.333348 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2921b6b4-ed5c-44b0-afa8-dcc713a1a7c1 None admin] PATH_INFO: `/v3/domains` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:23.334102 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2921b6b4-ed5c-44b0-afa8-dcc713a1a7c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:23.334408 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2921b6b4-ed5c-44b0-afa8-dcc713a1a7c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:23.336752 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2921b6b4-ed5c-44b0-afa8-dcc713a1a7c1 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:23.368062 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 291/616] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:38:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:38:23.372046 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2921b6b4-ed5c-44b0-afa8-dcc713a1a7c1 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:23.373044 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e850e456-1579-4298-913d-8cffa67abfe5 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:23.440388 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1753808-a904-43c4-ae0d-61ba550177d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:23.441076 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1753808-a904-43c4-ae0d-61ba550177d0 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:23.441355 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1753808-a904-43c4-ae0d-61ba550177d0 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:23.441628 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1753808-a904-43c4-ae0d-61ba550177d0 None admin] PATH_INFO: `/v3/roles` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:23.442411 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1753808-a904-43c4-ae0d-61ba550177d0 None admin] RBAC: Adding query filter params (name=service) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:23.442717 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1753808-a904-43c4-ae0d-61ba550177d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:23.444099 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1753808-a904-43c4-ae0d-61ba550177d0 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:23.465081 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 326/617] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 19 14:38:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:38:23.473581 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1753808-a904-43c4-ae0d-61ba550177d0 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:23.561693 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2921b6b4-ed5c-44b0-afa8-dcc713a1a7c1 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:23.623979 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4613606-b640-4db6-9683-4b896e4c5860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:23.625179 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4613606-b640-4db6-9683-4b896e4c5860 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:23.625774 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4613606-b640-4db6-9683-4b896e4c5860 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:23.626318 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4613606-b640-4db6-9683-4b896e4c5860 None admin] PATH_INFO: `/v3/users/swift` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:23.644593 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4613606-b640-4db6-9683-4b896e4c5860 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:23.646473 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4613606-b640-4db6-9683-4b896e4c5860 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:23.669436 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-f4613606-b640-4db6-9683-4b896e4c5860 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 14:38:23.669436 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:23.669436 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:23.669436 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:23.669436 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:23.669436 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:23.669436 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:23.669436 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:23.669436 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:23.669436 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:23.669436 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:23.669436 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:23.669436 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:38:23.669436 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:38:23.669436 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:38:23.669436 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:38:23.670563 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:23.670563 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:23.670563 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:38:23.670563 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:38:23.670563 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:38:23.670563 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:38:23.670563 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:23.670563 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:23.670563 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:23.670563 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:23.670563 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:23.670563 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:38:23.670563 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:23.670563 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:23.670563 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:23.670563 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:23.670563 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:23.670563 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:23.671597 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:23.671597 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:23.671597 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:38:23.671597 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:38:23.671597 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:38:23.671597 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:38:23.671597 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:38:23.671597 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:38:23.671597 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 14:38:23.671597 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:38:23.675596 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 292/618] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:38:23 2022] GET /identity/v3/users/swift => generated 83 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:38:23.678262 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1753808-a904-43c4-ae0d-61ba550177d0 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:23.682820 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4613606-b640-4db6-9683-4b896e4c5860 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:23.706559 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:23.711150 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:23.723377 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b0350c-7c70-4025-bea3-d3a73c570ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:23.724555 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b0350c-7c70-4025-bea3-d3a73c570ee2 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:23.725067 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b0350c-7c70-4025-bea3-d3a73c570ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:23.726676 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b0350c-7c70-4025-bea3-d3a73c570ee2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:23.745766 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b0350c-7c70-4025-bea3-d3a73c570ee2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:23.748277 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b0350c-7c70-4025-bea3-d3a73c570ee2 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:23.785590 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-96b0350c-7c70-4025-bea3-d3a73c570ee2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:23.785590 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:23.785590 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:23.785590 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:23.785590 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:23.785590 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:23.785590 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:23.785590 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:23.785590 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:23.785590 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:23.785590 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:23.785590 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:23.785590 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:38:23.785590 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:38:23.785590 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:38:23.786648 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:38:23.786648 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:23.786648 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:23.786648 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:23.786648 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:23.786648 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:23.786648 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:23.786648 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:23.786648 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:38:23.786648 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:23.786648 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:23.786648 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:23.786648 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:23.786648 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:23.786648 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:23.786648 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:23.786648 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:23.787641 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:38:23.787641 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:38:23.787641 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:23.787641 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:38:23.792553 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 327/619] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:38:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:23.799141 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b0350c-7c70-4025-bea3-d3a73c570ee2 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:23.911587 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b0350c-7c70-4025-bea3-d3a73c570ee2 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:23.920103 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4613606-b640-4db6-9683-4b896e4c5860 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:23.954133 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50155356-6d46-4c5f-a351-485136534fc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:23.955203 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50155356-6d46-4c5f-a351-485136534fc4 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:23.955736 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50155356-6d46-4c5f-a351-485136534fc4 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:23.956242 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50155356-6d46-4c5f-a351-485136534fc4 None admin] PATH_INFO: `/v3/domains` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:23.957206 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50155356-6d46-4c5f-a351-485136534fc4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:23.957736 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50155356-6d46-4c5f-a351-485136534fc4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:23.959514 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50155356-6d46-4c5f-a351-485136534fc4 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:23.974930 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 328/620] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:38:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:23.982659 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50155356-6d46-4c5f-a351-485136534fc4 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:24.015730 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf20a977-80f6-4ccf-bdeb-cdea1b1340e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:24.016434 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf20a977-80f6-4ccf-bdeb-cdea1b1340e2 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:24.016689 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf20a977-80f6-4ccf-bdeb-cdea1b1340e2 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:24.016969 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf20a977-80f6-4ccf-bdeb-cdea1b1340e2 None admin] PATH_INFO: `/v3/users` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:24.017817 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf20a977-80f6-4ccf-bdeb-cdea1b1340e2 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:24.018157 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf20a977-80f6-4ccf-bdeb-cdea1b1340e2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:24.019607 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf20a977-80f6-4ccf-bdeb-cdea1b1340e2 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:24.059707 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 293/621] 10.222.0.57 () {56 vars in 1135 bytes} [Thu May 19 14:38:23 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 369 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:24.061144 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf20a977-80f6-4ccf-bdeb-cdea1b1340e2 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:24.070300 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:24.150910 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:24.157291 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50155356-6d46-4c5f-a351-485136534fc4 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:24.198371 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf20a977-80f6-4ccf-bdeb-cdea1b1340e2 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:24.204181 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e28eb74-cada-4de8-b958-cf9e2cb37fbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:24.205170 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e28eb74-cada-4de8-b958-cf9e2cb37fbb None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:24.205614 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e28eb74-cada-4de8-b958-cf9e2cb37fbb None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:24.206081 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e28eb74-cada-4de8-b958-cf9e2cb37fbb None admin] PATH_INFO: `/v3/users/nova` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:24.216827 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e28eb74-cada-4de8-b958-cf9e2cb37fbb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:24.219184 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e28eb74-cada-4de8-b958-cf9e2cb37fbb None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:24.236156 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb368a0-a1a3-4784-93a3-83ad3174c892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:24.237041 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb368a0-a1a3-4784-93a3-83ad3174c892 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:24.237462 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb368a0-a1a3-4784-93a3-83ad3174c892 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:24.237814 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb368a0-a1a3-4784-93a3-83ad3174c892 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:24.238830 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-4e28eb74-cada-4de8-b958-cf9e2cb37fbb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 14:38:24.238830 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:24.238830 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:24.238830 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:24.238830 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:24.238830 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:24.238830 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:24.238830 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:24.238830 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:24.238830 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:24.238830 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:24.238830 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:24.238830 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:38:24.238830 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:38:24.238830 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:38:24.238830 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:38:24.239986 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:24.239986 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:24.239986 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:38:24.239986 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:38:24.239986 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:38:24.239986 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:38:24.239986 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:24.239986 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:24.239986 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:24.239986 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:24.239986 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:24.239986 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:38:24.239986 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:24.239986 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:24.239986 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:24.239986 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:24.239986 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:24.239986 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:24.241231 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:24.241231 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:24.241231 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:38:24.241231 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:38:24.241231 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:38:24.241231 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:38:24.241231 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:38:24.241231 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:38:24.241231 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 14:38:24.241231 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:38:24.244951 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 329/622] 10.222.0.57 () {56 vars in 1093 bytes} [Thu May 19 14:38:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:24.246418 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5621fbc8-ba39-495f-8278-c2ff1d695da0 None None] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:24.246851 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5621fbc8-ba39-495f-8278-c2ff1d695da0 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:24.247297 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5621fbc8-ba39-495f-8278-c2ff1d695da0 None None] PATH_INFO: `/` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:24.252822 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 330/623] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:38:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:38:24.254992 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bcc177-788f-4376-a99f-4a8f421eb459 None None] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:24.255281 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bcc177-788f-4376-a99f-4a8f421eb459 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:24.255558 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bcc177-788f-4376-a99f-4a8f421eb459 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:24.261405 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb368a0-a1a3-4784-93a3-83ad3174c892 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:24.262911 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb368a0-a1a3-4784-93a3-83ad3174c892 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:24.277337 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.auth.core [None req-34bcc177-788f-4376-a99f-4a8f421eb459 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:38:24.278933 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.fernet_utils [None req-34bcc177-788f-4376-a99f-4a8f421eb459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:38:24.281828 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-2fb368a0-a1a3-4784-93a3-83ad3174c892 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:24.281828 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:24.281828 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:24.281828 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:24.281828 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:24.281828 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:24.281828 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:24.281828 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:24.281828 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:24.281828 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:24.281828 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:24.281828 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:24.281828 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:38:24.281828 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:38:24.281828 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:38:24.282801 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:38:24.282801 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:24.282801 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:24.282801 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:24.282801 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:24.282801 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:24.282801 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:24.282801 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:24.282801 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:38:24.282801 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:24.282801 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:24.282801 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:24.282801 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:24.282801 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:24.282801 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:24.282801 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:24.282801 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:24.283839 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:38:24.283839 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:38:24.283839 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:24.283839 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:38:24.287733 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 294/624] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:38:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:24.288654 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb368a0-a1a3-4784-93a3-83ad3174c892 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:24.381865 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 331/625] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:38:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:38:24.381865 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bcc177-788f-4376-a99f-4a8f421eb459 None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:24.422383 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:24.427003 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb368a0-a1a3-4784-93a3-83ad3174c892 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:24.445779 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:24.466935 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eabb761-1079-49f6-9d2b-6776f51e5cfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:24.467943 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eabb761-1079-49f6-9d2b-6776f51e5cfe None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:24.468440 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eabb761-1079-49f6-9d2b-6776f51e5cfe None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:24.469008 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eabb761-1079-49f6-9d2b-6776f51e5cfe None admin] PATH_INFO: `/v3/users` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:24.470262 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eabb761-1079-49f6-9d2b-6776f51e5cfe None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:24.470789 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eabb761-1079-49f6-9d2b-6776f51e5cfe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:24.472460 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eabb761-1079-49f6-9d2b-6776f51e5cfe None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:24.497564 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bcc177-788f-4376-a99f-4a8f421eb459 None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:24.498545 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 295/626] 10.222.0.57 () {56 vars in 1133 bytes} [Thu May 19 14:38:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:24.499441 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eabb761-1079-49f6-9d2b-6776f51e5cfe None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:24.548421 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bd4703-7583-4906-809c-c3257de0db6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:24.549107 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bd4703-7583-4906-809c-c3257de0db6a None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:24.549387 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bd4703-7583-4906-809c-c3257de0db6a None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:24.549673 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bd4703-7583-4906-809c-c3257de0db6a None admin] PATH_INFO: `/v3/domains` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:24.550441 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bd4703-7583-4906-809c-c3257de0db6a None admin] RBAC: Adding query filter params (name=Default) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:24.550753 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bd4703-7583-4906-809c-c3257de0db6a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:24.552698 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bd4703-7583-4906-809c-c3257de0db6a None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:24.571957 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 332/627] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:38:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:24.572870 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bd4703-7583-4906-809c-c3257de0db6a None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:24.625823 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eabb761-1079-49f6-9d2b-6776f51e5cfe None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:24.665566 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4ac2b0-bd63-4e73-999d-004563e96d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:24.666517 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4ac2b0-bd63-4e73-999d-004563e96d0f None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:24.666809 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4ac2b0-bd63-4e73-999d-004563e96d0f None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:24.667093 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4ac2b0-bd63-4e73-999d-004563e96d0f None admin] PATH_INFO: `/v3/roles/service` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:24.670089 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:24.679673 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bd4703-7583-4906-809c-c3257de0db6a None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:24.680823 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4ac2b0-bd63-4e73-999d-004563e96d0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:38:24.689363 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-2a4ac2b0-bd63-4e73-999d-004563e96d0f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:38:24.689363 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:24.689363 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:24.689363 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:24.689363 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:24.689363 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:24.689363 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:24.689363 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:24.689363 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:24.689363 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:24.689363 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:24.689363 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:24.689363 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:38:24.689363 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:38:24.689363 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:38:24.689363 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:38:24.690449 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:38:24.690449 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:38:24.690449 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:38:24.690449 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:38:24.690449 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:24.690449 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:24.690449 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:24.690449 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:24.690449 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:24.690449 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:24.690449 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:24.690449 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:38:24.690449 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:24.690449 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:24.690449 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:24.690449 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:24.692236 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:24.692236 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:24.692236 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:24.692236 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:24.692236 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:38:24.692236 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:38:24.692236 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:38:24.692236 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:38:24.692236 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:38:24.692236 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:38:24.692236 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:38:24.692236 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:38:24.692236 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 296/628] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 14:38:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:24.692236 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4ac2b0-bd63-4e73-999d-004563e96d0f None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:24.719012 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1df1b9f-f0fe-4eb5-b270-8fa1d2fb2347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:24.720443 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1df1b9f-f0fe-4eb5-b270-8fa1d2fb2347 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:24.720955 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1df1b9f-f0fe-4eb5-b270-8fa1d2fb2347 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:24.721234 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1df1b9f-f0fe-4eb5-b270-8fa1d2fb2347 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:24.731727 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:24.799847 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1df1b9f-f0fe-4eb5-b270-8fa1d2fb2347 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:24.801626 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1df1b9f-f0fe-4eb5-b270-8fa1d2fb2347 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:24.802149 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4ac2b0-bd63-4e73-999d-004563e96d0f None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:24.839473 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d9dad2-abf0-4d78-ac8a-3fbabeeb790f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:24.840274 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d9dad2-abf0-4d78-ac8a-3fbabeeb790f None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:24.840559 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d9dad2-abf0-4d78-ac8a-3fbabeeb790f None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:24.840839 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d9dad2-abf0-4d78-ac8a-3fbabeeb790f None admin] PATH_INFO: `/v3/projects/service` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:24.848437 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d9dad2-abf0-4d78-ac8a-3fbabeeb790f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:24.849933 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d9dad2-abf0-4d78-ac8a-3fbabeeb790f None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:24.864906 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-26d9dad2-abf0-4d78-ac8a-3fbabeeb790f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:38:24.864906 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:24.864906 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:24.864906 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:24.864906 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:24.864906 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:24.864906 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:24.864906 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:24.864906 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:24.864906 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:24.864906 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:24.864906 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:24.864906 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:38:24.864906 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:38:24.864906 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:38:24.865981 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:38:24.865981 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:24.865981 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:24.865981 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:24.865981 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:24.865981 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:24.865981 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:24.865981 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:24.865981 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:38:24.865981 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:24.865981 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:24.865981 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:24.865981 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:24.865981 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:24.865981 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:24.865981 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:24.865981 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:24.868329 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:38:24.868329 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:38:24.868329 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:38:24.868329 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:38:24.868329 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:38:24.868329 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:38:24.868329 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:38:24.868329 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:38:24.868329 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 297/629] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:38:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:24.868329 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd91121a-b11a-4f10-af81-392d92f92276 None None] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:24.868329 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd91121a-b11a-4f10-af81-392d92f92276 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:24.869045 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd91121a-b11a-4f10-af81-392d92f92276 None None] PATH_INFO: `/` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:24.869887 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 298/630] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:38:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:38:24.874429 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd91121a-b11a-4f10-af81-392d92f92276 None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:24.908558 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:24.919741 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 333/631] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:38:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:38:24.924499 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1df1b9f-f0fe-4eb5-b270-8fa1d2fb2347 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:24.985520 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:24.993362 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd91121a-b11a-4f10-af81-392d92f92276 None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:25.034596 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a258ed-0951-4a5a-a422-69ad8cc1dca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:25.036159 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a258ed-0951-4a5a-a422-69ad8cc1dca7 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:25.036447 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a258ed-0951-4a5a-a422-69ad8cc1dca7 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:25.036722 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a258ed-0951-4a5a-a422-69ad8cc1dca7 None admin] PATH_INFO: `/v3/roles` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:25.037213 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1df1b9f-f0fe-4eb5-b270-8fa1d2fb2347 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:25.037471 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a258ed-0951-4a5a-a422-69ad8cc1dca7 None admin] RBAC: Adding query filter params (name=service) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:25.037778 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a258ed-0951-4a5a-a422-69ad8cc1dca7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:25.039727 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a258ed-0951-4a5a-a422-69ad8cc1dca7 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:25.052370 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 299/632] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 19 14:38:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:25.054332 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815353a5-983f-42ec-b12e-7fc69563526b None None] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:25.055267 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815353a5-983f-42ec-b12e-7fc69563526b None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:25.055555 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815353a5-983f-42ec-b12e-7fc69563526b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:25.073245 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.auth.core [None req-815353a5-983f-42ec-b12e-7fc69563526b None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:38:25.074473 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-815353a5-983f-42ec-b12e-7fc69563526b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:38:25.080143 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ace5b1a-8c91-413a-8970-3eea6cbd705e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:25.081035 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ace5b1a-8c91-413a-8970-3eea6cbd705e None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:25.081508 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ace5b1a-8c91-413a-8970-3eea6cbd705e None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:25.082044 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ace5b1a-8c91-413a-8970-3eea6cbd705e None admin] PATH_INFO: `/v3/projects` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:25.083462 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ace5b1a-8c91-413a-8970-3eea6cbd705e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:25.083974 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ace5b1a-8c91-413a-8970-3eea6cbd705e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:25.086401 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ace5b1a-8c91-413a-8970-3eea6cbd705e None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:25.104401 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 334/633] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:38:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:25.104401 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ace5b1a-8c91-413a-8970-3eea6cbd705e None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:25.157754 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 300/634] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:38:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:38:25.158931 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815353a5-983f-42ec-b12e-7fc69563526b None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:25.181454 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:25.234112 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ace5b1a-8c91-413a-8970-3eea6cbd705e None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:25.260897 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:25.270190 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87c8077-18ee-46f3-8787-473fae28c296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:25.271075 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87c8077-18ee-46f3-8787-473fae28c296 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:25.271369 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87c8077-18ee-46f3-8787-473fae28c296 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:25.271641 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87c8077-18ee-46f3-8787-473fae28c296 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:25.275865 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815353a5-983f-42ec-b12e-7fc69563526b None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:25.285143 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87c8077-18ee-46f3-8787-473fae28c296 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:25.287090 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87c8077-18ee-46f3-8787-473fae28c296 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:25.307170 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-e87c8077-18ee-46f3-8787-473fae28c296 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:25.307170 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:25.307170 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:25.307170 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:25.307170 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:25.307170 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:25.307170 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:25.307170 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:25.307170 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:25.307170 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:25.307170 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:25.307170 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:25.307170 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:38:25.307170 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:38:25.307170 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:38:25.308197 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:38:25.308197 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:25.308197 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:25.308197 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:25.308197 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:25.308197 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:25.308197 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:25.308197 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:25.308197 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:38:25.308197 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:25.308197 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:25.308197 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:25.308197 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:25.308197 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:25.308197 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:25.308197 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:25.308197 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:25.309215 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:38:25.309215 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:38:25.309215 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:25.309215 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:38:25.309215 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 335/635] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:38:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:25.309215 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87c8077-18ee-46f3-8787-473fae28c296 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:25.321036 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f681c97d-0725-4309-b9ee-ad4a0511c94c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:25.321906 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f681c97d-0725-4309-b9ee-ad4a0511c94c None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:25.322183 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f681c97d-0725-4309-b9ee-ad4a0511c94c None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:25.322466 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f681c97d-0725-4309-b9ee-ad4a0511c94c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:25.334665 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f681c97d-0725-4309-b9ee-ad4a0511c94c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:25.337095 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f681c97d-0725-4309-b9ee-ad4a0511c94c None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:25.359202 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-f681c97d-0725-4309-b9ee-ad4a0511c94c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:25.359202 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:25.359202 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:25.359202 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:25.359202 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:25.359202 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:25.359202 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:25.359202 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:25.359202 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:25.359202 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:25.359202 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:25.359202 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:25.359202 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:38:25.359202 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:38:25.359202 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:38:25.361115 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:38:25.361115 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:25.361115 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:25.361115 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:25.361115 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:25.361115 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:25.361115 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:25.361115 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:25.361115 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:38:25.361115 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:25.361115 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:25.361115 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:25.361115 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:25.361115 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:25.361115 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:25.361115 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:25.361115 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:25.362149 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:38:25.362149 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:38:25.362149 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:25.362149 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:38:25.362149 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 301/636] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:38:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:25.363000 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f681c97d-0725-4309-b9ee-ad4a0511c94c None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:25.427342 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87c8077-18ee-46f3-8787-473fae28c296 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:25.432098 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:25.468708 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd05bb6-0709-4562-834a-5901e337cf26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:25.469611 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd05bb6-0709-4562-834a-5901e337cf26 None admin] REQUEST_METHOD: `PUT` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:25.469981 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd05bb6-0709-4562-834a-5901e337cf26 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:25.470197 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd05bb6-0709-4562-834a-5901e337cf26 None admin] PATH_INFO: `/v3/projects/42f35b96b9d4499abfb5e5f7cca49adc/users/4f8e2e0909074a85bb493c49ac3a6d4d/roles/ffb8da90af53491980816087ee621ccb` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:25.472269 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f681c97d-0725-4309-b9ee-ad4a0511c94c None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:25.486850 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd05bb6-0709-4562-834a-5901e337cf26 None admin] RBAC: Authorizing `identity:create_grant(project_id=42f35b96b9d4499abfb5e5f7cca49adc, user_id=4f8e2e0909074a85bb493c49ac3a6d4d, role_id=ffb8da90af53491980816087ee621ccb)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:25.488335 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd05bb6-0709-4562-834a-5901e337cf26 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:25.502356 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 336/637] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 14:38:25 2022] PUT /identity/v3/projects/42f35b96b9d4499abfb5e5f7cca49adc/users/4f8e2e0909074a85bb493c49ac3a6d4d/roles/ffb8da90af53491980816087ee621ccb => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:38:25.503571 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd05bb6-0709-4562-834a-5901e337cf26 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:25.526765 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86dc682-790b-45e2-88d7-3f755a8d8a4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:25.527584 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86dc682-790b-45e2-88d7-3f755a8d8a4a None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:25.527986 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86dc682-790b-45e2-88d7-3f755a8d8a4a None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:25.528328 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86dc682-790b-45e2-88d7-3f755a8d8a4a None admin] PATH_INFO: `/v3/roles/service` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:25.538952 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:25.542376 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86dc682-790b-45e2-88d7-3f755a8d8a4a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:38:25.551414 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-c86dc682-790b-45e2-88d7-3f755a8d8a4a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:38:25.551414 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:25.551414 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:25.551414 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:25.551414 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:25.551414 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:25.551414 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:25.551414 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:25.551414 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:25.551414 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:25.551414 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:25.551414 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:25.551414 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:38:25.551414 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:38:25.551414 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:38:25.551414 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:38:25.552560 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:38:25.552560 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:38:25.552560 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:38:25.552560 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:38:25.552560 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:25.552560 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:25.552560 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:25.552560 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:25.552560 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:25.552560 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:25.552560 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:25.552560 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:38:25.552560 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:25.552560 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:25.552560 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:25.552560 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:25.559935 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:25.559935 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:25.559935 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:25.559935 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:25.559935 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:38:25.559935 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:38:25.559935 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:38:25.559935 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:38:25.559935 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:38:25.559935 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:38:25.559935 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:38:25.559935 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:38:25.559935 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 302/638] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 14:38:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:25.559935 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86dc682-790b-45e2-88d7-3f755a8d8a4a None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:25.637979 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd05bb6-0709-4562-834a-5901e337cf26 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:25.675934 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86dc682-790b-45e2-88d7-3f755a8d8a4a None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:25.679605 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976c66f6-affd-43f3-a385-cde9b9153051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:25.680297 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976c66f6-affd-43f3-a385-cde9b9153051 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:25.680615 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976c66f6-affd-43f3-a385-cde9b9153051 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:25.680890 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976c66f6-affd-43f3-a385-cde9b9153051 None admin] PATH_INFO: `/v3/domains` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:25.681637 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976c66f6-affd-43f3-a385-cde9b9153051 None admin] RBAC: Adding query filter params (name=Default) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:25.681976 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976c66f6-affd-43f3-a385-cde9b9153051 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:25.683379 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976c66f6-affd-43f3-a385-cde9b9153051 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:25.698386 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 337/639] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:38:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:25.700562 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad19848-6784-43e9-b3bb-8550e06ae089 None None] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:25.700858 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad19848-6784-43e9-b3bb-8550e06ae089 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:25.701139 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad19848-6784-43e9-b3bb-8550e06ae089 None None] PATH_INFO: `/` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:25.702783 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 338/640] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:38:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:38:25.703519 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad19848-6784-43e9-b3bb-8550e06ae089 None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:25.723815 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102eabd5-4dc5-4c87-bb37-8f6347d92635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:25.724546 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102eabd5-4dc5-4c87-bb37-8f6347d92635 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:25.724834 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102eabd5-4dc5-4c87-bb37-8f6347d92635 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:25.725139 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102eabd5-4dc5-4c87-bb37-8f6347d92635 None admin] PATH_INFO: `/v3/domains` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:25.725363 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:25.725948 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102eabd5-4dc5-4c87-bb37-8f6347d92635 None admin] RBAC: Adding query filter params (name=Default) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:25.726459 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102eabd5-4dc5-4c87-bb37-8f6347d92635 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:25.727855 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102eabd5-4dc5-4c87-bb37-8f6347d92635 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:25.740467 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 303/641] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:38:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:25.741954 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102eabd5-4dc5-4c87-bb37-8f6347d92635 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:25.807147 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:25.825677 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad19848-6784-43e9-b3bb-8550e06ae089 None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:25.853630 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102eabd5-4dc5-4c87-bb37-8f6347d92635 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:25.861342 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f7b536-05c4-4120-9bfa-e6a65a5885a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:25.862070 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f7b536-05c4-4120-9bfa-e6a65a5885a5 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:25.862346 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f7b536-05c4-4120-9bfa-e6a65a5885a5 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:25.862625 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f7b536-05c4-4120-9bfa-e6a65a5885a5 None admin] PATH_INFO: `/v3/roles` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:25.863376 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f7b536-05c4-4120-9bfa-e6a65a5885a5 None admin] RBAC: Adding query filter params (name=service) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:25.863684 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f7b536-05c4-4120-9bfa-e6a65a5885a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:25.865073 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f7b536-05c4-4120-9bfa-e6a65a5885a5 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:25.874332 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 339/642] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 19 14:38:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:25.875750 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363a63f6-d577-4d2c-bbbf-5264fa2b234d None None] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:25.876043 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363a63f6-d577-4d2c-bbbf-5264fa2b234d None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:25.876323 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363a63f6-d577-4d2c-bbbf-5264fa2b234d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:25.894755 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10026943-bb08-4a45-9b71-1fdad70cb416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:25.895214 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.auth.core [None req-363a63f6-d577-4d2c-bbbf-5264fa2b234d None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:38:25.896066 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10026943-bb08-4a45-9b71-1fdad70cb416 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:25.896429 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.fernet_utils [None req-363a63f6-d577-4d2c-bbbf-5264fa2b234d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:38:25.896752 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10026943-bb08-4a45-9b71-1fdad70cb416 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:25.897208 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10026943-bb08-4a45-9b71-1fdad70cb416 None admin] PATH_INFO: `/v3/projects/service` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:25.907283 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10026943-bb08-4a45-9b71-1fdad70cb416 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:25.908753 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10026943-bb08-4a45-9b71-1fdad70cb416 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:25.923182 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-10026943-bb08-4a45-9b71-1fdad70cb416 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:38:25.923182 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:25.923182 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:25.923182 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:25.923182 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:25.923182 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:25.923182 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:25.923182 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:25.923182 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:25.923182 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:25.923182 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:25.923182 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:25.923182 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:38:25.923182 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:38:25.923182 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:38:25.924386 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:38:25.924386 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:25.924386 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:25.924386 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:25.924386 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:25.924386 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:25.924386 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:25.924386 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:25.924386 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:38:25.924386 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:25.924386 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:25.924386 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:25.924386 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:25.924386 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:25.924386 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:25.924386 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:25.924386 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:25.925599 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:38:25.925599 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:38:25.925599 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:38:25.925599 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:38:25.925599 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:38:25.925599 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:38:25.925599 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:38:25.925599 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:38:25.927498 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 304/643] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:38:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:25.928440 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10026943-bb08-4a45-9b71-1fdad70cb416 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:25.978914 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 340/644] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:38:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:38:25.980092 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363a63f6-d577-4d2c-bbbf-5264fa2b234d None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:26.001819 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:26.047619 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10026943-bb08-4a45-9b71-1fdad70cb416 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:26.071009 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:26.088499 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e562d5-6b9c-4c04-9b62-a50f2a8138dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:26.089304 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e562d5-6b9c-4c04-9b62-a50f2a8138dc None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:26.089593 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e562d5-6b9c-4c04-9b62-a50f2a8138dc None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:26.089894 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e562d5-6b9c-4c04-9b62-a50f2a8138dc None admin] PATH_INFO: `/v3/users/glance` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:26.099306 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e562d5-6b9c-4c04-9b62-a50f2a8138dc None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:26.100213 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363a63f6-d577-4d2c-bbbf-5264fa2b234d None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:26.100931 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e562d5-6b9c-4c04-9b62-a50f2a8138dc None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:26.116145 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-30e562d5-6b9c-4c04-9b62-a50f2a8138dc None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 19 14:38:26.116145 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:26.116145 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:26.116145 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:26.116145 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:26.116145 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:26.116145 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:26.116145 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:26.116145 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:26.116145 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:26.116145 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:26.116145 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:26.116145 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:38:26.116145 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:38:26.116145 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:38:26.116145 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:38:26.117347 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:26.117347 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:26.117347 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:38:26.117347 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:38:26.117347 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:38:26.117347 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:38:26.117347 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:26.117347 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:26.117347 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:26.117347 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:26.117347 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:26.117347 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:38:26.117347 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:26.117347 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:26.117347 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:26.117347 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:26.117347 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:26.117347 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:26.118437 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:26.118437 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:26.118437 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:38:26.118437 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:38:26.118437 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:38:26.118437 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:38:26.118437 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:38:26.118437 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:38:26.118437 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 19 14:38:26.118437 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:38:26.118437 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 305/645] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:38:25 2022] GET /identity/v3/users/glance => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:26.119077 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e562d5-6b9c-4c04-9b62-a50f2a8138dc None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:26.141976 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28a0453-9edb-49cf-a8aa-820f17aa90a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:26.142777 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28a0453-9edb-49cf-a8aa-820f17aa90a2 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:26.143060 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28a0453-9edb-49cf-a8aa-820f17aa90a2 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:26.143334 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28a0453-9edb-49cf-a8aa-820f17aa90a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:26.154125 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28a0453-9edb-49cf-a8aa-820f17aa90a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:26.155586 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28a0453-9edb-49cf-a8aa-820f17aa90a2 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:26.183950 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-e28a0453-9edb-49cf-a8aa-820f17aa90a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:26.183950 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:26.183950 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:26.183950 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:26.183950 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:26.183950 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:26.183950 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:26.183950 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:26.183950 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:26.183950 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:26.183950 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:26.183950 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:26.183950 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:38:26.183950 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:38:26.183950 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:38:26.185103 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:38:26.185103 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:26.185103 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:26.185103 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:26.185103 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:26.185103 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:26.185103 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:26.185103 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:26.185103 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:38:26.185103 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:26.185103 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:26.185103 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:26.185103 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:26.185103 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:26.185103 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:26.185103 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:26.185103 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:26.186523 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:38:26.186523 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:38:26.186523 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:26.186523 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:38:26.186523 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 341/646] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:38:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:26.186523 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28a0453-9edb-49cf-a8aa-820f17aa90a2 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:26.244067 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e562d5-6b9c-4c04-9b62-a50f2a8138dc None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:26.246769 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:26.281086 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4329ddd3-921d-4f1f-8111-eec6a25628f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:26.281888 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4329ddd3-921d-4f1f-8111-eec6a25628f6 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:26.282111 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4329ddd3-921d-4f1f-8111-eec6a25628f6 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:26.282385 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4329ddd3-921d-4f1f-8111-eec6a25628f6 None admin] PATH_INFO: `/v3/projects` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:26.283174 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4329ddd3-921d-4f1f-8111-eec6a25628f6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:26.283547 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4329ddd3-921d-4f1f-8111-eec6a25628f6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:26.284986 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4329ddd3-921d-4f1f-8111-eec6a25628f6 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:26.294852 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28a0453-9edb-49cf-a8aa-820f17aa90a2 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:26.304111 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 306/647] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:38:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:26.304111 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4329ddd3-921d-4f1f-8111-eec6a25628f6 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:26.333114 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf0d44e-e864-4628-bc11-aaa4ab2fc3ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:26.334279 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf0d44e-e864-4628-bc11-aaa4ab2fc3ef None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:26.335004 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf0d44e-e864-4628-bc11-aaa4ab2fc3ef None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:26.335953 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf0d44e-e864-4628-bc11-aaa4ab2fc3ef None admin] PATH_INFO: `/v3/roles/service` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:26.347159 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf0d44e-e864-4628-bc11-aaa4ab2fc3ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:38:26.350153 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:26.355513 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-baf0d44e-e864-4628-bc11-aaa4ab2fc3ef None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:38:26.355513 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:26.355513 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:26.355513 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:26.355513 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:26.355513 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:26.355513 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:26.355513 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:26.355513 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:26.355513 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:26.355513 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:26.355513 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:26.355513 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:38:26.355513 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:38:26.355513 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:38:26.355513 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:38:26.356537 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:38:26.356537 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:38:26.356537 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:38:26.356537 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:38:26.356537 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:26.356537 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:26.356537 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:26.356537 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:26.356537 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:26.356537 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:26.356537 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:26.356537 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:38:26.356537 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:26.356537 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:26.356537 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:26.356537 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:26.357477 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:26.357477 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:26.357477 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:26.357477 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:26.357477 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:38:26.357477 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:38:26.357477 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:38:26.357477 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:38:26.357477 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:38:26.357477 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:38:26.357477 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:38:26.357477 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:38:26.357477 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 342/648] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 14:38:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:26.357477 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf0d44e-e864-4628-bc11-aaa4ab2fc3ef None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:26.417661 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4329ddd3-921d-4f1f-8111-eec6a25628f6 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:26.455316 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9342d8-a8f2-456c-8db7-3873ed5be266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:26.455992 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9342d8-a8f2-456c-8db7-3873ed5be266 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:26.456280 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9342d8-a8f2-456c-8db7-3873ed5be266 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:26.456564 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9342d8-a8f2-456c-8db7-3873ed5be266 None admin] PATH_INFO: `/v3/users` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:26.457408 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9342d8-a8f2-456c-8db7-3873ed5be266 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:26.457721 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9342d8-a8f2-456c-8db7-3873ed5be266 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:26.459199 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9342d8-a8f2-456c-8db7-3873ed5be266 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:26.470588 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf0d44e-e864-4628-bc11-aaa4ab2fc3ef None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:26.477662 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 307/649] 10.222.0.57 () {56 vars in 1137 bytes} [Thu May 19 14:38:26 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:26.478522 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9342d8-a8f2-456c-8db7-3873ed5be266 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:26.497796 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:26.514994 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5b806c-a968-4016-ba77-601cf6479b91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:26.515673 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5b806c-a968-4016-ba77-601cf6479b91 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:26.515955 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5b806c-a968-4016-ba77-601cf6479b91 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:26.516233 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5b806c-a968-4016-ba77-601cf6479b91 None admin] PATH_INFO: `/v3/domains` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:26.516952 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5b806c-a968-4016-ba77-601cf6479b91 None admin] RBAC: Adding query filter params (name=Default) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:26.517320 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5b806c-a968-4016-ba77-601cf6479b91 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:26.518748 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5b806c-a968-4016-ba77-601cf6479b91 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:26.535582 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 343/650] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:38:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:26.535582 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5b806c-a968-4016-ba77-601cf6479b91 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:26.582862 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:26.587927 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9342d8-a8f2-456c-8db7-3873ed5be266 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:26.624314 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6e4d1f-90e3-49e0-97e9-1dd44d57567b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:26.625588 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6e4d1f-90e3-49e0-97e9-1dd44d57567b None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:26.625936 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6e4d1f-90e3-49e0-97e9-1dd44d57567b None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:26.626227 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6e4d1f-90e3-49e0-97e9-1dd44d57567b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:26.627233 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6e4d1f-90e3-49e0-97e9-1dd44d57567b None admin] RBAC: Adding query filter params (role.id=ffb8da90af53491980816087ee621ccb, scope.project.id=42f35b96b9d4499abfb5e5f7cca49adc, user.id=a9591727b2f54244baf90b11a8e2277f) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:26.627553 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6e4d1f-90e3-49e0-97e9-1dd44d57567b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:26.629307 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6e4d1f-90e3-49e0-97e9-1dd44d57567b None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:26.649891 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 308/651] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:38:26 2022] GET /identity/v3/role_assignments?user.id=a9591727b2f54244baf90b11a8e2277f&scope.project.id=42f35b96b9d4499abfb5e5f7cca49adc&role.id=ffb8da90af53491980816087ee621ccb => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:26.649891 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6e4d1f-90e3-49e0-97e9-1dd44d57567b None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:26.650247 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5b806c-a968-4016-ba77-601cf6479b91 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:26.695657 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dc28ef-1340-4dca-abba-7c3589cb2db6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:26.697491 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dc28ef-1340-4dca-abba-7c3589cb2db6 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:26.697975 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dc28ef-1340-4dca-abba-7c3589cb2db6 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:26.698361 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dc28ef-1340-4dca-abba-7c3589cb2db6 None admin] PATH_INFO: `/v3/roles` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:26.699291 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dc28ef-1340-4dca-abba-7c3589cb2db6 None admin] RBAC: Adding query filter params (name=service) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:26.699793 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dc28ef-1340-4dca-abba-7c3589cb2db6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:26.701333 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dc28ef-1340-4dca-abba-7c3589cb2db6 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:26.716443 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 344/652] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 19 14:38:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:26.717617 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dc28ef-1340-4dca-abba-7c3589cb2db6 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:26.777190 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6e4d1f-90e3-49e0-97e9-1dd44d57567b None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:26.784115 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:26.823265 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefc448a-945b-4044-945e-7fb8ce787e33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:26.823988 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefc448a-945b-4044-945e-7fb8ce787e33 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:26.824512 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefc448a-945b-4044-945e-7fb8ce787e33 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:26.824796 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefc448a-945b-4044-945e-7fb8ce787e33 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:26.860819 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dc28ef-1340-4dca-abba-7c3589cb2db6 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:26.875260 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:26.909703 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a232a8-6bdb-4869-810c-f1a089f41fa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:26.911031 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a232a8-6bdb-4869-810c-f1a089f41fa9 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:26.911326 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a232a8-6bdb-4869-810c-f1a089f41fa9 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:26.911896 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a232a8-6bdb-4869-810c-f1a089f41fa9 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:26.923562 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a232a8-6bdb-4869-810c-f1a089f41fa9 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:26.925023 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a232a8-6bdb-4869-810c-f1a089f41fa9 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:26.937898 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefc448a-945b-4044-945e-7fb8ce787e33 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:26.939348 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefc448a-945b-4044-945e-7fb8ce787e33 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:26.948734 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-d0a232a8-6bdb-4869-810c-f1a089f41fa9 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 14:38:26.948734 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:26.948734 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:26.948734 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:26.948734 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:26.948734 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:26.948734 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:26.948734 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:26.948734 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:26.948734 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:26.948734 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:26.948734 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:26.948734 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:38:26.948734 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:38:26.948734 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:38:26.948734 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:38:26.949807 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:26.949807 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:26.949807 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:38:26.949807 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:38:26.949807 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:38:26.949807 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:38:26.949807 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:26.949807 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:26.949807 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:26.949807 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:26.949807 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:26.949807 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:38:26.949807 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:26.949807 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:26.949807 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:26.949807 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:26.949807 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:26.949807 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:26.951069 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:26.951069 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:26.951069 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:38:26.951069 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:38:26.951069 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:38:26.951069 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:38:26.951069 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:38:26.951069 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:38:26.951069 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 14:38:26.951069 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:38:26.951069 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 345/653] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:38:26 2022] GET /identity/v3/users/cinder => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:26.951708 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a232a8-6bdb-4869-810c-f1a089f41fa9 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:27.060461 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:27.063891 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a232a8-6bdb-4869-810c-f1a089f41fa9 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:27.074568 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 309/654] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:38:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:38:27.075963 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefc448a-945b-4044-945e-7fb8ce787e33 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:27.104977 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921cf0c7-dce3-4411-9886-db6111306777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:27.105976 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921cf0c7-dce3-4411-9886-db6111306777 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:27.106366 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921cf0c7-dce3-4411-9886-db6111306777 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:27.106747 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921cf0c7-dce3-4411-9886-db6111306777 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:27.119638 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921cf0c7-dce3-4411-9886-db6111306777 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:27.121182 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921cf0c7-dce3-4411-9886-db6111306777 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:27.140010 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-921cf0c7-dce3-4411-9886-db6111306777 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:27.140010 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:27.140010 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:27.140010 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:27.140010 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:27.140010 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:27.140010 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:27.140010 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:27.140010 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:27.140010 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:27.140010 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:27.140010 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:27.140010 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:38:27.140010 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:38:27.140010 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:38:27.141550 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:38:27.141550 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:27.141550 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:27.141550 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:27.141550 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:27.141550 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:27.141550 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:27.141550 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:27.141550 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:38:27.141550 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:27.141550 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:27.141550 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:27.141550 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:27.141550 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:27.141550 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:27.141550 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:27.141550 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:27.143713 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:38:27.143713 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:38:27.143713 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:27.143713 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:38:27.143713 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 346/655] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:38:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:27.147690 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921cf0c7-dce3-4411-9886-db6111306777 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:27.160188 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:27.211441 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefc448a-945b-4044-945e-7fb8ce787e33 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:27.248494 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b528f4-1992-47c0-88f9-cc7386a38b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:27.249176 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b528f4-1992-47c0-88f9-cc7386a38b92 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:27.249463 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b528f4-1992-47c0-88f9-cc7386a38b92 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:27.249745 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b528f4-1992-47c0-88f9-cc7386a38b92 None admin] PATH_INFO: `/v3/users` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:27.250640 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b528f4-1992-47c0-88f9-cc7386a38b92 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:27.251010 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b528f4-1992-47c0-88f9-cc7386a38b92 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:27.252613 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b528f4-1992-47c0-88f9-cc7386a38b92 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:27.259588 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921cf0c7-dce3-4411-9886-db6111306777 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:27.272574 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 310/656] 10.222.0.57 () {56 vars in 1137 bytes} [Thu May 19 14:38:27 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:27.273651 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b528f4-1992-47c0-88f9-cc7386a38b92 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:27.296535 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cbbe9a-6073-439b-8209-a2952d9d3f0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:27.297349 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cbbe9a-6073-439b-8209-a2952d9d3f0e None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:27.297633 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cbbe9a-6073-439b-8209-a2952d9d3f0e None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:27.297964 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cbbe9a-6073-439b-8209-a2952d9d3f0e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:27.311699 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cbbe9a-6073-439b-8209-a2952d9d3f0e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:27.313200 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cbbe9a-6073-439b-8209-a2952d9d3f0e None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:27.332293 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-38cbbe9a-6073-439b-8209-a2952d9d3f0e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:27.332293 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:27.332293 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:27.332293 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:27.332293 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:27.332293 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:27.332293 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:27.332293 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:27.332293 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:27.332293 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:27.332293 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:27.332293 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:27.332293 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:38:27.332293 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:38:27.332293 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:38:27.333351 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:38:27.333351 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:27.333351 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:27.333351 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:27.333351 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:27.333351 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:27.333351 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:27.333351 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:27.333351 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:38:27.333351 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:27.333351 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:27.333351 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:27.333351 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:27.333351 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:27.333351 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:27.333351 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:27.333351 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:27.334619 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:38:27.334619 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:38:27.334619 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:27.334619 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:38:27.334619 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 347/657] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:38:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:27.334619 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cbbe9a-6073-439b-8209-a2952d9d3f0e None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:27.338590 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:27.402718 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b528f4-1992-47c0-88f9-cc7386a38b92 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:27.429554 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:27.445876 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a102e5ca-c316-4a5e-b3c8-df140bd48eee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:27.446973 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a102e5ca-c316-4a5e-b3c8-df140bd48eee None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:27.447599 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a102e5ca-c316-4a5e-b3c8-df140bd48eee None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:27.447949 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a102e5ca-c316-4a5e-b3c8-df140bd48eee None admin] PATH_INFO: `/v3/domains` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:27.448765 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a102e5ca-c316-4a5e-b3c8-df140bd48eee None admin] RBAC: Adding query filter params (name=Default) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:27.449147 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a102e5ca-c316-4a5e-b3c8-df140bd48eee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:27.449271 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cbbe9a-6073-439b-8209-a2952d9d3f0e None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:27.450847 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a102e5ca-c316-4a5e-b3c8-df140bd48eee None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:27.467657 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 311/658] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:38:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:27.468892 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a102e5ca-c316-4a5e-b3c8-df140bd48eee None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:27.486873 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1ca2ef-a135-4bab-86f3-9cfa5df64d32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:27.487628 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1ca2ef-a135-4bab-86f3-9cfa5df64d32 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:27.487918 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1ca2ef-a135-4bab-86f3-9cfa5df64d32 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:27.488191 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1ca2ef-a135-4bab-86f3-9cfa5df64d32 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:27.577785 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1ca2ef-a135-4bab-86f3-9cfa5df64d32 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:27.580369 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1ca2ef-a135-4bab-86f3-9cfa5df64d32 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:27.584097 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:27.586944 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a102e5ca-c316-4a5e-b3c8-df140bd48eee None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:27.627464 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779f9aa0-b0c0-45b1-a822-2b03faf23692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:27.628535 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779f9aa0-b0c0-45b1-a822-2b03faf23692 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:27.629032 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779f9aa0-b0c0-45b1-a822-2b03faf23692 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:27.629501 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779f9aa0-b0c0-45b1-a822-2b03faf23692 None admin] PATH_INFO: `/v3/domains` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:27.630491 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779f9aa0-b0c0-45b1-a822-2b03faf23692 None admin] RBAC: Adding query filter params (name=Default) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:27.631024 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779f9aa0-b0c0-45b1-a822-2b03faf23692 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:27.633718 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779f9aa0-b0c0-45b1-a822-2b03faf23692 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:27.653996 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 312/659] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:38:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:27.655213 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779f9aa0-b0c0-45b1-a822-2b03faf23692 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:27.702462 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:27.719549 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 348/660] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:38:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:38:27.719549 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1ca2ef-a135-4bab-86f3-9cfa5df64d32 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:27.783443 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779f9aa0-b0c0-45b1-a822-2b03faf23692 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:27.828143 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befca54b-04f9-4e51-9d07-016c7bbe2ff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:27.829246 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1ca2ef-a135-4bab-86f3-9cfa5df64d32 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:27.829958 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befca54b-04f9-4e51-9d07-016c7bbe2ff3 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:27.830456 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befca54b-04f9-4e51-9d07-016c7bbe2ff3 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:27.830940 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befca54b-04f9-4e51-9d07-016c7bbe2ff3 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:27.832422 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:27.840437 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befca54b-04f9-4e51-9d07-016c7bbe2ff3 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:27.841910 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befca54b-04f9-4e51-9d07-016c7bbe2ff3 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:27.858461 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-befca54b-04f9-4e51-9d07-016c7bbe2ff3 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 19 14:38:27.858461 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:27.858461 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:27.858461 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:27.858461 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:27.858461 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:27.858461 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:27.858461 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:27.858461 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:27.858461 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:27.858461 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:27.858461 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:27.858461 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:38:27.858461 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:38:27.858461 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:38:27.858461 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:38:27.859526 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:27.859526 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:27.859526 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:38:27.859526 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:38:27.859526 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:38:27.859526 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:38:27.859526 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:27.859526 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:27.859526 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:27.859526 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:27.859526 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:27.859526 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:38:27.859526 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:27.859526 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:27.859526 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:27.859526 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:27.859526 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:27.859526 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:27.860567 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:27.860567 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:27.860567 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:38:27.860567 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:38:27.860567 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:38:27.860567 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:38:27.860567 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:38:27.860567 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:38:27.860567 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 19 14:38:27.860567 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:38:27.864122 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 313/661] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 14:38:27 2022] GET /identity/v3/users/neutron => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:27.864939 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befca54b-04f9-4e51-9d07-016c7bbe2ff3 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:27.869935 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec43a036-b965-44b5-be89-b2246c612e9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:27.870737 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec43a036-b965-44b5-be89-b2246c612e9d None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:27.871024 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec43a036-b965-44b5-be89-b2246c612e9d None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:27.871302 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec43a036-b965-44b5-be89-b2246c612e9d None admin] PATH_INFO: `/v3/projects/service` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:27.878263 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec43a036-b965-44b5-be89-b2246c612e9d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:27.879711 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec43a036-b965-44b5-be89-b2246c612e9d None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:27.894904 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-ec43a036-b965-44b5-be89-b2246c612e9d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:38:27.894904 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:27.894904 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:27.894904 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:27.894904 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:27.894904 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:27.894904 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:27.894904 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:27.894904 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:27.894904 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:27.894904 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:27.894904 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:27.894904 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:38:27.894904 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:38:27.894904 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:38:27.895889 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:38:27.895889 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:27.895889 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:27.895889 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:27.895889 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:27.895889 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:27.895889 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:27.895889 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:27.895889 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:38:27.895889 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:27.895889 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:27.895889 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:27.895889 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:27.895889 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:27.895889 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:27.895889 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:27.895889 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:27.896843 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:38:27.896843 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:38:27.896843 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:38:27.896843 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:38:27.896843 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:38:27.896843 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:38:27.896843 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:38:27.896843 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:38:27.896843 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 349/662] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:38:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:27.897387 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe2dff8-d72a-4625-9c78-971e5758cbb1 None None] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:27.897816 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe2dff8-d72a-4625-9c78-971e5758cbb1 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:27.898050 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe2dff8-d72a-4625-9c78-971e5758cbb1 None None] PATH_INFO: `/` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:27.899474 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 350/663] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:38:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:38:27.904167 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe2dff8-d72a-4625-9c78-971e5758cbb1 None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:27.981806 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:27.985539 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befca54b-04f9-4e51-9d07-016c7bbe2ff3 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:28.024195 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe2dff8-d72a-4625-9c78-971e5758cbb1 None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:28.025872 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ffef3e-e7ba-47b4-bb0d-bedca185a90f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:28.027506 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ffef3e-e7ba-47b4-bb0d-bedca185a90f None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:28.027799 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ffef3e-e7ba-47b4-bb0d-bedca185a90f None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:28.028081 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ffef3e-e7ba-47b4-bb0d-bedca185a90f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:28.040694 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ffef3e-e7ba-47b4-bb0d-bedca185a90f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:28.042193 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ffef3e-e7ba-47b4-bb0d-bedca185a90f None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:28.061038 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e53ae30-a8a2-49c5-b5f8-23026c5f0203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:28.061243 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-36ffef3e-e7ba-47b4-bb0d-bedca185a90f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:28.061243 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:28.061243 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:28.061243 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:28.061243 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:28.061243 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:28.061243 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:28.061243 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:28.061243 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:28.061243 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:28.061243 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:28.061243 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:28.061243 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:38:28.061243 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:38:28.061243 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:38:28.062473 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:38:28.062473 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:28.062473 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:28.062473 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:28.062473 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:28.062473 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:28.062473 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:28.062473 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:28.062473 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:38:28.062473 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:28.062473 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:28.062473 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:28.062473 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:28.062473 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:28.062473 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:28.062473 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:28.062473 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:28.063896 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:38:28.063896 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:38:28.063896 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:28.063896 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:38:28.064154 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e53ae30-a8a2-49c5-b5f8-23026c5f0203 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:28.064154 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e53ae30-a8a2-49c5-b5f8-23026c5f0203 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:28.064154 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e53ae30-a8a2-49c5-b5f8-23026c5f0203 None admin] PATH_INFO: `/v3/users` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:28.064154 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e53ae30-a8a2-49c5-b5f8-23026c5f0203 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:28.064154 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e53ae30-a8a2-49c5-b5f8-23026c5f0203 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:28.065087 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e53ae30-a8a2-49c5-b5f8-23026c5f0203 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:28.066940 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 314/664] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:38:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:28.067816 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ffef3e-e7ba-47b4-bb0d-bedca185a90f None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:28.088013 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 351/665] 10.222.0.57 () {56 vars in 1139 bytes} [Thu May 19 14:38:27 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:28.089277 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec9a4ae-95a1-4047-a033-3482368780c4 None None] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:28.089576 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec9a4ae-95a1-4047-a033-3482368780c4 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:28.089883 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec9a4ae-95a1-4047-a033-3482368780c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:28.107834 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.auth.core [None req-1ec9a4ae-95a1-4047-a033-3482368780c4 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:38:28.109044 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.fernet_utils [None req-1ec9a4ae-95a1-4047-a033-3482368780c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:38:28.127278 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:28.183534 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ffef3e-e7ba-47b4-bb0d-bedca185a90f None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:28.198231 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 352/666] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:38:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:38:28.198231 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec9a4ae-95a1-4047-a033-3482368780c4 None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:28.225526 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bbb42b-4cb7-4298-8cb1-84945e349880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:28.226371 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bbb42b-4cb7-4298-8cb1-84945e349880 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:28.226664 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bbb42b-4cb7-4298-8cb1-84945e349880 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:28.226890 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bbb42b-4cb7-4298-8cb1-84945e349880 None admin] PATH_INFO: `/v3/projects` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:28.227642 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bbb42b-4cb7-4298-8cb1-84945e349880 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:28.227955 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bbb42b-4cb7-4298-8cb1-84945e349880 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:28.229402 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bbb42b-4cb7-4298-8cb1-84945e349880 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:28.241113 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 315/667] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:38:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:28.242492 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bbb42b-4cb7-4298-8cb1-84945e349880 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:28.260034 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:28.312651 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec9a4ae-95a1-4047-a033-3482368780c4 None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:28.351216 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389a1eab-af29-4fa7-81ae-0d4ecbea04ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:28.351931 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389a1eab-af29-4fa7-81ae-0d4ecbea04ba None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:28.352221 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389a1eab-af29-4fa7-81ae-0d4ecbea04ba None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:28.352502 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389a1eab-af29-4fa7-81ae-0d4ecbea04ba None admin] PATH_INFO: `/v3/domains` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:28.353230 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389a1eab-af29-4fa7-81ae-0d4ecbea04ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:28.353539 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389a1eab-af29-4fa7-81ae-0d4ecbea04ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:28.354966 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389a1eab-af29-4fa7-81ae-0d4ecbea04ba None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:28.359617 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bbb42b-4cb7-4298-8cb1-84945e349880 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:28.367449 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 353/668] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:38:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:28.368285 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389a1eab-af29-4fa7-81ae-0d4ecbea04ba None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:28.383520 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:28.403880 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b210bbc-eefb-445d-9117-5ed2a7c83933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:28.404533 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b210bbc-eefb-445d-9117-5ed2a7c83933 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:28.405075 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b210bbc-eefb-445d-9117-5ed2a7c83933 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:28.405361 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b210bbc-eefb-445d-9117-5ed2a7c83933 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:28.477352 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389a1eab-af29-4fa7-81ae-0d4ecbea04ba None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:28.478756 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b210bbc-eefb-445d-9117-5ed2a7c83933 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:28.480227 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b210bbc-eefb-445d-9117-5ed2a7c83933 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:28.495450 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:28.513605 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f68ec89-4a10-4f6d-aa00-28ee72c578d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:28.514448 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f68ec89-4a10-4f6d-aa00-28ee72c578d8 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:28.514749 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f68ec89-4a10-4f6d-aa00-28ee72c578d8 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:28.515011 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f68ec89-4a10-4f6d-aa00-28ee72c578d8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:28.526006 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f68ec89-4a10-4f6d-aa00-28ee72c578d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:38:28.533913 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-9f68ec89-4a10-4f6d-aa00-28ee72c578d8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:38:28.533913 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:28.533913 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:28.533913 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:28.533913 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:28.533913 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:28.533913 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:28.533913 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:28.533913 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:28.533913 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:28.533913 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:28.533913 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:28.533913 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:38:28.533913 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:38:28.533913 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:38:28.533913 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:38:28.534963 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:38:28.534963 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:38:28.534963 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:38:28.534963 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:38:28.534963 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:28.534963 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:28.534963 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:28.534963 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:28.534963 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:28.534963 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:28.534963 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:28.534963 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:38:28.534963 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:28.534963 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:28.534963 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:28.534963 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:28.535884 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:28.535884 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:28.535884 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:28.535884 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:28.535884 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:38:28.535884 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:38:28.535884 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:38:28.535884 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:38:28.535884 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:38:28.535884 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:38:28.535884 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:38:28.535884 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:38:28.535884 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 354/669] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 14:38:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:28.536653 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f68ec89-4a10-4f6d-aa00-28ee72c578d8 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:28.593656 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 316/670] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:38:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:38:28.594855 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b210bbc-eefb-445d-9117-5ed2a7c83933 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:28.622903 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:28.658151 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f68ec89-4a10-4f6d-aa00-28ee72c578d8 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:28.694575 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0179acc6-dd68-4405-aa45-fdb508165235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:28.695483 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0179acc6-dd68-4405-aa45-fdb508165235 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:28.695925 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0179acc6-dd68-4405-aa45-fdb508165235 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:28.696461 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0179acc6-dd68-4405-aa45-fdb508165235 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:28.697606 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0179acc6-dd68-4405-aa45-fdb508165235 None admin] RBAC: Adding query filter params (role.id=ffb8da90af53491980816087ee621ccb, scope.project.id=42f35b96b9d4499abfb5e5f7cca49adc, user.id=ca35fa8e73ce489081eb7ea2ca5c5e09) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:28.698105 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0179acc6-dd68-4405-aa45-fdb508165235 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:28.699694 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0179acc6-dd68-4405-aa45-fdb508165235 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:28.699868 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b210bbc-eefb-445d-9117-5ed2a7c83933 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:28.713882 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 355/671] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:38:28 2022] GET /identity/v3/role_assignments?user.id=ca35fa8e73ce489081eb7ea2ca5c5e09&scope.project.id=42f35b96b9d4499abfb5e5f7cca49adc&role.id=ffb8da90af53491980816087ee621ccb => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:28.713882 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0179acc6-dd68-4405-aa45-fdb508165235 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:28.718870 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:28.753580 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a1624f-747d-4093-9e55-b102ffe2c263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:28.754340 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a1624f-747d-4093-9e55-b102ffe2c263 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:28.754622 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a1624f-747d-4093-9e55-b102ffe2c263 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:28.754897 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a1624f-747d-4093-9e55-b102ffe2c263 None admin] PATH_INFO: `/v3/projects/service` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:28.765712 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a1624f-747d-4093-9e55-b102ffe2c263 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:28.768066 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a1624f-747d-4093-9e55-b102ffe2c263 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:28.790741 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-38a1624f-747d-4093-9e55-b102ffe2c263 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:38:28.790741 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:28.790741 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:28.790741 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:28.790741 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:28.790741 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:28.790741 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:28.790741 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:28.790741 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:28.790741 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:28.790741 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:28.790741 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:28.790741 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:38:28.790741 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:38:28.790741 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:38:28.791833 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:38:28.791833 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:28.791833 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:28.791833 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:28.791833 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:28.791833 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:28.791833 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:28.791833 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:28.791833 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:38:28.791833 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:28.791833 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:28.791833 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:28.791833 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:28.791833 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:28.791833 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:28.791833 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:28.791833 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:28.792795 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:38:28.792795 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:38:28.792795 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:38:28.792795 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:38:28.792795 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:38:28.792795 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:38:28.792795 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:38:28.792795 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:38:28.794005 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 317/672] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:38:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:28.797865 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a1624f-747d-4093-9e55-b102ffe2c263 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:28.854297 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0179acc6-dd68-4405-aa45-fdb508165235 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:28.908237 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62df9b0a-581a-48c6-9533-84a8ce6aa3bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:28.909266 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62df9b0a-581a-48c6-9533-84a8ce6aa3bf None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:28.909759 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62df9b0a-581a-48c6-9533-84a8ce6aa3bf None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:28.910259 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62df9b0a-581a-48c6-9533-84a8ce6aa3bf None admin] PATH_INFO: `/v3/roles` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:28.911211 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62df9b0a-581a-48c6-9533-84a8ce6aa3bf None admin] RBAC: Adding query filter params (name=service) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:28.911706 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62df9b0a-581a-48c6-9533-84a8ce6aa3bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:28.913330 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62df9b0a-581a-48c6-9533-84a8ce6aa3bf None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:28.917253 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:28.926392 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 356/673] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 19 14:38:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:28.926848 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a1624f-747d-4093-9e55-b102ffe2c263 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:28.927454 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62df9b0a-581a-48c6-9533-84a8ce6aa3bf None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:28.986246 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a757780-6f4c-4448-8422-924b82e47c9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:28.987059 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a757780-6f4c-4448-8422-924b82e47c9f None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:28.989077 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a757780-6f4c-4448-8422-924b82e47c9f None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:28.989077 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a757780-6f4c-4448-8422-924b82e47c9f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:29.002295 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a757780-6f4c-4448-8422-924b82e47c9f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:29.004215 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a757780-6f4c-4448-8422-924b82e47c9f None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:29.026871 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-6a757780-6f4c-4448-8422-924b82e47c9f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:29.026871 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:29.026871 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:29.026871 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:29.026871 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:29.026871 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:29.026871 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:29.026871 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:29.026871 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:29.026871 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:29.026871 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:29.026871 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:29.026871 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:38:29.026871 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:38:29.026871 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:38:29.027898 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:38:29.027898 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:29.027898 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:29.027898 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:29.027898 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:29.027898 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:29.027898 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:29.027898 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:29.027898 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:38:29.027898 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:29.027898 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:29.027898 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:29.027898 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:29.027898 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:29.027898 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:29.027898 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:29.027898 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:29.028878 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:38:29.028878 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:38:29.028878 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:29.028878 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:38:29.030974 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 318/674] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:38:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:29.032012 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a757780-6f4c-4448-8422-924b82e47c9f None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:29.059864 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62df9b0a-581a-48c6-9533-84a8ce6aa3bf None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:29.068085 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:29.107015 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6418ffc-5605-4787-8f7c-95348b47382c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:29.108598 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6418ffc-5605-4787-8f7c-95348b47382c None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:29.109177 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6418ffc-5605-4787-8f7c-95348b47382c None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:29.109643 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6418ffc-5605-4787-8f7c-95348b47382c None admin] PATH_INFO: `/v3/projects` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:29.110920 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6418ffc-5605-4787-8f7c-95348b47382c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:29.111872 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6418ffc-5605-4787-8f7c-95348b47382c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:29.114078 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6418ffc-5605-4787-8f7c-95348b47382c None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:29.131374 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 357/675] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:38:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:29.133377 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6418ffc-5605-4787-8f7c-95348b47382c None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:29.161544 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a757780-6f4c-4448-8422-924b82e47c9f None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:29.205274 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:29.208215 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e5db0b-4b55-42fe-8e05-6365b2b42a90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:29.209378 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e5db0b-4b55-42fe-8e05-6365b2b42a90 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:29.210875 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e5db0b-4b55-42fe-8e05-6365b2b42a90 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:29.211012 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e5db0b-4b55-42fe-8e05-6365b2b42a90 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:29.222165 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e5db0b-4b55-42fe-8e05-6365b2b42a90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:29.223644 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e5db0b-4b55-42fe-8e05-6365b2b42a90 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:29.243027 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-99e5db0b-4b55-42fe-8e05-6365b2b42a90 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:29.243027 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:29.243027 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:29.243027 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:29.243027 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:29.243027 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:29.243027 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:29.243027 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:29.243027 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:29.243027 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:29.243027 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:29.243027 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:29.243027 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:38:29.243027 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:38:29.243027 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:38:29.244058 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:38:29.244058 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:29.244058 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:29.244058 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:29.244058 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:29.244058 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:29.244058 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:29.244058 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:29.244058 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:38:29.244058 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:29.244058 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:29.244058 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:29.244058 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:29.244058 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:29.244058 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:29.244058 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:29.244058 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:29.245059 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:38:29.245059 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:38:29.245059 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:29.245059 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:38:29.245059 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 319/676] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:38:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:29.245372 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e5db0b-4b55-42fe-8e05-6365b2b42a90 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:29.252666 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6418ffc-5605-4787-8f7c-95348b47382c None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:29.287740 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20b90b0-3d61-4ac7-8b9e-7e9eb4c85b74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:29.288358 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20b90b0-3d61-4ac7-8b9e-7e9eb4c85b74 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:29.288645 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20b90b0-3d61-4ac7-8b9e-7e9eb4c85b74 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:29.288923 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20b90b0-3d61-4ac7-8b9e-7e9eb4c85b74 None admin] PATH_INFO: `/v3/domains` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:29.289644 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20b90b0-3d61-4ac7-8b9e-7e9eb4c85b74 None admin] RBAC: Adding query filter params (name=Default) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:29.289981 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20b90b0-3d61-4ac7-8b9e-7e9eb4c85b74 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:29.291379 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20b90b0-3d61-4ac7-8b9e-7e9eb4c85b74 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:29.303858 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 358/677] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:38:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:29.305176 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f16894-e262-44a3-889c-8638ae15e609 None None] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:29.305465 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f16894-e262-44a3-889c-8638ae15e609 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:29.305742 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f16894-e262-44a3-889c-8638ae15e609 None None] PATH_INFO: `/` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:29.309968 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 359/678] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:38:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:38:29.309968 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f16894-e262-44a3-889c-8638ae15e609 None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:29.342497 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:29.369351 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e5db0b-4b55-42fe-8e05-6365b2b42a90 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:29.408858 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e43a92a-a530-40c5-af9f-8a23b4dcd261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:29.409536 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e43a92a-a530-40c5-af9f-8a23b4dcd261 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:29.409813 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e43a92a-a530-40c5-af9f-8a23b4dcd261 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:29.410139 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e43a92a-a530-40c5-af9f-8a23b4dcd261 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:29.410984 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e43a92a-a530-40c5-af9f-8a23b4dcd261 None admin] RBAC: Adding query filter params (role.id=ffb8da90af53491980816087ee621ccb, scope.project.id=42f35b96b9d4499abfb5e5f7cca49adc, user.id=aa7df5964bed4638a9bf19d37971ec20) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:29.411303 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e43a92a-a530-40c5-af9f-8a23b4dcd261 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:29.412760 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e43a92a-a530-40c5-af9f-8a23b4dcd261 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:29.429884 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 320/679] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:38:29 2022] GET /identity/v3/role_assignments?user.id=aa7df5964bed4638a9bf19d37971ec20&scope.project.id=42f35b96b9d4499abfb5e5f7cca49adc&role.id=ffb8da90af53491980816087ee621ccb => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:29.429884 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e43a92a-a530-40c5-af9f-8a23b4dcd261 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:29.442130 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f16894-e262-44a3-889c-8638ae15e609 None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:29.469419 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:29.497511 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19340a2e-d440-42cc-8c3d-c8e6d6850944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:29.498479 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19340a2e-d440-42cc-8c3d-c8e6d6850944 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:29.498765 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19340a2e-d440-42cc-8c3d-c8e6d6850944 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:29.499041 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19340a2e-d440-42cc-8c3d-c8e6d6850944 None admin] PATH_INFO: `/v3/domains` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:29.499776 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19340a2e-d440-42cc-8c3d-c8e6d6850944 None admin] RBAC: Adding query filter params (name=Default) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:29.500083 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19340a2e-d440-42cc-8c3d-c8e6d6850944 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:29.502046 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19340a2e-d440-42cc-8c3d-c8e6d6850944 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:29.516510 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 360/680] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:38:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:29.518023 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a826d8af-a280-46c2-91e2-30cefc9e981c None None] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:29.518501 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a826d8af-a280-46c2-91e2-30cefc9e981c None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:29.518792 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a826d8af-a280-46c2-91e2-30cefc9e981c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:29.537459 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.auth.core [None req-a826d8af-a280-46c2-91e2-30cefc9e981c None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:38:29.539184 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.fernet_utils [None req-a826d8af-a280-46c2-91e2-30cefc9e981c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:38:29.549071 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e43a92a-a530-40c5-af9f-8a23b4dcd261 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:29.601124 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455ca958-02ec-40fa-9944-2cc4006eafb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:29.602752 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455ca958-02ec-40fa-9944-2cc4006eafb5 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:29.603037 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455ca958-02ec-40fa-9944-2cc4006eafb5 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:29.603599 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455ca958-02ec-40fa-9944-2cc4006eafb5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:29.614413 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455ca958-02ec-40fa-9944-2cc4006eafb5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:29.617316 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455ca958-02ec-40fa-9944-2cc4006eafb5 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:29.618346 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 361/681] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:38:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:38:29.619878 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a826d8af-a280-46c2-91e2-30cefc9e981c None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:29.627969 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:29.645298 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-455ca958-02ec-40fa-9944-2cc4006eafb5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:38:29.645298 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:29.645298 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:29.645298 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:29.645298 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:29.645298 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:29.645298 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:29.645298 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:29.645298 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:29.645298 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:29.645298 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:29.645298 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:29.645298 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:38:29.645298 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:38:29.645298 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:38:29.646409 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:38:29.646409 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:29.646409 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:29.646409 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:29.646409 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:29.646409 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:29.646409 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:29.646409 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:29.646409 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:38:29.646409 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:29.646409 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:29.646409 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:29.646409 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:29.646409 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:29.646409 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:29.646409 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:29.646409 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:29.647403 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:38:29.647403 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:38:29.647403 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:38:29.647403 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:38:29.647403 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:38:29.647403 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:38:29.647403 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:38:29.647403 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:38:29.647904 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 321/682] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:38:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:29.649343 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455ca958-02ec-40fa-9944-2cc4006eafb5 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:29.756142 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a826d8af-a280-46c2-91e2-30cefc9e981c None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:29.766522 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:29.782661 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455ca958-02ec-40fa-9944-2cc4006eafb5 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:29.795794 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a601f554-7459-4e74-afb1-a2193ec90e40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:29.796529 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a601f554-7459-4e74-afb1-a2193ec90e40 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:29.796812 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a601f554-7459-4e74-afb1-a2193ec90e40 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:29.797085 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a601f554-7459-4e74-afb1-a2193ec90e40 None admin] PATH_INFO: `/v3/users/swift` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:29.806646 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a601f554-7459-4e74-afb1-a2193ec90e40 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:29.808140 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a601f554-7459-4e74-afb1-a2193ec90e40 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:29.824671 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-a601f554-7459-4e74-afb1-a2193ec90e40 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 14:38:29.824671 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:29.824671 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:29.824671 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:29.824671 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:29.824671 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:29.824671 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:29.824671 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:29.824671 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:29.824671 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:29.824671 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:29.824671 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:29.824671 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:38:29.824671 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:38:29.824671 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:38:29.824671 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:38:29.825769 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:29.825769 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:29.825769 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:38:29.825769 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:38:29.825769 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:38:29.825769 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:38:29.825769 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:29.825769 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:29.825769 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:29.825769 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:29.825769 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:29.825769 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:38:29.825769 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:29.825769 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:29.825769 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:29.825769 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:29.825769 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:29.825769 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:29.826854 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:29.826854 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:29.826854 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:38:29.826854 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:38:29.826854 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:38:29.826854 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:38:29.826854 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:38:29.826854 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:38:29.826854 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 14:38:29.826854 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:38:29.826854 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 362/683] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:38:29 2022] GET /identity/v3/users/swift => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:29.827507 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a601f554-7459-4e74-afb1-a2193ec90e40 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:29.949289 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a601f554-7459-4e74-afb1-a2193ec90e40 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:29.950133 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c83804-55f3-4d4a-ba4f-95470562e4aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:29.951160 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c83804-55f3-4d4a-ba4f-95470562e4aa None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:29.951672 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c83804-55f3-4d4a-ba4f-95470562e4aa None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:29.952160 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c83804-55f3-4d4a-ba4f-95470562e4aa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:29.963890 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c83804-55f3-4d4a-ba4f-95470562e4aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:38:29.971930 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-51c83804-55f3-4d4a-ba4f-95470562e4aa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:38:29.971930 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:29.971930 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:29.971930 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:29.971930 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:29.971930 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:29.971930 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:29.971930 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:29.971930 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:29.971930 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:29.971930 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:29.971930 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:29.971930 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:38:29.971930 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:38:29.971930 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:38:29.971930 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:38:29.972952 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:38:29.972952 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:38:29.972952 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:38:29.972952 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:38:29.972952 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:29.972952 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:29.972952 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:29.972952 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:29.972952 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:29.972952 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:29.972952 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:29.972952 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:38:29.972952 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:29.972952 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:29.972952 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:29.972952 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:29.974005 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:29.974005 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:29.974005 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:29.974005 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:29.974005 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:38:29.974005 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:38:29.974005 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:38:29.974005 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:38:29.974005 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:38:29.974005 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:38:29.974005 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:38:29.974005 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:38:29.977391 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 322/684] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:38:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:29.978448 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c83804-55f3-4d4a-ba4f-95470562e4aa None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:29.995390 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a442fe-088b-4943-8411-c99c768328a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:29.996558 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a442fe-088b-4943-8411-c99c768328a1 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:29.996558 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a442fe-088b-4943-8411-c99c768328a1 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:29.996746 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a442fe-088b-4943-8411-c99c768328a1 None admin] PATH_INFO: `/v3/projects` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:29.997615 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a442fe-088b-4943-8411-c99c768328a1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:29.998030 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a442fe-088b-4943-8411-c99c768328a1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:29.999698 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a442fe-088b-4943-8411-c99c768328a1 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:30.015301 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 363/685] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:38:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:30.016478 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a442fe-088b-4943-8411-c99c768328a1 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:30.049958 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:30.051162 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:30.097387 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c83804-55f3-4d4a-ba4f-95470562e4aa None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:30.135680 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a442fe-088b-4943-8411-c99c768328a1 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:30.135992 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ee7aaa-3422-4cfc-ad31-cc2b10205360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:30.136772 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ee7aaa-3422-4cfc-ad31-cc2b10205360 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:30.137157 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ee7aaa-3422-4cfc-ad31-cc2b10205360 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:30.137551 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ee7aaa-3422-4cfc-ad31-cc2b10205360 None admin] PATH_INFO: `/v3/users` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:30.138645 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ee7aaa-3422-4cfc-ad31-cc2b10205360 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:30.139078 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ee7aaa-3422-4cfc-ad31-cc2b10205360 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:30.140707 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ee7aaa-3422-4cfc-ad31-cc2b10205360 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:30.160133 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 323/686] 10.222.0.57 () {56 vars in 1135 bytes} [Thu May 19 14:38:29 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:30.161286 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ee7aaa-3422-4cfc-ad31-cc2b10205360 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:30.175544 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff528b4-4b93-441f-b849-73b47942d421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:30.176491 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff528b4-4b93-441f-b849-73b47942d421 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:30.176950 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff528b4-4b93-441f-b849-73b47942d421 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:30.177376 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff528b4-4b93-441f-b849-73b47942d421 None admin] PATH_INFO: `/v3/roles` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:30.178335 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff528b4-4b93-441f-b849-73b47942d421 None admin] RBAC: Adding query filter params (name=admin) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:30.179366 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff528b4-4b93-441f-b849-73b47942d421 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:30.181618 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff528b4-4b93-441f-b849-73b47942d421 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:30.196779 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 364/687] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:38:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:30.198055 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff528b4-4b93-441f-b849-73b47942d421 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:30.274249 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ee7aaa-3422-4cfc-ad31-cc2b10205360 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:30.293173 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:30.301010 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:30.313778 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1336d5-635d-430f-bb66-e4dc3a034389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:30.314384 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff528b4-4b93-441f-b849-73b47942d421 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:30.314709 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1336d5-635d-430f-bb66-e4dc3a034389 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:30.315102 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1336d5-635d-430f-bb66-e4dc3a034389 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:30.315473 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1336d5-635d-430f-bb66-e4dc3a034389 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:30.316511 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1336d5-635d-430f-bb66-e4dc3a034389 None admin] RBAC: Adding query filter params (role.id=ffb8da90af53491980816087ee621ccb, scope.project.id=42f35b96b9d4499abfb5e5f7cca49adc, user.id=c7cd031eea90473aaaa0f1375f04dc66) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:30.316941 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1336d5-635d-430f-bb66-e4dc3a034389 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:30.318676 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1336d5-635d-430f-bb66-e4dc3a034389 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:30.326419 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 324/688] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:38:30 2022] GET /identity/v3/role_assignments?user.id=c7cd031eea90473aaaa0f1375f04dc66&scope.project.id=42f35b96b9d4499abfb5e5f7cca49adc&role.id=ffb8da90af53491980816087ee621ccb => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:30.327370 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1336d5-635d-430f-bb66-e4dc3a034389 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:30.355801 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d828e8-1506-4062-96f7-26f761bca640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:30.356992 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d828e8-1506-4062-96f7-26f761bca640 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:30.357549 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d828e8-1506-4062-96f7-26f761bca640 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:30.358054 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d828e8-1506-4062-96f7-26f761bca640 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:30.442410 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1336d5-635d-430f-bb66-e4dc3a034389 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:30.480205 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c7b2ae-59ea-4fba-8b09-94184a78d3be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:30.481278 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c7b2ae-59ea-4fba-8b09-94184a78d3be None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:30.481667 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c7b2ae-59ea-4fba-8b09-94184a78d3be None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:30.482501 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c7b2ae-59ea-4fba-8b09-94184a78d3be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:30.484866 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d828e8-1506-4062-96f7-26f761bca640 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:30.490987 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d828e8-1506-4062-96f7-26f761bca640 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:30.505024 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c7b2ae-59ea-4fba-8b09-94184a78d3be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:30.506863 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c7b2ae-59ea-4fba-8b09-94184a78d3be None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:30.547214 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-a8c7b2ae-59ea-4fba-8b09-94184a78d3be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:30.547214 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:30.547214 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:30.547214 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:30.547214 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:30.547214 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:30.547214 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:30.547214 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:30.547214 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:30.547214 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:30.547214 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:30.547214 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:30.547214 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:38:30.547214 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:38:30.547214 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:38:30.548447 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:38:30.548447 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:30.548447 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:30.548447 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:30.548447 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:30.548447 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:30.548447 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:30.548447 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:30.548447 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:38:30.548447 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:30.548447 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:30.548447 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:30.548447 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:30.548447 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:30.548447 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:30.548447 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:30.548447 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:30.549640 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:38:30.549640 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:38:30.549640 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:30.549640 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:38:30.554403 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 325/689] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:38:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:30.561748 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c7b2ae-59ea-4fba-8b09-94184a78d3be None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:30.633709 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:30.671324 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 365/690] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:38:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:38:30.671650 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:30.678269 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d828e8-1506-4062-96f7-26f761bca640 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:30.715147 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c7b2ae-59ea-4fba-8b09-94184a78d3be None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:30.756041 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ede2844-bd88-4cd4-833e-74e3d742c0e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:30.756708 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ede2844-bd88-4cd4-833e-74e3d742c0e3 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:30.757871 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ede2844-bd88-4cd4-833e-74e3d742c0e3 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:30.758184 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ede2844-bd88-4cd4-833e-74e3d742c0e3 None admin] PATH_INFO: `/v3/domains` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:30.758927 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ede2844-bd88-4cd4-833e-74e3d742c0e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:30.759235 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ede2844-bd88-4cd4-833e-74e3d742c0e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:30.760625 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ede2844-bd88-4cd4-833e-74e3d742c0e3 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:30.775574 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 326/691] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:38:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:30.780538 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ede2844-bd88-4cd4-833e-74e3d742c0e3 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:30.822175 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d828e8-1506-4062-96f7-26f761bca640 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:30.872491 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8658b24-4960-484f-86b0-08b6b37b599d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:30.874354 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8658b24-4960-484f-86b0-08b6b37b599d None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:30.874860 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8658b24-4960-484f-86b0-08b6b37b599d None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:30.875503 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8658b24-4960-484f-86b0-08b6b37b599d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:30.895534 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8658b24-4960-484f-86b0-08b6b37b599d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:30.897283 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8658b24-4960-484f-86b0-08b6b37b599d None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:30.928321 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-a8658b24-4960-484f-86b0-08b6b37b599d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:30.928321 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:30.928321 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:30.928321 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:30.928321 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:30.928321 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:30.928321 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:30.928321 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:30.928321 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:30.928321 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:30.928321 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:30.928321 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:30.928321 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:38:30.928321 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:38:30.928321 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:38:30.930081 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:38:30.930081 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:30.930081 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:30.930081 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:30.930081 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:30.930081 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:30.930081 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:30.930081 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:30.930081 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:38:30.930081 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:30.930081 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:30.930081 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:30.930081 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:30.930081 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:30.930081 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:30.930081 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:30.930081 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:30.931174 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:38:30.931174 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:38:30.931174 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:30.931174 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:38:30.943133 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 366/692] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:38:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:30.946866 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:30.949431 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8658b24-4960-484f-86b0-08b6b37b599d None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:30.956898 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ede2844-bd88-4cd4-833e-74e3d742c0e3 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:31.029215 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123a6c43-787e-4130-930e-07dddf99935e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:31.030335 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123a6c43-787e-4130-930e-07dddf99935e None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:31.030809 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123a6c43-787e-4130-930e-07dddf99935e None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:31.036611 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123a6c43-787e-4130-930e-07dddf99935e None admin] PATH_INFO: `/v3/users/nova` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:31.048113 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123a6c43-787e-4130-930e-07dddf99935e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:31.050205 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:31.051103 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123a6c43-787e-4130-930e-07dddf99935e None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:31.073101 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-123a6c43-787e-4130-930e-07dddf99935e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 14:38:31.073101 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:31.073101 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:31.073101 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:31.073101 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:31.073101 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:31.073101 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:31.073101 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:31.073101 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:31.073101 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:31.073101 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:31.073101 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:31.073101 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:38:31.073101 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:38:31.073101 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:38:31.073101 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:38:31.074234 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:31.074234 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:31.074234 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:38:31.074234 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:38:31.074234 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:38:31.074234 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:38:31.074234 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:31.074234 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:31.074234 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:31.074234 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:31.074234 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:31.074234 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:38:31.074234 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:31.074234 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:31.074234 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:31.074234 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:31.074234 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:31.074234 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:31.075265 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:31.075265 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:31.075265 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:38:31.075265 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:38:31.075265 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:38:31.075265 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:38:31.075265 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:38:31.075265 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:38:31.075265 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 14:38:31.075265 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:38:31.075920 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 327/693] 10.222.0.57 () {56 vars in 1093 bytes} [Thu May 19 14:38:30 2022] GET /identity/v3/users/nova => generated 82 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:31.085783 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123a6c43-787e-4130-930e-07dddf99935e None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:31.152486 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8658b24-4960-484f-86b0-08b6b37b599d None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:31.196154 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13aac19d-ee19-4960-9f84-6c0e0e97175c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:31.196840 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aac19d-ee19-4960-9f84-6c0e0e97175c None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:31.197129 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aac19d-ee19-4960-9f84-6c0e0e97175c None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:31.197408 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aac19d-ee19-4960-9f84-6c0e0e97175c None admin] PATH_INFO: `/v3/domains` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:31.198194 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13aac19d-ee19-4960-9f84-6c0e0e97175c None admin] RBAC: Adding query filter params (name=Default) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:31.198507 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13aac19d-ee19-4960-9f84-6c0e0e97175c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:31.200568 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13aac19d-ee19-4960-9f84-6c0e0e97175c None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:31.215887 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 367/694] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:38:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:31.226868 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13aac19d-ee19-4960-9f84-6c0e0e97175c None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:31.289941 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123a6c43-787e-4130-930e-07dddf99935e None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:31.328052 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3329207-b8ce-4f4c-a57a-ab9dae40c02d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:31.328679 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3329207-b8ce-4f4c-a57a-ab9dae40c02d None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:31.328966 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3329207-b8ce-4f4c-a57a-ab9dae40c02d None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:31.329247 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3329207-b8ce-4f4c-a57a-ab9dae40c02d None admin] PATH_INFO: `/v3/users` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:31.330138 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3329207-b8ce-4f4c-a57a-ab9dae40c02d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:31.330455 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3329207-b8ce-4f4c-a57a-ab9dae40c02d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:31.331897 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3329207-b8ce-4f4c-a57a-ab9dae40c02d None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:31.351914 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 328/695] 10.222.0.57 () {56 vars in 1133 bytes} [Thu May 19 14:38:31 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:31.353420 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b364f50-55bc-40f7-a264-6b0d6b383f0b None None] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:31.353721 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b364f50-55bc-40f7-a264-6b0d6b383f0b None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:31.354044 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b364f50-55bc-40f7-a264-6b0d6b383f0b None None] PATH_INFO: `/` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:31.359516 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 329/696] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:38:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:38:31.360317 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b364f50-55bc-40f7-a264-6b0d6b383f0b None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:31.368007 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13aac19d-ee19-4960-9f84-6c0e0e97175c None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:31.370369 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:31.376640 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:31.420038 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184e6f97-9e26-4988-9bb4-57f57e5925a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:31.421096 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184e6f97-9e26-4988-9bb4-57f57e5925a0 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:31.421481 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184e6f97-9e26-4988-9bb4-57f57e5925a0 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:31.421966 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184e6f97-9e26-4988-9bb4-57f57e5925a0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:31.433599 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184e6f97-9e26-4988-9bb4-57f57e5925a0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:31.435243 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184e6f97-9e26-4988-9bb4-57f57e5925a0 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:31.456922 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-184e6f97-9e26-4988-9bb4-57f57e5925a0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:38:31.456922 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:31.456922 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:31.456922 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:31.456922 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:31.456922 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:31.456922 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:31.456922 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:31.456922 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:31.456922 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:31.456922 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:31.456922 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:31.456922 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:38:31.456922 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:38:31.456922 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:38:31.458192 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:38:31.458192 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:31.458192 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:31.458192 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:31.458192 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:31.458192 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:31.458192 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:31.458192 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:31.458192 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:38:31.458192 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:31.458192 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:31.458192 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:31.458192 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:31.458192 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:31.458192 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:31.458192 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:31.458192 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:31.459371 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:38:31.459371 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:38:31.459371 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:38:31.459371 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:38:31.459371 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:38:31.459371 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:38:31.459371 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:38:31.459371 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:38:31.463752 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 368/697] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:38:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:31.463883 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268d0854-9e47-4d6b-961e-96de5f91b9f8 None None] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:31.464092 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268d0854-9e47-4d6b-961e-96de5f91b9f8 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:31.464379 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268d0854-9e47-4d6b-961e-96de5f91b9f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:31.480113 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b364f50-55bc-40f7-a264-6b0d6b383f0b None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:31.489238 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.auth.core [None req-268d0854-9e47-4d6b-961e-96de5f91b9f8 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:38:31.491163 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.fernet_utils [None req-268d0854-9e47-4d6b-961e-96de5f91b9f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:38:31.517297 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c49902-3000-44c2-9a0b-c34fad2f9310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:31.518030 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c49902-3000-44c2-9a0b-c34fad2f9310 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:31.518319 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c49902-3000-44c2-9a0b-c34fad2f9310 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:31.518599 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c49902-3000-44c2-9a0b-c34fad2f9310 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:31.589514 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 369/698] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:38:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:38:31.590497 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268d0854-9e47-4d6b-961e-96de5f91b9f8 None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:31.608655 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c49902-3000-44c2-9a0b-c34fad2f9310 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:31.610254 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c49902-3000-44c2-9a0b-c34fad2f9310 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:31.635525 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:31.722363 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268d0854-9e47-4d6b-961e-96de5f91b9f8 None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:31.726165 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:31.743257 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 330/699] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:38:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:38:31.744291 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c49902-3000-44c2-9a0b-c34fad2f9310 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:31.764634 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2202a0-69ee-44dd-8056-028fd081e864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:31.765655 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2202a0-69ee-44dd-8056-028fd081e864 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:31.766052 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2202a0-69ee-44dd-8056-028fd081e864 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:31.766763 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2202a0-69ee-44dd-8056-028fd081e864 None admin] PATH_INFO: `/v3/projects` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:31.767626 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2202a0-69ee-44dd-8056-028fd081e864 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:31.768014 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2202a0-69ee-44dd-8056-028fd081e864 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:31.770099 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2202a0-69ee-44dd-8056-028fd081e864 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:31.793422 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 370/700] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:38:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:31.794589 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2202a0-69ee-44dd-8056-028fd081e864 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:31.867073 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c49902-3000-44c2-9a0b-c34fad2f9310 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:31.882566 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:31.909678 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b9c533-0a9a-4b97-84fc-7bf659fc352d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:31.910602 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b9c533-0a9a-4b97-84fc-7bf659fc352d None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:31.910959 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b9c533-0a9a-4b97-84fc-7bf659fc352d None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:31.911308 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b9c533-0a9a-4b97-84fc-7bf659fc352d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:31.923115 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b9c533-0a9a-4b97-84fc-7bf659fc352d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:31.924687 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b9c533-0a9a-4b97-84fc-7bf659fc352d None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:31.941348 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2202a0-69ee-44dd-8056-028fd081e864 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:31.943956 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-43b9c533-0a9a-4b97-84fc-7bf659fc352d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:31.943956 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:31.943956 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:31.943956 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:31.943956 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:31.943956 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:31.943956 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:31.943956 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:31.943956 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:31.943956 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:31.943956 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:31.943956 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:31.943956 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:38:31.943956 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:38:31.943956 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:38:31.945982 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:38:31.945982 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:31.945982 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:31.945982 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:31.945982 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:31.945982 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:31.945982 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:31.945982 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:31.945982 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:38:31.945982 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:31.945982 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:31.945982 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:31.945982 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:31.945982 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:31.945982 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:31.945982 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:31.945982 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:31.947183 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:38:31.947183 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:38:31.947183 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:31.947183 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:38:31.949962 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 331/701] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:38:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:31.949962 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b9c533-0a9a-4b97-84fc-7bf659fc352d None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:31.986977 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f326524-f08c-4183-8443-91e178b2f1d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:31.988214 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f326524-f08c-4183-8443-91e178b2f1d8 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:31.988677 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f326524-f08c-4183-8443-91e178b2f1d8 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:31.988959 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f326524-f08c-4183-8443-91e178b2f1d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:32.002442 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f326524-f08c-4183-8443-91e178b2f1d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:32.003963 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f326524-f08c-4183-8443-91e178b2f1d8 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:32.022877 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-1f326524-f08c-4183-8443-91e178b2f1d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:32.022877 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:32.022877 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:32.022877 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:32.022877 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:32.022877 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:32.022877 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:32.022877 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:32.022877 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:32.022877 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:32.022877 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:32.022877 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:32.022877 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:38:32.022877 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:38:32.022877 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:38:32.025539 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:38:32.025539 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:32.025539 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:32.025539 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:32.025539 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:32.025539 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:32.025539 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:32.025539 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:32.025539 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:38:32.025539 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:32.025539 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:32.025539 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:32.025539 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:32.025539 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:32.025539 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:32.025539 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:32.025539 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:32.027641 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:38:32.027641 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:38:32.027641 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:32.027641 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:38:32.027641 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 371/702] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:38:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:32.027641 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f326524-f08c-4183-8443-91e178b2f1d8 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:32.033483 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:32.076341 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b9c533-0a9a-4b97-84fc-7bf659fc352d None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:32.114002 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f483c58c-20f2-4d9e-8b57-37171f975fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:32.114663 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f483c58c-20f2-4d9e-8b57-37171f975fe2 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:32.114942 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f483c58c-20f2-4d9e-8b57-37171f975fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:32.115217 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f483c58c-20f2-4d9e-8b57-37171f975fe2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:32.116053 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f483c58c-20f2-4d9e-8b57-37171f975fe2 None admin] RBAC: Adding query filter params (role.id=ffb8da90af53491980816087ee621ccb, scope.project.id=42f35b96b9d4499abfb5e5f7cca49adc, user.id=4f8e2e0909074a85bb493c49ac3a6d4d) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:32.116458 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f483c58c-20f2-4d9e-8b57-37171f975fe2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:32.117992 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f483c58c-20f2-4d9e-8b57-37171f975fe2 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:32.134243 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 332/703] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:38:31 2022] GET /identity/v3/role_assignments?user.id=4f8e2e0909074a85bb493c49ac3a6d4d&scope.project.id=42f35b96b9d4499abfb5e5f7cca49adc&role.id=ffb8da90af53491980816087ee621ccb => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:32.135042 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f483c58c-20f2-4d9e-8b57-37171f975fe2 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:32.146975 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f326524-f08c-4183-8443-91e178b2f1d8 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:32.183725 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:32.196621 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38491df4-1501-417d-9c94-2d003075dcb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:32.198379 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38491df4-1501-417d-9c94-2d003075dcb8 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:32.198379 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38491df4-1501-417d-9c94-2d003075dcb8 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:32.198379 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38491df4-1501-417d-9c94-2d003075dcb8 None admin] PATH_INFO: `/v3/domains` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:32.199179 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38491df4-1501-417d-9c94-2d003075dcb8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:32.199680 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38491df4-1501-417d-9c94-2d003075dcb8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:32.201383 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38491df4-1501-417d-9c94-2d003075dcb8 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:32.222753 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 372/704] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:38:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:32.222753 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c5a78c-7d53-4338-b23d-13bc09dc3a0f None None] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:32.222753 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c5a78c-7d53-4338-b23d-13bc09dc3a0f None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:32.222753 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c5a78c-7d53-4338-b23d-13bc09dc3a0f None None] PATH_INFO: `/` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:32.227207 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 373/705] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:38:32 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:38:32.228241 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c5a78c-7d53-4338-b23d-13bc09dc3a0f None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:32.304663 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f483c58c-20f2-4d9e-8b57-37171f975fe2 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:32.308468 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:32.357797 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa28455d-af2c-4da9-9bc4-2e823a0001de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:32.358756 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa28455d-af2c-4da9-9bc4-2e823a0001de None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:32.359200 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa28455d-af2c-4da9-9bc4-2e823a0001de None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:32.359345 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c5a78c-7d53-4338-b23d-13bc09dc3a0f None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:32.359787 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa28455d-af2c-4da9-9bc4-2e823a0001de None admin] PATH_INFO: `/v3/domains` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:32.360689 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa28455d-af2c-4da9-9bc4-2e823a0001de None admin] RBAC: Adding query filter params (name=Default) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:32.361321 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa28455d-af2c-4da9-9bc4-2e823a0001de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:32.363622 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa28455d-af2c-4da9-9bc4-2e823a0001de None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:32.379462 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 333/706] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:38:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:32.381589 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7498923-c05f-422f-a0bf-bf2ee7a39c88 None None] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:32.382765 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7498923-c05f-422f-a0bf-bf2ee7a39c88 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:32.383217 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7498923-c05f-422f-a0bf-bf2ee7a39c88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:32.401606 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.auth.core [None req-b7498923-c05f-422f-a0bf-bf2ee7a39c88 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:38:32.402897 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-b7498923-c05f-422f-a0bf-bf2ee7a39c88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:38:32.410056 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b175a836-2f7f-40cc-9207-3fbcf16ba463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:32.411496 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b175a836-2f7f-40cc-9207-3fbcf16ba463 None admin] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:32.412000 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b175a836-2f7f-40cc-9207-3fbcf16ba463 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:32.412487 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b175a836-2f7f-40cc-9207-3fbcf16ba463 None admin] PATH_INFO: `/v3/users` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:32.414324 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b175a836-2f7f-40cc-9207-3fbcf16ba463 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:32.416671 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b175a836-2f7f-40cc-9207-3fbcf16ba463 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:32.445792 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 374/707] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 19 14:38:32 2022] POST /identity/v3/users => generated 254 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:38:32.447165 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b175a836-2f7f-40cc-9207-3fbcf16ba463 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:32.487197 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 334/708] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:38:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:38:32.512804 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7498923-c05f-422f-a0bf-bf2ee7a39c88 None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:32.534662 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:32.575777 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b175a836-2f7f-40cc-9207-3fbcf16ba463 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:32.600209 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:32.617960 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1336117-0bd3-42da-9936-b9547cbaebee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:32.619384 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1336117-0bd3-42da-9936-b9547cbaebee None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:32.619890 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1336117-0bd3-42da-9936-b9547cbaebee None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:32.620425 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1336117-0bd3-42da-9936-b9547cbaebee None admin] PATH_INFO: `/v3/projects/service` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:32.642737 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1336117-0bd3-42da-9936-b9547cbaebee None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:32.644747 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1336117-0bd3-42da-9936-b9547cbaebee None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:32.670309 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-a1336117-0bd3-42da-9936-b9547cbaebee None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:38:32.670309 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:32.670309 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:32.670309 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:32.670309 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:32.670309 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:32.670309 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:32.670309 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:32.670309 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:32.670309 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:32.670309 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:32.670309 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:32.670309 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:38:32.670309 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:38:32.670309 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:38:32.671372 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:38:32.671372 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:32.671372 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:32.671372 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:32.671372 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:32.671372 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:32.671372 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:32.671372 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:32.671372 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:38:32.671372 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:32.671372 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:32.671372 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:32.671372 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:32.671372 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:32.671372 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:32.671372 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:32.671372 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:32.672350 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:38:32.672350 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:38:32.672350 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:38:32.672350 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:38:32.672350 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:38:32.672350 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:38:32.672350 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:38:32.672350 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:38:32.675097 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 375/709] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:38:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:32.682980 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1336117-0bd3-42da-9936-b9547cbaebee None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:32.688783 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7498923-c05f-422f-a0bf-bf2ee7a39c88 None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:32.736147 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab12846d-2496-446c-a04d-cb1acbc5834c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:32.737046 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab12846d-2496-446c-a04d-cb1acbc5834c None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:32.737376 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab12846d-2496-446c-a04d-cb1acbc5834c None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:32.737649 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab12846d-2496-446c-a04d-cb1acbc5834c None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:32.738346 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab12846d-2496-446c-a04d-cb1acbc5834c None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:32.739723 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab12846d-2496-446c-a04d-cb1acbc5834c None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:32.754987 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-ab12846d-2496-446c-a04d-cb1acbc5834c None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 14:38:32.754987 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:32.754987 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:38:32.754987 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:38:32.754987 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:38:32.754987 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:38:32.754987 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:38:32.754987 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:38:32.754987 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 14:38:32.754987 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:38:32.754987 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:38:32.754987 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:38:32.754987 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:32.754987 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:32.754987 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:32.754987 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:32.756111 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:32.756111 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:32.756111 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:32.756111 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:32.756111 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:32.756111 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:32.756111 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:32.756111 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:38:32.756111 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:38:32.756111 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:38:32.756111 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:38:32.756111 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:32.756111 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:32.756111 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:32.756111 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:32.756111 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:32.757677 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:32.757677 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:32.757677 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:38:32.757677 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:32.757677 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:32.757677 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:32.757677 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:32.757677 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:32.757677 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:32.757677 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:32.757677 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:32.757677 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:38:32.757677 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:38:32.757677 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 14:38:32.757677 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:38:32.757677 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 335/710] 10.222.0.57 () {56 vars in 1129 bytes} [Thu May 19 14:38:32 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:38:32.763145 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab12846d-2496-446c-a04d-cb1acbc5834c None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:32.798931 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1336117-0bd3-42da-9936-b9547cbaebee None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:32.841628 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20b5dc3-581e-429c-89e8-27e01d141d20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:32.842782 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20b5dc3-581e-429c-89e8-27e01d141d20 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:32.843297 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20b5dc3-581e-429c-89e8-27e01d141d20 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:32.843786 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20b5dc3-581e-429c-89e8-27e01d141d20 None admin] PATH_INFO: `/v3/projects` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:32.844831 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20b5dc3-581e-429c-89e8-27e01d141d20 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:32.845382 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20b5dc3-581e-429c-89e8-27e01d141d20 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:32.847168 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20b5dc3-581e-429c-89e8-27e01d141d20 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:32.861674 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 376/711] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:38:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:38:32.863206 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:32.870948 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20b5dc3-581e-429c-89e8-27e01d141d20 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:32.892014 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab12846d-2496-446c-a04d-cb1acbc5834c None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:32.932539 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:32.936834 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e8e2ed-ac20-4dbd-8081-5430d29ac0be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:32.937731 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e8e2ed-ac20-4dbd-8081-5430d29ac0be None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:32.938162 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e8e2ed-ac20-4dbd-8081-5430d29ac0be None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:32.938528 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e8e2ed-ac20-4dbd-8081-5430d29ac0be None admin] PATH_INFO: `/v3/services` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:32.939321 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e8e2ed-ac20-4dbd-8081-5430d29ac0be None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:32.939697 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e8e2ed-ac20-4dbd-8081-5430d29ac0be None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:32.941147 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e8e2ed-ac20-4dbd-8081-5430d29ac0be None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:32.947355 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 336/712] 10.222.0.57 () {56 vars in 1124 bytes} [Thu May 19 14:38:32 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:32.952845 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e8e2ed-ac20-4dbd-8081-5430d29ac0be None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:33.017319 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20b5dc3-581e-429c-89e8-27e01d141d20 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:33.055443 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85853528-fe4d-4394-902d-a1dcf5363aa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:33.056224 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85853528-fe4d-4394-902d-a1dcf5363aa4 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:33.056504 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85853528-fe4d-4394-902d-a1dcf5363aa4 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:33.056778 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85853528-fe4d-4394-902d-a1dcf5363aa4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:33.057728 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85853528-fe4d-4394-902d-a1dcf5363aa4 None admin] RBAC: Adding query filter params (role.id=c2935b1a25ca46fcba419fe3d1715330, scope.project.id=42f35b96b9d4499abfb5e5f7cca49adc, user.id=a9591727b2f54244baf90b11a8e2277f) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:33.058092 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85853528-fe4d-4394-902d-a1dcf5363aa4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:33.059521 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85853528-fe4d-4394-902d-a1dcf5363aa4 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:33.069867 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 377/713] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:38:32 2022] GET /identity/v3/role_assignments?user.id=a9591727b2f54244baf90b11a8e2277f&scope.project.id=42f35b96b9d4499abfb5e5f7cca49adc&role.id=c2935b1a25ca46fcba419fe3d1715330 => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:33.069867 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c7233f-6dd9-4638-a2bb-0ff7cf957a8b None None] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:33.069867 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c7233f-6dd9-4638-a2bb-0ff7cf957a8b None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:33.069867 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c7233f-6dd9-4638-a2bb-0ff7cf957a8b None None] PATH_INFO: `/` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:33.070486 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 378/714] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:38:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:38:33.075725 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae35d622-6dd0-483f-829c-278b529a018b None None] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:33.076016 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae35d622-6dd0-483f-829c-278b529a018b None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:33.076299 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae35d622-6dd0-483f-829c-278b529a018b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:33.095031 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.auth.core [None req-ae35d622-6dd0-483f-829c-278b529a018b None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:38:33.096259 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.fernet_utils [None req-ae35d622-6dd0-483f-829c-278b529a018b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:38:33.103859 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e8e2ed-ac20-4dbd-8081-5430d29ac0be None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:33.174017 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 379/715] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:38:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:38:33.199211 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae35d622-6dd0-483f-829c-278b529a018b None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:33.200732 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a073859d-1546-4259-8bba-dbce5ab76379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:33.201799 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a073859d-1546-4259-8bba-dbce5ab76379 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:33.202428 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a073859d-1546-4259-8bba-dbce5ab76379 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:33.202937 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a073859d-1546-4259-8bba-dbce5ab76379 None admin] PATH_INFO: `/v3/services` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:33.203904 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a073859d-1546-4259-8bba-dbce5ab76379 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:33.204435 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a073859d-1546-4259-8bba-dbce5ab76379 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:33.206302 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a073859d-1546-4259-8bba-dbce5ab76379 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:33.217986 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 337/716] 10.222.0.57 () {56 vars in 1124 bytes} [Thu May 19 14:38:32 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:33.239228 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:33.321253 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae35d622-6dd0-483f-829c-278b529a018b None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:33.488690 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2331c229-7f4d-49b4-bc7c-3051eeba71ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:33.489422 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2331c229-7f4d-49b4-bc7c-3051eeba71ac None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:33.489706 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2331c229-7f4d-49b4-bc7c-3051eeba71ac None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:33.490016 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2331c229-7f4d-49b4-bc7c-3051eeba71ac None admin] PATH_INFO: `/v3/services/network` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:33.490681 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2331c229-7f4d-49b4-bc7c-3051eeba71ac None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:33.492601 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2331c229-7f4d-49b4-bc7c-3051eeba71ac None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:33.507545 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-2331c229-7f4d-49b4-bc7c-3051eeba71ac None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 19 14:38:33.507545 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:33.507545 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:38:33.507545 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:38:33.507545 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:38:33.507545 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:38:33.507545 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:38:33.507545 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:38:33.507545 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 14:38:33.507545 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:38:33.507545 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:38:33.507545 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:38:33.507545 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:33.507545 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:33.507545 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:33.507545 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:33.507545 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:33.509259 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:33.509259 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:33.509259 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:33.509259 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:33.509259 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:33.509259 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:33.509259 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:38:33.509259 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:38:33.509259 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:38:33.509259 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:38:33.509259 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:33.509259 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:33.509259 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:33.509259 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:33.509259 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:33.509259 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:33.510467 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:33.510467 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:38:33.510467 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:33.510467 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:33.510467 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:33.510467 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:33.510467 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:33.510467 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:33.510467 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:33.510467 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:33.510467 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:38:33.510467 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:38:33.510467 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 14:38:33.510467 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:38:33.510467 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 380/717] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:38:33 2022] GET /identity/v3/services/network => generated 88 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:33.514731 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a073859d-1546-4259-8bba-dbce5ab76379 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:33.542211 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:33.782673 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a073859d-1546-4259-8bba-dbce5ab76379 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:33.819726 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae4b049-0bc8-49b9-9b76-79d76927effb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:33.820420 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae4b049-0bc8-49b9-9b76-79d76927effb None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:33.820802 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae4b049-0bc8-49b9-9b76-79d76927effb None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:33.821087 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae4b049-0bc8-49b9-9b76-79d76927effb None admin] PATH_INFO: `/v3/services` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:33.821813 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae4b049-0bc8-49b9-9b76-79d76927effb None admin] RBAC: Adding query filter params (name=network) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:33.822156 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae4b049-0bc8-49b9-9b76-79d76927effb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:33.823545 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae4b049-0bc8-49b9-9b76-79d76927effb None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:33.828676 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 338/718] 10.222.0.57 () {56 vars in 1112 bytes} [Thu May 19 14:38:33 2022] GET /identity/v3/services?name=network => generated 124 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:33.835946 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2331c229-7f4d-49b4-bc7c-3051eeba71ac None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:33.986145 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:33.990121 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2331c229-7f4d-49b4-bc7c-3051eeba71ac None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:34.045655 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9744a618-e10a-43e1-8482-49696a6ae099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:34.046296 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9744a618-e10a-43e1-8482-49696a6ae099 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:34.046567 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9744a618-e10a-43e1-8482-49696a6ae099 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:34.046840 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9744a618-e10a-43e1-8482-49696a6ae099 None admin] PATH_INFO: `/v3/services` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:34.047570 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9744a618-e10a-43e1-8482-49696a6ae099 None admin] RBAC: Adding query filter params (type=network) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:34.047881 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9744a618-e10a-43e1-8482-49696a6ae099 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:34.049289 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9744a618-e10a-43e1-8482-49696a6ae099 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:34.058969 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 381/719] 10.222.0.57 () {56 vars in 1112 bytes} [Thu May 19 14:38:33 2022] GET /identity/v3/services?type=network => generated 124 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:35.350756 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fc5e27-4f72-4910-ae41-b4d20b7d84a6 None None] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:35.351540 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fc5e27-4f72-4910-ae41-b4d20b7d84a6 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:35.351540 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fc5e27-4f72-4910-ae41-b4d20b7d84a6 None None] PATH_INFO: `/` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:35.352941 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 339/720] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:38:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:38:35.363486 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707d90ae-2254-4854-b176-b234419c41c9 None None] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:35.364396 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707d90ae-2254-4854-b176-b234419c41c9 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:35.364396 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707d90ae-2254-4854-b176-b234419c41c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:35.394938 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.auth.core [None req-707d90ae-2254-4854-b176-b234419c41c9 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:38:35.396157 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.fernet_utils [None req-707d90ae-2254-4854-b176-b234419c41c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:38:35.480308 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 382/721] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:38:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:38:35.523390 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fc5e27-4f72-4910-ae41-b4d20b7d84a6 None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:35.560149 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5fd424-0776-410b-b20a-b15f8bf86d0e None None] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:35.560565 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5fd424-0776-410b-b20a-b15f8bf86d0e None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:35.560728 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5fd424-0776-410b-b20a-b15f8bf86d0e None None] PATH_INFO: `/` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:35.562357 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 383/722] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:38:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:38:35.571537 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b3c7b1-fa78-42d4-bb93-a03af1e0f428 None None] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:35.572389 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b3c7b1-fa78-42d4-bb93-a03af1e0f428 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:35.573157 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b3c7b1-fa78-42d4-bb93-a03af1e0f428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:35.607498 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.auth.core [None req-e5b3c7b1-fa78-42d4-bb93-a03af1e0f428 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:38:35.608689 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.fernet_utils [None req-e5b3c7b1-fa78-42d4-bb93-a03af1e0f428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:38:35.642609 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:35.699009 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fc5e27-4f72-4910-ae41-b4d20b7d84a6 None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:35.743017 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 384/723] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:38:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:38:35.756029 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f95557f-d0ba-49d2-be93-52eb9c094106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:35.757720 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f95557f-d0ba-49d2-be93-52eb9c094106 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:35.757720 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f95557f-d0ba-49d2-be93-52eb9c094106 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:35.758203 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f95557f-d0ba-49d2-be93-52eb9c094106 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:35.769335 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f95557f-d0ba-49d2-be93-52eb9c094106 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:38:35.777277 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-3f95557f-d0ba-49d2-be93-52eb9c094106 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:38:35.777277 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:35.777277 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:35.777277 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:35.777277 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:35.777277 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:35.777277 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:35.777277 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:35.777277 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:35.777277 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:35.777277 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:35.777277 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:35.777277 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:38:35.777277 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:38:35.777277 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:38:35.777277 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:38:35.779225 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:38:35.779225 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:38:35.779225 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:38:35.779225 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:38:35.779225 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:35.779225 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:35.779225 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:35.779225 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:35.779225 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:35.779225 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:35.779225 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:35.779225 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:38:35.779225 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:35.779225 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:35.779225 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:35.779225 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:35.781150 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:35.781150 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:35.781150 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:35.781150 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:35.781150 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:38:35.781150 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:38:35.781150 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:38:35.781150 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:38:35.781150 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:38:35.781150 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:38:35.781150 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:38:35.781150 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:38:35.781150 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 340/724] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:38:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:35.787496 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b3c7b1-fa78-42d4-bb93-a03af1e0f428 None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:35.808915 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f95557f-d0ba-49d2-be93-52eb9c094106 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:35.827779 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:35.961456 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b3c7b1-fa78-42d4-bb93-a03af1e0f428 None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:35.989266 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f95557f-d0ba-49d2-be93-52eb9c094106 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:36.029228 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e26602-b0c9-44f2-8457-5f9b2c374f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:36.029795 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e26602-b0c9-44f2-8457-5f9b2c374f66 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:36.030087 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e26602-b0c9-44f2-8457-5f9b2c374f66 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:36.030367 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e26602-b0c9-44f2-8457-5f9b2c374f66 None admin] PATH_INFO: `/v3/roles` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:36.031121 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e26602-b0c9-44f2-8457-5f9b2c374f66 None admin] RBAC: Adding query filter params (name=admin) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:36.031431 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e26602-b0c9-44f2-8457-5f9b2c374f66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:36.032838 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e26602-b0c9-44f2-8457-5f9b2c374f66 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:36.050885 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 385/725] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:38:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:36.057815 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f1c2c3-a2e1-49c2-8674-edc2c723cd30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:36.058648 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f1c2c3-a2e1-49c2-8674-edc2c723cd30 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:36.058930 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f1c2c3-a2e1-49c2-8674-edc2c723cd30 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:36.059190 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f1c2c3-a2e1-49c2-8674-edc2c723cd30 None admin] PATH_INFO: `/v3/roles/service` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:36.061391 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e26602-b0c9-44f2-8457-5f9b2c374f66 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:36.068036 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:36.090410 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f1c2c3-a2e1-49c2-8674-edc2c723cd30 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:38:36.103151 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-f7f1c2c3-a2e1-49c2-8674-edc2c723cd30 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:38:36.103151 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:36.103151 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:36.103151 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:36.103151 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:36.103151 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:36.103151 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:36.103151 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:36.103151 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:36.103151 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:36.103151 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:36.103151 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:36.103151 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:38:36.103151 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:38:36.103151 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:38:36.103151 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:38:36.106475 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:38:36.106475 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:38:36.106475 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:38:36.106475 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:38:36.106475 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:36.106475 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:36.106475 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:36.106475 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:36.106475 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:36.106475 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:36.106475 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:36.106475 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:38:36.106475 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:36.106475 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:36.106475 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:36.106475 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:36.109444 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:36.109444 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:36.109444 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:36.109444 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:36.109444 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:38:36.109444 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:38:36.109444 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:38:36.109444 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:38:36.109444 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:38:36.109444 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:38:36.109444 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:38:36.109444 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:38:36.109444 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 341/726] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 14:38:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:36.111983 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f1c2c3-a2e1-49c2-8674-edc2c723cd30 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:36.175480 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:36.195797 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e26602-b0c9-44f2-8457-5f9b2c374f66 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:36.235345 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22186ba7-6e13-4c74-8c55-8cfd8f320599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:36.236205 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22186ba7-6e13-4c74-8c55-8cfd8f320599 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:36.236473 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22186ba7-6e13-4c74-8c55-8cfd8f320599 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:36.236814 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22186ba7-6e13-4c74-8c55-8cfd8f320599 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:36.250140 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22186ba7-6e13-4c74-8c55-8cfd8f320599 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:36.252913 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22186ba7-6e13-4c74-8c55-8cfd8f320599 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:36.285435 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-22186ba7-6e13-4c74-8c55-8cfd8f320599 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:36.285435 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:36.285435 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:36.285435 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:36.285435 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:36.285435 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:36.285435 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:36.285435 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:36.285435 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:36.285435 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:36.285435 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:36.285435 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:36.285435 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:38:36.285435 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:38:36.285435 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:38:36.287579 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:38:36.287579 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:36.287579 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:36.287579 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:36.287579 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:36.287579 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:36.287579 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:36.287579 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:36.287579 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:38:36.287579 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:36.287579 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:36.287579 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:36.287579 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:36.287579 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:36.287579 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:36.287579 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:36.287579 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:36.290411 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:38:36.290411 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:38:36.290411 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:36.290411 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:38:36.293890 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 386/727] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:38:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:36.304302 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22186ba7-6e13-4c74-8c55-8cfd8f320599 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:36.333871 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f1c2c3-a2e1-49c2-8674-edc2c723cd30 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:36.399151 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2010b79-8f79-4a8b-992b-d941dbf3034a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:36.400173 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2010b79-8f79-4a8b-992b-d941dbf3034a None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:36.400173 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2010b79-8f79-4a8b-992b-d941dbf3034a None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:36.400501 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2010b79-8f79-4a8b-992b-d941dbf3034a None admin] PATH_INFO: `/v3/roles` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:36.401199 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2010b79-8f79-4a8b-992b-d941dbf3034a None admin] RBAC: Adding query filter params (name=service) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:36.401517 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2010b79-8f79-4a8b-992b-d941dbf3034a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:36.402936 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2010b79-8f79-4a8b-992b-d941dbf3034a None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:36.432268 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 342/728] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 19 14:38:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:38:36.442899 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2010b79-8f79-4a8b-992b-d941dbf3034a None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:36.483041 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22186ba7-6e13-4c74-8c55-8cfd8f320599 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:36.515583 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:36.522972 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe47c8f-a256-48a4-a3f0-0af0ef1c6fb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:36.523863 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe47c8f-a256-48a4-a3f0-0af0ef1c6fb3 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:36.524082 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe47c8f-a256-48a4-a3f0-0af0ef1c6fb3 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:36.524361 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe47c8f-a256-48a4-a3f0-0af0ef1c6fb3 None admin] PATH_INFO: `/v3/domains` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:36.525091 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe47c8f-a256-48a4-a3f0-0af0ef1c6fb3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:36.525397 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe47c8f-a256-48a4-a3f0-0af0ef1c6fb3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:36.526799 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe47c8f-a256-48a4-a3f0-0af0ef1c6fb3 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:36.529655 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:36.540678 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 387/729] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:38:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:38:36.546187 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e2e6f6-a4ec-4b8f-9609-50a91aba18fa None None] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:36.546399 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e2e6f6-a4ec-4b8f-9609-50a91aba18fa None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:36.546673 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e2e6f6-a4ec-4b8f-9609-50a91aba18fa None None] PATH_INFO: `/` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:36.547989 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 388/730] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:38:36 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:38:36.549062 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ab764a-451a-438e-95b7-32ad7f0249f9 None None] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:36.549400 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ab764a-451a-438e-95b7-32ad7f0249f9 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:36.549678 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ab764a-451a-438e-95b7-32ad7f0249f9 None None] PATH_INFO: `/` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:36.554166 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 389/731] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:38:36 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:38:36.554166 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ab764a-451a-438e-95b7-32ad7f0249f9 None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:36.575154 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2010b79-8f79-4a8b-992b-d941dbf3034a None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:36.613199 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d8e205-371b-4315-a80a-eb9018a91072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:36.614033 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d8e205-371b-4315-a80a-eb9018a91072 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:36.614315 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d8e205-371b-4315-a80a-eb9018a91072 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:36.614585 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d8e205-371b-4315-a80a-eb9018a91072 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:36.625577 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d8e205-371b-4315-a80a-eb9018a91072 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:36.627139 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d8e205-371b-4315-a80a-eb9018a91072 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:36.645393 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-f5d8e205-371b-4315-a80a-eb9018a91072 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:36.645393 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:36.645393 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:36.645393 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:36.645393 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:36.645393 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:36.645393 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:36.645393 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:36.645393 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:36.645393 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:36.645393 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:36.645393 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:36.645393 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:38:36.645393 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:38:36.645393 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:38:36.646763 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:38:36.646763 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:36.646763 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:36.646763 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:36.646763 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:36.646763 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:36.646763 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:36.646763 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:36.646763 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:38:36.646763 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:36.646763 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:36.646763 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:36.646763 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:36.646763 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:36.646763 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:36.646763 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:36.646763 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:36.648533 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:38:36.648533 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:38:36.648533 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:36.648533 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:38:36.648533 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 343/732] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:38:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:36.648853 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efb96fb-148b-42a0-ad07-f496f1350021 None None] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:36.648936 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efb96fb-148b-42a0-ad07-f496f1350021 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:36.649537 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efb96fb-148b-42a0-ad07-f496f1350021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:36.668915 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.auth.core [None req-7efb96fb-148b-42a0-ad07-f496f1350021 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:38:36.670143 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-7efb96fb-148b-42a0-ad07-f496f1350021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:38:36.671013 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ab764a-451a-438e-95b7-32ad7f0249f9 None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:36.707733 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6e190a-860d-4b54-b2b8-f2f870d0112b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:36.708674 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6e190a-860d-4b54-b2b8-f2f870d0112b None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:36.708963 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6e190a-860d-4b54-b2b8-f2f870d0112b None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:36.709279 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6e190a-860d-4b54-b2b8-f2f870d0112b None admin] PATH_INFO: `/v3/users/swift` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:36.718504 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6e190a-860d-4b54-b2b8-f2f870d0112b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:36.719836 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6e190a-860d-4b54-b2b8-f2f870d0112b None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:36.734768 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-ac6e190a-860d-4b54-b2b8-f2f870d0112b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 14:38:36.734768 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:36.734768 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:36.734768 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:36.734768 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:36.734768 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:36.734768 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:36.734768 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:36.734768 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:36.734768 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:36.734768 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:36.734768 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:36.734768 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:38:36.734768 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:38:36.734768 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:38:36.734768 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:38:36.735844 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:36.735844 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:36.735844 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:38:36.735844 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:38:36.735844 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:38:36.735844 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:38:36.735844 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:36.735844 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:36.735844 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:36.735844 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:36.735844 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:36.735844 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:38:36.735844 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:36.735844 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:36.735844 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:36.735844 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:36.735844 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:36.735844 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:36.736855 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:36.736855 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:36.736855 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:38:36.736855 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:38:36.736855 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:38:36.736855 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:38:36.736855 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:38:36.736855 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:38:36.736855 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 14:38:36.736855 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:38:36.736855 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 390/733] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:38:36 2022] GET /identity/v3/users/swift => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:36.737501 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a3f6b8-153b-4ad4-86d2-6b09a40d7ebe None None] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:36.737501 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a3f6b8-153b-4ad4-86d2-6b09a40d7ebe None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:36.738355 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a3f6b8-153b-4ad4-86d2-6b09a40d7ebe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:36.751668 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 344/734] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:38:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:38:36.754769 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efb96fb-148b-42a0-ad07-f496f1350021 None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:36.756282 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.auth.core [None req-d4a3f6b8-153b-4ad4-86d2-6b09a40d7ebe None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:38:36.758213 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.fernet_utils [None req-d4a3f6b8-153b-4ad4-86d2-6b09a40d7ebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:38:36.800623 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:36.828414 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:36.855500 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 391/735] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:38:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:38:36.856557 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a3f6b8-153b-4ad4-86d2-6b09a40d7ebe None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:36.876404 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efb96fb-148b-42a0-ad07-f496f1350021 None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:36.916160 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b436245c-33a7-44d3-b064-20356dd59aeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:36.917257 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b436245c-33a7-44d3-b064-20356dd59aeb None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:36.917545 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b436245c-33a7-44d3-b064-20356dd59aeb None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:36.918191 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b436245c-33a7-44d3-b064-20356dd59aeb None admin] PATH_INFO: `/v3/domains` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:36.918939 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b436245c-33a7-44d3-b064-20356dd59aeb None admin] RBAC: Adding query filter params (name=Default) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:36.919417 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b436245c-33a7-44d3-b064-20356dd59aeb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:36.921137 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b436245c-33a7-44d3-b064-20356dd59aeb None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:36.937599 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 345/736] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:38:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:36.937599 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b436245c-33a7-44d3-b064-20356dd59aeb None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:36.970693 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a3f6b8-153b-4ad4-86d2-6b09a40d7ebe None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:37.009338 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e7723e-3b23-4556-9c5a-4509f2bd1e2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:37.010055 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e7723e-3b23-4556-9c5a-4509f2bd1e2c None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:37.010345 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e7723e-3b23-4556-9c5a-4509f2bd1e2c None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:37.010624 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e7723e-3b23-4556-9c5a-4509f2bd1e2c None admin] PATH_INFO: `/v3/users` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:37.011475 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e7723e-3b23-4556-9c5a-4509f2bd1e2c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:37.011793 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e7723e-3b23-4556-9c5a-4509f2bd1e2c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:37.013240 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e7723e-3b23-4556-9c5a-4509f2bd1e2c None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:37.032062 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 392/737] 10.222.0.57 () {56 vars in 1135 bytes} [Thu May 19 14:38:36 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:37.032898 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e7723e-3b23-4556-9c5a-4509f2bd1e2c None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:37.052096 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:37.056454 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b436245c-33a7-44d3-b064-20356dd59aeb None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:37.084161 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:37.093081 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1f0d9b-394a-4936-a2d5-07560d112164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:37.094366 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1f0d9b-394a-4936-a2d5-07560d112164 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:37.094721 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1f0d9b-394a-4936-a2d5-07560d112164 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:37.095009 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1f0d9b-394a-4936-a2d5-07560d112164 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:37.106370 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1f0d9b-394a-4936-a2d5-07560d112164 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:38:37.113946 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-bd1f0d9b-394a-4936-a2d5-07560d112164 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:38:37.113946 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:37.113946 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:37.113946 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:37.113946 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:37.113946 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:37.113946 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:37.113946 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:37.113946 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:37.113946 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:37.113946 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:37.113946 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:37.113946 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:38:37.113946 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:38:37.113946 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:38:37.113946 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:38:37.115158 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:38:37.115158 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:38:37.115158 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:38:37.115158 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:38:37.115158 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:37.115158 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:37.115158 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:37.115158 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:37.115158 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:37.115158 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:37.115158 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:37.115158 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:38:37.115158 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:37.115158 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:37.115158 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:37.115158 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:37.116867 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:37.116867 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:37.116867 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:37.116867 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:37.116867 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:38:37.116867 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:38:37.116867 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:38:37.116867 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:38:37.116867 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:38:37.116867 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:38:37.116867 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:38:37.116867 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:38:37.116867 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 346/738] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:38:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:37.116867 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1f0d9b-394a-4936-a2d5-07560d112164 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:37.143891 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e7723e-3b23-4556-9c5a-4509f2bd1e2c None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:37.183779 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067257e3-3269-462d-b450-da2ea6735f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:37.184450 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067257e3-3269-462d-b450-da2ea6735f42 None admin] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:37.184731 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067257e3-3269-462d-b450-da2ea6735f42 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:37.185011 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067257e3-3269-462d-b450-da2ea6735f42 None admin] PATH_INFO: `/v3/services` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:37.185734 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067257e3-3269-462d-b450-da2ea6735f42 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:37.187163 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067257e3-3269-462d-b450-da2ea6735f42 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:37.210496 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 393/739] 10.222.0.57 () {60 vars in 1141 bytes} [Thu May 19 14:38:37 2022] POST /identity/v3/services => generated 251 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:38:37.210496 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067257e3-3269-462d-b450-da2ea6735f42 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:37.236477 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1f0d9b-394a-4936-a2d5-07560d112164 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:37.291676 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797dfdd7-2838-4b86-91a2-d189e9f1419c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:37.292685 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797dfdd7-2838-4b86-91a2-d189e9f1419c None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:37.292971 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797dfdd7-2838-4b86-91a2-d189e9f1419c None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:37.293254 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797dfdd7-2838-4b86-91a2-d189e9f1419c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:37.301686 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797dfdd7-2838-4b86-91a2-d189e9f1419c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:37.303311 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797dfdd7-2838-4b86-91a2-d189e9f1419c None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:37.319729 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-797dfdd7-2838-4b86-91a2-d189e9f1419c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 14:38:37.319729 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:37.319729 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:37.319729 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:37.319729 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:37.319729 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:37.319729 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:37.319729 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:37.319729 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:37.319729 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:37.319729 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:37.319729 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:37.319729 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:38:37.319729 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:38:37.319729 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:38:37.319729 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:38:37.320843 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:37.320843 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:37.320843 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:38:37.320843 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:38:37.320843 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:38:37.320843 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:38:37.320843 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:37.320843 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:37.320843 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:37.320843 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:37.320843 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:37.320843 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:38:37.320843 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:37.320843 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:37.320843 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:37.320843 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:37.320843 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:37.320843 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:37.321914 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:37.321914 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:37.321914 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:38:37.321914 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:38:37.321914 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:38:37.321914 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:38:37.321914 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:38:37.321914 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:38:37.321914 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 14:38:37.321914 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:38:37.321914 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 347/740] 10.222.0.57 () {56 vars in 1117 bytes} [Thu May 19 14:38:37 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:37.322965 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aa8c68-95e8-48ae-92a5-77312257ca4f None None] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:37.322965 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aa8c68-95e8-48ae-92a5-77312257ca4f None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:37.322965 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aa8c68-95e8-48ae-92a5-77312257ca4f None None] PATH_INFO: `/` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:37.324378 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 348/741] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:38:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:38:37.325073 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96aa8c68-95e8-48ae-92a5-77312257ca4f None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:37.330660 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067257e3-3269-462d-b450-da2ea6735f42 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:37.333831 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:37.346207 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:37.377709 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbdc7da-98e0-46b1-a40a-c5fd502c9e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:37.378476 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbdc7da-98e0-46b1-a40a-c5fd502c9e70 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:37.379101 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbdc7da-98e0-46b1-a40a-c5fd502c9e70 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:37.379388 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbdc7da-98e0-46b1-a40a-c5fd502c9e70 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:37.449857 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96aa8c68-95e8-48ae-92a5-77312257ca4f None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:37.470618 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbdc7da-98e0-46b1-a40a-c5fd502c9e70 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:37.472921 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbdc7da-98e0-46b1-a40a-c5fd502c9e70 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:37.484704 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea7f07b-c9d0-45f9-8fe2-aa2a78446118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:37.485380 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea7f07b-c9d0-45f9-8fe2-aa2a78446118 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:37.485666 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea7f07b-c9d0-45f9-8fe2-aa2a78446118 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:37.485975 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea7f07b-c9d0-45f9-8fe2-aa2a78446118 None admin] PATH_INFO: `/v3/roles` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:37.486730 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea7f07b-c9d0-45f9-8fe2-aa2a78446118 None admin] RBAC: Adding query filter params (name=admin) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:37.487041 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea7f07b-c9d0-45f9-8fe2-aa2a78446118 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:37.488411 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea7f07b-c9d0-45f9-8fe2-aa2a78446118 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:37.501922 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 349/742] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:38:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:37.501922 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea7f07b-c9d0-45f9-8fe2-aa2a78446118 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:37.594683 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 394/743] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:38:37 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:38:37.595208 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:37.595708 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3753a306-e54b-497d-827d-1b694d525251 None None] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:37.596002 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3753a306-e54b-497d-827d-1b694d525251 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:37.596280 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3753a306-e54b-497d-827d-1b694d525251 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:37.604015 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:37.615687 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.auth.core [None req-3753a306-e54b-497d-827d-1b694d525251 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:38:37.618301 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.fernet_utils [None req-3753a306-e54b-497d-827d-1b694d525251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:38:37.628831 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea7f07b-c9d0-45f9-8fe2-aa2a78446118 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:37.666121 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a89dfd-5c6e-474d-a988-d8e0d677dd05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:37.666847 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a89dfd-5c6e-474d-a988-d8e0d677dd05 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:37.667136 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a89dfd-5c6e-474d-a988-d8e0d677dd05 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:37.667411 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a89dfd-5c6e-474d-a988-d8e0d677dd05 None admin] PATH_INFO: `/v3/users` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:37.668260 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a89dfd-5c6e-474d-a988-d8e0d677dd05 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:37.668573 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a89dfd-5c6e-474d-a988-d8e0d677dd05 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:37.670041 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a89dfd-5c6e-474d-a988-d8e0d677dd05 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:37.697882 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 350/744] 10.222.0.57 () {56 vars in 1149 bytes} [Thu May 19 14:38:37 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:37.697882 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a89dfd-5c6e-474d-a988-d8e0d677dd05 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:37.698165 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 395/745] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:38:37 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:38:37.698165 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3753a306-e54b-497d-827d-1b694d525251 None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:37.805716 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3753a306-e54b-497d-827d-1b694d525251 None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:37.812124 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a89dfd-5c6e-474d-a988-d8e0d677dd05 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:37.844176 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1632eae-a349-409e-a720-ed7f7d8f26df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:37.845578 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1632eae-a349-409e-a720-ed7f7d8f26df None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:37.846121 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1632eae-a349-409e-a720-ed7f7d8f26df None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:37.846715 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1632eae-a349-409e-a720-ed7f7d8f26df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:37.847056 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce73d6ae-5701-4407-a4df-e8cc1279fc7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:37.847876 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce73d6ae-5701-4407-a4df-e8cc1279fc7e None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:37.848146 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce73d6ae-5701-4407-a4df-e8cc1279fc7e None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:37.848427 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce73d6ae-5701-4407-a4df-e8cc1279fc7e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:37.849957 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:37.850468 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:37.860948 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce73d6ae-5701-4407-a4df-e8cc1279fc7e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:37.861640 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1632eae-a349-409e-a720-ed7f7d8f26df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:37.862452 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce73d6ae-5701-4407-a4df-e8cc1279fc7e None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:37.863328 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1632eae-a349-409e-a720-ed7f7d8f26df None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:37.880824 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-ce73d6ae-5701-4407-a4df-e8cc1279fc7e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:37.880824 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:37.880824 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:37.880824 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:37.880824 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:37.880824 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:37.880824 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:37.880824 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:37.880824 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:37.880824 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:37.880824 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:37.880824 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:37.880824 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:38:37.880824 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:38:37.880824 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:38:37.882450 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:38:37.882450 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:37.882450 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:37.882450 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:37.882450 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:37.882450 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:37.882450 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:37.882450 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:37.882450 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:38:37.882450 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:37.882450 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:37.882450 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:37.882450 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:37.882450 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:37.882450 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:37.882450 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:37.882450 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:37.883431 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:38:37.883431 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:38:37.883431 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:37.883431 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:38:37.883671 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-a1632eae-a349-409e-a720-ed7f7d8f26df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:37.883671 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:37.883671 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:37.883671 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:37.883671 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:37.883671 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:37.883671 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:37.883671 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:37.883671 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:37.883671 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:37.883671 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:37.883671 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:37.883671 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:38:37.883671 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:38:37.883671 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:38:37.884524 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:38:37.884524 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:37.884524 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:37.884524 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:37.884524 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:37.884524 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:37.884524 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:37.884524 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:37.884524 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:38:37.884524 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:37.884524 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:37.884524 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:37.884524 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:37.884524 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:37.884524 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:37.884524 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:37.884524 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:37.885499 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:38:37.885499 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:38:37.885499 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:37.885499 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:38:37.887270 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 351/746] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:38:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:37.887702 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 396/747] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:38:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:37.888018 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce73d6ae-5701-4407-a4df-e8cc1279fc7e None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:37.888422 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1632eae-a349-409e-a720-ed7f7d8f26df None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:37.995435 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1632eae-a349-409e-a720-ed7f7d8f26df None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:37.995695 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce73d6ae-5701-4407-a4df-e8cc1279fc7e None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:38.032078 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70616296-7484-4c88-8569-5f679be6f3ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:38.032298 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3bb155-5395-4bca-93f7-7147af0d5383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:38.032733 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3bb155-5395-4bca-93f7-7147af0d5383 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:38.032867 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70616296-7484-4c88-8569-5f679be6f3ac None admin] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:38.033039 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70616296-7484-4c88-8569-5f679be6f3ac None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:38.033146 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3bb155-5395-4bca-93f7-7147af0d5383 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:38.033299 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70616296-7484-4c88-8569-5f679be6f3ac None admin] PATH_INFO: `/v3/services` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:38.033396 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3bb155-5395-4bca-93f7-7147af0d5383 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:38.034038 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70616296-7484-4c88-8569-5f679be6f3ac None admin] RBAC: Authorizing `identity:create_service()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:38.035415 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70616296-7484-4c88-8569-5f679be6f3ac None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:38.054868 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 397/748] 10.222.0.57 () {60 vars in 1141 bytes} [Thu May 19 14:38:37 2022] POST /identity/v3/services => generated 240 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:38:38.055733 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70616296-7484-4c88-8569-5f679be6f3ac None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:38.117134 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:38.127357 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:38.135479 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3bb155-5395-4bca-93f7-7147af0d5383 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:38.137704 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3bb155-5395-4bca-93f7-7147af0d5383 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:38.233199 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70616296-7484-4c88-8569-5f679be6f3ac None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:38.277679 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24987207-9f37-42be-b494-46743ea51339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:38.278971 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24987207-9f37-42be-b494-46743ea51339 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:38.279324 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24987207-9f37-42be-b494-46743ea51339 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:38.279752 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24987207-9f37-42be-b494-46743ea51339 None admin] PATH_INFO: `/v3/domains` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:38.280497 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24987207-9f37-42be-b494-46743ea51339 None admin] RBAC: Adding query filter params (name=Default) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:38.280810 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24987207-9f37-42be-b494-46743ea51339 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:38.287187 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24987207-9f37-42be-b494-46743ea51339 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:38.287507 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 352/749] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:38:37 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:38:38.287507 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3bb155-5395-4bca-93f7-7147af0d5383 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:38.298430 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 398/750] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:38:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:38.299234 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24987207-9f37-42be-b494-46743ea51339 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:38.412356 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:38.423078 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24987207-9f37-42be-b494-46743ea51339 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:38.425490 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:38.438954 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3bb155-5395-4bca-93f7-7147af0d5383 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:38.475268 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77757e0-04cc-4384-9167-afbe3065a136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:38.476689 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77757e0-04cc-4384-9167-afbe3065a136 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:38.476980 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77757e0-04cc-4384-9167-afbe3065a136 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:38.477225 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77757e0-04cc-4384-9167-afbe3065a136 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:38.479285 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e19252-3b3e-472b-9ba8-0ed630b12a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:38.480527 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e19252-3b3e-472b-9ba8-0ed630b12a01 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:38.481021 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e19252-3b3e-472b-9ba8-0ed630b12a01 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:38.481522 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e19252-3b3e-472b-9ba8-0ed630b12a01 None admin] PATH_INFO: `/v3/domains` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:38.482525 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e19252-3b3e-472b-9ba8-0ed630b12a01 None admin] RBAC: Adding query filter params (name=Default) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:38.483061 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e19252-3b3e-472b-9ba8-0ed630b12a01 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:38.484658 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e19252-3b3e-472b-9ba8-0ed630b12a01 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:38.489696 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77757e0-04cc-4384-9167-afbe3065a136 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:38.491410 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77757e0-04cc-4384-9167-afbe3065a136 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:38.502087 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 353/751] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:38:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:38.503249 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e19252-3b3e-472b-9ba8-0ed630b12a01 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:38.512985 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-d77757e0-04cc-4384-9167-afbe3065a136 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:38.512985 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:38.512985 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:38.512985 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:38.512985 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:38.512985 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:38.512985 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:38.512985 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:38.512985 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:38.512985 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:38.512985 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:38.512985 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:38.512985 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:38:38.512985 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:38:38.512985 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:38:38.514096 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:38:38.514096 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:38.514096 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:38.514096 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:38.514096 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:38.514096 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:38.514096 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:38.514096 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:38.514096 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:38:38.514096 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:38.514096 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:38.514096 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:38.514096 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:38.514096 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:38.514096 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:38.514096 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:38.514096 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:38.515073 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:38:38.515073 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:38:38.515073 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:38.515073 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:38:38.515073 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 399/752] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:38:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:38.515495 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77757e0-04cc-4384-9167-afbe3065a136 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:38.638592 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77757e0-04cc-4384-9167-afbe3065a136 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:38.641297 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e19252-3b3e-472b-9ba8-0ed630b12a01 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:38.678277 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3cf2de-9c37-4e18-8275-a6f3a1d1e74a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:38.679474 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3cf2de-9c37-4e18-8275-a6f3a1d1e74a None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:38.679971 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3cf2de-9c37-4e18-8275-a6f3a1d1e74a None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:38.680480 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3cf2de-9c37-4e18-8275-a6f3a1d1e74a None admin] PATH_INFO: `/v3/projects/service` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:38.687325 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8239c04a-051f-40b3-b2e1-10abad296d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:38.688407 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8239c04a-051f-40b3-b2e1-10abad296d28 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:38.688696 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8239c04a-051f-40b3-b2e1-10abad296d28 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:38.688976 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8239c04a-051f-40b3-b2e1-10abad296d28 None admin] PATH_INFO: `/v3/users/nova` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:38.689270 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3cf2de-9c37-4e18-8275-a6f3a1d1e74a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:38.691162 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3cf2de-9c37-4e18-8275-a6f3a1d1e74a None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:38.698520 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8239c04a-051f-40b3-b2e1-10abad296d28 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:38.699968 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8239c04a-051f-40b3-b2e1-10abad296d28 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:38.707016 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-ad3cf2de-9c37-4e18-8275-a6f3a1d1e74a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:38:38.707016 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:38.707016 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:38.707016 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:38.707016 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:38.707016 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:38.707016 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:38.707016 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:38.707016 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:38.707016 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:38.707016 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:38.707016 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:38.707016 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:38:38.707016 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:38:38.707016 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:38:38.707980 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:38:38.707980 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:38.707980 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:38.707980 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:38.707980 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:38.707980 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:38.707980 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:38.707980 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:38.707980 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:38:38.707980 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:38.707980 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:38.707980 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:38.707980 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:38.707980 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:38.707980 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:38.707980 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:38.707980 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:38.708970 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:38:38.708970 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:38:38.708970 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:38:38.708970 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:38:38.708970 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:38:38.708970 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:38:38.708970 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:38:38.708970 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:38:38.716445 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-8239c04a-051f-40b3-b2e1-10abad296d28 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 14:38:38.716445 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:38.716445 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:38.716445 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:38.716445 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:38.716445 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:38.716445 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:38.716445 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:38.716445 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:38.716445 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:38.716445 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:38.716445 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:38.716445 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:38:38.716445 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:38:38.716445 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:38:38.716445 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:38:38.717591 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:38.717591 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:38.717591 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:38:38.717591 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:38:38.717591 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:38:38.717591 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:38:38.717591 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:38.717591 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:38.717591 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:38.717591 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:38.717591 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:38.717591 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:38:38.717591 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:38.717591 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:38.717591 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:38.717591 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:38.717591 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:38.717591 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:38.718954 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:38.718954 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:38.718954 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:38:38.718954 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:38:38.718954 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:38:38.718954 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:38:38.718954 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:38:38.718954 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:38:38.718954 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 14:38:38.718954 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:38:38.718954 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 354/753] 10.222.0.57 () {56 vars in 1093 bytes} [Thu May 19 14:38:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:38.718954 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8239c04a-051f-40b3-b2e1-10abad296d28 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:38.720256 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 400/754] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:38:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:38.721394 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3cf2de-9c37-4e18-8275-a6f3a1d1e74a None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:38.729560 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:38.741726 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:38.843598 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3cf2de-9c37-4e18-8275-a6f3a1d1e74a None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:38.862144 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8239c04a-051f-40b3-b2e1-10abad296d28 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:38.885691 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd81df0-6825-49a5-966e-a2c204794e59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:38.886818 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd81df0-6825-49a5-966e-a2c204794e59 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:38.887316 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd81df0-6825-49a5-966e-a2c204794e59 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:38.887820 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd81df0-6825-49a5-966e-a2c204794e59 None admin] PATH_INFO: `/v3/projects` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:38.888920 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd81df0-6825-49a5-966e-a2c204794e59 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:38.889461 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd81df0-6825-49a5-966e-a2c204794e59 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:38.891175 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd81df0-6825-49a5-966e-a2c204794e59 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:38.905826 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f237d64-b46d-4c30-bd2f-c96b46431432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:38.906539 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f237d64-b46d-4c30-bd2f-c96b46431432 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:38.906829 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f237d64-b46d-4c30-bd2f-c96b46431432 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:38.907109 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f237d64-b46d-4c30-bd2f-c96b46431432 None admin] PATH_INFO: `/v3/domains` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:38.909886 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f237d64-b46d-4c30-bd2f-c96b46431432 None admin] RBAC: Adding query filter params (name=Default) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:38.909886 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f237d64-b46d-4c30-bd2f-c96b46431432 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:38.909886 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f237d64-b46d-4c30-bd2f-c96b46431432 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:38.912393 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 401/755] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:38:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:38.914045 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd81df0-6825-49a5-966e-a2c204794e59 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:38.931708 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 355/756] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:38:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:38.931708 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f237d64-b46d-4c30-bd2f-c96b46431432 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:39.005618 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:39.026456 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:39.042887 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd81df0-6825-49a5-966e-a2c204794e59 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:39.078537 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f237d64-b46d-4c30-bd2f-c96b46431432 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:39.086048 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3449d9fb-e0fe-4ffc-916a-aa680c0f7fc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:39.086738 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3449d9fb-e0fe-4ffc-916a-aa680c0f7fc1 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:39.087025 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3449d9fb-e0fe-4ffc-916a-aa680c0f7fc1 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:39.087311 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3449d9fb-e0fe-4ffc-916a-aa680c0f7fc1 None admin] PATH_INFO: `/v3/users` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:39.088158 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3449d9fb-e0fe-4ffc-916a-aa680c0f7fc1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:39.088469 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3449d9fb-e0fe-4ffc-916a-aa680c0f7fc1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:39.090426 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3449d9fb-e0fe-4ffc-916a-aa680c0f7fc1 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:39.114455 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 402/757] 10.222.0.57 () {56 vars in 1133 bytes} [Thu May 19 14:38:38 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:39.115392 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3449d9fb-e0fe-4ffc-916a-aa680c0f7fc1 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:39.120735 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91841ee-18bf-4b36-9ca8-b05a86f609f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:39.121804 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91841ee-18bf-4b36-9ca8-b05a86f609f0 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:39.122334 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91841ee-18bf-4b36-9ca8-b05a86f609f0 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:39.122824 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91841ee-18bf-4b36-9ca8-b05a86f609f0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:39.123920 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91841ee-18bf-4b36-9ca8-b05a86f609f0 None admin] RBAC: Adding query filter params (role.id=c2935b1a25ca46fcba419fe3d1715330, scope.project.id=42f35b96b9d4499abfb5e5f7cca49adc, user.id=4f8e2e0909074a85bb493c49ac3a6d4d) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:39.124461 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91841ee-18bf-4b36-9ca8-b05a86f609f0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:39.126171 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91841ee-18bf-4b36-9ca8-b05a86f609f0 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:39.137876 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 356/758] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:38:38 2022] GET /identity/v3/role_assignments?user.id=4f8e2e0909074a85bb493c49ac3a6d4d&scope.project.id=42f35b96b9d4499abfb5e5f7cca49adc&role.id=c2935b1a25ca46fcba419fe3d1715330 => generated 259 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:39.137876 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91841ee-18bf-4b36-9ca8-b05a86f609f0 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:39.323391 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91841ee-18bf-4b36-9ca8-b05a86f609f0 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:39.337345 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3449d9fb-e0fe-4ffc-916a-aa680c0f7fc1 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:39.371731 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:39.379377 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cb8148-cc5a-40a6-be14-2387da7744a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:39.380808 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cb8148-cc5a-40a6-be14-2387da7744a6 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:39.381452 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cb8148-cc5a-40a6-be14-2387da7744a6 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:39.382830 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cb8148-cc5a-40a6-be14-2387da7744a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:39.393704 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:39.400148 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cb8148-cc5a-40a6-be14-2387da7744a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:39.401972 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cb8148-cc5a-40a6-be14-2387da7744a6 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:39.409585 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7088183b-63fd-45a1-a26a-354bd41b21c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:39.409585 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7088183b-63fd-45a1-a26a-354bd41b21c4 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:39.409585 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7088183b-63fd-45a1-a26a-354bd41b21c4 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:39.409585 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7088183b-63fd-45a1-a26a-354bd41b21c4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:39.415627 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7088183b-63fd-45a1-a26a-354bd41b21c4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:39.418778 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7088183b-63fd-45a1-a26a-354bd41b21c4 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:39.432772 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-7088183b-63fd-45a1-a26a-354bd41b21c4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:38:39.432772 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:39.432772 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:39.432772 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:39.432772 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:39.432772 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:39.432772 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:39.432772 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:39.432772 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:39.432772 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:39.432772 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:39.432772 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:39.432772 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:38:39.432772 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:38:39.432772 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:38:39.433809 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:38:39.433809 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:39.433809 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:39.433809 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:39.433809 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:39.433809 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:39.433809 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:39.433809 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:39.433809 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:38:39.433809 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:39.433809 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:39.433809 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:39.433809 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:39.433809 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:39.433809 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:39.433809 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:39.433809 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:39.434852 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:38:39.434852 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:38:39.434852 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:38:39.434852 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:38:39.434852 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:38:39.434852 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:38:39.434852 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:38:39.434852 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:38:39.434852 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 403/759] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:38:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:39.438073 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-07cb8148-cc5a-40a6-be14-2387da7744a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:39.438073 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:39.438073 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:39.438073 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:39.438073 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:39.438073 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:39.438073 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:39.438073 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:39.438073 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:39.438073 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:39.438073 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:39.438073 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:39.438073 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:38:39.438073 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:38:39.438073 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:38:39.439056 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:38:39.439056 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:39.439056 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:39.439056 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:39.439056 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:39.439056 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:39.439056 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:39.439056 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:39.439056 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:38:39.439056 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:39.439056 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:39.439056 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:39.439056 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:39.439056 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:39.439056 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:39.439056 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:39.439056 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:39.440075 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:38:39.440075 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:38:39.440075 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:39.440075 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:38:39.440332 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7088183b-63fd-45a1-a26a-354bd41b21c4 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:39.442567 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 357/760] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:38:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:39.449196 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cb8148-cc5a-40a6-be14-2387da7744a6 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:39.591012 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7088183b-63fd-45a1-a26a-354bd41b21c4 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:39.627813 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fed42a-a6e8-4057-b9ce-8de84de3fe8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:39.628240 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cb8148-cc5a-40a6-be14-2387da7744a6 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:39.628688 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fed42a-a6e8-4057-b9ce-8de84de3fe8c None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:39.628774 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fed42a-a6e8-4057-b9ce-8de84de3fe8c None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:39.629016 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fed42a-a6e8-4057-b9ce-8de84de3fe8c None admin] PATH_INFO: `/v3/projects` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:39.630023 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fed42a-a6e8-4057-b9ce-8de84de3fe8c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:39.630340 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fed42a-a6e8-4057-b9ce-8de84de3fe8c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:39.633586 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fed42a-a6e8-4057-b9ce-8de84de3fe8c None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:39.650074 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 404/761] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:38:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:39.651496 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98915bbd-a3de-405f-a053-b5e9efdad941 None None] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:39.651790 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98915bbd-a3de-405f-a053-b5e9efdad941 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:39.652072 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98915bbd-a3de-405f-a053-b5e9efdad941 None None] PATH_INFO: `/` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:39.653592 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 405/762] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:38:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:38:39.655776 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98915bbd-a3de-405f-a053-b5e9efdad941 None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:39.683015 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b60bff-f008-4e35-a493-8863506c5f94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:39.683733 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b60bff-f008-4e35-a493-8863506c5f94 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:39.684019 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b60bff-f008-4e35-a493-8863506c5f94 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:39.684347 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b60bff-f008-4e35-a493-8863506c5f94 None admin] PATH_INFO: `/v3/domains` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:39.685086 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b60bff-f008-4e35-a493-8863506c5f94 None admin] RBAC: Adding query filter params (name=Default) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:39.685392 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b60bff-f008-4e35-a493-8863506c5f94 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:39.686813 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b60bff-f008-4e35-a493-8863506c5f94 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:39.700287 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 358/763] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:38:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:39.701680 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab7d7cb-eea3-4cb9-9a17-a7096694023f None None] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:39.702004 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab7d7cb-eea3-4cb9-9a17-a7096694023f None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:39.702287 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab7d7cb-eea3-4cb9-9a17-a7096694023f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:39.720134 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.auth.core [None req-7ab7d7cb-eea3-4cb9-9a17-a7096694023f None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:38:39.721065 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:39.721326 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-7ab7d7cb-eea3-4cb9-9a17-a7096694023f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:38:39.762132 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:39.775385 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98915bbd-a3de-405f-a053-b5e9efdad941 None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:39.806159 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 359/764] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:38:39 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:38:39.807297 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab7d7cb-eea3-4cb9-9a17-a7096694023f None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:39.812487 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef61fef-f45b-4aee-8268-3c18180d1ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:39.813320 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef61fef-f45b-4aee-8268-3c18180d1ad7 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:39.813707 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef61fef-f45b-4aee-8268-3c18180d1ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:39.814123 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef61fef-f45b-4aee-8268-3c18180d1ad7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:39.815243 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef61fef-f45b-4aee-8268-3c18180d1ad7 None admin] RBAC: Adding query filter params (role.id=ffb8da90af53491980816087ee621ccb, scope.project.id=42f35b96b9d4499abfb5e5f7cca49adc, user.id=2cbf483c6556421fa36179722100f8b9) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:39.815659 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef61fef-f45b-4aee-8268-3c18180d1ad7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:39.817887 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef61fef-f45b-4aee-8268-3c18180d1ad7 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:39.824887 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 406/765] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:38:39 2022] GET /identity/v3/role_assignments?user.id=2cbf483c6556421fa36179722100f8b9&scope.project.id=42f35b96b9d4499abfb5e5f7cca49adc&role.id=ffb8da90af53491980816087ee621ccb => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:39.843238 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef61fef-f45b-4aee-8268-3c18180d1ad7 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:39.998458 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef61fef-f45b-4aee-8268-3c18180d1ad7 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:40.017650 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:40.040802 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bffc5a-d30d-4b6f-8523-44c870b93116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:40.042156 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bffc5a-d30d-4b6f-8523-44c870b93116 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:40.042659 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bffc5a-d30d-4b6f-8523-44c870b93116 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:40.043150 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bffc5a-d30d-4b6f-8523-44c870b93116 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:40.044063 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bffc5a-d30d-4b6f-8523-44c870b93116 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:40.045784 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bffc5a-d30d-4b6f-8523-44c870b93116 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:40.052370 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab7d7cb-eea3-4cb9-9a17-a7096694023f None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:40.059870 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-00bffc5a-d30d-4b6f-8523-44c870b93116 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 14:38:40.059870 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:40.059870 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:38:40.059870 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:38:40.059870 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:38:40.059870 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:38:40.059870 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:38:40.059870 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:38:40.059870 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 14:38:40.059870 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:38:40.059870 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:38:40.059870 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:38:40.059870 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:40.059870 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:40.059870 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:40.059870 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:40.061210 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:40.061210 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:40.061210 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:40.061210 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:40.061210 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:40.061210 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:40.061210 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:40.061210 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:38:40.061210 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:38:40.061210 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:38:40.061210 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:38:40.061210 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:40.061210 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:40.061210 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:40.061210 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:40.061210 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:40.062317 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:40.062317 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:40.062317 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:38:40.062317 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:40.062317 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:40.062317 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:40.062317 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:40.062317 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:40.062317 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:40.062317 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:40.062317 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:40.062317 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:38:40.062317 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:38:40.062317 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 14:38:40.062317 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:38:40.069699 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 407/766] 10.222.0.57 () {56 vars in 1129 bytes} [Thu May 19 14:38:39 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:38:40.071252 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bffc5a-d30d-4b6f-8523-44c870b93116 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:40.130573 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fe7408-61a5-40e7-b2a9-f0a7f36000d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:40.131708 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fe7408-61a5-40e7-b2a9-f0a7f36000d7 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:40.132456 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fe7408-61a5-40e7-b2a9-f0a7f36000d7 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:40.132840 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fe7408-61a5-40e7-b2a9-f0a7f36000d7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:40.142789 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fe7408-61a5-40e7-b2a9-f0a7f36000d7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:40.151189 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fe7408-61a5-40e7-b2a9-f0a7f36000d7 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:40.180029 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-a4fe7408-61a5-40e7-b2a9-f0a7f36000d7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:38:40.180029 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:40.180029 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:40.180029 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:40.180029 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:40.180029 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:40.180029 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:40.180029 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:40.180029 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:40.180029 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:40.180029 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:40.180029 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:40.180029 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:38:40.180029 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:38:40.180029 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:38:40.180997 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:38:40.180997 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:40.180997 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:40.180997 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:40.180997 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:40.180997 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:40.180997 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:40.180997 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:40.180997 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:38:40.180997 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:40.180997 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:40.180997 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:40.180997 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:40.180997 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:40.180997 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:40.180997 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:40.180997 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:40.182584 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:38:40.182584 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:38:40.182584 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:38:40.182584 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:38:40.182584 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:38:40.182584 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:38:40.182584 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:38:40.182584 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:38:40.182584 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 360/767] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:38:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 375 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:40.184421 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bffc5a-d30d-4b6f-8523-44c870b93116 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:40.187141 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fe7408-61a5-40e7-b2a9-f0a7f36000d7 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:40.221621 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee52bc3c-760f-43b0-afd1-be8b2fb56a61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:40.222817 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee52bc3c-760f-43b0-afd1-be8b2fb56a61 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:40.223338 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee52bc3c-760f-43b0-afd1-be8b2fb56a61 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:40.223821 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee52bc3c-760f-43b0-afd1-be8b2fb56a61 None admin] PATH_INFO: `/v3/services` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:40.224793 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee52bc3c-760f-43b0-afd1-be8b2fb56a61 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:40.225311 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee52bc3c-760f-43b0-afd1-be8b2fb56a61 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:40.227519 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee52bc3c-760f-43b0-afd1-be8b2fb56a61 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:40.243217 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 408/768] 10.222.0.57 () {56 vars in 1124 bytes} [Thu May 19 14:38:40 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:38:40.248143 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee52bc3c-760f-43b0-afd1-be8b2fb56a61 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:40.263985 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:40.338801 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:40.400753 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee52bc3c-760f-43b0-afd1-be8b2fb56a61 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:40.429637 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fe7408-61a5-40e7-b2a9-f0a7f36000d7 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:40.450481 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab7d60b-67e7-4bf1-b356-758a4e14b4c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:40.451600 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab7d60b-67e7-4bf1-b356-758a4e14b4c3 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:40.452094 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab7d60b-67e7-4bf1-b356-758a4e14b4c3 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:40.452577 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab7d60b-67e7-4bf1-b356-758a4e14b4c3 None admin] PATH_INFO: `/v3/services` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:40.453753 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab7d60b-67e7-4bf1-b356-758a4e14b4c3 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:40.454269 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab7d60b-67e7-4bf1-b356-758a4e14b4c3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:40.455974 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab7d60b-67e7-4bf1-b356-758a4e14b4c3 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:40.468450 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 409/769] 10.222.0.57 () {56 vars in 1124 bytes} [Thu May 19 14:38:40 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:40.470260 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab7d60b-67e7-4bf1-b356-758a4e14b4c3 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:40.475006 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e38d24-a08f-4e30-9fe5-7f3c8a8615e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:40.476394 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e38d24-a08f-4e30-9fe5-7f3c8a8615e0 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:40.476907 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e38d24-a08f-4e30-9fe5-7f3c8a8615e0 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:40.477433 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e38d24-a08f-4e30-9fe5-7f3c8a8615e0 None admin] PATH_INFO: `/v3/projects` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:40.478514 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e38d24-a08f-4e30-9fe5-7f3c8a8615e0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:40.479077 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e38d24-a08f-4e30-9fe5-7f3c8a8615e0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:40.480912 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e38d24-a08f-4e30-9fe5-7f3c8a8615e0 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:40.498731 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 361/770] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:38:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:40.506472 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e38d24-a08f-4e30-9fe5-7f3c8a8615e0 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:40.654357 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:40.667065 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e38d24-a08f-4e30-9fe5-7f3c8a8615e0 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:40.680393 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab7d60b-67e7-4bf1-b356-758a4e14b4c3 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:40.690453 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:40.704982 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d280abc5-cd1d-4d0f-9243-638b19e2c745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:40.705893 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d280abc5-cd1d-4d0f-9243-638b19e2c745 None admin] REQUEST_METHOD: `PUT` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:40.706184 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d280abc5-cd1d-4d0f-9243-638b19e2c745 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:40.706461 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d280abc5-cd1d-4d0f-9243-638b19e2c745 None admin] PATH_INFO: `/v3/projects/42f35b96b9d4499abfb5e5f7cca49adc/users/a9591727b2f54244baf90b11a8e2277f/roles/c2935b1a25ca46fcba419fe3d1715330` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:40.708481 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d280abc5-cd1d-4d0f-9243-638b19e2c745 None admin] RBAC: Authorizing `identity:create_grant(project_id=42f35b96b9d4499abfb5e5f7cca49adc, user_id=a9591727b2f54244baf90b11a8e2277f, role_id=c2935b1a25ca46fcba419fe3d1715330)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:40.710237 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d280abc5-cd1d-4d0f-9243-638b19e2c745 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:40.726463 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e50cf2-fc31-49a6-b096-ccb2d588d11c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:40.732731 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 362/771] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 14:38:40 2022] PUT /identity/v3/projects/42f35b96b9d4499abfb5e5f7cca49adc/users/a9591727b2f54244baf90b11a8e2277f/roles/c2935b1a25ca46fcba419fe3d1715330 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:38:40.732731 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ddcda1-c6a9-4a6c-999f-066aa74d8b75 None None] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:40.732731 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ddcda1-c6a9-4a6c-999f-066aa74d8b75 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:40.732731 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ddcda1-c6a9-4a6c-999f-066aa74d8b75 None None] PATH_INFO: `/` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:40.732731 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 363/772] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:38:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:38:40.734308 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e50cf2-fc31-49a6-b096-ccb2d588d11c None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:40.734929 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e50cf2-fc31-49a6-b096-ccb2d588d11c None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:40.735419 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e50cf2-fc31-49a6-b096-ccb2d588d11c None admin] PATH_INFO: `/v3/endpoints` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:40.736559 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e50cf2-fc31-49a6-b096-ccb2d588d11c None admin] RBAC: Adding query filter params (interface=public, service_id=d6aa37fc857a450392fcd5e65de46f5d, region_id=RegionOne) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:40.737166 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e50cf2-fc31-49a6-b096-ccb2d588d11c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:40.738907 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e50cf2-fc31-49a6-b096-ccb2d588d11c None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:40.749437 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a6c5fb-c22a-4e40-a063-d854553862ca None None] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:40.751402 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a6c5fb-c22a-4e40-a063-d854553862ca None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:40.752270 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a6c5fb-c22a-4e40-a063-d854553862ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:40.752545 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 410/773] 10.222.0.57 () {56 vars in 1251 bytes} [Thu May 19 14:38:40 2022] GET /identity/v3/endpoints?service_id=d6aa37fc857a450392fcd5e65de46f5d&interface=public®ion_id=RegionOne => generated 194 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:38:40.757479 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e50cf2-fc31-49a6-b096-ccb2d588d11c None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:40.790501 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.auth.core [None req-82a6c5fb-c22a-4e40-a063-d854553862ca None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:38:40.793257 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-82a6c5fb-c22a-4e40-a063-d854553862ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:38:40.936767 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 364/774] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:38:40 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:38:40.957023 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e50cf2-fc31-49a6-b096-ccb2d588d11c None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:40.984557 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a6c5fb-c22a-4e40-a063-d854553862ca None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:41.015364 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d774219-bbcd-412e-a67d-ec4e976a2b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:41.016059 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d774219-bbcd-412e-a67d-ec4e976a2b37 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:41.016348 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d774219-bbcd-412e-a67d-ec4e976a2b37 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:41.016629 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d774219-bbcd-412e-a67d-ec4e976a2b37 None admin] PATH_INFO: `/v3/services` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:41.017329 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d774219-bbcd-412e-a67d-ec4e976a2b37 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:41.018908 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d774219-bbcd-412e-a67d-ec4e976a2b37 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:41.027756 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 411/775] 10.222.0.57 () {56 vars in 1087 bytes} [Thu May 19 14:38:40 2022] GET /identity/v3/services => generated 772 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:41.091459 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:41.113568 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a6c5fb-c22a-4e40-a063-d854553862ca None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:41.153477 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30993a5b-2cc1-43ec-86f5-08be202fed6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:41.154528 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30993a5b-2cc1-43ec-86f5-08be202fed6c None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:41.155103 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30993a5b-2cc1-43ec-86f5-08be202fed6c None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:41.155480 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30993a5b-2cc1-43ec-86f5-08be202fed6c None admin] PATH_INFO: `/v3/services/network` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:41.161779 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30993a5b-2cc1-43ec-86f5-08be202fed6c None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:41.169175 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30993a5b-2cc1-43ec-86f5-08be202fed6c None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:41.183236 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-30993a5b-2cc1-43ec-86f5-08be202fed6c None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 19 14:38:41.183236 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:41.183236 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:38:41.183236 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:38:41.183236 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:38:41.183236 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:38:41.183236 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:38:41.183236 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:38:41.183236 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 14:38:41.183236 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:38:41.183236 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:38:41.183236 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:38:41.183236 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:41.183236 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:41.183236 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:41.183236 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:41.183236 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:41.184628 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:41.184628 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:41.184628 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:41.184628 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:41.184628 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:41.184628 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:41.184628 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:38:41.184628 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:38:41.184628 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:38:41.184628 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:38:41.184628 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:41.184628 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:41.184628 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:41.184628 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:41.184628 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:41.184628 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:41.185868 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:41.185868 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:38:41.185868 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:41.185868 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:41.185868 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:41.185868 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:41.185868 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:41.185868 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:41.185868 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:41.185868 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:41.185868 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:38:41.185868 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:38:41.185868 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 14:38:41.185868 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:38:41.189962 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 365/776] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:38:40 2022] GET /identity/v3/services/network => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:38:41.194417 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d774219-bbcd-412e-a67d-ec4e976a2b37 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:41.235279 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:41.322528 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d774219-bbcd-412e-a67d-ec4e976a2b37 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:41.360275 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb335b1-f9e4-4fb5-96af-4589631f644a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:41.361012 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb335b1-f9e4-4fb5-96af-4589631f644a None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:41.361298 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb335b1-f9e4-4fb5-96af-4589631f644a None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:41.361575 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb335b1-f9e4-4fb5-96af-4589631f644a None admin] PATH_INFO: `/v3/services` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:41.362364 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb335b1-f9e4-4fb5-96af-4589631f644a None admin] RBAC: Adding query filter params (name=network) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:41.362674 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb335b1-f9e4-4fb5-96af-4589631f644a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:41.364065 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb335b1-f9e4-4fb5-96af-4589631f644a None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:41.370231 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 412/777] 10.222.0.57 () {56 vars in 1112 bytes} [Thu May 19 14:38:41 2022] GET /identity/v3/services?name=network => generated 124 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:41.375815 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30993a5b-2cc1-43ec-86f5-08be202fed6c None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:41.521720 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30993a5b-2cc1-43ec-86f5-08be202fed6c None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:41.576106 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305a880c-cfdf-4250-b9c9-c266185d0651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:41.577207 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305a880c-cfdf-4250-b9c9-c266185d0651 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:41.577724 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305a880c-cfdf-4250-b9c9-c266185d0651 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:41.578289 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305a880c-cfdf-4250-b9c9-c266185d0651 None admin] PATH_INFO: `/v3/services` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:41.579277 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305a880c-cfdf-4250-b9c9-c266185d0651 None admin] RBAC: Adding query filter params (type=network) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:41.579860 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305a880c-cfdf-4250-b9c9-c266185d0651 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:41.581497 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305a880c-cfdf-4250-b9c9-c266185d0651 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:41.590316 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 366/778] 10.222.0.57 () {56 vars in 1112 bytes} [Thu May 19 14:38:41 2022] GET /identity/v3/services?type=network => generated 350 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:41.596183 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb335b1-f9e4-4fb5-96af-4589631f644a None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:41.719033 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:41.722057 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb335b1-f9e4-4fb5-96af-4589631f644a None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:41.768189 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3ff4b7-f810-4057-a5d6-02062407738b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:41.768854 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3ff4b7-f810-4057-a5d6-02062407738b None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:41.769138 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3ff4b7-f810-4057-a5d6-02062407738b None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:41.769415 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3ff4b7-f810-4057-a5d6-02062407738b None admin] PATH_INFO: `/v3/endpoints` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:41.770244 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3ff4b7-f810-4057-a5d6-02062407738b None admin] RBAC: Adding query filter params (interface=public, service_id=9c71693e95c54eae90beb3709e4d200e, region_id=RegionOne) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:41.770556 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3ff4b7-f810-4057-a5d6-02062407738b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:41.771996 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3ff4b7-f810-4057-a5d6-02062407738b None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:41.777632 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 413/779] 10.222.0.57 () {56 vars in 1251 bytes} [Thu May 19 14:38:41 2022] GET /identity/v3/endpoints?service_id=9c71693e95c54eae90beb3709e4d200e&interface=public®ion_id=RegionOne => generated 194 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:41.778058 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83734e0a-dad6-4395-baf3-1dc63942a575 None None] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:41.778917 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83734e0a-dad6-4395-baf3-1dc63942a575 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:41.779341 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83734e0a-dad6-4395-baf3-1dc63942a575 None None] PATH_INFO: `/` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:41.781091 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 367/780] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:38:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:38:41.785951 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3ff4b7-f810-4057-a5d6-02062407738b None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:41.787548 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381f50e8-cfc9-4262-b3b4-4597d1e37fd9 None None] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:41.788370 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381f50e8-cfc9-4262-b3b4-4597d1e37fd9 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:41.788682 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381f50e8-cfc9-4262-b3b4-4597d1e37fd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:41.830890 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.auth.core [None req-381f50e8-cfc9-4262-b3b4-4597d1e37fd9 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:38:41.832100 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-381f50e8-cfc9-4262-b3b4-4597d1e37fd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:38:41.842216 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:41.895214 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3ff4b7-f810-4057-a5d6-02062407738b None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:41.943593 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369380e6-4d87-4763-9ca0-70715fbe8c84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:41.944892 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369380e6-4d87-4763-9ca0-70715fbe8c84 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:41.945178 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369380e6-4d87-4763-9ca0-70715fbe8c84 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:41.945614 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369380e6-4d87-4763-9ca0-70715fbe8c84 None admin] PATH_INFO: `/v3/services` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:41.946338 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369380e6-4d87-4763-9ca0-70715fbe8c84 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:41.947738 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369380e6-4d87-4763-9ca0-70715fbe8c84 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:41.962189 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 414/781] 10.222.0.57 () {56 vars in 1087 bytes} [Thu May 19 14:38:41 2022] GET /identity/v3/services => generated 772 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:41.982185 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 368/782] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:38:41 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:38:42.017521 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369380e6-4d87-4763-9ca0-70715fbe8c84 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:42.066032 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:42.173639 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369380e6-4d87-4763-9ca0-70715fbe8c84 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:42.209285 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f65c2f9-c222-4900-a114-76059d654fa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:42.210928 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f65c2f9-c222-4900-a114-76059d654fa4 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:42.211265 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f65c2f9-c222-4900-a114-76059d654fa4 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:42.211727 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f65c2f9-c222-4900-a114-76059d654fa4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:42.222888 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f65c2f9-c222-4900-a114-76059d654fa4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:38:42.231495 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-7f65c2f9-c222-4900-a114-76059d654fa4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:38:42.231495 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:42.231495 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:42.231495 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:42.231495 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:42.231495 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:42.231495 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:42.231495 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:42.231495 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:42.231495 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:42.231495 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:42.231495 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:42.231495 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:38:42.231495 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:38:42.231495 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:38:42.231495 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:38:42.232527 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:38:42.232527 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:38:42.232527 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:38:42.232527 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:38:42.232527 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:42.232527 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:42.232527 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:42.232527 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:42.232527 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:42.232527 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:42.232527 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:42.232527 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:38:42.232527 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:42.232527 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:42.232527 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:42.232527 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:42.233464 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:42.233464 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:42.233464 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:42.233464 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:42.233464 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:38:42.233464 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:38:42.233464 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:38:42.233464 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:38:42.233464 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:38:42.233464 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:38:42.233464 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:38:42.233464 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:38:42.233464 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 415/783] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:38:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:42.243406 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381f50e8-cfc9-4262-b3b4-4597d1e37fd9 None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:42.356414 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381f50e8-cfc9-4262-b3b4-4597d1e37fd9 None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:42.393648 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d9c730-1874-44af-b2f8-b502856cc98e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:42.394349 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d9c730-1874-44af-b2f8-b502856cc98e None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:42.394707 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d9c730-1874-44af-b2f8-b502856cc98e None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:42.394988 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d9c730-1874-44af-b2f8-b502856cc98e None admin] PATH_INFO: `/v3/roles` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:42.395745 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d9c730-1874-44af-b2f8-b502856cc98e None admin] RBAC: Adding query filter params (name=admin) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:42.396054 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d9c730-1874-44af-b2f8-b502856cc98e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:42.398105 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d9c730-1874-44af-b2f8-b502856cc98e None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:42.407272 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 369/784] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:38:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:42.419235 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f65c2f9-c222-4900-a114-76059d654fa4 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:42.566294 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:42.579439 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f65c2f9-c222-4900-a114-76059d654fa4 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:42.634619 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448e9e54-c306-4422-80ec-fea65a4e364a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:42.635446 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448e9e54-c306-4422-80ec-fea65a4e364a None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:42.635740 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448e9e54-c306-4422-80ec-fea65a4e364a None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:42.636019 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448e9e54-c306-4422-80ec-fea65a4e364a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:42.653716 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448e9e54-c306-4422-80ec-fea65a4e364a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:42.655233 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448e9e54-c306-4422-80ec-fea65a4e364a None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:42.681175 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-448e9e54-c306-4422-80ec-fea65a4e364a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:42.681175 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:42.681175 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:42.681175 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:42.681175 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:42.681175 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:42.681175 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:42.681175 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:42.681175 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:42.681175 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:42.681175 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:42.681175 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:42.681175 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:38:42.681175 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:38:42.681175 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:38:42.682258 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:38:42.682258 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:42.682258 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:42.682258 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:42.682258 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:42.682258 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:42.682258 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:42.682258 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:42.682258 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:38:42.682258 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:42.682258 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:42.682258 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:42.682258 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:42.682258 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:42.682258 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:42.682258 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:42.682258 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:42.683233 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:38:42.683233 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:38:42.683233 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:42.683233 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:38:42.683233 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 416/785] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:38:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:42.688228 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d9c730-1874-44af-b2f8-b502856cc98e None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:42.691202 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:42.712450 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac8219b-46c1-4602-bc80-9572aed03c2d None None] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:42.712787 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac8219b-46c1-4602-bc80-9572aed03c2d None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:42.713018 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac8219b-46c1-4602-bc80-9572aed03c2d None None] PATH_INFO: `/` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:42.717866 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 417/786] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:38:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:38:42.735606 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef6532b-3351-4815-8f47-fcbb9eeee392 None None] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:42.736249 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef6532b-3351-4815-8f47-fcbb9eeee392 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:42.736249 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef6532b-3351-4815-8f47-fcbb9eeee392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:42.777457 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.auth.core [None req-eef6532b-3351-4815-8f47-fcbb9eeee392 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:38:42.779258 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.fernet_utils [None req-eef6532b-3351-4815-8f47-fcbb9eeee392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:38:42.836295 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d9c730-1874-44af-b2f8-b502856cc98e None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:42.871925 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5af98a-4978-4252-a0a8-6c2c264df650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:42.872593 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5af98a-4978-4252-a0a8-6c2c264df650 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:42.872881 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5af98a-4978-4252-a0a8-6c2c264df650 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:42.873224 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5af98a-4978-4252-a0a8-6c2c264df650 None admin] PATH_INFO: `/v3/domains` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:42.873994 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5af98a-4978-4252-a0a8-6c2c264df650 None admin] RBAC: Adding query filter params (name=Default) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:42.874453 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5af98a-4978-4252-a0a8-6c2c264df650 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:42.875842 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5af98a-4978-4252-a0a8-6c2c264df650 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:42.890260 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 370/787] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:38:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:42.895523 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5af98a-4978-4252-a0a8-6c2c264df650 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:42.932933 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 418/788] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:38:42 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 364 bytes (3 switches on core 0) May 19 14:38:42.967621 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef6532b-3351-4815-8f47-fcbb9eeee392 None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:43.034059 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:43.082386 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5af98a-4978-4252-a0a8-6c2c264df650 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:43.104328 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:43.151506 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce04d12-86a5-44d2-a3c2-96dc782fbfb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:43.152293 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce04d12-86a5-44d2-a3c2-96dc782fbfb2 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:43.152577 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce04d12-86a5-44d2-a3c2-96dc782fbfb2 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:43.153540 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce04d12-86a5-44d2-a3c2-96dc782fbfb2 None admin] PATH_INFO: `/v3/users/swift` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:43.164067 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce04d12-86a5-44d2-a3c2-96dc782fbfb2 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:43.165721 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce04d12-86a5-44d2-a3c2-96dc782fbfb2 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:43.170015 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef6532b-3351-4815-8f47-fcbb9eeee392 None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:43.195870 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-8ce04d12-86a5-44d2-a3c2-96dc782fbfb2 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 14:38:43.195870 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:43.195870 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:43.195870 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:43.195870 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:43.195870 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:43.195870 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:43.195870 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:43.195870 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:43.195870 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:43.195870 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:43.195870 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:43.195870 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:38:43.195870 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:38:43.195870 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:38:43.195870 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:38:43.196970 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:43.196970 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:43.196970 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:38:43.196970 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:38:43.196970 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:38:43.196970 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:38:43.196970 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:43.196970 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:43.196970 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:43.196970 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:43.196970 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:43.196970 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:38:43.196970 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:43.196970 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:43.196970 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:43.196970 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:43.196970 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:43.196970 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:43.198497 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:43.198497 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:43.198497 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:38:43.198497 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:38:43.198497 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:38:43.198497 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:38:43.198497 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:38:43.198497 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:38:43.198497 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 14:38:43.198497 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:38:43.199109 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 371/789] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:38:42 2022] GET /identity/v3/users/swift => generated 83 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:43.208082 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce04d12-86a5-44d2-a3c2-96dc782fbfb2 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:43.240458 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd5fa1e-b4f9-45d8-945b-2c794c143495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:43.241303 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd5fa1e-b4f9-45d8-945b-2c794c143495 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:43.241649 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd5fa1e-b4f9-45d8-945b-2c794c143495 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:43.241952 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd5fa1e-b4f9-45d8-945b-2c794c143495 None admin] PATH_INFO: `/v3/roles/service` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:43.262574 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd5fa1e-b4f9-45d8-945b-2c794c143495 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:38:43.271662 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-1fd5fa1e-b4f9-45d8-945b-2c794c143495 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:38:43.271662 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:43.271662 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:43.271662 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:43.271662 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:43.271662 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:43.271662 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:43.271662 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:43.271662 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:43.271662 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:43.271662 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:43.271662 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:43.271662 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:38:43.271662 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:38:43.271662 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:38:43.271662 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:38:43.273278 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:38:43.273278 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:38:43.273278 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:38:43.273278 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:38:43.273278 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:43.273278 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:43.273278 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:43.273278 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:43.273278 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:43.273278 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:43.273278 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:43.273278 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:38:43.273278 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:43.273278 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:43.273278 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:43.273278 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:43.274356 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:43.274356 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:43.274356 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:43.274356 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:43.274356 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:38:43.274356 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:38:43.274356 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:38:43.274356 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:38:43.274356 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:38:43.274356 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:38:43.274356 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:38:43.274356 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:38:43.274356 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 419/790] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 14:38:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:38:43.279520 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd5fa1e-b4f9-45d8-945b-2c794c143495 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:43.391029 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce04d12-86a5-44d2-a3c2-96dc782fbfb2 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:43.416528 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:43.430185 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff003dc9-ab1e-4ff9-a77f-a299b7ff5cc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:43.430864 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff003dc9-ab1e-4ff9-a77f-a299b7ff5cc3 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:43.431148 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff003dc9-ab1e-4ff9-a77f-a299b7ff5cc3 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:43.431425 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff003dc9-ab1e-4ff9-a77f-a299b7ff5cc3 None admin] PATH_INFO: `/v3/users` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:43.432279 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff003dc9-ab1e-4ff9-a77f-a299b7ff5cc3 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:43.432593 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff003dc9-ab1e-4ff9-a77f-a299b7ff5cc3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:43.434063 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff003dc9-ab1e-4ff9-a77f-a299b7ff5cc3 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:43.452854 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 372/791] 10.222.0.57 () {56 vars in 1135 bytes} [Thu May 19 14:38:43 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:43.462940 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff003dc9-ab1e-4ff9-a77f-a299b7ff5cc3 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:43.484393 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd5fa1e-b4f9-45d8-945b-2c794c143495 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:43.529125 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:43.546554 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65062476-3b47-4292-ac15-0a08491598d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:43.547308 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65062476-3b47-4292-ac15-0a08491598d6 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:43.547601 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65062476-3b47-4292-ac15-0a08491598d6 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:43.547882 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65062476-3b47-4292-ac15-0a08491598d6 None admin] PATH_INFO: `/v3/roles` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:43.548641 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65062476-3b47-4292-ac15-0a08491598d6 None admin] RBAC: Adding query filter params (name=service) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:43.548955 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65062476-3b47-4292-ac15-0a08491598d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:43.550383 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65062476-3b47-4292-ac15-0a08491598d6 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:43.567765 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 420/792] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 19 14:38:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:43.573531 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65062476-3b47-4292-ac15-0a08491598d6 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:43.578926 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff003dc9-ab1e-4ff9-a77f-a299b7ff5cc3 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:43.620127 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d9579b-bdb8-4c67-8c7d-213969394787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:43.621021 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d9579b-bdb8-4c67-8c7d-213969394787 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:43.621314 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d9579b-bdb8-4c67-8c7d-213969394787 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:43.621592 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d9579b-bdb8-4c67-8c7d-213969394787 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:43.634875 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d9579b-bdb8-4c67-8c7d-213969394787 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:43.636506 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d9579b-bdb8-4c67-8c7d-213969394787 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:43.657685 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-67d9579b-bdb8-4c67-8c7d-213969394787 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:43.657685 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:43.657685 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:43.657685 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:43.657685 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:43.657685 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:43.657685 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:43.657685 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:43.657685 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:43.657685 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:43.657685 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:43.657685 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:43.657685 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:38:43.657685 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:38:43.657685 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:38:43.658766 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:38:43.658766 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:43.658766 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:43.658766 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:43.658766 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:43.658766 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:43.658766 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:43.658766 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:43.658766 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:38:43.658766 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:43.658766 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:43.658766 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:43.658766 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:43.658766 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:43.658766 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:43.658766 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:43.658766 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:43.659771 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:38:43.659771 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:38:43.659771 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:43.659771 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:38:43.664383 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 373/793] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:38:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:43.669359 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d9579b-bdb8-4c67-8c7d-213969394787 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:43.734708 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:43.769917 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65062476-3b47-4292-ac15-0a08491598d6 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:43.813025 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6d24c5-31f9-4bea-9a4e-f412afeae2d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:43.813906 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6d24c5-31f9-4bea-9a4e-f412afeae2d9 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:43.814165 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6d24c5-31f9-4bea-9a4e-f412afeae2d9 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:43.814426 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6d24c5-31f9-4bea-9a4e-f412afeae2d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:43.827304 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6d24c5-31f9-4bea-9a4e-f412afeae2d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:43.828785 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6d24c5-31f9-4bea-9a4e-f412afeae2d9 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:43.850975 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d9579b-bdb8-4c67-8c7d-213969394787 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:43.854566 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-bb6d24c5-31f9-4bea-9a4e-f412afeae2d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:43.854566 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:43.854566 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:43.854566 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:43.854566 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:43.854566 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:43.854566 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:43.854566 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:43.854566 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:43.854566 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:43.854566 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:43.854566 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:43.854566 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:38:43.854566 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:38:43.854566 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:38:43.855554 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:38:43.855554 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:43.855554 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:43.855554 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:43.855554 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:43.855554 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:43.855554 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:43.855554 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:43.855554 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:38:43.855554 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:43.855554 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:43.855554 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:43.855554 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:43.855554 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:43.855554 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:43.855554 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:43.855554 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:43.856541 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:38:43.856541 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:38:43.856541 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:43.856541 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:38:43.856541 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 421/794] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:38:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:38:43.862190 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99380541-0dba-4c06-ab11-fb8dae339aa4 None None] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:43.862190 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99380541-0dba-4c06-ab11-fb8dae339aa4 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:43.862190 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99380541-0dba-4c06-ab11-fb8dae339aa4 None None] PATH_INFO: `/` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:43.862190 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 422/795] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:38:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:38:43.864865 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99380541-0dba-4c06-ab11-fb8dae339aa4 None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:43.896736 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aa3cc8-05b4-45b1-ac6a-9385216fc8f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:43.897465 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aa3cc8-05b4-45b1-ac6a-9385216fc8f7 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:43.897797 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aa3cc8-05b4-45b1-ac6a-9385216fc8f7 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:43.898109 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aa3cc8-05b4-45b1-ac6a-9385216fc8f7 None admin] PATH_INFO: `/v3/domains` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:43.898836 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29aa3cc8-05b4-45b1-ac6a-9385216fc8f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:43.899149 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29aa3cc8-05b4-45b1-ac6a-9385216fc8f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:43.900524 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29aa3cc8-05b4-45b1-ac6a-9385216fc8f7 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:43.911991 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 374/796] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:38:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:43.913484 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64a2d5d-198c-443b-9acd-7cc40afb6b64 None None] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:43.913945 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64a2d5d-198c-443b-9acd-7cc40afb6b64 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:43.914236 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64a2d5d-198c-443b-9acd-7cc40afb6b64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:43.919518 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:43.933573 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.auth.core [None req-f64a2d5d-198c-443b-9acd-7cc40afb6b64 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:38:43.934820 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-f64a2d5d-198c-443b-9acd-7cc40afb6b64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:38:43.982226 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99380541-0dba-4c06-ab11-fb8dae339aa4 None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:44.005127 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 375/797] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:38:43 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:38:44.007336 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02aa6fc-693b-47ee-8542-bd77d8c61acc None None] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:44.007336 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02aa6fc-693b-47ee-8542-bd77d8c61acc None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:44.007608 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02aa6fc-693b-47ee-8542-bd77d8c61acc None None] PATH_INFO: `/` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:44.009274 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 376/798] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:38:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:38:44.013999 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02aa6fc-693b-47ee-8542-bd77d8c61acc None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:44.033141 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13373d20-71eb-441a-b608-5d9fad3ed2fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:44.033815 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13373d20-71eb-441a-b608-5d9fad3ed2fd None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:44.034141 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13373d20-71eb-441a-b608-5d9fad3ed2fd None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:44.034415 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13373d20-71eb-441a-b608-5d9fad3ed2fd None admin] PATH_INFO: `/v3/domains` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:44.035139 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13373d20-71eb-441a-b608-5d9fad3ed2fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:44.035446 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13373d20-71eb-441a-b608-5d9fad3ed2fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:44.036820 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13373d20-71eb-441a-b608-5d9fad3ed2fd None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:44.041299 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:44.050150 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 423/799] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:38:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:44.051526 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7385bb91-3393-4854-9c65-d9382f07d796 None None] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:44.051818 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7385bb91-3393-4854-9c65-d9382f07d796 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:44.052091 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7385bb91-3393-4854-9c65-d9382f07d796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:44.070131 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.auth.core [None req-7385bb91-3393-4854-9c65-d9382f07d796 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:38:44.071558 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.fernet_utils [None req-7385bb91-3393-4854-9c65-d9382f07d796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:38:44.137085 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02aa6fc-693b-47ee-8542-bd77d8c61acc None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:44.145383 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 424/800] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:38:44 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:38:44.146513 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7385bb91-3393-4854-9c65-d9382f07d796 None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:44.186095 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe8f996-815c-4548-a410-eef5d3376d1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:44.186915 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe8f996-815c-4548-a410-eef5d3376d1f None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:44.187285 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe8f996-815c-4548-a410-eef5d3376d1f None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:44.187630 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe8f996-815c-4548-a410-eef5d3376d1f None admin] PATH_INFO: `/v3/projects/service` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:44.195730 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:44.195897 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe8f996-815c-4548-a410-eef5d3376d1f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:44.197356 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe8f996-815c-4548-a410-eef5d3376d1f None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:44.213297 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-6fe8f996-815c-4548-a410-eef5d3376d1f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:38:44.213297 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:44.213297 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:44.213297 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:44.213297 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:44.213297 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:44.213297 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:44.213297 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:44.213297 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:44.213297 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:44.213297 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:44.213297 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:44.213297 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:38:44.213297 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:38:44.213297 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:38:44.214396 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:38:44.214396 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:44.214396 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:44.214396 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:44.214396 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:44.214396 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:44.214396 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:44.214396 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:44.214396 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:38:44.214396 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:44.214396 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:44.214396 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:44.214396 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:44.214396 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:44.214396 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:44.214396 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:44.214396 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:44.215395 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:38:44.215395 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:38:44.215395 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:38:44.215395 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:38:44.215395 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:38:44.215395 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:38:44.215395 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:38:44.215395 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:38:44.215395 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 377/801] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:38:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:44.215395 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe8f996-815c-4548-a410-eef5d3376d1f None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:44.276727 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7385bb91-3393-4854-9c65-d9382f07d796 None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:44.308936 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:44.314414 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d34103-8022-49b7-8de3-5679c23c8df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:44.315220 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d34103-8022-49b7-8de3-5679c23c8df8 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:44.315500 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d34103-8022-49b7-8de3-5679c23c8df8 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:44.315855 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d34103-8022-49b7-8de3-5679c23c8df8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:44.327354 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d34103-8022-49b7-8de3-5679c23c8df8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:38:44.330313 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe8f996-815c-4548-a410-eef5d3376d1f None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:44.335877 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-e3d34103-8022-49b7-8de3-5679c23c8df8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:38:44.335877 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:44.335877 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:44.335877 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:44.335877 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:44.335877 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:44.335877 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:44.335877 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:44.335877 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:44.335877 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:44.335877 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:44.335877 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:44.335877 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:38:44.335877 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:38:44.335877 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:38:44.335877 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:38:44.336888 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:38:44.336888 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:38:44.336888 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:38:44.336888 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:38:44.336888 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:44.336888 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:44.336888 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:44.336888 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:44.336888 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:44.336888 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:44.336888 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:44.336888 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:38:44.336888 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:44.336888 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:44.336888 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:44.336888 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:44.343346 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:44.343346 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:44.343346 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:44.343346 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:44.343346 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:38:44.343346 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:38:44.343346 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:38:44.343346 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:38:44.343346 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:38:44.343346 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:38:44.343346 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:38:44.343346 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:38:44.343346 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 425/802] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:38:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:44.343346 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d34103-8022-49b7-8de3-5679c23c8df8 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:44.376156 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762acfc5-3791-49e8-b942-0155292b9aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:44.376982 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762acfc5-3791-49e8-b942-0155292b9aa3 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:44.377271 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762acfc5-3791-49e8-b942-0155292b9aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:44.377556 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762acfc5-3791-49e8-b942-0155292b9aa3 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:44.385569 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762acfc5-3791-49e8-b942-0155292b9aa3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:44.387062 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762acfc5-3791-49e8-b942-0155292b9aa3 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:44.402460 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-762acfc5-3791-49e8-b942-0155292b9aa3 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 14:38:44.402460 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:44.402460 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:44.402460 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:44.402460 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:44.402460 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:44.402460 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:44.402460 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:44.402460 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:44.402460 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:44.402460 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:44.402460 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:44.402460 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:38:44.402460 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:38:44.402460 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:38:44.402460 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:38:44.403552 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:44.403552 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:44.403552 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:38:44.403552 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:38:44.403552 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:38:44.403552 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:38:44.403552 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:44.403552 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:44.403552 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:44.403552 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:44.403552 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:44.403552 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:38:44.403552 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:44.403552 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:44.403552 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:44.403552 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:44.403552 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:44.403552 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:44.404583 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:44.404583 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:44.404583 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:38:44.404583 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:38:44.404583 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:38:44.404583 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:38:44.404583 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:38:44.404583 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:38:44.404583 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 14:38:44.404583 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:38:44.404583 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 378/803] 10.222.0.57 () {56 vars in 1117 bytes} [Thu May 19 14:38:44 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:44.404583 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762acfc5-3791-49e8-b942-0155292b9aa3 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:44.451467 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:44.461027 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d34103-8022-49b7-8de3-5679c23c8df8 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:44.497992 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d72195-475a-443a-8715-57eb46a061b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:44.498812 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d72195-475a-443a-8715-57eb46a061b6 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:44.499097 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d72195-475a-443a-8715-57eb46a061b6 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:44.499377 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d72195-475a-443a-8715-57eb46a061b6 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:44.500043 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d72195-475a-443a-8715-57eb46a061b6 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:44.501439 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d72195-475a-443a-8715-57eb46a061b6 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:44.514067 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-91d72195-475a-443a-8715-57eb46a061b6 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 14:38:44.514067 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:44.514067 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:38:44.514067 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:38:44.514067 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:38:44.514067 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:38:44.514067 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:38:44.514067 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:38:44.514067 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 14:38:44.514067 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:38:44.514067 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:38:44.514067 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:38:44.514067 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:44.514067 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:44.514067 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:44.514067 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:44.515360 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:44.515360 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:44.515360 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:44.515360 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:44.515360 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:44.515360 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:44.515360 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:44.515360 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:38:44.515360 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:38:44.515360 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:38:44.515360 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:38:44.515360 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:44.515360 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:44.515360 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:44.515360 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:44.515360 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:44.516314 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:44.516314 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:44.516314 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:38:44.516314 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:44.516314 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:44.516314 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:44.516314 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:44.516314 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:44.516314 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:44.516314 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:44.516314 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:44.516314 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:38:44.516314 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:38:44.516314 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 14:38:44.516314 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:38:44.516314 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 426/804] 10.222.0.57 () {56 vars in 1129 bytes} [Thu May 19 14:38:44 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:44.517256 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d72195-475a-443a-8715-57eb46a061b6 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:44.531335 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762acfc5-3791-49e8-b942-0155292b9aa3 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:44.573463 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c43922-5db5-4b8f-a97d-8f5312abdf52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:44.574042 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c43922-5db5-4b8f-a97d-8f5312abdf52 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:44.574333 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c43922-5db5-4b8f-a97d-8f5312abdf52 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:44.574645 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c43922-5db5-4b8f-a97d-8f5312abdf52 None admin] PATH_INFO: `/v3/projects` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:44.575413 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c43922-5db5-4b8f-a97d-8f5312abdf52 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:44.575729 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c43922-5db5-4b8f-a97d-8f5312abdf52 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:44.577172 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c43922-5db5-4b8f-a97d-8f5312abdf52 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:44.591187 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 379/805] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:38:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:44.591955 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c43922-5db5-4b8f-a97d-8f5312abdf52 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:44.594787 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:44.638731 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d72195-475a-443a-8715-57eb46a061b6 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:44.675831 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46da3774-b438-49de-9b40-6934fee6baf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:44.676510 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46da3774-b438-49de-9b40-6934fee6baf4 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:44.676796 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46da3774-b438-49de-9b40-6934fee6baf4 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:44.677076 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46da3774-b438-49de-9b40-6934fee6baf4 None admin] PATH_INFO: `/v3/roles` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:44.677825 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46da3774-b438-49de-9b40-6934fee6baf4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:44.678169 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46da3774-b438-49de-9b40-6934fee6baf4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:44.680399 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46da3774-b438-49de-9b40-6934fee6baf4 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:44.689124 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 427/806] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:38:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:44.689974 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46da3774-b438-49de-9b40-6934fee6baf4 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:44.707669 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c43922-5db5-4b8f-a97d-8f5312abdf52 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:44.716647 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:44.744351 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b0ea16-3a8f-4fe1-8e52-1f7960f0bd18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:44.745027 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b0ea16-3a8f-4fe1-8e52-1f7960f0bd18 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:44.745315 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b0ea16-3a8f-4fe1-8e52-1f7960f0bd18 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:44.745620 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b0ea16-3a8f-4fe1-8e52-1f7960f0bd18 None admin] PATH_INFO: `/v3/users` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:44.746529 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b0ea16-3a8f-4fe1-8e52-1f7960f0bd18 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:44.746844 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b0ea16-3a8f-4fe1-8e52-1f7960f0bd18 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:44.748297 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b0ea16-3a8f-4fe1-8e52-1f7960f0bd18 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:44.767055 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 380/807] 10.222.0.57 () {56 vars in 1149 bytes} [Thu May 19 14:38:44 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:44.767867 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b0ea16-3a8f-4fe1-8e52-1f7960f0bd18 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:44.804417 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46da3774-b438-49de-9b40-6934fee6baf4 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:44.827269 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:44.837884 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f60719-996f-4bc1-822b-bc5904061c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:44.838553 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f60719-996f-4bc1-822b-bc5904061c83 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:44.838837 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f60719-996f-4bc1-822b-bc5904061c83 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:44.839111 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f60719-996f-4bc1-822b-bc5904061c83 None admin] PATH_INFO: `/v3/services` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:44.839813 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f60719-996f-4bc1-822b-bc5904061c83 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:44.840119 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f60719-996f-4bc1-822b-bc5904061c83 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:44.841495 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f60719-996f-4bc1-822b-bc5904061c83 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:44.845942 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 428/808] 10.222.0.57 () {56 vars in 1124 bytes} [Thu May 19 14:38:44 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:44.846721 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f60719-996f-4bc1-822b-bc5904061c83 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:44.871711 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b0ea16-3a8f-4fe1-8e52-1f7960f0bd18 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:44.905198 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1f2182-de63-49c9-bb64-4ac13f8c8a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:44.906118 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1f2182-de63-49c9-bb64-4ac13f8c8a9b None admin] REQUEST_METHOD: `PUT` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:44.906393 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1f2182-de63-49c9-bb64-4ac13f8c8a9b None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:44.906662 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1f2182-de63-49c9-bb64-4ac13f8c8a9b None admin] PATH_INFO: `/v3/projects/42f35b96b9d4499abfb5e5f7cca49adc/users/4f8e2e0909074a85bb493c49ac3a6d4d/roles/c2935b1a25ca46fcba419fe3d1715330` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:44.908609 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1f2182-de63-49c9-bb64-4ac13f8c8a9b None admin] RBAC: Authorizing `identity:create_grant(project_id=42f35b96b9d4499abfb5e5f7cca49adc, user_id=4f8e2e0909074a85bb493c49ac3a6d4d, role_id=c2935b1a25ca46fcba419fe3d1715330)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:44.910057 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1f2182-de63-49c9-bb64-4ac13f8c8a9b None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:44.923113 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 381/809] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 14:38:44 2022] PUT /identity/v3/projects/42f35b96b9d4499abfb5e5f7cca49adc/users/4f8e2e0909074a85bb493c49ac3a6d4d/roles/c2935b1a25ca46fcba419fe3d1715330 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:38:44.924361 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b72f65f-3ae0-43f3-a69f-d1aa07dbcb35 None None] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:44.924649 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b72f65f-3ae0-43f3-a69f-d1aa07dbcb35 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:44.924928 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b72f65f-3ae0-43f3-a69f-d1aa07dbcb35 None None] PATH_INFO: `/` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:44.926425 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 382/810] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:38:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:38:44.926928 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b72f65f-3ae0-43f3-a69f-d1aa07dbcb35 None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:44.935180 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:44.954147 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f60719-996f-4bc1-822b-bc5904061c83 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:44.998876 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e159269-687b-4b8f-9286-e74bd999bb04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:44.999732 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e159269-687b-4b8f-9286-e74bd999bb04 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:45.000016 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e159269-687b-4b8f-9286-e74bd999bb04 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:45.000297 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e159269-687b-4b8f-9286-e74bd999bb04 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:45.012779 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e159269-687b-4b8f-9286-e74bd999bb04 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:45.014305 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e159269-687b-4b8f-9286-e74bd999bb04 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:45.033554 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-2e159269-687b-4b8f-9286-e74bd999bb04 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:45.033554 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:45.033554 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:45.033554 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:45.033554 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:45.033554 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:45.033554 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:45.033554 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:45.033554 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:45.033554 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:45.033554 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:45.033554 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:45.033554 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:38:45.033554 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:38:45.033554 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:38:45.034907 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:38:45.034907 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:45.034907 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:45.034907 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:45.034907 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:45.034907 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:45.034907 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:45.034907 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:45.034907 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:38:45.034907 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:45.034907 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:45.034907 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:45.034907 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:45.034907 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:45.034907 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:45.034907 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:45.034907 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:45.036126 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:38:45.036126 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:38:45.036126 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:45.036126 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:38:45.040243 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 429/811] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:38:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:45.041048 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e159269-687b-4b8f-9286-e74bd999bb04 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:45.067311 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b72f65f-3ae0-43f3-a69f-d1aa07dbcb35 None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:45.090600 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:45.109127 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64024a54-ead8-4770-b0c1-56d5f377b023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:45.109975 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64024a54-ead8-4770-b0c1-56d5f377b023 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:45.110265 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64024a54-ead8-4770-b0c1-56d5f377b023 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:45.110544 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64024a54-ead8-4770-b0c1-56d5f377b023 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:45.124404 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64024a54-ead8-4770-b0c1-56d5f377b023 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:45.125960 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64024a54-ead8-4770-b0c1-56d5f377b023 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:45.145622 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-64024a54-ead8-4770-b0c1-56d5f377b023 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:45.145622 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:45.145622 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:45.145622 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:45.145622 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:45.145622 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:45.145622 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:45.145622 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:45.145622 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:45.145622 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:45.145622 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:45.145622 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:45.145622 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:38:45.145622 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:38:45.145622 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:38:45.146777 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:38:45.146777 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:45.146777 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:45.146777 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:45.146777 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:45.146777 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:45.146777 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:45.146777 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:45.146777 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:38:45.146777 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:45.146777 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:45.146777 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:45.146777 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:45.146777 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:45.146777 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:45.146777 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:45.146777 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:45.148322 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:38:45.148322 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:38:45.148322 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:45.148322 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:38:45.148322 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 383/812] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:38:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:45.149397 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa5369c-8316-4c3c-bc71-4c8bccc21281 None None] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:45.149693 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa5369c-8316-4c3c-bc71-4c8bccc21281 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:45.150798 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa5369c-8316-4c3c-bc71-4c8bccc21281 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:45.174793 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.auth.core [None req-cfa5369c-8316-4c3c-bc71-4c8bccc21281 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:38:45.175442 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-cfa5369c-8316-4c3c-bc71-4c8bccc21281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:38:45.187928 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e159269-687b-4b8f-9286-e74bd999bb04 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:45.223855 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec97792-34d0-4635-b2e1-3b5c9866ade4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:45.224648 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec97792-34d0-4635-b2e1-3b5c9866ade4 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:45.224934 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec97792-34d0-4635-b2e1-3b5c9866ade4 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:45.226029 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec97792-34d0-4635-b2e1-3b5c9866ade4 None admin] PATH_INFO: `/v3/services` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:45.226734 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec97792-34d0-4635-b2e1-3b5c9866ade4 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:45.227045 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec97792-34d0-4635-b2e1-3b5c9866ade4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:45.228567 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec97792-34d0-4635-b2e1-3b5c9866ade4 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:45.231213 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:45.240431 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 430/813] 10.222.0.57 () {56 vars in 1124 bytes} [Thu May 19 14:38:45 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:45.241383 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec97792-34d0-4635-b2e1-3b5c9866ade4 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:45.257946 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 384/814] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:38:45 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:38:45.257946 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa5369c-8316-4c3c-bc71-4c8bccc21281 None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:45.365677 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec97792-34d0-4635-b2e1-3b5c9866ade4 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:45.368149 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa5369c-8316-4c3c-bc71-4c8bccc21281 None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:45.370760 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:45.403205 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275da9a0-28f6-4660-88a5-be339563a413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:45.403877 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275da9a0-28f6-4660-88a5-be339563a413 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:45.404419 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275da9a0-28f6-4660-88a5-be339563a413 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:45.404419 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275da9a0-28f6-4660-88a5-be339563a413 None admin] PATH_INFO: `/v3/domains` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:45.405284 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275da9a0-28f6-4660-88a5-be339563a413 None admin] RBAC: Adding query filter params (name=Default) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:45.405598 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275da9a0-28f6-4660-88a5-be339563a413 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:45.407015 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275da9a0-28f6-4660-88a5-be339563a413 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:45.413265 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e649f4-9941-4f87-826f-cb5e1f10ae6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:45.414038 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e649f4-9941-4f87-826f-cb5e1f10ae6a None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:45.414250 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e649f4-9941-4f87-826f-cb5e1f10ae6a None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:45.414535 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e649f4-9941-4f87-826f-cb5e1f10ae6a None admin] PATH_INFO: `/v3/domains` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:45.415269 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e649f4-9941-4f87-826f-cb5e1f10ae6a None admin] RBAC: Adding query filter params (name=Default) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:45.415586 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e649f4-9941-4f87-826f-cb5e1f10ae6a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:45.416967 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e649f4-9941-4f87-826f-cb5e1f10ae6a None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:45.424796 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 431/815] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:38:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:45.424796 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275da9a0-28f6-4660-88a5-be339563a413 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:45.429743 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 385/816] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:38:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:45.430502 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e649f4-9941-4f87-826f-cb5e1f10ae6a None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:45.499207 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:45.541561 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e649f4-9941-4f87-826f-cb5e1f10ae6a None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:45.544745 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275da9a0-28f6-4660-88a5-be339563a413 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:45.579532 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ec5f7d-7134-4ec2-9b98-f42bc597589d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:45.580358 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ec5f7d-7134-4ec2-9b98-f42bc597589d None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:45.580643 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ec5f7d-7134-4ec2-9b98-f42bc597589d None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:45.580917 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ec5f7d-7134-4ec2-9b98-f42bc597589d None admin] PATH_INFO: `/v3/services/network` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:45.581044 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541949fa-80d4-4523-8791-6613de2f1a82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:45.581572 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ec5f7d-7134-4ec2-9b98-f42bc597589d None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:45.581675 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541949fa-80d4-4523-8791-6613de2f1a82 None admin] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:45.581938 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541949fa-80d4-4523-8791-6613de2f1a82 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:45.582220 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541949fa-80d4-4523-8791-6613de2f1a82 None admin] PATH_INFO: `/v3/endpoints` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:45.582960 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541949fa-80d4-4523-8791-6613de2f1a82 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:45.583065 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ec5f7d-7134-4ec2-9b98-f42bc597589d None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:45.584329 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541949fa-80d4-4523-8791-6613de2f1a82 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:45.596097 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-09ec5f7d-7134-4ec2-9b98-f42bc597589d None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 19 14:38:45.596097 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:45.596097 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:38:45.596097 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:38:45.596097 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:38:45.596097 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:38:45.596097 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:38:45.596097 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:38:45.596097 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 14:38:45.596097 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:38:45.596097 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:38:45.596097 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:38:45.596097 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:45.596097 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:45.596097 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:45.596097 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:45.596097 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:45.597240 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:45.597240 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:45.597240 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:45.597240 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:45.597240 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:45.597240 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:45.597240 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:38:45.597240 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:38:45.597240 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:38:45.597240 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:38:45.597240 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:45.597240 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:45.597240 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:45.597240 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:45.597240 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:45.597240 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:45.598199 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:45.598199 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:38:45.598199 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:45.598199 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:45.598199 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:45.598199 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:45.598199 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:45.598199 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:45.598199 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:45.598199 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:45.598199 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:38:45.598199 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:38:45.598199 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 14:38:45.598199 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:38:45.598199 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 386/817] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:38:45 2022] GET /identity/v3/services/network => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:45.599815 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ec5f7d-7134-4ec2-9b98-f42bc597589d None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:45.613053 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 432/818] 10.222.0.57 () {60 vars in 1144 bytes} [Thu May 19 14:38:45 2022] POST /identity/v3/endpoints => generated 346 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:38:45.622969 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541949fa-80d4-4523-8791-6613de2f1a82 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:45.635749 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:45.726739 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ec5f7d-7134-4ec2-9b98-f42bc597589d None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:45.780007 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e5ebfc-ffa1-4a8d-b5cb-28b409098bb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:45.782189 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e5ebfc-ffa1-4a8d-b5cb-28b409098bb0 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:45.783027 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e5ebfc-ffa1-4a8d-b5cb-28b409098bb0 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:45.783560 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e5ebfc-ffa1-4a8d-b5cb-28b409098bb0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:45.785266 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541949fa-80d4-4523-8791-6613de2f1a82 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:45.795796 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e5ebfc-ffa1-4a8d-b5cb-28b409098bb0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:45.798646 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e5ebfc-ffa1-4a8d-b5cb-28b409098bb0 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:45.806600 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:45.820233 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-a9e5ebfc-ffa1-4a8d-b5cb-28b409098bb0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 14:38:45.820233 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:45.820233 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:45.820233 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:45.820233 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:45.820233 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:45.820233 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:45.820233 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:45.820233 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:45.820233 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:45.820233 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:45.820233 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:45.820233 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:38:45.820233 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:38:45.820233 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:38:45.820233 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:38:45.821497 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:45.821497 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:45.821497 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:38:45.821497 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:38:45.821497 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:38:45.821497 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:38:45.821497 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:45.821497 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:45.821497 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:45.821497 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:45.821497 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:45.821497 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:38:45.821497 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:45.821497 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:45.821497 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:45.821497 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:45.821497 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:45.821497 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:45.822845 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:45.822845 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:45.822845 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:38:45.822845 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:38:45.822845 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:38:45.822845 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:38:45.822845 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:38:45.822845 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:38:45.822845 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 14:38:45.822845 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:38:45.825925 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 387/819] 10.222.0.57 () {56 vars in 1093 bytes} [Thu May 19 14:38:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:45.827048 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e5ebfc-ffa1-4a8d-b5cb-28b409098bb0 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:45.833197 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49614fd8-3ea3-4b72-a479-41b0a40ffead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:45.834395 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49614fd8-3ea3-4b72-a479-41b0a40ffead None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:45.835647 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49614fd8-3ea3-4b72-a479-41b0a40ffead None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:45.835931 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49614fd8-3ea3-4b72-a479-41b0a40ffead None admin] PATH_INFO: `/v3/projects/service` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:45.843546 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49614fd8-3ea3-4b72-a479-41b0a40ffead None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:45.845108 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49614fd8-3ea3-4b72-a479-41b0a40ffead None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:45.862210 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-49614fd8-3ea3-4b72-a479-41b0a40ffead None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:38:45.862210 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:45.862210 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:45.862210 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:45.862210 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:45.862210 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:45.862210 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:45.862210 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:45.862210 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:45.862210 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:45.862210 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:45.862210 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:45.862210 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:38:45.862210 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:38:45.862210 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:38:45.863408 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:38:45.863408 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:45.863408 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:45.863408 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:45.863408 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:45.863408 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:45.863408 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:45.863408 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:45.863408 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:38:45.863408 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:45.863408 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:45.863408 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:45.863408 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:45.863408 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:45.863408 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:45.863408 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:45.863408 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:45.864637 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:38:45.864637 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:38:45.864637 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:38:45.864637 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:38:45.864637 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:38:45.864637 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:38:45.864637 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:38:45.864637 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:38:45.864637 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 433/820] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:38:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:45.864637 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49614fd8-3ea3-4b72-a479-41b0a40ffead None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:45.937288 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:45.959939 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e5ebfc-ffa1-4a8d-b5cb-28b409098bb0 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:45.999235 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49614fd8-3ea3-4b72-a479-41b0a40ffead None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:45.999774 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce01b1de-0cc3-4fbd-8f70-761e87b4c73e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:46.000601 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce01b1de-0cc3-4fbd-8f70-761e87b4c73e None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:46.000892 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce01b1de-0cc3-4fbd-8f70-761e87b4c73e None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:46.001175 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce01b1de-0cc3-4fbd-8f70-761e87b4c73e None admin] PATH_INFO: `/v3/services` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:46.001916 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce01b1de-0cc3-4fbd-8f70-761e87b4c73e None admin] RBAC: Adding query filter params (name=network) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:46.002236 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce01b1de-0cc3-4fbd-8f70-761e87b4c73e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:46.003680 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce01b1de-0cc3-4fbd-8f70-761e87b4c73e None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:46.009663 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 388/821] 10.222.0.57 () {56 vars in 1112 bytes} [Thu May 19 14:38:45 2022] GET /identity/v3/services?name=network => generated 124 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:46.014688 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce01b1de-0cc3-4fbd-8f70-761e87b4c73e None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:46.051584 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14466142-f96f-455b-bdf5-40f560a127a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:46.053205 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14466142-f96f-455b-bdf5-40f560a127a2 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:46.054188 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14466142-f96f-455b-bdf5-40f560a127a2 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:46.054620 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14466142-f96f-455b-bdf5-40f560a127a2 None admin] PATH_INFO: `/v3/users` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:46.055624 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14466142-f96f-455b-bdf5-40f560a127a2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:46.056364 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14466142-f96f-455b-bdf5-40f560a127a2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:46.058914 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14466142-f96f-455b-bdf5-40f560a127a2 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:46.084033 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 434/822] 10.222.0.57 () {56 vars in 1133 bytes} [Thu May 19 14:38:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:46.086608 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14466142-f96f-455b-bdf5-40f560a127a2 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:46.135331 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:46.138981 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce01b1de-0cc3-4fbd-8f70-761e87b4c73e None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:46.185896 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff712f71-d1f1-4776-94bc-72f0ce6c0484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:46.185896 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff712f71-d1f1-4776-94bc-72f0ce6c0484 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:46.185896 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff712f71-d1f1-4776-94bc-72f0ce6c0484 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:46.185896 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff712f71-d1f1-4776-94bc-72f0ce6c0484 None admin] PATH_INFO: `/v3/projects` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:46.186377 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff712f71-d1f1-4776-94bc-72f0ce6c0484 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:46.186377 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff712f71-d1f1-4776-94bc-72f0ce6c0484 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:46.187058 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff712f71-d1f1-4776-94bc-72f0ce6c0484 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:46.206083 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 389/823] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:38:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:46.207037 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff712f71-d1f1-4776-94bc-72f0ce6c0484 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:46.218229 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:46.224195 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14466142-f96f-455b-bdf5-40f560a127a2 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:46.267127 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905ed051-ea72-40d8-821f-98d856c2e214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:46.268081 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905ed051-ea72-40d8-821f-98d856c2e214 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:46.268372 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905ed051-ea72-40d8-821f-98d856c2e214 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:46.268654 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905ed051-ea72-40d8-821f-98d856c2e214 None admin] PATH_INFO: `/v3/services` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:46.269793 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905ed051-ea72-40d8-821f-98d856c2e214 None admin] RBAC: Adding query filter params (type=network) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:46.270180 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905ed051-ea72-40d8-821f-98d856c2e214 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:46.271580 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905ed051-ea72-40d8-821f-98d856c2e214 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:46.284597 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 435/824] 10.222.0.57 () {56 vars in 1112 bytes} [Thu May 19 14:38:46 2022] GET /identity/v3/services?type=network => generated 350 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:46.285874 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905ed051-ea72-40d8-821f-98d856c2e214 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:46.338424 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff712f71-d1f1-4776-94bc-72f0ce6c0484 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:46.374018 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0b98db-e133-4f36-8eb8-5033d7f6c55a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:46.374975 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0b98db-e133-4f36-8eb8-5033d7f6c55a None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:46.375297 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0b98db-e133-4f36-8eb8-5033d7f6c55a None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:46.375581 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0b98db-e133-4f36-8eb8-5033d7f6c55a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:46.405509 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:46.409906 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905ed051-ea72-40d8-821f-98d856c2e214 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:46.450894 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b287fd-4ef3-4c0a-9347-a297ead0ac9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:46.452050 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b287fd-4ef3-4c0a-9347-a297ead0ac9a None admin] REQUEST_METHOD: `PUT` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:46.452459 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b287fd-4ef3-4c0a-9347-a297ead0ac9a None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:46.453133 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b287fd-4ef3-4c0a-9347-a297ead0ac9a None admin] PATH_INFO: `/v3/projects/42f35b96b9d4499abfb5e5f7cca49adc/users/2cbf483c6556421fa36179722100f8b9/roles/ffb8da90af53491980816087ee621ccb` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:46.455496 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0b98db-e133-4f36-8eb8-5033d7f6c55a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:46.457238 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0b98db-e133-4f36-8eb8-5033d7f6c55a None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:46.467102 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:46.471192 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b287fd-4ef3-4c0a-9347-a297ead0ac9a None admin] RBAC: Authorizing `identity:create_grant(project_id=42f35b96b9d4499abfb5e5f7cca49adc, user_id=2cbf483c6556421fa36179722100f8b9, role_id=ffb8da90af53491980816087ee621ccb)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:46.472917 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b287fd-4ef3-4c0a-9347-a297ead0ac9a None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:46.489356 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 436/825] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 14:38:46 2022] PUT /identity/v3/projects/42f35b96b9d4499abfb5e5f7cca49adc/users/2cbf483c6556421fa36179722100f8b9/roles/ffb8da90af53491980816087ee621ccb => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:38:46.490720 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b287fd-4ef3-4c0a-9347-a297ead0ac9a None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:46.591291 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 390/826] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:38:46 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:38:46.611454 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0b98db-e133-4f36-8eb8-5033d7f6c55a None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:46.640453 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b287fd-4ef3-4c0a-9347-a297ead0ac9a None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:46.718696 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4e21ea-6a8b-4f89-99e4-b2613d0c2568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:46.719404 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4e21ea-6a8b-4f89-99e4-b2613d0c2568 None admin] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:46.719704 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4e21ea-6a8b-4f89-99e4-b2613d0c2568 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:46.719986 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4e21ea-6a8b-4f89-99e4-b2613d0c2568 None admin] PATH_INFO: `/v3/endpoints` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:46.720758 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4e21ea-6a8b-4f89-99e4-b2613d0c2568 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:46.722236 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4e21ea-6a8b-4f89-99e4-b2613d0c2568 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:46.753289 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 437/827] 10.222.0.57 () {60 vars in 1144 bytes} [Thu May 19 14:38:46 2022] POST /identity/v3/endpoints => generated 327 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:38:46.773973 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0b98db-e133-4f36-8eb8-5033d7f6c55a None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:46.783840 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:46.827043 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbedd0a0-8c44-4ae5-8eaa-6c34f6a5ed2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:46.827920 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbedd0a0-8c44-4ae5-8eaa-6c34f6a5ed2a None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:46.828354 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbedd0a0-8c44-4ae5-8eaa-6c34f6a5ed2a None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:46.828637 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbedd0a0-8c44-4ae5-8eaa-6c34f6a5ed2a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:46.842048 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbedd0a0-8c44-4ae5-8eaa-6c34f6a5ed2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:46.843131 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbedd0a0-8c44-4ae5-8eaa-6c34f6a5ed2a None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:46.865117 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-bbedd0a0-8c44-4ae5-8eaa-6c34f6a5ed2a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:46.865117 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:46.865117 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:46.865117 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:46.865117 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:46.865117 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:46.865117 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:46.865117 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:46.865117 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:46.865117 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:46.865117 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:46.865117 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:46.865117 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:38:46.865117 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:38:46.865117 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:38:46.866765 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:38:46.866765 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:46.866765 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:46.866765 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:46.866765 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:46.866765 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:46.866765 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:46.866765 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:46.866765 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:38:46.866765 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:46.866765 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:46.866765 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:46.866765 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:46.866765 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:46.866765 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:46.866765 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:46.866765 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:46.868428 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:38:46.868428 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:38:46.868428 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:46.868428 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:38:46.868428 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 391/828] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:38:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:46.874596 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4e21ea-6a8b-4f89-99e4-b2613d0c2568 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:46.878912 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:47.010113 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4e21ea-6a8b-4f89-99e4-b2613d0c2568 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:47.047861 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5cfed0-0a49-4180-9e92-dc48fc67ca2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:47.048536 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5cfed0-0a49-4180-9e92-dc48fc67ca2f None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:47.048822 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5cfed0-0a49-4180-9e92-dc48fc67ca2f None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:47.049104 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5cfed0-0a49-4180-9e92-dc48fc67ca2f None admin] PATH_INFO: `/v3/domains` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:47.049834 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5cfed0-0a49-4180-9e92-dc48fc67ca2f None admin] RBAC: Adding query filter params (name=Default) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:47.050178 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5cfed0-0a49-4180-9e92-dc48fc67ca2f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:47.051644 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5cfed0-0a49-4180-9e92-dc48fc67ca2f None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:47.065457 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 438/829] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:38:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:47.071430 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbedd0a0-8c44-4ae5-8eaa-6c34f6a5ed2a None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:47.187759 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbedd0a0-8c44-4ae5-8eaa-6c34f6a5ed2a None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:47.230404 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d02f6bf-975e-4cbe-a52e-87355df1d6c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:47.231234 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d02f6bf-975e-4cbe-a52e-87355df1d6c1 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:47.231524 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d02f6bf-975e-4cbe-a52e-87355df1d6c1 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:47.231804 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d02f6bf-975e-4cbe-a52e-87355df1d6c1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:47.240063 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d02f6bf-975e-4cbe-a52e-87355df1d6c1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:47.242554 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d02f6bf-975e-4cbe-a52e-87355df1d6c1 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:47.259478 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-6d02f6bf-975e-4cbe-a52e-87355df1d6c1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:38:47.259478 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:47.259478 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:47.259478 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:47.259478 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:47.259478 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:47.259478 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:47.259478 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:47.259478 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:47.259478 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:47.259478 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:47.259478 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:47.259478 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:38:47.259478 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:38:47.259478 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:38:47.260528 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:38:47.260528 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:47.260528 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:47.260528 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:47.260528 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:47.260528 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:47.260528 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:47.260528 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:47.260528 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:38:47.260528 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:47.260528 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:47.260528 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:47.260528 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:47.260528 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:47.260528 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:47.260528 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:47.260528 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:47.261958 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:38:47.261958 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:38:47.261958 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:38:47.261958 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:38:47.261958 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:38:47.261958 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:38:47.261958 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:38:47.261958 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:38:47.261958 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 392/830] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:38:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:47.268518 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5cfed0-0a49-4180-9e92-dc48fc67ca2f None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:47.346085 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:47.398124 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5cfed0-0a49-4180-9e92-dc48fc67ca2f None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:47.434155 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ce71ca-8b7d-4774-b7aa-f863c0a8d156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:47.434745 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ce71ca-8b7d-4774-b7aa-f863c0a8d156 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:47.435032 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ce71ca-8b7d-4774-b7aa-f863c0a8d156 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:47.435306 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ce71ca-8b7d-4774-b7aa-f863c0a8d156 None admin] PATH_INFO: `/v3/projects` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:47.436103 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ce71ca-8b7d-4774-b7aa-f863c0a8d156 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:47.436412 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ce71ca-8b7d-4774-b7aa-f863c0a8d156 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:47.437875 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ce71ca-8b7d-4774-b7aa-f863c0a8d156 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:47.450314 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 439/831] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:38:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:47.458950 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d02f6bf-975e-4cbe-a52e-87355df1d6c1 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:47.468044 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b7364d-09d4-4bff-b847-9bcb329f2ad7 None None] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:47.468353 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b7364d-09d4-4bff-b847-9bcb329f2ad7 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:47.468617 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b7364d-09d4-4bff-b847-9bcb329f2ad7 None None] PATH_INFO: `/` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:47.470374 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 440/832] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:38:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:38:47.477767 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecab22eb-233e-46c5-aedd-305d7e35bd65 None None] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:47.478096 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecab22eb-233e-46c5-aedd-305d7e35bd65 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:47.478383 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecab22eb-233e-46c5-aedd-305d7e35bd65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:47.480008 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:47.500518 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.auth.core [None req-ecab22eb-233e-46c5-aedd-305d7e35bd65 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:38:47.502552 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.fernet_utils [None req-ecab22eb-233e-46c5-aedd-305d7e35bd65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:38:47.578585 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d02f6bf-975e-4cbe-a52e-87355df1d6c1 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:47.580367 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 441/833] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:38:47 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (3 switches on core 0) May 19 14:38:47.605585 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecab22eb-233e-46c5-aedd-305d7e35bd65 None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:47.616273 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dcc7b1-dcf5-43c9-b0fd-140d8ae397a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:47.617162 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dcc7b1-dcf5-43c9-b0fd-140d8ae397a1 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:47.617450 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dcc7b1-dcf5-43c9-b0fd-140d8ae397a1 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:47.617977 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dcc7b1-dcf5-43c9-b0fd-140d8ae397a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:47.618723 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dcc7b1-dcf5-43c9-b0fd-140d8ae397a1 None admin] RBAC: Adding query filter params (role.id=c2935b1a25ca46fcba419fe3d1715330, scope.project.id=42f35b96b9d4499abfb5e5f7cca49adc, user.id=a9591727b2f54244baf90b11a8e2277f) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:47.619035 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dcc7b1-dcf5-43c9-b0fd-140d8ae397a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:47.620616 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dcc7b1-dcf5-43c9-b0fd-140d8ae397a1 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:47.627897 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 393/834] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:38:47 2022] GET /identity/v3/role_assignments?user.id=a9591727b2f54244baf90b11a8e2277f&scope.project.id=42f35b96b9d4499abfb5e5f7cca49adc&role.id=c2935b1a25ca46fcba419fe3d1715330 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:47.671502 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:47.722438 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecab22eb-233e-46c5-aedd-305d7e35bd65 None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:47.760993 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f5c151-d989-4224-b14c-f7790b425121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:47.761822 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f5c151-d989-4224-b14c-f7790b425121 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:47.762141 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f5c151-d989-4224-b14c-f7790b425121 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:47.762419 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f5c151-d989-4224-b14c-f7790b425121 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:47.774996 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f5c151-d989-4224-b14c-f7790b425121 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:38:47.782999 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-b8f5c151-d989-4224-b14c-f7790b425121 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:38:47.782999 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:47.782999 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:47.782999 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:47.782999 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:47.782999 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:47.782999 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:47.782999 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:47.782999 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:47.782999 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:47.782999 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:47.782999 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:47.782999 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:38:47.782999 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:38:47.782999 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:38:47.782999 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:38:47.784174 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:38:47.784174 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:38:47.784174 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:38:47.784174 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:38:47.784174 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:47.784174 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:47.784174 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:47.784174 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:47.784174 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:47.784174 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:47.784174 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:47.784174 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:38:47.784174 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:47.784174 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:47.784174 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:47.784174 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:47.785784 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:47.785784 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:47.785784 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:47.785784 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:47.785784 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:38:47.785784 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:38:47.785784 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:38:47.785784 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:38:47.785784 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:38:47.785784 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:38:47.785784 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:38:47.785784 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:38:47.785784 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 442/835] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:38:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:38:47.792060 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dcc7b1-dcf5-43c9-b0fd-140d8ae397a1 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:47.882669 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:47.909153 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dcc7b1-dcf5-43c9-b0fd-140d8ae397a1 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:47.950416 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5119b21d-2393-40f0-8712-e648368f17cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:47.951155 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5119b21d-2393-40f0-8712-e648368f17cc None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:47.951446 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5119b21d-2393-40f0-8712-e648368f17cc None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:47.951727 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5119b21d-2393-40f0-8712-e648368f17cc None admin] PATH_INFO: `/v3/roles` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:47.952489 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5119b21d-2393-40f0-8712-e648368f17cc None admin] RBAC: Adding query filter params (name=admin) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:47.952802 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5119b21d-2393-40f0-8712-e648368f17cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:47.954221 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5119b21d-2393-40f0-8712-e648368f17cc None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:47.963982 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 394/836] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:38:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:38:47.969546 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f5c151-d989-4224-b14c-f7790b425121 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:48.094700 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f5c151-d989-4224-b14c-f7790b425121 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:48.122252 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:48.130833 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f857c263-19af-4492-9d4f-7c4400b6fd2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:48.132467 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f857c263-19af-4492-9d4f-7c4400b6fd2a None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:48.132793 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f857c263-19af-4492-9d4f-7c4400b6fd2a None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:48.133087 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f857c263-19af-4492-9d4f-7c4400b6fd2a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:48.144849 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d823a63b-0548-4cc3-a33a-f39d4084b451 None None] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:48.145387 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d823a63b-0548-4cc3-a33a-f39d4084b451 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:48.145490 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d823a63b-0548-4cc3-a33a-f39d4084b451 None None] PATH_INFO: `/` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:48.146812 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 395/837] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:38:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:38:48.148795 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f857c263-19af-4492-9d4f-7c4400b6fd2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:48.150326 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f857c263-19af-4492-9d4f-7c4400b6fd2a None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:48.155712 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20053de-9969-41ba-968f-c0f5984de4e9 None None] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:48.159464 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20053de-9969-41ba-968f-c0f5984de4e9 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:48.159464 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20053de-9969-41ba-968f-c0f5984de4e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:48.169553 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-f857c263-19af-4492-9d4f-7c4400b6fd2a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:48.169553 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:48.169553 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:48.169553 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:48.169553 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:48.169553 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:48.169553 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:48.169553 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:48.169553 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:48.169553 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:48.169553 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:48.169553 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:48.169553 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:38:48.169553 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:38:48.169553 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:38:48.170564 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:38:48.170564 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:48.170564 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:48.170564 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:48.170564 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:48.170564 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:48.170564 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:48.170564 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:48.170564 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:38:48.170564 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:48.170564 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:48.170564 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:48.170564 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:48.170564 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:48.170564 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:48.170564 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:48.170564 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:48.171921 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:38:48.171921 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:38:48.171921 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:48.171921 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:38:48.171921 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 443/838] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:38:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:48.176619 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f857c263-19af-4492-9d4f-7c4400b6fd2a None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:48.243084 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.auth.core [None req-e20053de-9969-41ba-968f-c0f5984de4e9 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:38:48.244313 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-e20053de-9969-41ba-968f-c0f5984de4e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:38:48.289615 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f857c263-19af-4492-9d4f-7c4400b6fd2a None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:48.326876 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0188f2e-249d-4796-9c97-c0865d524d3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:48.327549 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0188f2e-249d-4796-9c97-c0865d524d3b None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:48.327837 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0188f2e-249d-4796-9c97-c0865d524d3b None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:48.328118 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0188f2e-249d-4796-9c97-c0865d524d3b None admin] PATH_INFO: `/v3/domains` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:48.328853 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0188f2e-249d-4796-9c97-c0865d524d3b None admin] RBAC: Adding query filter params (name=Default) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:48.329205 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0188f2e-249d-4796-9c97-c0865d524d3b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:48.330679 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0188f2e-249d-4796-9c97-c0865d524d3b None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:48.348165 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 444/839] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:38:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:48.349025 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0188f2e-249d-4796-9c97-c0865d524d3b None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:48.397015 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:48.417601 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 396/840] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:38:48 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:38:48.439533 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20053de-9969-41ba-968f-c0f5984de4e9 None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:48.540119 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0188f2e-249d-4796-9c97-c0865d524d3b None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:48.562117 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:48.580875 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a638b0b0-4461-4a63-be2a-4acfa3641fc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:48.582262 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a638b0b0-4461-4a63-be2a-4acfa3641fc3 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:48.582776 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a638b0b0-4461-4a63-be2a-4acfa3641fc3 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:48.583283 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a638b0b0-4461-4a63-be2a-4acfa3641fc3 None admin] PATH_INFO: `/v3/users/swift` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:48.592895 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a638b0b0-4461-4a63-be2a-4acfa3641fc3 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:48.595152 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a638b0b0-4461-4a63-be2a-4acfa3641fc3 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:48.599474 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20053de-9969-41ba-968f-c0f5984de4e9 None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:48.612493 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-a638b0b0-4461-4a63-be2a-4acfa3641fc3 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 14:38:48.612493 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:48.612493 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:48.612493 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:48.612493 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:48.612493 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:48.612493 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:48.612493 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:48.612493 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:48.612493 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:48.612493 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:48.612493 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:48.612493 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:38:48.612493 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:38:48.612493 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:38:48.612493 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:38:48.613577 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:48.613577 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:48.613577 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:38:48.613577 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:38:48.613577 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:38:48.613577 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:38:48.613577 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:48.613577 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:48.613577 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:48.613577 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:48.613577 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:48.613577 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:38:48.613577 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:48.613577 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:48.613577 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:48.613577 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:48.613577 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:48.613577 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:48.614688 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:48.614688 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:48.614688 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:38:48.614688 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:38:48.614688 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:38:48.614688 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:38:48.614688 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:38:48.614688 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:38:48.614688 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 14:38:48.614688 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:38:48.618984 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 445/841] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:38:48 2022] GET /identity/v3/users/swift => generated 83 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:48.624942 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a638b0b0-4461-4a63-be2a-4acfa3641fc3 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:48.645111 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e50e2d-2306-4a6e-bd4f-a410426791d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:48.646276 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e50e2d-2306-4a6e-bd4f-a410426791d5 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:48.646414 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e50e2d-2306-4a6e-bd4f-a410426791d5 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:48.646701 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e50e2d-2306-4a6e-bd4f-a410426791d5 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:48.647941 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e50e2d-2306-4a6e-bd4f-a410426791d5 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:48.649451 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e50e2d-2306-4a6e-bd4f-a410426791d5 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:48.662727 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-c0e50e2d-2306-4a6e-bd4f-a410426791d5 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 14:38:48.662727 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:48.662727 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:38:48.662727 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:38:48.662727 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:38:48.662727 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:38:48.662727 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:38:48.662727 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:38:48.662727 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 14:38:48.662727 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:38:48.662727 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:38:48.662727 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:38:48.662727 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:48.662727 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:48.662727 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:48.662727 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:48.662727 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:48.664109 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:48.664109 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:48.664109 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:48.664109 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:48.664109 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:48.664109 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:48.664109 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:38:48.664109 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:38:48.664109 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:38:48.664109 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:38:48.664109 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:48.664109 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:48.664109 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:48.664109 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:48.664109 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:48.664109 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:48.665044 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:48.665044 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:38:48.665044 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:48.665044 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:48.665044 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:48.665044 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:48.665044 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:48.665044 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:48.665044 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:48.665044 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:48.665044 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:38:48.665044 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:38:48.665044 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 14:38:48.665044 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:38:48.669278 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 397/842] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 19 14:38:48 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:48.670421 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e50e2d-2306-4a6e-bd4f-a410426791d5 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:48.731465 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:48.751481 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a638b0b0-4461-4a63-be2a-4acfa3641fc3 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:48.787257 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e50e2d-2306-4a6e-bd4f-a410426791d5 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:48.793640 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf8fcd5-6aaf-469a-94a3-6411b9a74d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:48.794579 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf8fcd5-6aaf-469a-94a3-6411b9a74d76 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:48.794985 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf8fcd5-6aaf-469a-94a3-6411b9a74d76 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:48.795333 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf8fcd5-6aaf-469a-94a3-6411b9a74d76 None admin] PATH_INFO: `/v3/users` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:48.796338 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf8fcd5-6aaf-469a-94a3-6411b9a74d76 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:48.796687 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf8fcd5-6aaf-469a-94a3-6411b9a74d76 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:48.798399 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf8fcd5-6aaf-469a-94a3-6411b9a74d76 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:48.824067 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 446/843] 10.222.0.57 () {56 vars in 1135 bytes} [Thu May 19 14:38:48 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:38:48.825137 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf8fcd5-6aaf-469a-94a3-6411b9a74d76 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:48.837102 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e64bad-f7ce-4352-9d1e-cccba700faf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:48.837788 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e64bad-f7ce-4352-9d1e-cccba700faf6 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:48.838110 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e64bad-f7ce-4352-9d1e-cccba700faf6 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:48.838407 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e64bad-f7ce-4352-9d1e-cccba700faf6 None admin] PATH_INFO: `/v3/services` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:48.839682 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e64bad-f7ce-4352-9d1e-cccba700faf6 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:48.839995 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e64bad-f7ce-4352-9d1e-cccba700faf6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:48.841878 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e64bad-f7ce-4352-9d1e-cccba700faf6 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:48.843288 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:48.854000 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 398/844] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 19 14:38:48 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:38:48.856458 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e64bad-f7ce-4352-9d1e-cccba700faf6 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:48.978881 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf8fcd5-6aaf-469a-94a3-6411b9a74d76 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:48.992532 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e64bad-f7ce-4352-9d1e-cccba700faf6 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:49.016063 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6aa3ca4-6e51-4f02-9915-ec9c810ae20f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:49.017264 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aa3ca4-6e51-4f02-9915-ec9c810ae20f None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:49.017754 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aa3ca4-6e51-4f02-9915-ec9c810ae20f None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:49.018257 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aa3ca4-6e51-4f02-9915-ec9c810ae20f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:49.035027 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:49.037186 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218c697f-0fde-4796-82a9-ce8bd71383b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:49.038791 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218c697f-0fde-4796-82a9-ce8bd71383b7 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:49.040214 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218c697f-0fde-4796-82a9-ce8bd71383b7 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:49.040561 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218c697f-0fde-4796-82a9-ce8bd71383b7 None admin] PATH_INFO: `/v3/services` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:49.041362 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218c697f-0fde-4796-82a9-ce8bd71383b7 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:49.041752 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218c697f-0fde-4796-82a9-ce8bd71383b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:49.043980 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218c697f-0fde-4796-82a9-ce8bd71383b7 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:49.054004 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 399/845] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 19 14:38:48 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:49.110323 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6aa3ca4-6e51-4f02-9915-ec9c810ae20f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:49.112176 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6aa3ca4-6e51-4f02-9915-ec9c810ae20f None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:49.234312 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 447/846] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:38:48 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:38:49.240419 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218c697f-0fde-4796-82a9-ce8bd71383b7 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:49.318305 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:49.327006 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450845ba-a6df-4e42-ad34-db5230f77e93 None None] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:49.327322 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450845ba-a6df-4e42-ad34-db5230f77e93 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:49.327571 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450845ba-a6df-4e42-ad34-db5230f77e93 None None] PATH_INFO: `/` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:49.329801 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 448/847] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:38:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:38:49.339719 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34b74c8-467c-4f4d-adf4-9f2054579865 None None] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:49.340015 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34b74c8-467c-4f4d-adf4-9f2054579865 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:49.340297 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34b74c8-467c-4f4d-adf4-9f2054579865 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:49.359052 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.auth.core [None req-f34b74c8-467c-4f4d-adf4-9f2054579865 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:38:49.360298 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.fernet_utils [None req-f34b74c8-467c-4f4d-adf4-9f2054579865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:38:49.368551 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218c697f-0fde-4796-82a9-ce8bd71383b7 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:49.405124 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:49.409812 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17242e5-e75c-44c2-b6d4-cce1cbda0853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:49.410902 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17242e5-e75c-44c2-b6d4-cce1cbda0853 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:49.411354 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17242e5-e75c-44c2-b6d4-cce1cbda0853 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:49.411798 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17242e5-e75c-44c2-b6d4-cce1cbda0853 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:49.425742 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17242e5-e75c-44c2-b6d4-cce1cbda0853 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:49.427697 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17242e5-e75c-44c2-b6d4-cce1cbda0853 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:49.436590 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 449/848] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:38:49 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:38:49.448377 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-f17242e5-e75c-44c2-b6d4-cce1cbda0853 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:49.448377 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:49.448377 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:49.448377 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:49.448377 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:49.448377 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:49.448377 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:49.448377 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:49.448377 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:49.448377 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:49.448377 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:49.448377 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:49.448377 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:38:49.448377 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:38:49.448377 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:38:49.449599 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:38:49.449599 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:49.449599 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:49.449599 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:49.449599 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:49.449599 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:49.449599 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:49.449599 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:49.449599 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:38:49.449599 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:49.449599 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:49.449599 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:49.449599 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:49.449599 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:49.449599 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:49.449599 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:49.449599 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:49.450934 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:38:49.450934 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:38:49.450934 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:49.450934 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:38:49.452859 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 400/849] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:38:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:49.458904 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34b74c8-467c-4f4d-adf4-9f2054579865 None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:49.461463 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17242e5-e75c-44c2-b6d4-cce1cbda0853 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:49.568441 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34b74c8-467c-4f4d-adf4-9f2054579865 None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:49.576299 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17242e5-e75c-44c2-b6d4-cce1cbda0853 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:49.592757 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:49.608873 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dac0ec-6daa-4e4a-8bd7-895ee6fd236a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:49.609982 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dac0ec-6daa-4e4a-8bd7-895ee6fd236a None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:49.610501 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dac0ec-6daa-4e4a-8bd7-895ee6fd236a None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:49.610999 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dac0ec-6daa-4e4a-8bd7-895ee6fd236a None admin] PATH_INFO: `/v3/domains` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:49.611967 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dac0ec-6daa-4e4a-8bd7-895ee6fd236a None admin] RBAC: Adding query filter params (name=Default) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:49.612498 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dac0ec-6daa-4e4a-8bd7-895ee6fd236a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:49.614316 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dac0ec-6daa-4e4a-8bd7-895ee6fd236a None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:49.617368 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0c2e4f-e635-4737-8ae4-6508380aa33c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:49.618680 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0c2e4f-e635-4737-8ae4-6508380aa33c None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:49.618808 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0c2e4f-e635-4737-8ae4-6508380aa33c None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:49.618999 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0c2e4f-e635-4737-8ae4-6508380aa33c None admin] PATH_INFO: `/v3/roles/service` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:49.628940 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 450/850] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:38:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:49.634761 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dac0ec-6daa-4e4a-8bd7-895ee6fd236a None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:49.635696 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0c2e4f-e635-4737-8ae4-6508380aa33c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:38:49.644884 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-1e0c2e4f-e635-4737-8ae4-6508380aa33c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:38:49.644884 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:49.644884 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:49.644884 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:49.644884 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:49.644884 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:49.644884 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:49.644884 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:49.644884 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:49.644884 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:49.644884 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:49.644884 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:49.644884 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:38:49.644884 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:38:49.644884 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:38:49.644884 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:38:49.646007 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:38:49.646007 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:38:49.646007 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:38:49.646007 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:38:49.646007 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:49.646007 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:49.646007 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:49.646007 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:49.646007 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:49.646007 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:49.646007 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:49.646007 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:38:49.646007 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:49.646007 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:49.646007 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:49.646007 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:49.646943 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:49.646943 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:49.646943 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:49.646943 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:49.646943 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:38:49.646943 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:38:49.646943 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:38:49.646943 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:38:49.646943 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:38:49.646943 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:38:49.646943 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:38:49.646943 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:38:49.649969 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 401/851] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 14:38:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:49.653406 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0c2e4f-e635-4737-8ae4-6508380aa33c None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:49.686190 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:49.771406 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dac0ec-6daa-4e4a-8bd7-895ee6fd236a None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:49.799221 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0c2e4f-e635-4737-8ae4-6508380aa33c None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:49.809828 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e8544f-c35f-4b54-b205-ecaf0f57baae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:49.810652 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e8544f-c35f-4b54-b205-ecaf0f57baae None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:49.810931 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e8544f-c35f-4b54-b205-ecaf0f57baae None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:49.811201 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e8544f-c35f-4b54-b205-ecaf0f57baae None admin] PATH_INFO: `/v3/projects/service` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:49.818602 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e8544f-c35f-4b54-b205-ecaf0f57baae None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:49.820883 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e8544f-c35f-4b54-b205-ecaf0f57baae None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:49.836501 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-00e8544f-c35f-4b54-b205-ecaf0f57baae None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:38:49.836501 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:49.836501 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:49.836501 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:49.836501 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:49.836501 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:49.836501 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:49.836501 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:49.836501 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:49.836501 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:49.836501 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:49.836501 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:49.836501 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:38:49.836501 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:38:49.836501 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:38:49.837570 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:38:49.837570 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:49.837570 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:49.837570 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:49.837570 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:49.837570 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:49.837570 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:49.837570 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:49.837570 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:38:49.837570 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:49.837570 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:49.837570 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:49.837570 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:49.837570 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:49.837570 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:49.837570 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:49.837570 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:49.838628 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:38:49.838628 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:38:49.838628 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:38:49.838628 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:38:49.838628 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:38:49.838628 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:38:49.838628 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:38:49.838628 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:38:49.838628 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 451/852] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:38:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:38:49.845340 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e8544f-c35f-4b54-b205-ecaf0f57baae None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:49.848075 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5078b78c-4c6d-4004-8548-7adcdddd01e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:49.848810 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5078b78c-4c6d-4004-8548-7adcdddd01e8 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:49.849078 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5078b78c-4c6d-4004-8548-7adcdddd01e8 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:49.849388 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5078b78c-4c6d-4004-8548-7adcdddd01e8 None admin] PATH_INFO: `/v3/roles` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:49.850240 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5078b78c-4c6d-4004-8548-7adcdddd01e8 None admin] RBAC: Adding query filter params (name=service) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:49.850548 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5078b78c-4c6d-4004-8548-7adcdddd01e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:49.852113 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5078b78c-4c6d-4004-8548-7adcdddd01e8 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:49.866691 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 402/853] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 19 14:38:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:38:49.868110 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5078b78c-4c6d-4004-8548-7adcdddd01e8 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:49.913828 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:49.958956 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:49.967722 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e8544f-c35f-4b54-b205-ecaf0f57baae None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:49.989339 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5078b78c-4c6d-4004-8548-7adcdddd01e8 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:50.005356 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307c4989-b2a0-4472-b555-091afef7633b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:50.006078 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307c4989-b2a0-4472-b555-091afef7633b None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:50.006398 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307c4989-b2a0-4472-b555-091afef7633b None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:50.007423 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307c4989-b2a0-4472-b555-091afef7633b None admin] PATH_INFO: `/v3/projects` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:50.008344 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307c4989-b2a0-4472-b555-091afef7633b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:50.008657 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307c4989-b2a0-4472-b555-091afef7633b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:50.010260 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307c4989-b2a0-4472-b555-091afef7633b None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:50.024279 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 452/854] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:38:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:38:50.034915 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307c4989-b2a0-4472-b555-091afef7633b None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:50.035111 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e2b581-82dc-45f5-b515-e607faa0ff9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:50.035111 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e2b581-82dc-45f5-b515-e607faa0ff9b None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:50.035111 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e2b581-82dc-45f5-b515-e607faa0ff9b None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:50.035111 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e2b581-82dc-45f5-b515-e607faa0ff9b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:50.044995 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e2b581-82dc-45f5-b515-e607faa0ff9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:50.046619 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e2b581-82dc-45f5-b515-e607faa0ff9b None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:50.069966 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-64e2b581-82dc-45f5-b515-e607faa0ff9b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:50.069966 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:50.069966 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:50.069966 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:50.069966 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:50.069966 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:50.069966 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:50.069966 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:50.069966 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:50.069966 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:50.069966 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:50.069966 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:50.069966 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:38:50.069966 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:38:50.069966 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:38:50.071270 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:38:50.071270 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:50.071270 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:50.071270 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:50.071270 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:50.071270 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:50.071270 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:50.071270 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:50.071270 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:38:50.071270 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:50.071270 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:50.071270 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:50.071270 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:50.071270 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:50.071270 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:50.071270 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:50.071270 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:50.072451 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:38:50.072451 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:38:50.072451 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:50.072451 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:38:50.072451 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 403/855] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:38:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:50.074857 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f62348-2f31-426c-8e8f-c285c047386e None None] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:50.074857 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f62348-2f31-426c-8e8f-c285c047386e None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:50.074857 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f62348-2f31-426c-8e8f-c285c047386e None None] PATH_INFO: `/` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:50.075920 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 404/856] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:38:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:38:50.077547 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f62348-2f31-426c-8e8f-c285c047386e None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:50.145658 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307c4989-b2a0-4472-b555-091afef7633b None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:50.182129 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:50.191644 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0461d1c-0743-4019-9aa1-0cdfbde2cfa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:50.192305 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0461d1c-0743-4019-9aa1-0cdfbde2cfa2 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:50.192588 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0461d1c-0743-4019-9aa1-0cdfbde2cfa2 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:50.192867 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0461d1c-0743-4019-9aa1-0cdfbde2cfa2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:50.193691 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0461d1c-0743-4019-9aa1-0cdfbde2cfa2 None admin] RBAC: Adding query filter params (role.id=c2935b1a25ca46fcba419fe3d1715330, scope.project.id=42f35b96b9d4499abfb5e5f7cca49adc, user.id=4f8e2e0909074a85bb493c49ac3a6d4d) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:50.194132 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0461d1c-0743-4019-9aa1-0cdfbde2cfa2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:50.195575 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0461d1c-0743-4019-9aa1-0cdfbde2cfa2 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:50.198791 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f62348-2f31-426c-8e8f-c285c047386e None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:50.201507 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 453/857] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:38:50 2022] GET /identity/v3/role_assignments?user.id=4f8e2e0909074a85bb493c49ac3a6d4d&scope.project.id=42f35b96b9d4499abfb5e5f7cca49adc&role.id=c2935b1a25ca46fcba419fe3d1715330 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:38:50.206042 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b3afdd-ebe0-46c5-a86c-bce0e7596e7d None None] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:50.206042 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b3afdd-ebe0-46c5-a86c-bce0e7596e7d None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:50.206042 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b3afdd-ebe0-46c5-a86c-bce0e7596e7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:50.227397 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.auth.core [None req-a5b3afdd-ebe0-46c5-a86c-bce0e7596e7d None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:38:50.229087 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.fernet_utils [None req-a5b3afdd-ebe0-46c5-a86c-bce0e7596e7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:38:50.245983 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:50.246642 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f9fb6f-2e81-4499-bbd2-90e9fe1d447f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:50.248857 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f9fb6f-2e81-4499-bbd2-90e9fe1d447f None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:50.249374 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f9fb6f-2e81-4499-bbd2-90e9fe1d447f None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:50.249979 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f9fb6f-2e81-4499-bbd2-90e9fe1d447f None admin] PATH_INFO: `/v3/domains` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:50.251675 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f9fb6f-2e81-4499-bbd2-90e9fe1d447f None admin] RBAC: Adding query filter params (name=Default) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:50.252420 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f9fb6f-2e81-4499-bbd2-90e9fe1d447f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:50.254064 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f9fb6f-2e81-4499-bbd2-90e9fe1d447f None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:50.268771 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 405/858] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:38:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:50.275041 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f9fb6f-2e81-4499-bbd2-90e9fe1d447f None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:50.322872 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 454/859] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:38:50 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:38:50.344558 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b3afdd-ebe0-46c5-a86c-bce0e7596e7d None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:50.416222 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f9fb6f-2e81-4499-bbd2-90e9fe1d447f None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:50.457895 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1676d1-ad79-4cc2-b360-8966b41724ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:50.458670 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1676d1-ad79-4cc2-b360-8966b41724ee None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:50.458958 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1676d1-ad79-4cc2-b360-8966b41724ee None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:50.459238 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1676d1-ad79-4cc2-b360-8966b41724ee None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:50.465424 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b3afdd-ebe0-46c5-a86c-bce0e7596e7d None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:50.467574 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1676d1-ad79-4cc2-b360-8966b41724ee None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:50.469055 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1676d1-ad79-4cc2-b360-8966b41724ee None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:50.485207 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-9c1676d1-ad79-4cc2-b360-8966b41724ee None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 14:38:50.485207 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:50.485207 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:50.485207 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:50.485207 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:50.485207 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:50.485207 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:50.485207 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:50.485207 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:50.485207 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:50.485207 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:50.485207 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:50.485207 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:38:50.485207 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:38:50.485207 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:38:50.485207 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:38:50.486298 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:50.486298 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:50.486298 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:38:50.486298 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:38:50.486298 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:38:50.486298 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:38:50.486298 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:50.486298 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:50.486298 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:50.486298 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:50.486298 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:50.486298 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:38:50.486298 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:50.486298 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:50.486298 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:50.486298 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:50.486298 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:50.486298 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:50.487347 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:50.487347 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:50.487347 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:38:50.487347 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:38:50.487347 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:38:50.487347 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:38:50.487347 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:38:50.487347 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:38:50.487347 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 14:38:50.487347 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:38:50.487959 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 406/860] 10.222.0.57 () {56 vars in 1117 bytes} [Thu May 19 14:38:50 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:50.494142 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1676d1-ad79-4cc2-b360-8966b41724ee None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:50.506496 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fec114-22f1-4b7f-b21c-ae2edf584b01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:50.507283 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:50.507676 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fec114-22f1-4b7f-b21c-ae2edf584b01 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:50.508115 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fec114-22f1-4b7f-b21c-ae2edf584b01 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:50.508411 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fec114-22f1-4b7f-b21c-ae2edf584b01 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:50.509232 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fec114-22f1-4b7f-b21c-ae2edf584b01 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:50.510824 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fec114-22f1-4b7f-b21c-ae2edf584b01 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:50.524348 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-35fec114-22f1-4b7f-b21c-ae2edf584b01 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 14:38:50.524348 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:50.524348 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:38:50.524348 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:38:50.524348 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:38:50.524348 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:38:50.524348 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:38:50.524348 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:38:50.524348 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 14:38:50.524348 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:38:50.524348 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:38:50.524348 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:38:50.524348 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:50.524348 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:50.524348 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:50.524348 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:50.525501 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:50.525501 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:50.525501 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:50.525501 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:50.525501 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:50.525501 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:50.525501 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:50.525501 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:38:50.525501 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:38:50.525501 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:38:50.525501 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:38:50.525501 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:50.525501 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:50.525501 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:50.525501 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:50.525501 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:50.526893 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:50.526893 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:50.526893 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:38:50.526893 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:50.526893 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:50.526893 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:50.526893 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:50.526893 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:50.526893 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:50.526893 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:50.526893 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:50.526893 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:38:50.526893 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:38:50.526893 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 14:38:50.526893 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:38:50.526893 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 455/861] 10.222.0.57 () {56 vars in 1132 bytes} [Thu May 19 14:38:50 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:38:50.531695 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fec114-22f1-4b7f-b21c-ae2edf584b01 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:50.555236 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:50.624902 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1676d1-ad79-4cc2-b360-8966b41724ee None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:50.658417 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fec114-22f1-4b7f-b21c-ae2edf584b01 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:50.674642 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee91d8c-1321-40e1-ac96-0d4ab4651d60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:50.675348 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee91d8c-1321-40e1-ac96-0d4ab4651d60 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:50.675617 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee91d8c-1321-40e1-ac96-0d4ab4651d60 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:50.675894 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee91d8c-1321-40e1-ac96-0d4ab4651d60 None admin] PATH_INFO: `/v3/users` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:50.677138 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee91d8c-1321-40e1-ac96-0d4ab4651d60 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:50.677457 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee91d8c-1321-40e1-ac96-0d4ab4651d60 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:50.679626 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee91d8c-1321-40e1-ac96-0d4ab4651d60 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:50.699590 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6703f26-8183-4e90-be81-ef7c3ee15ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:50.700721 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6703f26-8183-4e90-be81-ef7c3ee15ac7 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:50.701241 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6703f26-8183-4e90-be81-ef7c3ee15ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:50.701755 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6703f26-8183-4e90-be81-ef7c3ee15ac7 None admin] PATH_INFO: `/v3/services` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:50.702801 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 407/862] 10.222.0.57 () {56 vars in 1149 bytes} [Thu May 19 14:38:50 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:38:50.702945 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6703f26-8183-4e90-be81-ef7c3ee15ac7 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:50.706058 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6703f26-8183-4e90-be81-ef7c3ee15ac7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:50.706058 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6703f26-8183-4e90-be81-ef7c3ee15ac7 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:50.708009 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee91d8c-1321-40e1-ac96-0d4ab4651d60 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:50.712292 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 456/863] 10.222.0.57 () {56 vars in 1126 bytes} [Thu May 19 14:38:50 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:38:50.718657 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6703f26-8183-4e90-be81-ef7c3ee15ac7 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:50.796156 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:50.820952 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:50.844239 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee91d8c-1321-40e1-ac96-0d4ab4651d60 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:50.853141 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6703f26-8183-4e90-be81-ef7c3ee15ac7 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:50.883131 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa75287-ad38-45e0-bd08-ac9a90fa2481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:50.884165 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa75287-ad38-45e0-bd08-ac9a90fa2481 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:50.884664 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa75287-ad38-45e0-bd08-ac9a90fa2481 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:50.885157 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa75287-ad38-45e0-bd08-ac9a90fa2481 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:50.900957 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f2cce1-fc75-47d1-be62-d5634ccb5a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:50.901643 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f2cce1-fc75-47d1-be62-d5634ccb5a64 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:50.901961 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f2cce1-fc75-47d1-be62-d5634ccb5a64 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:50.902244 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f2cce1-fc75-47d1-be62-d5634ccb5a64 None admin] PATH_INFO: `/v3/services` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:50.903535 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f2cce1-fc75-47d1-be62-d5634ccb5a64 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:50.903856 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f2cce1-fc75-47d1-be62-d5634ccb5a64 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:50.905818 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f2cce1-fc75-47d1-be62-d5634ccb5a64 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:50.913137 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 457/864] 10.222.0.57 () {56 vars in 1126 bytes} [Thu May 19 14:38:50 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:50.972204 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa75287-ad38-45e0-bd08-ac9a90fa2481 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:50.973991 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa75287-ad38-45e0-bd08-ac9a90fa2481 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:51.051048 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:51.102459 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 408/865] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:38:50 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 19 14:38:51.108363 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f2cce1-fc75-47d1-be62-d5634ccb5a64 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:51.231058 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f2cce1-fc75-47d1-be62-d5634ccb5a64 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:51.268084 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf2fc49-5b98-4756-853c-dd93c4385833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:51.268892 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf2fc49-5b98-4756-853c-dd93c4385833 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:51.269227 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf2fc49-5b98-4756-853c-dd93c4385833 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:51.269511 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf2fc49-5b98-4756-853c-dd93c4385833 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:51.279799 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:51.283706 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf2fc49-5b98-4756-853c-dd93c4385833 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:51.284938 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf2fc49-5b98-4756-853c-dd93c4385833 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:51.304479 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-1bf2fc49-5b98-4756-853c-dd93c4385833 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:51.304479 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:51.304479 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:51.304479 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:51.304479 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:51.304479 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:51.304479 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:51.304479 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:51.304479 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:51.304479 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:51.304479 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:51.304479 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:51.304479 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:38:51.304479 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:38:51.304479 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:38:51.305563 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:38:51.305563 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:51.305563 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:51.305563 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:51.305563 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:51.305563 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:51.305563 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:51.305563 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:51.305563 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:38:51.305563 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:51.305563 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:51.305563 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:51.305563 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:51.305563 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:51.305563 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:51.305563 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:51.305563 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:51.307049 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:38:51.307049 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:38:51.307049 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:51.307049 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:38:51.307049 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 458/866] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:38:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:51.311712 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa75287-ad38-45e0-bd08-ac9a90fa2481 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:51.421201 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa75287-ad38-45e0-bd08-ac9a90fa2481 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:51.458621 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0016d12b-360f-453e-9f7e-1de5c5829f8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:51.459349 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0016d12b-360f-453e-9f7e-1de5c5829f8b None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:51.459931 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0016d12b-360f-453e-9f7e-1de5c5829f8b None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:51.460207 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0016d12b-360f-453e-9f7e-1de5c5829f8b None admin] PATH_INFO: `/v3/domains` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:51.460944 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0016d12b-360f-453e-9f7e-1de5c5829f8b None admin] RBAC: Adding query filter params (name=Default) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:51.461252 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0016d12b-360f-453e-9f7e-1de5c5829f8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:51.462663 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0016d12b-360f-453e-9f7e-1de5c5829f8b None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:51.475619 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 409/867] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:38:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:51.481419 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf2fc49-5b98-4756-853c-dd93c4385833 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:51.519171 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5763404f-c603-47ac-9ec6-ba0683a3ed5a None None] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:51.519529 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5763404f-c603-47ac-9ec6-ba0683a3ed5a None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:51.519742 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5763404f-c603-47ac-9ec6-ba0683a3ed5a None None] PATH_INFO: `/` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:51.521415 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 410/868] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:38:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:38:51.527145 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394d7e52-b933-488d-b66e-5a5173efc337 None None] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:51.527565 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394d7e52-b933-488d-b66e-5a5173efc337 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:51.527952 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394d7e52-b933-488d-b66e-5a5173efc337 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:51.545880 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.auth.core [None req-394d7e52-b933-488d-b66e-5a5173efc337 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:38:51.547212 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-394d7e52-b933-488d-b66e-5a5173efc337 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:38:51.570209 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:51.593258 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf2fc49-5b98-4756-853c-dd93c4385833 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:51.626006 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 411/869] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:38:51 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:38:51.633489 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac87d3f3-5533-4c75-9b62-21379f096e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:51.634567 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac87d3f3-5533-4c75-9b62-21379f096e70 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:51.635022 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac87d3f3-5533-4c75-9b62-21379f096e70 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:51.635452 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac87d3f3-5533-4c75-9b62-21379f096e70 None admin] PATH_INFO: `/v3/projects/service` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:51.643130 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac87d3f3-5533-4c75-9b62-21379f096e70 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:51.644770 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac87d3f3-5533-4c75-9b62-21379f096e70 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:51.648245 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394d7e52-b933-488d-b66e-5a5173efc337 None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:51.668400 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-ac87d3f3-5533-4c75-9b62-21379f096e70 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:38:51.668400 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:51.668400 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:51.668400 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:51.668400 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:51.668400 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:51.668400 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:51.668400 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:51.668400 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:51.668400 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:51.668400 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:51.668400 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:51.668400 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:38:51.668400 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:38:51.668400 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:38:51.669451 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:38:51.669451 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:51.669451 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:51.669451 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:51.669451 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:51.669451 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:51.669451 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:51.669451 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:51.669451 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:38:51.669451 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:51.669451 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:51.669451 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:51.669451 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:51.669451 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:51.669451 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:51.669451 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:51.669451 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:51.670502 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:38:51.670502 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:38:51.670502 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:38:51.670502 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:38:51.670502 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:38:51.670502 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:38:51.670502 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:38:51.670502 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:38:51.676760 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 459/870] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:38:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:51.677194 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac87d3f3-5533-4c75-9b62-21379f096e70 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:51.762336 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:51.779582 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394d7e52-b933-488d-b66e-5a5173efc337 None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:51.816914 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac87d3f3-5533-4c75-9b62-21379f096e70 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:51.826084 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2b99e1-2c7c-4f77-a7b0-33dd1c4f0efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:51.826756 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2b99e1-2c7c-4f77-a7b0-33dd1c4f0efc None admin] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:51.827044 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2b99e1-2c7c-4f77-a7b0-33dd1c4f0efc None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:51.827325 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2b99e1-2c7c-4f77-a7b0-33dd1c4f0efc None admin] PATH_INFO: `/v3/services` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:51.828058 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2b99e1-2c7c-4f77-a7b0-33dd1c4f0efc None admin] RBAC: Authorizing `identity:create_service()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:51.829430 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2b99e1-2c7c-4f77-a7b0-33dd1c4f0efc None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:51.852201 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 412/871] 10.222.0.57 () {60 vars in 1141 bytes} [Thu May 19 14:38:51 2022] POST /identity/v3/services => generated 251 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:38:51.873795 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c33f4a-c699-44b9-b223-76e2abaf8682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:51.874573 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c33f4a-c699-44b9-b223-76e2abaf8682 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:51.874857 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c33f4a-c699-44b9-b223-76e2abaf8682 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:51.875219 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c33f4a-c699-44b9-b223-76e2abaf8682 None admin] PATH_INFO: `/v3/projects` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:51.876085 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c33f4a-c699-44b9-b223-76e2abaf8682 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:51.876451 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c33f4a-c699-44b9-b223-76e2abaf8682 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:51.878015 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c33f4a-c699-44b9-b223-76e2abaf8682 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:51.892698 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 460/872] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:38:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:38:51.899781 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2b99e1-2c7c-4f77-a7b0-33dd1c4f0efc None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:51.924160 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:52.028973 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2b99e1-2c7c-4f77-a7b0-33dd1c4f0efc None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:52.068458 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331047ce-0110-4964-a5d1-90327a668a30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:52.069331 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331047ce-0110-4964-a5d1-90327a668a30 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:52.069616 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331047ce-0110-4964-a5d1-90327a668a30 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:52.069927 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331047ce-0110-4964-a5d1-90327a668a30 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:52.070782 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331047ce-0110-4964-a5d1-90327a668a30 None admin] RBAC: Adding query filter params (role.id=ffb8da90af53491980816087ee621ccb, scope.project.id=42f35b96b9d4499abfb5e5f7cca49adc, user.id=2cbf483c6556421fa36179722100f8b9) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:52.071160 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331047ce-0110-4964-a5d1-90327a668a30 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:52.072671 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331047ce-0110-4964-a5d1-90327a668a30 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:52.080494 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 413/873] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:38:51 2022] GET /identity/v3/role_assignments?user.id=2cbf483c6556421fa36179722100f8b9&scope.project.id=42f35b96b9d4499abfb5e5f7cca49adc&role.id=ffb8da90af53491980816087ee621ccb => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:52.643673 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d1f581-0776-4132-beed-be75747ee31e None None] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:52.644245 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d1f581-0776-4132-beed-be75747ee31e None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:52.644245 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d1f581-0776-4132-beed-be75747ee31e None None] PATH_INFO: `/` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:52.646024 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 461/874] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:38:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:38:52.652094 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aab4381-7e8e-4e64-90cd-866104194553 None None] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:52.652535 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aab4381-7e8e-4e64-90cd-866104194553 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:52.652726 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aab4381-7e8e-4e64-90cd-866104194553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:52.670666 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.auth.core [None req-0aab4381-7e8e-4e64-90cd-866104194553 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:38:52.671876 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-0aab4381-7e8e-4e64-90cd-866104194553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:38:52.746619 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 414/875] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:38:52 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:38:52.769901 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d1f581-0776-4132-beed-be75747ee31e None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:52.878506 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d1f581-0776-4132-beed-be75747ee31e None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:52.888113 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:52.915486 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa67791-90b0-43eb-ac8b-eb0cfea66e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:52.916222 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa67791-90b0-43eb-ac8b-eb0cfea66e6f None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:52.916504 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa67791-90b0-43eb-ac8b-eb0cfea66e6f None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:52.916779 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa67791-90b0-43eb-ac8b-eb0cfea66e6f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:52.917436 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa67791-90b0-43eb-ac8b-eb0cfea66e6f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:52.918867 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa67791-90b0-43eb-ac8b-eb0cfea66e6f None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:52.931061 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-cfa67791-90b0-43eb-ac8b-eb0cfea66e6f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 14:38:52.931061 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:52.931061 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:38:52.931061 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:38:52.931061 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:38:52.931061 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:38:52.931061 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:38:52.931061 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:38:52.931061 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 14:38:52.931061 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:38:52.931061 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:38:52.931061 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:38:52.931061 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:52.931061 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:52.931061 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:52.931061 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:52.931061 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:52.932142 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:52.932142 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:52.932142 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:52.932142 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:52.932142 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:52.932142 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:52.932142 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:38:52.932142 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:38:52.932142 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:38:52.932142 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:38:52.932142 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:52.932142 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:52.932142 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:52.932142 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:52.932142 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:52.932142 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:52.933061 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:52.933061 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:38:52.933061 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:52.933061 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:52.933061 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:52.933061 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:52.933061 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:52.933061 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:52.933061 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:52.933061 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:52.933061 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:38:52.933061 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:38:52.933061 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 14:38:52.933061 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:38:52.933061 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 462/876] 10.222.0.57 () {56 vars in 1126 bytes} [Thu May 19 14:38:52 2022] GET /identity/v3/services/object-store => generated 93 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:52.938653 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aab4381-7e8e-4e64-90cd-866104194553 None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:52.954574 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:53.048938 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aab4381-7e8e-4e64-90cd-866104194553 None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:53.090941 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5702301-3eb2-48be-bfc1-afb6a7f2ecce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:53.091615 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5702301-3eb2-48be-bfc1-afb6a7f2ecce None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:53.091900 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5702301-3eb2-48be-bfc1-afb6a7f2ecce None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:53.092179 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5702301-3eb2-48be-bfc1-afb6a7f2ecce None admin] PATH_INFO: `/v3/services` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:53.092893 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5702301-3eb2-48be-bfc1-afb6a7f2ecce None admin] RBAC: Adding query filter params (name=object-store) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:53.093199 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5702301-3eb2-48be-bfc1-afb6a7f2ecce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:53.094635 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5702301-3eb2-48be-bfc1-afb6a7f2ecce None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:53.100019 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 415/877] 10.222.0.57 () {56 vars in 1122 bytes} [Thu May 19 14:38:52 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:53.105694 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa67791-90b0-43eb-ac8b-eb0cfea66e6f None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:53.219694 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa67791-90b0-43eb-ac8b-eb0cfea66e6f None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:53.254804 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a914a17f-a706-40dc-bde2-9e598a9ba2d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:53.255171 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1c7cae-98e5-4a4c-9efa-192c50c3af91 None None] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:53.255438 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1c7cae-98e5-4a4c-9efa-192c50c3af91 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:53.255556 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a914a17f-a706-40dc-bde2-9e598a9ba2d7 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:53.255739 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1c7cae-98e5-4a4c-9efa-192c50c3af91 None None] PATH_INFO: `/` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:53.255856 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a914a17f-a706-40dc-bde2-9e598a9ba2d7 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:53.256020 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a914a17f-a706-40dc-bde2-9e598a9ba2d7 None admin] PATH_INFO: `/v3/services` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:53.257430 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a914a17f-a706-40dc-bde2-9e598a9ba2d7 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:53.257830 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a914a17f-a706-40dc-bde2-9e598a9ba2d7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:53.259230 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a914a17f-a706-40dc-bde2-9e598a9ba2d7 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:53.261921 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 416/878] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:38:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:38:53.263159 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6b5a47-7f7c-4f0f-8a64-4990cf3bf8c6 None None] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:53.263452 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6b5a47-7f7c-4f0f-8a64-4990cf3bf8c6 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:53.263735 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6b5a47-7f7c-4f0f-8a64-4990cf3bf8c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:53.264591 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 463/879] 10.222.0.57 () {56 vars in 1122 bytes} [Thu May 19 14:38:53 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:53.283931 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.auth.core [None req-4c6b5a47-7f7c-4f0f-8a64-4990cf3bf8c6 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:38:53.285188 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-4c6b5a47-7f7c-4f0f-8a64-4990cf3bf8c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:38:53.367707 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 417/880] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:38:53 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:38:53.391254 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a914a17f-a706-40dc-bde2-9e598a9ba2d7 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:53.432782 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:53.503332 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a914a17f-a706-40dc-bde2-9e598a9ba2d7 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:53.539439 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ffbbfe-8ca1-455a-abc0-c5240203e735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:53.540117 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ffbbfe-8ca1-455a-abc0-c5240203e735 None admin] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:53.540403 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ffbbfe-8ca1-455a-abc0-c5240203e735 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:53.540681 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ffbbfe-8ca1-455a-abc0-c5240203e735 None admin] PATH_INFO: `/v3/services` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:53.541427 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ffbbfe-8ca1-455a-abc0-c5240203e735 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:53.542843 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ffbbfe-8ca1-455a-abc0-c5240203e735 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:53.556728 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 464/881] 10.222.0.57 () {60 vars in 1141 bytes} [Thu May 19 14:38:53 2022] POST /identity/v3/services => generated 269 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:38:54.301037 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c59f551-bf92-4139-8d34-1a4d5f7cf4ed None None] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:54.302017 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c59f551-bf92-4139-8d34-1a4d5f7cf4ed None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:54.302511 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c59f551-bf92-4139-8d34-1a4d5f7cf4ed None None] PATH_INFO: `/` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:54.304940 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 418/882] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:38:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:38:54.310864 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5768a710-6e98-46f1-be94-da94995a86cb None None] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:54.311699 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5768a710-6e98-46f1-be94-da94995a86cb None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:54.312193 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5768a710-6e98-46f1-be94-da94995a86cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:54.331714 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.auth.core [None req-5768a710-6e98-46f1-be94-da94995a86cb None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:38:54.333276 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.fernet_utils [None req-5768a710-6e98-46f1-be94-da94995a86cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:38:54.416087 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 465/883] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:38:54 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:38:54.439391 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c59f551-bf92-4139-8d34-1a4d5f7cf4ed None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:54.445050 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:54.525514 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce05074-2b40-4944-8ce4-dc9c4ee2cf4b None None] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:54.526147 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce05074-2b40-4944-8ce4-dc9c4ee2cf4b None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:54.527136 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce05074-2b40-4944-8ce4-dc9c4ee2cf4b None None] PATH_INFO: `/` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:54.529406 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 466/884] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:38:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:38:54.535875 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7a9dcd-7706-4488-ab35-56ced370ce1d None None] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:54.536605 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7a9dcd-7706-4488-ab35-56ced370ce1d None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:54.537131 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7a9dcd-7706-4488-ab35-56ced370ce1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:54.558822 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c59f551-bf92-4139-8d34-1a4d5f7cf4ed None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:54.559758 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.auth.core [None req-1d7a9dcd-7706-4488-ab35-56ced370ce1d None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:38:54.562429 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.fernet_utils [None req-1d7a9dcd-7706-4488-ab35-56ced370ce1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:38:54.586243 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:54.606085 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143d6947-1523-456a-8b7c-47e380bd677b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:54.606884 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143d6947-1523-456a-8b7c-47e380bd677b None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:54.607165 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143d6947-1523-456a-8b7c-47e380bd677b None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:54.607437 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143d6947-1523-456a-8b7c-47e380bd677b None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:54.608089 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143d6947-1523-456a-8b7c-47e380bd677b None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:54.609554 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143d6947-1523-456a-8b7c-47e380bd677b None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:54.622977 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-143d6947-1523-456a-8b7c-47e380bd677b None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 14:38:54.622977 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:54.622977 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:38:54.622977 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:38:54.622977 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:38:54.622977 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:38:54.622977 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:38:54.622977 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:38:54.622977 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 14:38:54.622977 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:38:54.622977 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:38:54.622977 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:38:54.622977 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:54.622977 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:54.622977 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:54.622977 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:54.622977 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:54.624111 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:54.624111 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:54.624111 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:54.624111 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:54.624111 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:54.624111 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:54.624111 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:38:54.624111 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:38:54.624111 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:38:54.624111 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:38:54.624111 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:54.624111 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:54.624111 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:54.624111 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:54.624111 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:54.624111 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:54.625457 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:54.625457 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:38:54.625457 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:54.625457 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:54.625457 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:54.625457 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:54.625457 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:54.625457 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:54.625457 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:54.625457 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:54.625457 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:38:54.625457 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:38:54.625457 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 14:38:54.625457 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:38:54.625457 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 419/885] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 19 14:38:54 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:54.630367 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143d6947-1523-456a-8b7c-47e380bd677b None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:54.673397 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 467/886] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:38:54 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:38:54.698199 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7a9dcd-7706-4488-ab35-56ced370ce1d None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:54.711551 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:54.759405 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143d6947-1523-456a-8b7c-47e380bd677b None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:54.798454 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2a3243-3f0f-44f2-902e-9dc58a727586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:54.799133 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2a3243-3f0f-44f2-902e-9dc58a727586 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:54.799421 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2a3243-3f0f-44f2-902e-9dc58a727586 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:54.799702 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2a3243-3f0f-44f2-902e-9dc58a727586 None admin] PATH_INFO: `/v3/services` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:54.800544 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2a3243-3f0f-44f2-902e-9dc58a727586 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:54.800862 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2a3243-3f0f-44f2-902e-9dc58a727586 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:54.802280 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2a3243-3f0f-44f2-902e-9dc58a727586 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:54.807929 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 420/887] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 19 14:38:54 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:38:54.813140 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2a3243-3f0f-44f2-902e-9dc58a727586 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:54.832742 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7a9dcd-7706-4488-ab35-56ced370ce1d None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:54.883707 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840ecebc-fb21-468c-a9f7-4554f11c1fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:54.885596 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840ecebc-fb21-468c-a9f7-4554f11c1fc5 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:54.886165 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840ecebc-fb21-468c-a9f7-4554f11c1fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:54.886809 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840ecebc-fb21-468c-a9f7-4554f11c1fc5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:54.901746 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840ecebc-fb21-468c-a9f7-4554f11c1fc5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:38:54.913802 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-840ecebc-fb21-468c-a9f7-4554f11c1fc5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 14:38:54.913802 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:54.913802 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:54.913802 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:54.913802 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:54.913802 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:54.913802 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:54.913802 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:54.913802 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:54.913802 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:54.913802 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:54.913802 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:54.913802 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:38:54.913802 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:38:54.913802 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:38:54.913802 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:38:54.914917 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:38:54.914917 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:38:54.914917 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:38:54.914917 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:38:54.914917 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:54.914917 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:54.914917 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:54.914917 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:54.914917 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:54.914917 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:54.914917 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:54.914917 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:38:54.914917 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:54.914917 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:54.914917 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:54.914917 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:54.915852 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:54.915852 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:54.915852 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:54.915852 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:54.915852 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:38:54.915852 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:38:54.915852 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:38:54.915852 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:38:54.915852 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:38:54.915852 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:38:54.915852 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 14:38:54.915852 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:38:54.921999 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 468/888] 10.222.0.57 () {56 vars in 1120 bytes} [Thu May 19 14:38:54 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:38:54.925918 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840ecebc-fb21-468c-a9f7-4554f11c1fc5 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:54.935103 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2a3243-3f0f-44f2-902e-9dc58a727586 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:54.947353 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:54.968857 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:54.976658 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a1f974-fa5e-4f28-9f6f-939496862874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:54.977604 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a1f974-fa5e-4f28-9f6f-939496862874 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:54.978145 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a1f974-fa5e-4f28-9f6f-939496862874 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:54.978613 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a1f974-fa5e-4f28-9f6f-939496862874 None admin] PATH_INFO: `/v3/services` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:54.979628 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a1f974-fa5e-4f28-9f6f-939496862874 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:54.980153 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a1f974-fa5e-4f28-9f6f-939496862874 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:54.982027 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a1f974-fa5e-4f28-9f6f-939496862874 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:54.993711 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 421/889] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 19 14:38:54 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:38:54.994918 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a1f974-fa5e-4f28-9f6f-939496862874 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:55.063891 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840ecebc-fb21-468c-a9f7-4554f11c1fc5 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:55.106945 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a1f974-fa5e-4f28-9f6f-939496862874 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:55.108998 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe21e70-7182-4239-bcc0-cc6c5f70133a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:55.109766 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe21e70-7182-4239-bcc0-cc6c5f70133a None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:55.110974 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe21e70-7182-4239-bcc0-cc6c5f70133a None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:55.111335 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe21e70-7182-4239-bcc0-cc6c5f70133a None admin] PATH_INFO: `/v3/roles` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:55.112174 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe21e70-7182-4239-bcc0-cc6c5f70133a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:55.112580 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe21e70-7182-4239-bcc0-cc6c5f70133a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:55.114116 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe21e70-7182-4239-bcc0-cc6c5f70133a None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:55.129931 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 469/890] 10.222.0.57 () {56 vars in 1115 bytes} [Thu May 19 14:38:54 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:38:55.132469 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe21e70-7182-4239-bcc0-cc6c5f70133a None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:55.151266 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74439789-7db0-47b9-9fe1-76692ea43ff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:55.152386 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74439789-7db0-47b9-9fe1-76692ea43ff3 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:55.152882 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74439789-7db0-47b9-9fe1-76692ea43ff3 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:55.153339 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74439789-7db0-47b9-9fe1-76692ea43ff3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:55.154379 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74439789-7db0-47b9-9fe1-76692ea43ff3 None admin] RBAC: Adding query filter params (interface=public, service_id=54da9fbe472c4669ae020ab3cc870aa2, region_id=RegionOne) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:55.154948 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74439789-7db0-47b9-9fe1-76692ea43ff3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:55.156762 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74439789-7db0-47b9-9fe1-76692ea43ff3 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:55.170474 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 422/891] 10.222.0.57 () {56 vars in 1251 bytes} [Thu May 19 14:38:54 2022] GET /identity/v3/endpoints?service_id=54da9fbe472c4669ae020ab3cc870aa2&interface=public®ion_id=RegionOne => generated 194 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:55.172453 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74439789-7db0-47b9-9fe1-76692ea43ff3 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:55.240585 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:55.250760 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:55.279636 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe21e70-7182-4239-bcc0-cc6c5f70133a None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:55.306552 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74439789-7db0-47b9-9fe1-76692ea43ff3 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:55.319966 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff5e132-61b9-441f-9dd0-ab23c2a52fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:55.320837 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff5e132-61b9-441f-9dd0-ab23c2a52fe6 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:55.321120 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff5e132-61b9-441f-9dd0-ab23c2a52fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:55.321392 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff5e132-61b9-441f-9dd0-ab23c2a52fe6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:55.332901 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff5e132-61b9-441f-9dd0-ab23c2a52fe6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:55.334457 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff5e132-61b9-441f-9dd0-ab23c2a52fe6 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:55.348908 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f1f525-0d70-4841-bc5b-0ce42fcc8854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:55.350717 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f1f525-0d70-4841-bc5b-0ce42fcc8854 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:55.351604 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f1f525-0d70-4841-bc5b-0ce42fcc8854 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:55.352109 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f1f525-0d70-4841-bc5b-0ce42fcc8854 None admin] PATH_INFO: `/v3/services` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:55.353028 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f1f525-0d70-4841-bc5b-0ce42fcc8854 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:55.354331 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-8ff5e132-61b9-441f-9dd0-ab23c2a52fe6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:55.354331 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:55.354331 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:55.354331 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:55.354331 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:55.354331 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:55.354331 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:55.354331 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:55.354331 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:55.354331 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:55.354331 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:55.354331 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:55.354331 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:38:55.354331 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:38:55.354331 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:38:55.355294 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:38:55.355294 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:55.355294 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:55.355294 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:55.355294 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:55.355294 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:55.355294 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:55.355294 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:55.355294 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:38:55.355294 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:55.355294 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:55.355294 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:55.355294 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:55.355294 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:55.355294 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:55.355294 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:55.355294 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:55.356277 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:38:55.356277 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:38:55.356277 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:55.356277 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:38:55.357281 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f1f525-0d70-4841-bc5b-0ce42fcc8854 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:55.360782 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 470/892] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:38:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:55.362118 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff5e132-61b9-441f-9dd0-ab23c2a52fe6 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:55.370163 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 423/893] 10.222.0.57 () {56 vars in 1087 bytes} [Thu May 19 14:38:55 2022] GET /identity/v3/services => generated 1268 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 14:38:55.487255 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff5e132-61b9-441f-9dd0-ab23c2a52fe6 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:55.516657 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:55.526324 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce96154-e1c8-4407-97c9-0184fea7f242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:55.527567 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce96154-e1c8-4407-97c9-0184fea7f242 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:55.527848 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce96154-e1c8-4407-97c9-0184fea7f242 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:55.528119 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce96154-e1c8-4407-97c9-0184fea7f242 None admin] PATH_INFO: `/v3/domains` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:55.528855 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce96154-e1c8-4407-97c9-0184fea7f242 None admin] RBAC: Adding query filter params (name=Default) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:55.529162 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce96154-e1c8-4407-97c9-0184fea7f242 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:55.530863 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce96154-e1c8-4407-97c9-0184fea7f242 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:55.550008 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 471/894] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:38:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:55.551779 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f1f525-0d70-4841-bc5b-0ce42fcc8854 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:55.620219 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136ad551-e0d9-4280-b331-7eb844b786e8 None None] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:55.620442 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136ad551-e0d9-4280-b331-7eb844b786e8 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:55.620883 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136ad551-e0d9-4280-b331-7eb844b786e8 None None] PATH_INFO: `/` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:55.622311 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 472/895] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:38:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:38:55.627502 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afde2667-55fa-46d9-9ecc-eaf7aeacab29 None None] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:55.627793 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afde2667-55fa-46d9-9ecc-eaf7aeacab29 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:55.628136 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afde2667-55fa-46d9-9ecc-eaf7aeacab29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:55.646490 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.auth.core [None req-afde2667-55fa-46d9-9ecc-eaf7aeacab29 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:38:55.648226 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.fernet_utils [None req-afde2667-55fa-46d9-9ecc-eaf7aeacab29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:38:55.672731 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f1f525-0d70-4841-bc5b-0ce42fcc8854 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:55.700667 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:55.716532 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fbb8f4-ec82-49ea-8017-7fb2605a4d90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:55.717613 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fbb8f4-ec82-49ea-8017-7fb2605a4d90 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:55.718040 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fbb8f4-ec82-49ea-8017-7fb2605a4d90 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:55.718444 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fbb8f4-ec82-49ea-8017-7fb2605a4d90 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:55.728968 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fbb8f4-ec82-49ea-8017-7fb2605a4d90 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:55.730757 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fbb8f4-ec82-49ea-8017-7fb2605a4d90 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:55.744736 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 473/896] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:38:55 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:38:55.757545 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-a8fbb8f4-ec82-49ea-8017-7fb2605a4d90 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 14:38:55.757545 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:55.757545 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:55.757545 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:55.757545 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:55.757545 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:55.757545 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:55.757545 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:55.757545 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:55.757545 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:55.757545 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:55.757545 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:55.757545 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:38:55.757545 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:38:55.757545 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:38:55.757545 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:38:55.758664 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:55.758664 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:55.758664 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:38:55.758664 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:38:55.758664 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:38:55.758664 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:38:55.758664 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:55.758664 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:55.758664 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:55.758664 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:55.758664 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:55.758664 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:38:55.758664 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:55.758664 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:55.758664 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:55.758664 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:55.758664 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:55.758664 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:55.759916 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:55.759916 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:55.759916 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:38:55.759916 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:38:55.759916 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:38:55.759916 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:38:55.759916 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:38:55.759916 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:38:55.759916 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 14:38:55.759916 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:38:55.759916 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 424/897] 10.222.0.57 () {56 vars in 1117 bytes} [Thu May 19 14:38:55 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:38:55.765065 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afde2667-55fa-46d9-9ecc-eaf7aeacab29 None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:55.774189 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fbb8f4-ec82-49ea-8017-7fb2605a4d90 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:55.895711 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afde2667-55fa-46d9-9ecc-eaf7aeacab29 None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:55.896659 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fbb8f4-ec82-49ea-8017-7fb2605a4d90 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:55.914931 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:55.935975 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2060f1f7-3423-48f4-8be5-1c0916c4a7f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:55.936450 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2060f1f7-3423-48f4-8be5-1c0916c4a7f5 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:55.936735 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2060f1f7-3423-48f4-8be5-1c0916c4a7f5 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:55.937008 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2060f1f7-3423-48f4-8be5-1c0916c4a7f5 None admin] PATH_INFO: `/v3/users` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:55.937867 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2060f1f7-3423-48f4-8be5-1c0916c4a7f5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:55.938209 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2060f1f7-3423-48f4-8be5-1c0916c4a7f5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:55.939665 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2060f1f7-3423-48f4-8be5-1c0916c4a7f5 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:55.941602 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17be7b8-f759-45ea-a8b8-edbc02d52783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:55.943293 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17be7b8-f759-45ea-a8b8-edbc02d52783 None admin] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:55.943765 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17be7b8-f759-45ea-a8b8-edbc02d52783 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:55.944921 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17be7b8-f759-45ea-a8b8-edbc02d52783 None admin] PATH_INFO: `/v3/services` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:55.945863 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17be7b8-f759-45ea-a8b8-edbc02d52783 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:55.947763 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17be7b8-f759-45ea-a8b8-edbc02d52783 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:55.960513 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 474/898] 10.222.0.57 () {56 vars in 1149 bytes} [Thu May 19 14:38:55 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:55.961072 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9606cf20-3722-412c-aa4f-368cc91214b2 None None] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:55.961369 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9606cf20-3722-412c-aa4f-368cc91214b2 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:55.961650 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9606cf20-3722-412c-aa4f-368cc91214b2 None None] PATH_INFO: `/` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:55.963275 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 475/899] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:38:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:38:55.964751 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9606cf20-3722-412c-aa4f-368cc91214b2 None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:55.967527 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 425/900] 10.222.0.57 () {60 vars in 1141 bytes} [Thu May 19 14:38:55 2022] POST /identity/v3/services => generated 241 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 14:38:55.969016 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2f6bd2-00a8-4640-aea4-20e368f86404 None None] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:55.969306 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2f6bd2-00a8-4640-aea4-20e368f86404 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:55.969580 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2f6bd2-00a8-4640-aea4-20e368f86404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:55.990097 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.auth.core [None req-1f2f6bd2-00a8-4640-aea4-20e368f86404 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:38:55.991593 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-1f2f6bd2-00a8-4640-aea4-20e368f86404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:38:56.046273 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:56.086358 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 426/901] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:38:55 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:38:56.092018 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9606cf20-3722-412c-aa4f-368cc91214b2 None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:56.110664 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2f6bd2-00a8-4640-aea4-20e368f86404 None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:56.133550 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffbf7b2-6b02-4372-a325-fd6757ed5d07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:56.134311 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffbf7b2-6b02-4372-a325-fd6757ed5d07 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:56.134603 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffbf7b2-6b02-4372-a325-fd6757ed5d07 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:56.134883 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffbf7b2-6b02-4372-a325-fd6757ed5d07 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:56.192262 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:56.226963 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffbf7b2-6b02-4372-a325-fd6757ed5d07 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:56.228560 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffbf7b2-6b02-4372-a325-fd6757ed5d07 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:56.240442 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2f6bd2-00a8-4640-aea4-20e368f86404 None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:56.298374 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c438b2-2347-48da-bcfe-2939a690e823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:56.300011 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c438b2-2347-48da-bcfe-2939a690e823 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:56.301049 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c438b2-2347-48da-bcfe-2939a690e823 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:56.301892 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c438b2-2347-48da-bcfe-2939a690e823 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:56.304005 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c438b2-2347-48da-bcfe-2939a690e823 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:56.306903 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c438b2-2347-48da-bcfe-2939a690e823 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:56.326880 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-15c438b2-2347-48da-bcfe-2939a690e823 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 14:38:56.326880 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:56.326880 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:38:56.326880 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:38:56.326880 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:38:56.326880 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:38:56.326880 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:38:56.326880 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:38:56.326880 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 14:38:56.326880 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:38:56.326880 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:38:56.326880 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:38:56.326880 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:56.326880 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:56.326880 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:56.326880 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:56.327964 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:56.327964 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:56.327964 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:56.327964 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:56.327964 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:56.327964 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:56.327964 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:56.327964 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:38:56.327964 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:38:56.327964 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:38:56.327964 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:38:56.327964 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:56.327964 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:56.327964 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:56.327964 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:56.327964 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:56.329183 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:56.329183 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:56.329183 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:38:56.329183 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:56.329183 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:56.329183 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:56.329183 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:56.329183 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:56.329183 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:56.329183 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:56.329183 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:56.329183 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:38:56.329183 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:38:56.329183 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 14:38:56.329183 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:38:56.334239 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 427/902] 10.222.0.57 () {56 vars in 1132 bytes} [Thu May 19 14:38:56 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:38:56.336308 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c438b2-2347-48da-bcfe-2939a690e823 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:56.371612 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 476/903] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:38:55 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 19 14:38:56.377187 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffbf7b2-6b02-4372-a325-fd6757ed5d07 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:56.394194 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:56.462075 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:56.479756 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c438b2-2347-48da-bcfe-2939a690e823 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:56.490955 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffbf7b2-6b02-4372-a325-fd6757ed5d07 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:56.528615 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad271ba-e0ed-40a6-b411-28ce0407c828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:56.529412 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad271ba-e0ed-40a6-b411-28ce0407c828 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:56.529910 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad271ba-e0ed-40a6-b411-28ce0407c828 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:56.530270 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad271ba-e0ed-40a6-b411-28ce0407c828 None admin] PATH_INFO: `/v3/services` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:56.531130 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad271ba-e0ed-40a6-b411-28ce0407c828 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:56.531521 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad271ba-e0ed-40a6-b411-28ce0407c828 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:56.532535 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e83b398-1c65-479c-ad37-ce65780a0dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:56.533044 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad271ba-e0ed-40a6-b411-28ce0407c828 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:56.533747 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e83b398-1c65-479c-ad37-ce65780a0dc6 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:56.534443 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e83b398-1c65-479c-ad37-ce65780a0dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:56.534931 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e83b398-1c65-479c-ad37-ce65780a0dc6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:56.539232 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 428/904] 10.222.0.57 () {56 vars in 1126 bytes} [Thu May 19 14:38:56 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:56.545393 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad271ba-e0ed-40a6-b411-28ce0407c828 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:56.555552 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e83b398-1c65-479c-ad37-ce65780a0dc6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:56.557090 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e83b398-1c65-479c-ad37-ce65780a0dc6 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:56.580010 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-1e83b398-1c65-479c-ad37-ce65780a0dc6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:56.580010 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:56.580010 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:56.580010 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:56.580010 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:56.580010 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:56.580010 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:56.580010 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:56.580010 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:56.580010 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:56.580010 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:56.580010 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:56.580010 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:38:56.580010 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:38:56.580010 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:38:56.581048 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:38:56.581048 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:56.581048 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:56.581048 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:56.581048 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:56.581048 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:56.581048 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:56.581048 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:56.581048 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:38:56.581048 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:56.581048 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:56.581048 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:56.581048 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:56.581048 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:56.581048 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:56.581048 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:56.581048 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:56.582081 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:38:56.582081 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:38:56.582081 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:56.582081 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:38:56.586621 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 477/905] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:38:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:38:56.587537 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e83b398-1c65-479c-ad37-ce65780a0dc6 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:56.665066 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:56.670020 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad271ba-e0ed-40a6-b411-28ce0407c828 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:56.699671 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e83b398-1c65-479c-ad37-ce65780a0dc6 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:56.706810 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd4e779-3582-4caf-bb57-f255a070bc38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:56.707534 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd4e779-3582-4caf-bb57-f255a070bc38 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:56.707828 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd4e779-3582-4caf-bb57-f255a070bc38 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:56.708112 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd4e779-3582-4caf-bb57-f255a070bc38 None admin] PATH_INFO: `/v3/services` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:56.708833 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd4e779-3582-4caf-bb57-f255a070bc38 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:56.709147 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd4e779-3582-4caf-bb57-f255a070bc38 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:56.711128 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd4e779-3582-4caf-bb57-f255a070bc38 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:56.721404 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 429/906] 10.222.0.57 () {56 vars in 1126 bytes} [Thu May 19 14:38:56 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:56.726820 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd4e779-3582-4caf-bb57-f255a070bc38 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:56.746232 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:56.753068 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a109932-afed-49da-b0bd-a4f3a7c01926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:56.753873 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a109932-afed-49da-b0bd-a4f3a7c01926 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:56.754341 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a109932-afed-49da-b0bd-a4f3a7c01926 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:56.754693 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a109932-afed-49da-b0bd-a4f3a7c01926 None admin] PATH_INFO: `/v3/domains` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:56.755590 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a109932-afed-49da-b0bd-a4f3a7c01926 None admin] RBAC: Adding query filter params (name=Default) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:56.755965 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a109932-afed-49da-b0bd-a4f3a7c01926 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:56.758105 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a109932-afed-49da-b0bd-a4f3a7c01926 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:56.771386 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 478/907] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:38:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:56.776808 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a109932-afed-49da-b0bd-a4f3a7c01926 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:56.869271 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd4e779-3582-4caf-bb57-f255a070bc38 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:56.895459 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a109932-afed-49da-b0bd-a4f3a7c01926 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:56.916796 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b12429c-decc-4a73-9a17-e291c81954b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:56.917495 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b12429c-decc-4a73-9a17-e291c81954b8 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:56.917785 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b12429c-decc-4a73-9a17-e291c81954b8 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:56.918097 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b12429c-decc-4a73-9a17-e291c81954b8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:56.919348 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b12429c-decc-4a73-9a17-e291c81954b8 None admin] RBAC: Adding query filter params (interface=public, service_id=dbbd881951ff453491a83e7ac837fe18, region_id=RegionOne) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:56.919670 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b12429c-decc-4a73-9a17-e291c81954b8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:56.921373 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b12429c-decc-4a73-9a17-e291c81954b8 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:56.930732 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 430/908] 10.222.0.57 () {56 vars in 1251 bytes} [Thu May 19 14:38:56 2022] GET /identity/v3/endpoints?service_id=dbbd881951ff453491a83e7ac837fe18&interface=public®ion_id=RegionOne => generated 194 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:56.935246 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b12429c-decc-4a73-9a17-e291c81954b8 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:56.941301 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e6c5d5-a5d1-4808-b13d-f2844e700ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:56.942166 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e6c5d5-a5d1-4808-b13d-f2844e700ed6 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:56.942467 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e6c5d5-a5d1-4808-b13d-f2844e700ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:56.942746 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e6c5d5-a5d1-4808-b13d-f2844e700ed6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:56.950406 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e6c5d5-a5d1-4808-b13d-f2844e700ed6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:56.951703 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e6c5d5-a5d1-4808-b13d-f2844e700ed6 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:56.967427 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-29e6c5d5-a5d1-4808-b13d-f2844e700ed6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:38:56.967427 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:56.967427 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:56.967427 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:56.967427 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:56.967427 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:56.967427 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:56.967427 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:56.967427 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:56.967427 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:56.967427 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:56.967427 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:56.967427 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:38:56.967427 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:38:56.967427 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:38:56.968420 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:38:56.968420 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:56.968420 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:56.968420 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:56.968420 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:56.968420 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:56.968420 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:56.968420 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:56.968420 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:38:56.968420 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:56.968420 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:56.968420 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:56.968420 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:56.968420 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:56.968420 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:56.968420 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:56.968420 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:56.970022 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:38:56.970022 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:38:56.970022 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:38:56.970022 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:38:56.970022 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:38:56.970022 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:38:56.970022 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:38:56.970022 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:38:56.973010 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:56.973605 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 479/909] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:38:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:56.974471 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e6c5d5-a5d1-4808-b13d-f2844e700ed6 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:57.030910 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:57.077729 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b12429c-decc-4a73-9a17-e291c81954b8 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:57.092702 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e6c5d5-a5d1-4808-b13d-f2844e700ed6 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:57.121714 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99df245f-6607-41ef-ac8b-eb4ed27a640d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:57.122834 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99df245f-6607-41ef-ac8b-eb4ed27a640d None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:57.123412 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99df245f-6607-41ef-ac8b-eb4ed27a640d None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:57.123913 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99df245f-6607-41ef-ac8b-eb4ed27a640d None admin] PATH_INFO: `/v3/services` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:57.124815 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99df245f-6607-41ef-ac8b-eb4ed27a640d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:57.126678 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99df245f-6607-41ef-ac8b-eb4ed27a640d None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:57.131381 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a2c380-93a4-4aca-a2b6-476a1292e49e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:57.132709 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a2c380-93a4-4aca-a2b6-476a1292e49e None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:57.133044 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a2c380-93a4-4aca-a2b6-476a1292e49e None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:57.133335 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a2c380-93a4-4aca-a2b6-476a1292e49e None admin] PATH_INFO: `/v3/projects` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:57.134170 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a2c380-93a4-4aca-a2b6-476a1292e49e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:57.134528 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a2c380-93a4-4aca-a2b6-476a1292e49e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:57.136016 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a2c380-93a4-4aca-a2b6-476a1292e49e None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:57.142002 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 431/910] 10.222.0.57 () {56 vars in 1087 bytes} [Thu May 19 14:38:56 2022] GET /identity/v3/services => generated 1497 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 14:38:57.149586 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 480/911] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:38:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:38:57.155751 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99df245f-6607-41ef-ac8b-eb4ed27a640d None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:57.265125 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:57.278065 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99df245f-6607-41ef-ac8b-eb4ed27a640d None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:57.322552 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8cca6f-e0a3-4ca6-b5ca-1193e7d03ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:57.323235 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8cca6f-e0a3-4ca6-b5ca-1193e7d03ed4 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:57.323503 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8cca6f-e0a3-4ca6-b5ca-1193e7d03ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:57.323778 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8cca6f-e0a3-4ca6-b5ca-1193e7d03ed4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:57.324637 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8cca6f-e0a3-4ca6-b5ca-1193e7d03ed4 None admin] RBAC: Adding query filter params (role.id=db02c08ef7c847c0ba25ed9ddc05974c, scope.project.id=42f35b96b9d4499abfb5e5f7cca49adc, user.id=2cbf483c6556421fa36179722100f8b9) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:57.324954 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8cca6f-e0a3-4ca6-b5ca-1193e7d03ed4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:57.326486 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8cca6f-e0a3-4ca6-b5ca-1193e7d03ed4 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:57.335843 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 432/912] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:38:57 2022] GET /identity/v3/role_assignments?user.id=2cbf483c6556421fa36179722100f8b9&scope.project.id=42f35b96b9d4499abfb5e5f7cca49adc&role.id=db02c08ef7c847c0ba25ed9ddc05974c => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:57.845011 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544e1aac-2255-4d97-91e7-85073e1b59e9 None None] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:57.845585 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544e1aac-2255-4d97-91e7-85073e1b59e9 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:57.845688 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544e1aac-2255-4d97-91e7-85073e1b59e9 None None] PATH_INFO: `/` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:57.847286 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 481/913] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:38:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:38:57.853016 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288f4bb1-26b9-4f9c-90b0-3acd5d828ef5 None None] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:57.853316 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288f4bb1-26b9-4f9c-90b0-3acd5d828ef5 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:57.853602 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288f4bb1-26b9-4f9c-90b0-3acd5d828ef5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:57.871913 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.auth.core [None req-288f4bb1-26b9-4f9c-90b0-3acd5d828ef5 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:38:57.873129 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-288f4bb1-26b9-4f9c-90b0-3acd5d828ef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:38:57.946532 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 433/914] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:38:57 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:38:57.970850 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544e1aac-2255-4d97-91e7-85073e1b59e9 None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:58.084929 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544e1aac-2255-4d97-91e7-85073e1b59e9 None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:58.092328 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:58.122291 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e4bbdc-e4e2-4ee5-b239-c5af0d0e825b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:58.123009 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e4bbdc-e4e2-4ee5-b239-c5af0d0e825b None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:58.123293 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e4bbdc-e4e2-4ee5-b239-c5af0d0e825b None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:58.123569 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e4bbdc-e4e2-4ee5-b239-c5af0d0e825b None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:58.124225 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e4bbdc-e4e2-4ee5-b239-c5af0d0e825b None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:58.125703 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e4bbdc-e4e2-4ee5-b239-c5af0d0e825b None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:58.138624 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-17e4bbdc-e4e2-4ee5-b239-c5af0d0e825b None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 14:38:58.138624 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:58.138624 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:38:58.138624 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:38:58.138624 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:38:58.138624 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:38:58.138624 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:38:58.138624 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:38:58.138624 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 14:38:58.138624 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:38:58.138624 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:38:58.138624 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:38:58.138624 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:58.138624 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:58.138624 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:58.138624 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:58.138624 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:58.139744 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:58.139744 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:58.139744 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:58.139744 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:58.139744 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:58.139744 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:58.139744 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:38:58.139744 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:38:58.139744 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:38:58.139744 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:38:58.139744 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:58.139744 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:58.139744 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:58.139744 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:58.139744 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:58.139744 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:58.140830 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:58.140830 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:38:58.140830 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:58.140830 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:58.140830 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:58.140830 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:58.140830 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:58.140830 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:58.140830 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:58.140830 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:58.140830 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:38:58.140830 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:38:58.140830 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 14:38:58.140830 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:38:58.140830 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 482/915] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 19 14:38:57 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:58.146292 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288f4bb1-26b9-4f9c-90b0-3acd5d828ef5 None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:58.258501 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:58.276164 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288f4bb1-26b9-4f9c-90b0-3acd5d828ef5 None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:58.311327 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675c74cd-c654-47fd-850a-33897bf61c15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:58.312074 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675c74cd-c654-47fd-850a-33897bf61c15 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:58.312356 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675c74cd-c654-47fd-850a-33897bf61c15 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:58.312625 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675c74cd-c654-47fd-850a-33897bf61c15 None admin] PATH_INFO: `/v3/services` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:58.313331 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675c74cd-c654-47fd-850a-33897bf61c15 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:58.313636 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675c74cd-c654-47fd-850a-33897bf61c15 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:58.315106 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675c74cd-c654-47fd-850a-33897bf61c15 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:58.321072 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 434/916] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 19 14:38:58 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:58.326701 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e4bbdc-e4e2-4ee5-b239-c5af0d0e825b None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:58.404096 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589f7185-41b7-4603-b527-1119d767d4f2 None None] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:58.404668 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589f7185-41b7-4603-b527-1119d767d4f2 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:58.404668 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589f7185-41b7-4603-b527-1119d767d4f2 None None] PATH_INFO: `/` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:58.406322 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 435/917] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:38:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:38:58.411353 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d83f29-6169-400f-95d8-defc30f016d5 None None] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:58.411644 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d83f29-6169-400f-95d8-defc30f016d5 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:58.411921 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d83f29-6169-400f-95d8-defc30f016d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:58.431812 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.auth.core [None req-50d83f29-6169-400f-95d8-defc30f016d5 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:38:58.433336 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-50d83f29-6169-400f-95d8-defc30f016d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:38:58.445412 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e4bbdc-e4e2-4ee5-b239-c5af0d0e825b None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:58.487688 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de020b73-e934-426b-b8ab-016c44ad0c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:58.488754 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de020b73-e934-426b-b8ab-016c44ad0c8e None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:58.489044 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de020b73-e934-426b-b8ab-016c44ad0c8e None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:58.489317 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de020b73-e934-426b-b8ab-016c44ad0c8e None admin] PATH_INFO: `/v3/services` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:58.490384 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de020b73-e934-426b-b8ab-016c44ad0c8e None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:58.491726 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de020b73-e934-426b-b8ab-016c44ad0c8e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:58.493557 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de020b73-e934-426b-b8ab-016c44ad0c8e None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:58.501915 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 483/918] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 19 14:38:58 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:58.509879 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de020b73-e934-426b-b8ab-016c44ad0c8e None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:58.522509 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 436/919] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:38:58 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:38:58.547563 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d83f29-6169-400f-95d8-defc30f016d5 None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:58.555119 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:58.630928 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:58.633009 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de020b73-e934-426b-b8ab-016c44ad0c8e None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:58.655174 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d83f29-6169-400f-95d8-defc30f016d5 None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:58.671727 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b52349-5b03-4a22-81cf-f70447b5a4ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:58.672393 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b52349-5b03-4a22-81cf-f70447b5a4ab None admin] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:58.672785 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b52349-5b03-4a22-81cf-f70447b5a4ab None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:58.673069 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b52349-5b03-4a22-81cf-f70447b5a4ab None admin] PATH_INFO: `/v3/endpoints` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:58.673942 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b52349-5b03-4a22-81cf-f70447b5a4ab None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:58.675399 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b52349-5b03-4a22-81cf-f70447b5a4ab None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:58.692467 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ee27de-b039-4138-a1b1-72a6005e5fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:58.693278 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ee27de-b039-4138-a1b1-72a6005e5fe4 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:58.693601 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ee27de-b039-4138-a1b1-72a6005e5fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:58.694732 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ee27de-b039-4138-a1b1-72a6005e5fe4 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:58.695503 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ee27de-b039-4138-a1b1-72a6005e5fe4 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:58.696987 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ee27de-b039-4138-a1b1-72a6005e5fe4 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:58.697776 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 484/920] 10.222.0.57 () {60 vars in 1144 bytes} [Thu May 19 14:38:58 2022] POST /identity/v3/endpoints => generated 346 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 14:38:58.715860 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-a3ee27de-b039-4138-a1b1-72a6005e5fe4 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 14:38:58.715860 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:58.715860 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:38:58.715860 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:38:58.715860 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:38:58.715860 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:38:58.715860 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:38:58.715860 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:38:58.715860 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 14:38:58.715860 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:38:58.715860 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:38:58.715860 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:38:58.715860 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:58.715860 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:58.715860 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:58.715860 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:58.715860 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:58.718481 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:58.718481 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:58.718481 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:58.718481 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:58.718481 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:58.718481 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:58.718481 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:38:58.718481 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:38:58.718481 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:38:58.718481 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:38:58.718481 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:58.718481 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:58.718481 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:58.718481 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:58.718481 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:58.718481 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:58.720639 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:58.720639 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:38:58.720639 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:58.720639 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:58.720639 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:58.720639 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:58.720639 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:58.720639 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:58.720639 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:58.720639 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:58.720639 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:38:58.720639 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:38:58.720639 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 14:38:58.720639 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:38:58.720639 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 437/921] 10.222.0.57 () {56 vars in 1126 bytes} [Thu May 19 14:38:58 2022] GET /identity/v3/services/object-store => generated 93 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:58.724548 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b52349-5b03-4a22-81cf-f70447b5a4ab None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:58.844020 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:58.862582 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b52349-5b03-4a22-81cf-f70447b5a4ab None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:58.903226 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d70650-ed0c-4dcc-bc76-61b58b531923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:58.903899 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d70650-ed0c-4dcc-bc76-61b58b531923 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:58.904182 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d70650-ed0c-4dcc-bc76-61b58b531923 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:58.904461 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d70650-ed0c-4dcc-bc76-61b58b531923 None admin] PATH_INFO: `/v3/services` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:58.905231 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d70650-ed0c-4dcc-bc76-61b58b531923 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:58.905485 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d70650-ed0c-4dcc-bc76-61b58b531923 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:58.906914 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d70650-ed0c-4dcc-bc76-61b58b531923 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:58.912274 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 485/922] 10.222.0.57 () {56 vars in 1122 bytes} [Thu May 19 14:38:58 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:58.918044 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ee27de-b039-4138-a1b1-72a6005e5fe4 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:59.030652 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ee27de-b039-4138-a1b1-72a6005e5fe4 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:59.068058 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551ba3db-d229-466f-a5e9-d273de176b30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:59.069301 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551ba3db-d229-466f-a5e9-d273de176b30 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:59.069618 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551ba3db-d229-466f-a5e9-d273de176b30 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:59.069935 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551ba3db-d229-466f-a5e9-d273de176b30 None admin] PATH_INFO: `/v3/services` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:59.070659 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551ba3db-d229-466f-a5e9-d273de176b30 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:59.070972 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551ba3db-d229-466f-a5e9-d273de176b30 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:59.072434 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551ba3db-d229-466f-a5e9-d273de176b30 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:59.078049 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 438/923] 10.222.0.57 () {56 vars in 1122 bytes} [Thu May 19 14:38:58 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:38:59.083670 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d70650-ed0c-4dcc-bc76-61b58b531923 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:59.211576 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d70650-ed0c-4dcc-bc76-61b58b531923 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:59.248759 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e304bddd-997a-4d2a-86e1-302b57706863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:59.249433 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e304bddd-997a-4d2a-86e1-302b57706863 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:59.249720 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e304bddd-997a-4d2a-86e1-302b57706863 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:59.250038 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e304bddd-997a-4d2a-86e1-302b57706863 None admin] PATH_INFO: `/v3/endpoints` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:59.250908 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e304bddd-997a-4d2a-86e1-302b57706863 None admin] RBAC: Adding query filter params (interface=public, service_id=ede24f9b42b4401dbca7f94c7c6fd5cb, region_id=RegionOne) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:59.251232 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e304bddd-997a-4d2a-86e1-302b57706863 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:59.252623 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e304bddd-997a-4d2a-86e1-302b57706863 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:59.259196 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 486/924] 10.222.0.57 () {56 vars in 1251 bytes} [Thu May 19 14:38:59 2022] GET /identity/v3/endpoints?service_id=ede24f9b42b4401dbca7f94c7c6fd5cb&interface=public®ion_id=RegionOne => generated 194 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:59.266954 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551ba3db-d229-466f-a5e9-d273de176b30 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:59.277368 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:59.383386 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551ba3db-d229-466f-a5e9-d273de176b30 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:59.420997 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c4fcf3-e2df-4011-b4ab-5b8a1086cf79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:59.421757 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c4fcf3-e2df-4011-b4ab-5b8a1086cf79 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:59.422066 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c4fcf3-e2df-4011-b4ab-5b8a1086cf79 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:59.422353 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c4fcf3-e2df-4011-b4ab-5b8a1086cf79 None admin] PATH_INFO: `/v3/services` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:59.423103 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c4fcf3-e2df-4011-b4ab-5b8a1086cf79 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:59.424480 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c4fcf3-e2df-4011-b4ab-5b8a1086cf79 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:59.431910 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 439/925] 10.222.0.57 () {56 vars in 1087 bytes} [Thu May 19 14:38:59 2022] GET /identity/v3/services => generated 1497 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 14:38:59.561888 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f493b74b-48ab-44b5-af8b-cd5c8a30b0ac None None] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:59.561888 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f493b74b-48ab-44b5-af8b-cd5c8a30b0ac None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:59.561888 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f493b74b-48ab-44b5-af8b-cd5c8a30b0ac None None] PATH_INFO: `/` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:59.563540 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 487/926] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:38:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:38:59.570641 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e603e5-25ef-4425-bb70-029b490d330f None None] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:59.570954 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e603e5-25ef-4425-bb70-029b490d330f None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:59.571200 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e603e5-25ef-4425-bb70-029b490d330f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:59.590113 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.auth.core [None req-b0e603e5-25ef-4425-bb70-029b490d330f None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:38:59.591341 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-b0e603e5-25ef-4425-bb70-029b490d330f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:38:59.659803 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:59.665441 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 440/927] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:38:59 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:38:59.667867 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7ebbae-2314-40ac-8371-f5dbe90fc7b0 None None] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:59.668233 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7ebbae-2314-40ac-8371-f5dbe90fc7b0 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:59.668586 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7ebbae-2314-40ac-8371-f5dbe90fc7b0 None None] PATH_INFO: `/` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:59.673677 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 488/928] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:38:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:38:59.677279 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f9103e-b828-4724-bd59-728c3c29b2e4 None None] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:59.677279 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f9103e-b828-4724-bd59-728c3c29b2e4 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:59.677279 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f9103e-b828-4724-bd59-728c3c29b2e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:59.695372 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.auth.core [None req-e6f9103e-b828-4724-bd59-728c3c29b2e4 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:38:59.696562 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-e6f9103e-b828-4724-bd59-728c3c29b2e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:38:59.697494 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7ebbae-2314-40ac-8371-f5dbe90fc7b0 None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:59.715494 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:59.778613 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 441/929] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:38:59 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:38:59.801377 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f9103e-b828-4724-bd59-728c3c29b2e4 None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:59.825660 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7ebbae-2314-40ac-8371-f5dbe90fc7b0 None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:59.863103 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac3260c-36a5-4c53-92e1-d2e3e5c69709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:59.864084 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac3260c-36a5-4c53-92e1-d2e3e5c69709 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:59.864439 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac3260c-36a5-4c53-92e1-d2e3e5c69709 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:59.864855 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac3260c-36a5-4c53-92e1-d2e3e5c69709 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:59.865548 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac3260c-36a5-4c53-92e1-d2e3e5c69709 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:59.867747 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac3260c-36a5-4c53-92e1-d2e3e5c69709 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:59.881776 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-2ac3260c-36a5-4c53-92e1-d2e3e5c69709 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 14:38:59.881776 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:59.881776 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:38:59.881776 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:38:59.881776 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:38:59.881776 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:38:59.881776 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:38:59.881776 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:38:59.881776 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 14:38:59.881776 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:38:59.881776 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:38:59.881776 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:38:59.881776 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:59.881776 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:59.881776 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:59.881776 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:59.883028 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:59.883028 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:59.883028 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:59.883028 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:59.883028 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:59.883028 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:59.883028 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:59.883028 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:38:59.883028 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:38:59.883028 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:38:59.883028 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:38:59.883028 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:59.883028 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:59.883028 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:59.883028 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:59.883028 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:59.883950 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:59.883950 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:59.883950 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:38:59.883950 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:59.883950 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:59.883950 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:59.883950 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:59.883950 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:59.883950 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:59.883950 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:59.883950 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:59.883950 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:38:59.883950 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:38:59.883950 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 14:38:59.883950 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:38:59.888292 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 489/930] 10.222.0.57 () {56 vars in 1132 bytes} [Thu May 19 14:38:59 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:59.889466 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac3260c-36a5-4c53-92e1-d2e3e5c69709 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:59.933492 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f9103e-b828-4724-bd59-728c3c29b2e4 None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:59.974361 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:59.983391 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:59.985073 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8291eef6-2a4d-4e4d-bed0-17940d5418ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:59.986035 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8291eef6-2a4d-4e4d-bed0-17940d5418ac None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:59.986482 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8291eef6-2a4d-4e4d-bed0-17940d5418ac None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:59.986839 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8291eef6-2a4d-4e4d-bed0-17940d5418ac None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:00.000664 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8291eef6-2a4d-4e4d-bed0-17940d5418ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:39:00.010138 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-8291eef6-2a4d-4e4d-bed0-17940d5418ac None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 14:39:00.010138 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:39:00.010138 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:39:00.010138 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:39:00.010138 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:39:00.010138 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:39:00.010138 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:39:00.010138 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:39:00.010138 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:39:00.010138 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:39:00.010138 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:39:00.010138 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:39:00.010138 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:39:00.010138 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:39:00.010138 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:39:00.010138 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:39:00.011249 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:39:00.011249 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:39:00.011249 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:39:00.011249 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:39:00.011249 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:39:00.011249 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:39:00.011249 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:39:00.011249 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:39:00.011249 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:39:00.011249 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:39:00.011249 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:39:00.011249 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:39:00.011249 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:39:00.011249 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:39:00.011249 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:39:00.011249 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:39:00.012185 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:39:00.012185 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:39:00.012185 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:39:00.012185 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:39:00.012185 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:39:00.012185 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:39:00.012185 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:39:00.012185 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:39:00.012185 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:39:00.012185 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:39:00.012185 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 14:39:00.012185 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:39:00.016377 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac3260c-36a5-4c53-92e1-d2e3e5c69709 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:00.017112 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 442/931] 10.222.0.57 () {56 vars in 1120 bytes} [Thu May 19 14:38:59 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:39:00.018187 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8291eef6-2a4d-4e4d-bed0-17940d5418ac None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:00.065663 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcdf77a-3290-4534-98e8-7b91bae96e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:00.067262 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcdf77a-3290-4534-98e8-7b91bae96e0c None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:00.067737 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcdf77a-3290-4534-98e8-7b91bae96e0c None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:00.068222 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcdf77a-3290-4534-98e8-7b91bae96e0c None admin] PATH_INFO: `/v3/services` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:00.069490 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcdf77a-3290-4534-98e8-7b91bae96e0c None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:39:00.070455 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcdf77a-3290-4534-98e8-7b91bae96e0c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:00.072313 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcdf77a-3290-4534-98e8-7b91bae96e0c None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:00.079276 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 490/932] 10.222.0.57 () {56 vars in 1126 bytes} [Thu May 19 14:38:59 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:39:00.084920 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcdf77a-3290-4534-98e8-7b91bae96e0c None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:00.133700 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8291eef6-2a4d-4e4d-bed0-17940d5418ac None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:00.172103 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69783621-718e-46fb-96c7-ac2dc6f5a44c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:00.173303 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69783621-718e-46fb-96c7-ac2dc6f5a44c None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:00.173808 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69783621-718e-46fb-96c7-ac2dc6f5a44c None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:00.174336 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69783621-718e-46fb-96c7-ac2dc6f5a44c None admin] PATH_INFO: `/v3/roles` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:00.175322 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69783621-718e-46fb-96c7-ac2dc6f5a44c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:39:00.175958 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69783621-718e-46fb-96c7-ac2dc6f5a44c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:00.177740 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69783621-718e-46fb-96c7-ac2dc6f5a44c None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:00.192921 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 443/933] 10.222.0.57 () {56 vars in 1115 bytes} [Thu May 19 14:39:00 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:39:00.194312 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69783621-718e-46fb-96c7-ac2dc6f5a44c None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:00.214280 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcdf77a-3290-4534-98e8-7b91bae96e0c None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:00.255210 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:00.257808 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0eb02c-f229-47ed-84b2-46b1ed84aed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:00.258803 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0eb02c-f229-47ed-84b2-46b1ed84aed5 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:00.259089 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0eb02c-f229-47ed-84b2-46b1ed84aed5 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:00.259675 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0eb02c-f229-47ed-84b2-46b1ed84aed5 None admin] PATH_INFO: `/v3/services` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:00.260400 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0eb02c-f229-47ed-84b2-46b1ed84aed5 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:39:00.260707 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0eb02c-f229-47ed-84b2-46b1ed84aed5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:00.262311 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0eb02c-f229-47ed-84b2-46b1ed84aed5 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:00.273696 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 491/934] 10.222.0.57 () {56 vars in 1126 bytes} [Thu May 19 14:39:00 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:39:00.275747 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0eb02c-f229-47ed-84b2-46b1ed84aed5 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:00.285520 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:00.323842 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69783621-718e-46fb-96c7-ac2dc6f5a44c None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:00.362876 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b8cd4c-52fd-4272-8c8d-2f98874fa248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:00.364237 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b8cd4c-52fd-4272-8c8d-2f98874fa248 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:00.364735 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b8cd4c-52fd-4272-8c8d-2f98874fa248 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:00.365243 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b8cd4c-52fd-4272-8c8d-2f98874fa248 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:00.378040 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b8cd4c-52fd-4272-8c8d-2f98874fa248 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:00.380132 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b8cd4c-52fd-4272-8c8d-2f98874fa248 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:00.399649 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0eb02c-f229-47ed-84b2-46b1ed84aed5 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:00.401656 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-29b8cd4c-52fd-4272-8c8d-2f98874fa248 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:39:00.401656 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:39:00.401656 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:39:00.401656 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:39:00.401656 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:39:00.401656 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:39:00.401656 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:39:00.401656 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:39:00.401656 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:39:00.401656 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:39:00.401656 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:39:00.401656 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:39:00.401656 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:39:00.401656 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:39:00.401656 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:39:00.402746 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:39:00.402746 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:39:00.402746 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:39:00.402746 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:39:00.402746 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:39:00.402746 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:39:00.402746 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:39:00.402746 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:39:00.402746 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:39:00.402746 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:39:00.402746 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:39:00.402746 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:39:00.402746 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:39:00.402746 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:39:00.402746 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:39:00.402746 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:39:00.402746 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:39:00.403741 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:39:00.403741 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:39:00.403741 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:39:00.403741 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:39:00.405913 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 444/935] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:39:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:39:00.409642 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b8cd4c-52fd-4272-8c8d-2f98874fa248 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:00.444563 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cc7c1d-462c-4a8e-9e1b-adad6117dd42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:00.445876 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cc7c1d-462c-4a8e-9e1b-adad6117dd42 None admin] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:00.446430 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cc7c1d-462c-4a8e-9e1b-adad6117dd42 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:00.446920 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cc7c1d-462c-4a8e-9e1b-adad6117dd42 None admin] PATH_INFO: `/v3/endpoints` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:00.447875 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cc7c1d-462c-4a8e-9e1b-adad6117dd42 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:00.449698 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cc7c1d-462c-4a8e-9e1b-adad6117dd42 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:00.482273 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 492/936] 10.222.0.57 () {60 vars in 1144 bytes} [Thu May 19 14:39:00 2022] POST /identity/v3/endpoints => generated 347 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 14:39:00.538300 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b8cd4c-52fd-4272-8c8d-2f98874fa248 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:00.541132 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:00.587522 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20c5bdb-106e-4e65-93bc-17f1ee9fd8d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:00.588204 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20c5bdb-106e-4e65-93bc-17f1ee9fd8d8 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:00.588494 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20c5bdb-106e-4e65-93bc-17f1ee9fd8d8 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:00.588775 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20c5bdb-106e-4e65-93bc-17f1ee9fd8d8 None admin] PATH_INFO: `/v3/domains` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:00.589579 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20c5bdb-106e-4e65-93bc-17f1ee9fd8d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:39:00.589922 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20c5bdb-106e-4e65-93bc-17f1ee9fd8d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:00.591413 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20c5bdb-106e-4e65-93bc-17f1ee9fd8d8 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:00.609996 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 445/937] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:39:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:39:00.612114 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cc7c1d-462c-4a8e-9e1b-adad6117dd42 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:00.685246 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:00.732426 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cc7c1d-462c-4a8e-9e1b-adad6117dd42 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:00.769601 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3027d22e-e01b-4491-b47a-c6759fe23a44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:00.770843 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3027d22e-e01b-4491-b47a-c6759fe23a44 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:00.771362 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3027d22e-e01b-4491-b47a-c6759fe23a44 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:00.772478 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3027d22e-e01b-4491-b47a-c6759fe23a44 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:00.781358 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3027d22e-e01b-4491-b47a-c6759fe23a44 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:00.783197 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3027d22e-e01b-4491-b47a-c6759fe23a44 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:00.799283 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-3027d22e-e01b-4491-b47a-c6759fe23a44 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 14:39:00.799283 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:39:00.799283 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:39:00.799283 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:39:00.799283 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:39:00.799283 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:39:00.799283 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:39:00.799283 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:39:00.799283 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:39:00.799283 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:39:00.799283 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:39:00.799283 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:39:00.799283 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:39:00.799283 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:39:00.799283 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:39:00.799283 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:39:00.800380 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:39:00.800380 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:39:00.800380 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:39:00.800380 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:39:00.800380 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:39:00.800380 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:39:00.800380 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:39:00.800380 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:39:00.800380 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:39:00.800380 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:39:00.800380 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:39:00.800380 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:39:00.800380 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:39:00.800380 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:39:00.800380 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:39:00.800380 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:39:00.800380 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:39:00.800380 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:39:00.801467 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:39:00.801467 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:39:00.801467 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:39:00.801467 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:39:00.801467 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:39:00.801467 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:39:00.801467 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:39:00.801467 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:39:00.801467 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 14:39:00.801467 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:39:00.804295 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 493/938] 10.222.0.57 () {56 vars in 1117 bytes} [Thu May 19 14:39:00 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:39:00.809971 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20c5bdb-106e-4e65-93bc-17f1ee9fd8d8 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:00.918718 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20c5bdb-106e-4e65-93bc-17f1ee9fd8d8 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:00.955869 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8faa20a-a7b7-436c-af8e-612d9adab954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:00.956557 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8faa20a-a7b7-436c-af8e-612d9adab954 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:00.956845 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8faa20a-a7b7-436c-af8e-612d9adab954 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:00.957124 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8faa20a-a7b7-436c-af8e-612d9adab954 None admin] PATH_INFO: `/v3/users` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:00.958013 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8faa20a-a7b7-436c-af8e-612d9adab954 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:39:00.958335 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8faa20a-a7b7-436c-af8e-612d9adab954 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:00.960110 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8faa20a-a7b7-436c-af8e-612d9adab954 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:00.985571 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 446/939] 10.222.0.57 () {56 vars in 1149 bytes} [Thu May 19 14:39:00 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:39:00.987033 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3027d22e-e01b-4491-b47a-c6759fe23a44 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:01.102359 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3027d22e-e01b-4491-b47a-c6759fe23a44 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:01.126201 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:01.138791 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffb81fe-95ed-4d3e-b589-1f46be0ba3af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:01.139606 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffb81fe-95ed-4d3e-b589-1f46be0ba3af None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:01.139895 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffb81fe-95ed-4d3e-b589-1f46be0ba3af None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:01.140175 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffb81fe-95ed-4d3e-b589-1f46be0ba3af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:01.151990 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffb81fe-95ed-4d3e-b589-1f46be0ba3af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:01.153545 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffb81fe-95ed-4d3e-b589-1f46be0ba3af None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:01.172013 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e88670-4c35-46bf-8106-08df3b6431a8 None None] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:01.172309 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e88670-4c35-46bf-8106-08df3b6431a8 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:01.172584 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e88670-4c35-46bf-8106-08df3b6431a8 None None] PATH_INFO: `/` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:01.174309 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 447/940] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:39:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:39:01.176439 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-4ffb81fe-95ed-4d3e-b589-1f46be0ba3af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:39:01.176439 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:39:01.176439 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:39:01.176439 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:39:01.176439 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:39:01.176439 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:39:01.176439 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:39:01.176439 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:39:01.176439 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:39:01.176439 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:39:01.176439 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:39:01.176439 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:39:01.176439 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:39:01.176439 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:39:01.176439 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:39:01.177575 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:39:01.177575 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:39:01.177575 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:39:01.177575 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:39:01.177575 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:39:01.177575 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:39:01.177575 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:39:01.177575 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:39:01.177575 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:39:01.177575 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:39:01.177575 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:39:01.177575 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:39:01.177575 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:39:01.177575 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:39:01.177575 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:39:01.177575 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:39:01.177575 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:39:01.178853 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:39:01.178853 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:39:01.178853 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:39:01.178853 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:39:01.180533 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 494/941] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:39:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:39:01.186295 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b052c87a-925d-4b90-a3ac-3b25638a7201 None None] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:01.186811 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffb81fe-95ed-4d3e-b589-1f46be0ba3af None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:01.187054 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b052c87a-925d-4b90-a3ac-3b25638a7201 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:01.187525 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b052c87a-925d-4b90-a3ac-3b25638a7201 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:01.205341 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.auth.core [None req-b052c87a-925d-4b90-a3ac-3b25638a7201 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:39:01.207131 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-b052c87a-925d-4b90-a3ac-3b25638a7201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:39:01.224734 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:01.306370 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 448/942] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:39:01 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:39:01.318252 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffb81fe-95ed-4d3e-b589-1f46be0ba3af None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:01.331403 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b052c87a-925d-4b90-a3ac-3b25638a7201 None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:01.361789 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7174694c-183b-4e2a-970f-695dba2343d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:01.362885 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7174694c-183b-4e2a-970f-695dba2343d4 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:01.363490 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7174694c-183b-4e2a-970f-695dba2343d4 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:01.364007 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7174694c-183b-4e2a-970f-695dba2343d4 None admin] PATH_INFO: `/v3/domains` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:01.364992 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7174694c-183b-4e2a-970f-695dba2343d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:39:01.365560 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7174694c-183b-4e2a-970f-695dba2343d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:01.367681 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7174694c-183b-4e2a-970f-695dba2343d4 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:01.383231 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 495/943] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:39:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:39:01.388796 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7174694c-183b-4e2a-970f-695dba2343d4 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:01.435608 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:01.462769 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b052c87a-925d-4b90-a3ac-3b25638a7201 None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:01.501755 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0de172-7778-4cb2-ac21-58701e013761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:01.502458 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0de172-7778-4cb2-ac21-58701e013761 None admin] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:01.502750 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0de172-7778-4cb2-ac21-58701e013761 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:01.503030 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0de172-7778-4cb2-ac21-58701e013761 None admin] PATH_INFO: `/v3/projects` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:01.503757 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0de172-7778-4cb2-ac21-58701e013761 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:01.505505 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0de172-7778-4cb2-ac21-58701e013761 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:01.507178 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: WARNING keystone.server.flask.common [None req-cc0de172-7778-4cb2-ac21-58701e013761 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 19 14:39:01.519604 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7174694c-183b-4e2a-970f-695dba2343d4 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:01.527099 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.sql.core [None req-cc0de172-7778-4cb2-ac21-58701e013761 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 19 14:39:01.527099 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 19 14:39:01.527099 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [parameters: {'id': '543559e767ae40c7b0b1b189ade13eb0', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 19 14:39:01.527099 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=218624) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 19 14:39:01.535187 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-cc0de172-7778-4cb2-ac21-58701e013761 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 543559e767ae40c7b0b1b189ade13eb0.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 543559e767ae40c7b0b1b189ade13eb0. May 19 14:39:01.535187 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:39:01.535187 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 14:39:01.535187 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 14:39:01.535187 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 14:39:01.535187 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 14:39:01.535187 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 14:39:01.535187 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application result = self._query(query) May 19 14:39:01.535187 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 14:39:01.535187 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application conn.query(q) May 19 14:39:01.535187 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 14:39:01.535187 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 14:39:01.535187 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 14:39:01.536076 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application result.read() May 19 14:39:01.536076 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 14:39:01.536076 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 14:39:01.536076 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 14:39:01.536076 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 14:39:01.536076 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 14:39:01.536076 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 14:39:01.536076 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 14:39:01.536076 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 14:39:01.536076 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 19 14:39:01.536076 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:39:01.536076 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 19 14:39:01.536076 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:39:01.536076 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:39:01.536076 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 19 14:39:01.536076 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 19 14:39:01.536076 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project May 19 14:39:01.536076 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return project_ref.to_dict() May 19 14:39:01.537168 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 14:39:01.537168 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application next(self.gen) May 19 14:39:01.537168 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 19 14:39:01.537168 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application yield resource May 19 14:39:01.537168 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 14:39:01.537168 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application next(self.gen) May 19 14:39:01.537168 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 19 14:39:01.537168 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application self.session.rollback() May 19 14:39:01.537168 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 19 14:39:01.537168 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application self.force_reraise() May 19 14:39:01.537168 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 19 14:39:01.537168 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise self.value May 19 14:39:01.537168 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 19 14:39:01.537168 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 19 14:39:01.537168 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 19 14:39:01.537168 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application session.commit() May 19 14:39:01.537168 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 19 14:39:01.538344 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 19 14:39:01.538344 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 19 14:39:01.538344 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application self._prepare_impl() May 19 14:39:01.538344 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 19 14:39:01.538344 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application self.session.flush() May 19 14:39:01.538344 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 19 14:39:01.538344 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application self._flush(objects) May 19 14:39:01.538344 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 19 14:39:01.538344 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 19 14:39:01.538344 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 19 14:39:01.538344 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application compat.raise_( May 19 14:39:01.538344 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 14:39:01.538344 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception May 19 14:39:01.538344 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 19 14:39:01.538344 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application flush_context.execute() May 19 14:39:01.538344 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 19 14:39:01.538344 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rec.execute(self) May 19 14:39:01.539797 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 19 14:39:01.539797 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 19 14:39:01.539797 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 19 14:39:01.539797 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application _emit_insert_statements( May 19 14:39:01.539797 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 19 14:39:01.539797 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application c = connection._execute_20( May 19 14:39:01.539797 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 19 14:39:01.539797 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 19 14:39:01.539797 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 19 14:39:01.539797 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 19 14:39:01.539797 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 19 14:39:01.539797 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application ret = self._execute_context( May 19 14:39:01.539797 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 19 14:39:01.539797 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 19 14:39:01.539797 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 19 14:39:01.540716 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 19 14:39:01.540716 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 14:39:01.540716 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception May 19 14:39:01.540716 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 14:39:01.540716 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 14:39:01.540716 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 14:39:01.540716 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 14:39:01.540716 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 14:39:01.540716 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application result = self._query(query) May 19 14:39:01.540716 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 14:39:01.540716 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application conn.query(q) May 19 14:39:01.540716 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 14:39:01.540716 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 14:39:01.540716 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 14:39:01.540716 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application result.read() May 19 14:39:01.540716 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 14:39:01.540716 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 14:39:01.541697 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 14:39:01.541697 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 14:39:01.541697 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 14:39:01.541697 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 14:39:01.541697 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 14:39:01.541697 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 14:39:01.541697 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 19 14:39:01.541697 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 19 14:39:01.541697 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [parameters: {'id': '543559e767ae40c7b0b1b189ade13eb0', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 19 14:39:01.541697 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 19 14:39:01.541697 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:39:01.541697 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:39:01.541697 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:39:01.541697 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:39:01.541697 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project May 19 14:39:01.544464 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) May 19 14:39:01.544464 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 19 14:39:01.544464 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 19 14:39:01.544464 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. May 19 14:39:01.544464 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:39:01.544464 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:39:01.544464 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:39:01.544464 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:39:01.544464 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:39:01.544464 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:39:01.544464 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:39:01.544464 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:39:01.544464 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:39:01.544464 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:39:01.544464 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:39:01.544464 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:39:01.544464 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:39:01.547155 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:39:01.547155 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post May 19 14:39:01.547155 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( May 19 14:39:01.547155 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:39:01.547155 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:39:01.547155 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project May 19 14:39:01.547155 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.Conflict( May 19 14:39:01.547155 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 543559e767ae40c7b0b1b189ade13eb0. May 19 14:39:01.547155 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:39:01.547155 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 449/944] 10.222.0.57 () {60 vars in 1140 bytes} [Thu May 19 14:39:01 2022] POST /identity/v3/projects => generated 262 bytes in 211 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 19 14:39:01.547155 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0de172-7778-4cb2-ac21-58701e013761 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:01.549745 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:01.577260 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbf109a-8d8a-46fe-94a2-6679d2123c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:01.578547 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbf109a-8d8a-46fe-94a2-6679d2123c38 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:01.579063 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbf109a-8d8a-46fe-94a2-6679d2123c38 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:01.579549 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbf109a-8d8a-46fe-94a2-6679d2123c38 None admin] PATH_INFO: `/v3/projects/service` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:01.588443 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbf109a-8d8a-46fe-94a2-6679d2123c38 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:01.590259 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbf109a-8d8a-46fe-94a2-6679d2123c38 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:01.606634 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-8fbf109a-8d8a-46fe-94a2-6679d2123c38 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:39:01.606634 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:39:01.606634 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:39:01.606634 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:39:01.606634 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:39:01.606634 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:39:01.606634 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:39:01.606634 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:39:01.606634 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:39:01.606634 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:39:01.606634 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:39:01.606634 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:39:01.606634 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:39:01.606634 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:39:01.606634 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:39:01.607714 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:39:01.607714 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:39:01.607714 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:39:01.607714 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:39:01.607714 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:39:01.607714 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:39:01.607714 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:39:01.607714 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:39:01.607714 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:39:01.607714 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:39:01.607714 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:39:01.607714 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:39:01.607714 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:39:01.607714 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:39:01.607714 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:39:01.607714 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:39:01.607714 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:39:01.608704 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:39:01.608704 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:39:01.608704 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:39:01.608704 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:39:01.608704 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:39:01.608704 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:39:01.608704 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:39:01.608704 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:39:01.611624 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 496/945] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:39:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:39:01.618725 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbf109a-8d8a-46fe-94a2-6679d2123c38 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:01.666818 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0de172-7778-4cb2-ac21-58701e013761 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:01.704231 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b72cb4-8985-46f1-9af4-c8ee39ae34e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:01.705041 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b72cb4-8985-46f1-9af4-c8ee39ae34e9 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:01.705328 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b72cb4-8985-46f1-9af4-c8ee39ae34e9 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:01.705609 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b72cb4-8985-46f1-9af4-c8ee39ae34e9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:01.713179 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b72cb4-8985-46f1-9af4-c8ee39ae34e9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:01.714950 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b72cb4-8985-46f1-9af4-c8ee39ae34e9 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:01.727999 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:01.732215 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-f5b72cb4-8985-46f1-9af4-c8ee39ae34e9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:39:01.732215 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:39:01.732215 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:39:01.732215 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:39:01.732215 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:39:01.732215 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:39:01.732215 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:39:01.732215 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:39:01.732215 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:39:01.732215 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:39:01.732215 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:39:01.732215 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:39:01.732215 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:39:01.732215 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:39:01.732215 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:39:01.733181 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:39:01.733181 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:39:01.733181 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:39:01.733181 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:39:01.733181 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:39:01.733181 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:39:01.733181 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:39:01.733181 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:39:01.733181 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:39:01.733181 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:39:01.733181 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:39:01.733181 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:39:01.733181 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:39:01.733181 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:39:01.733181 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:39:01.733181 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:39:01.733181 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:39:01.734272 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:39:01.734272 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:39:01.734272 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:39:01.734272 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:39:01.734272 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:39:01.734272 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:39:01.734272 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:39:01.734272 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:39:01.738357 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 450/946] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:39:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:39:01.739358 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbf109a-8d8a-46fe-94a2-6679d2123c38 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:01.739861 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b72cb4-8985-46f1-9af4-c8ee39ae34e9 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:01.783335 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfda9c4-f297-44a1-b675-74c6d4a02feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:01.784381 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfda9c4-f297-44a1-b675-74c6d4a02feb None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:01.784898 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfda9c4-f297-44a1-b675-74c6d4a02feb None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:01.785394 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfda9c4-f297-44a1-b675-74c6d4a02feb None admin] PATH_INFO: `/v3/projects` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:01.786428 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfda9c4-f297-44a1-b675-74c6d4a02feb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:39:01.786962 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfda9c4-f297-44a1-b675-74c6d4a02feb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:01.789787 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfda9c4-f297-44a1-b675-74c6d4a02feb None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:01.805384 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 497/947] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:39:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:39:01.807169 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023bf079-9d35-4764-8db3-c5203ef48b92 None None] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:01.807667 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023bf079-9d35-4764-8db3-c5203ef48b92 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:01.808170 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023bf079-9d35-4764-8db3-c5203ef48b92 None None] PATH_INFO: `/` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:01.809610 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 498/948] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:39:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:39:01.812285 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023bf079-9d35-4764-8db3-c5203ef48b92 None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:01.814029 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:01.858109 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b72cb4-8985-46f1-9af4-c8ee39ae34e9 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:01.897221 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce79307-0ca8-4f10-89f1-944e3fd603d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:01.897911 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce79307-0ca8-4f10-89f1-944e3fd603d9 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:01.898233 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce79307-0ca8-4f10-89f1-944e3fd603d9 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:01.898508 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce79307-0ca8-4f10-89f1-944e3fd603d9 None admin] PATH_INFO: `/v3/projects` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:01.899255 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce79307-0ca8-4f10-89f1-944e3fd603d9 None admin] RBAC: Adding query filter params (name=service) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:39:01.899567 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce79307-0ca8-4f10-89f1-944e3fd603d9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:01.901015 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce79307-0ca8-4f10-89f1-944e3fd603d9 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:01.921561 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 451/949] 10.222.0.57 () {56 vars in 1112 bytes} [Thu May 19 14:39:01 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:39:01.922926 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2e8d42-901e-482e-8c6d-ef535cb88f97 None None] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:01.923313 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2e8d42-901e-482e-8c6d-ef535cb88f97 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:01.923686 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2e8d42-901e-482e-8c6d-ef535cb88f97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:01.927364 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023bf079-9d35-4764-8db3-c5203ef48b92 None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:01.950879 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.auth.core [None req-0d2e8d42-901e-482e-8c6d-ef535cb88f97 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:39:01.952756 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-0d2e8d42-901e-482e-8c6d-ef535cb88f97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:39:01.984201 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859e7aec-0aa2-4981-a777-c2cd06402988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:01.985964 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859e7aec-0aa2-4981-a777-c2cd06402988 None admin] REQUEST_METHOD: `PUT` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:01.986831 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859e7aec-0aa2-4981-a777-c2cd06402988 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:01.987280 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859e7aec-0aa2-4981-a777-c2cd06402988 None admin] PATH_INFO: `/v3/projects/42f35b96b9d4499abfb5e5f7cca49adc/users/2cbf483c6556421fa36179722100f8b9/roles/db02c08ef7c847c0ba25ed9ddc05974c` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:01.991706 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859e7aec-0aa2-4981-a777-c2cd06402988 None admin] RBAC: Authorizing `identity:create_grant(project_id=42f35b96b9d4499abfb5e5f7cca49adc, user_id=2cbf483c6556421fa36179722100f8b9, role_id=db02c08ef7c847c0ba25ed9ddc05974c)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:01.994071 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859e7aec-0aa2-4981-a777-c2cd06402988 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:02.023013 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 499/950] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 14:39:01 2022] PUT /identity/v3/projects/42f35b96b9d4499abfb5e5f7cca49adc/users/2cbf483c6556421fa36179722100f8b9/roles/db02c08ef7c847c0ba25ed9ddc05974c => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 14:39:02.049005 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 452/951] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:39:01 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:39:02.074595 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859e7aec-0aa2-4981-a777-c2cd06402988 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:02.095531 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:02.212260 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859e7aec-0aa2-4981-a777-c2cd06402988 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:02.252172 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb4632c-5744-4193-bb80-bdb0bc732512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:02.253270 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb4632c-5744-4193-bb80-bdb0bc732512 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:02.253819 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb4632c-5744-4193-bb80-bdb0bc732512 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:02.254278 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb4632c-5744-4193-bb80-bdb0bc732512 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:02.255165 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb4632c-5744-4193-bb80-bdb0bc732512 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:02.257319 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb4632c-5744-4193-bb80-bdb0bc732512 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:02.270934 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-7cb4632c-5744-4193-bb80-bdb0bc732512 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 14:39:02.270934 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:39:02.270934 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:39:02.270934 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:39:02.270934 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:39:02.270934 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:39:02.270934 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:39:02.270934 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:39:02.270934 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 14:39:02.270934 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:39:02.270934 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:39:02.270934 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:39:02.270934 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:39:02.270934 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:39:02.270934 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:39:02.270934 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:39:02.270934 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:39:02.272886 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:39:02.272886 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:39:02.272886 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:39:02.272886 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:39:02.272886 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:39:02.272886 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:39:02.272886 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:39:02.272886 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:39:02.272886 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:39:02.272886 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:39:02.272886 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:39:02.272886 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:39:02.272886 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:39:02.272886 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:39:02.272886 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:39:02.272886 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:39:02.274290 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:39:02.274290 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:39:02.274290 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:39:02.274290 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:39:02.274290 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:39:02.274290 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:39:02.274290 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:39:02.274290 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:39:02.274290 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:39:02.274290 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:39:02.274290 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:39:02.274290 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:39:02.274290 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 14:39:02.274290 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:39:02.276946 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 500/952] 10.222.0.57 () {56 vars in 1126 bytes} [Thu May 19 14:39:02 2022] GET /identity/v3/services/object-store => generated 93 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:39:02.283295 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2e8d42-901e-482e-8c6d-ef535cb88f97 None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:02.342984 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:02.392151 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2e8d42-901e-482e-8c6d-ef535cb88f97 None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:02.429646 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad177c6-bca4-48ac-a4a1-d3a6e6fe2295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:02.430280 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad177c6-bca4-48ac-a4a1-d3a6e6fe2295 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:02.430570 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad177c6-bca4-48ac-a4a1-d3a6e6fe2295 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:02.430850 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad177c6-bca4-48ac-a4a1-d3a6e6fe2295 None admin] PATH_INFO: `/v3/services` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:02.431567 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad177c6-bca4-48ac-a4a1-d3a6e6fe2295 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:39:02.431879 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad177c6-bca4-48ac-a4a1-d3a6e6fe2295 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:02.433257 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad177c6-bca4-48ac-a4a1-d3a6e6fe2295 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:02.438828 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 453/953] 10.222.0.57 () {56 vars in 1122 bytes} [Thu May 19 14:39:02 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:39:02.444635 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb4632c-5744-4193-bb80-bdb0bc732512 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:02.530543 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:02.555609 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb4632c-5744-4193-bb80-bdb0bc732512 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:02.593454 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cee2055-3ddc-4b06-8151-b0d41f83c5f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:02.594171 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cee2055-3ddc-4b06-8151-b0d41f83c5f6 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:02.594471 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cee2055-3ddc-4b06-8151-b0d41f83c5f6 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:02.594796 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cee2055-3ddc-4b06-8151-b0d41f83c5f6 None admin] PATH_INFO: `/v3/services` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:02.595509 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cee2055-3ddc-4b06-8151-b0d41f83c5f6 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:39:02.595870 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cee2055-3ddc-4b06-8151-b0d41f83c5f6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:02.597273 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cee2055-3ddc-4b06-8151-b0d41f83c5f6 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:02.603873 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 501/954] 10.222.0.57 () {56 vars in 1122 bytes} [Thu May 19 14:39:02 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:39:02.609256 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad177c6-bca4-48ac-a4a1-d3a6e6fe2295 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:02.717912 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad177c6-bca4-48ac-a4a1-d3a6e6fe2295 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:02.738799 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:02.758482 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a899ba-b3ee-4f27-a916-d18b32283309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:02.759221 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a899ba-b3ee-4f27-a916-d18b32283309 None admin] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:02.759511 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a899ba-b3ee-4f27-a916-d18b32283309 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:02.759791 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a899ba-b3ee-4f27-a916-d18b32283309 None admin] PATH_INFO: `/v3/endpoints` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:02.760557 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a899ba-b3ee-4f27-a916-d18b32283309 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:02.761954 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a899ba-b3ee-4f27-a916-d18b32283309 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:02.786245 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 454/955] 10.222.0.57 () {60 vars in 1144 bytes} [Thu May 19 14:39:02 2022] POST /identity/v3/endpoints => generated 349 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:39:02.903846 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be358602-1244-4cba-b153-eb267e79eab6 None None] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:02.904132 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be358602-1244-4cba-b153-eb267e79eab6 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:02.904412 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be358602-1244-4cba-b153-eb267e79eab6 None None] PATH_INFO: `/` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:02.906202 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 502/956] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:39:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:39:02.912049 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562d9745-7e4a-42f2-bd82-983002443317 None None] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:02.912383 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562d9745-7e4a-42f2-bd82-983002443317 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:02.912679 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562d9745-7e4a-42f2-bd82-983002443317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:02.931625 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.auth.core [None req-562d9745-7e4a-42f2-bd82-983002443317 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:39:02.932877 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-562d9745-7e4a-42f2-bd82-983002443317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:39:03.014618 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 455/957] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:39:02 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:39:03.037865 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be358602-1244-4cba-b153-eb267e79eab6 None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:03.143968 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be358602-1244-4cba-b153-eb267e79eab6 None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:03.192177 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c38f2f-6a9c-4e7f-a462-a00ced2e113d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:03.193121 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c38f2f-6a9c-4e7f-a462-a00ced2e113d None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:03.193412 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c38f2f-6a9c-4e7f-a462-a00ced2e113d None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:03.193864 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c38f2f-6a9c-4e7f-a462-a00ced2e113d None admin] PATH_INFO: `/v3/services/compute` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:03.195154 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c38f2f-6a9c-4e7f-a462-a00ced2e113d None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:03.196563 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c38f2f-6a9c-4e7f-a462-a00ced2e113d None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:03.209082 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-c8c38f2f-6a9c-4e7f-a462-a00ced2e113d None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 19 14:39:03.209082 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:39:03.209082 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:39:03.209082 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:39:03.209082 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:39:03.209082 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:39:03.209082 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:39:03.209082 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:39:03.209082 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 14:39:03.209082 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:39:03.209082 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:39:03.209082 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:39:03.209082 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:39:03.209082 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:39:03.209082 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:39:03.209082 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:39:03.209082 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:39:03.210518 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:39:03.210518 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:39:03.210518 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:39:03.210518 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:39:03.210518 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:39:03.210518 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:39:03.210518 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:39:03.210518 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:39:03.210518 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:39:03.210518 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:39:03.210518 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:39:03.210518 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:39:03.210518 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:39:03.210518 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:39:03.210518 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:39:03.210518 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:39:03.211448 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:39:03.211448 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:39:03.211448 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:39:03.211448 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:39:03.211448 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:39:03.211448 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:39:03.211448 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:39:03.211448 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:39:03.211448 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:39:03.211448 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:39:03.211448 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:39:03.211448 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:39:03.211448 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 14:39:03.211448 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:39:03.211448 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 503/958] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:39:03 2022] GET /identity/v3/services/compute => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:39:03.217177 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562d9745-7e4a-42f2-bd82-983002443317 None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:03.323342 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:03.326511 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562d9745-7e4a-42f2-bd82-983002443317 None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:03.363921 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a7f94d-91a2-4dff-aa0a-124a217ca72b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:03.364595 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a7f94d-91a2-4dff-aa0a-124a217ca72b None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:03.364878 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a7f94d-91a2-4dff-aa0a-124a217ca72b None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:03.365156 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a7f94d-91a2-4dff-aa0a-124a217ca72b None admin] PATH_INFO: `/v3/services` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:03.366453 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a7f94d-91a2-4dff-aa0a-124a217ca72b None admin] RBAC: Adding query filter params (name=compute) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:39:03.366966 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a7f94d-91a2-4dff-aa0a-124a217ca72b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:03.368356 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a7f94d-91a2-4dff-aa0a-124a217ca72b None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:03.373800 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 456/959] 10.222.0.57 () {56 vars in 1112 bytes} [Thu May 19 14:39:03 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:39:03.379877 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c38f2f-6a9c-4e7f-a462-a00ced2e113d None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:03.393933 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:03.495188 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c38f2f-6a9c-4e7f-a462-a00ced2e113d None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:03.535102 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092c1ecf-a450-4e11-ba47-c75f5c7fa831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:03.535775 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092c1ecf-a450-4e11-ba47-c75f5c7fa831 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:03.536062 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092c1ecf-a450-4e11-ba47-c75f5c7fa831 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:03.536340 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092c1ecf-a450-4e11-ba47-c75f5c7fa831 None admin] PATH_INFO: `/v3/services` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:03.537050 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092c1ecf-a450-4e11-ba47-c75f5c7fa831 None admin] RBAC: Adding query filter params (type=compute) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:39:03.537395 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092c1ecf-a450-4e11-ba47-c75f5c7fa831 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:03.539027 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092c1ecf-a450-4e11-ba47-c75f5c7fa831 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:03.544175 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 504/960] 10.222.0.57 () {56 vars in 1112 bytes} [Thu May 19 14:39:03 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:39:04.372126 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d440821f-81b1-46f5-ad60-41e82f5b0ad2 None None] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:04.373108 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d440821f-81b1-46f5-ad60-41e82f5b0ad2 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:04.373622 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d440821f-81b1-46f5-ad60-41e82f5b0ad2 None None] PATH_INFO: `/` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:04.375565 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 457/961] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:39:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:39:04.381737 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5147e4b2-0beb-459f-a7c1-f73cf9e9e0db None None] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:04.382661 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5147e4b2-0beb-459f-a7c1-f73cf9e9e0db None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:04.383228 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5147e4b2-0beb-459f-a7c1-f73cf9e9e0db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:04.402308 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.auth.core [None req-5147e4b2-0beb-459f-a7c1-f73cf9e9e0db None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:39:04.403884 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.fernet_utils [None req-5147e4b2-0beb-459f-a7c1-f73cf9e9e0db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:39:04.472474 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:04.478132 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 505/962] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:39:04 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:39:04.484479 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8161a9dd-4961-4793-9d1f-42b6832cb5a3 None None] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:04.484846 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8161a9dd-4961-4793-9d1f-42b6832cb5a3 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:04.485201 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8161a9dd-4961-4793-9d1f-42b6832cb5a3 None None] PATH_INFO: `/` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:04.486983 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 458/963] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:39:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:39:04.495329 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf52b8e-1d83-43e5-ad85-b7509adf2eb4 None None] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:04.495731 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf52b8e-1d83-43e5-ad85-b7509adf2eb4 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:04.496022 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf52b8e-1d83-43e5-ad85-b7509adf2eb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:04.507657 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8161a9dd-4961-4793-9d1f-42b6832cb5a3 None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:04.515690 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.auth.core [None req-9cf52b8e-1d83-43e5-ad85-b7509adf2eb4 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:39:04.516897 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.fernet_utils [None req-9cf52b8e-1d83-43e5-ad85-b7509adf2eb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:39:04.602083 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 506/964] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:39:04 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:39:04.626564 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf52b8e-1d83-43e5-ad85-b7509adf2eb4 None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:04.644350 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8161a9dd-4961-4793-9d1f-42b6832cb5a3 None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:04.691816 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c39640-f226-468b-96a9-22c700dafbe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:04.692494 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c39640-f226-468b-96a9-22c700dafbe5 None admin] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:04.692786 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c39640-f226-468b-96a9-22c700dafbe5 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:04.693077 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c39640-f226-468b-96a9-22c700dafbe5 None admin] PATH_INFO: `/v3/users` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:04.694553 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c39640-f226-468b-96a9-22c700dafbe5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:04.696005 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c39640-f226-468b-96a9-22c700dafbe5 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:04.711746 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.sql.core [None req-b4c39640-f226-468b-96a9-22c700dafbe5 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 19 14:39:04.711746 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 19 14:39:04.711746 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [parameters: {'user_id': 'e9ee5642b6a444baa826e850a7ac0eed', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 19 14:39:04.711746 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=218624) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 19 14:39:04.721872 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-b4c39640-f226-468b-96a9-22c700dafbe5 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 19 14:39:04.721872 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:39:04.721872 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 14:39:04.721872 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 14:39:04.721872 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 14:39:04.721872 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 14:39:04.721872 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 14:39:04.721872 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application result = self._query(query) May 19 14:39:04.721872 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 14:39:04.721872 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application conn.query(q) May 19 14:39:04.721872 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 14:39:04.721872 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 14:39:04.721872 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 14:39:04.721872 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application result.read() May 19 14:39:04.721872 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 14:39:04.722929 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 14:39:04.722929 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 14:39:04.722929 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 14:39:04.722929 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 14:39:04.722929 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 14:39:04.722929 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 14:39:04.722929 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 14:39:04.722929 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 19 14:39:04.722929 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:39:04.722929 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 19 14:39:04.722929 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:39:04.722929 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:39:04.722929 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 19 14:39:04.722929 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 19 14:39:04.722929 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user May 19 14:39:04.722929 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) May 19 14:39:04.722929 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 14:39:04.722929 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application next(self.gen) May 19 14:39:04.724093 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 19 14:39:04.724093 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application yield resource May 19 14:39:04.724093 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 14:39:04.724093 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application next(self.gen) May 19 14:39:04.724093 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 19 14:39:04.724093 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application self.session.rollback() May 19 14:39:04.724093 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 19 14:39:04.724093 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application self.force_reraise() May 19 14:39:04.724093 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 19 14:39:04.724093 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise self.value May 19 14:39:04.724093 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 19 14:39:04.724093 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 19 14:39:04.724093 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 19 14:39:04.724093 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application session.commit() May 19 14:39:04.724093 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 19 14:39:04.724093 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 19 14:39:04.724093 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 19 14:39:04.725144 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application self._prepare_impl() May 19 14:39:04.725144 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 19 14:39:04.725144 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application self.session.flush() May 19 14:39:04.725144 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 19 14:39:04.725144 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application self._flush(objects) May 19 14:39:04.725144 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 19 14:39:04.725144 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 19 14:39:04.725144 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 19 14:39:04.725144 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application compat.raise_( May 19 14:39:04.725144 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 14:39:04.725144 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception May 19 14:39:04.725144 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 19 14:39:04.725144 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application flush_context.execute() May 19 14:39:04.725144 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 19 14:39:04.725144 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rec.execute(self) May 19 14:39:04.725144 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 19 14:39:04.725144 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 19 14:39:04.726196 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 19 14:39:04.726196 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application _emit_insert_statements( May 19 14:39:04.726196 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements May 19 14:39:04.726196 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application result = connection._execute_20( May 19 14:39:04.726196 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 19 14:39:04.726196 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 19 14:39:04.726196 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 19 14:39:04.726196 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 19 14:39:04.726196 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 19 14:39:04.726196 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application ret = self._execute_context( May 19 14:39:04.726196 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 19 14:39:04.726196 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 19 14:39:04.726196 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 19 14:39:04.726196 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 19 14:39:04.726196 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 14:39:04.726196 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception May 19 14:39:04.727144 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 14:39:04.727144 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 14:39:04.727144 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 14:39:04.727144 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 14:39:04.727144 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 14:39:04.727144 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application result = self._query(query) May 19 14:39:04.727144 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 14:39:04.727144 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application conn.query(q) May 19 14:39:04.727144 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 14:39:04.727144 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 14:39:04.727144 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 14:39:04.727144 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application result.read() May 19 14:39:04.727144 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 14:39:04.727144 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 14:39:04.727144 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 14:39:04.727144 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 14:39:04.728060 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 14:39:04.728060 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 14:39:04.728060 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 14:39:04.728060 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 14:39:04.728060 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 19 14:39:04.728060 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 19 14:39:04.728060 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [parameters: {'user_id': 'e9ee5642b6a444baa826e850a7ac0eed', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 19 14:39:04.728060 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 19 14:39:04.728060 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:39:04.728060 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:39:04.728060 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:39:04.728060 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:39:04.728060 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:39:04.728060 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:39:04.728060 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:39:04.731253 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:39:04.731253 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:39:04.731253 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:39:04.731253 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:39:04.731253 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:39:04.731253 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:39:04.731253 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:39:04.731253 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post May 19 14:39:04.731253 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( May 19 14:39:04.731253 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:39:04.731253 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:39:04.731253 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:39:04.731253 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:39:04.731253 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:39:04.731253 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:39:04.731253 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user May 19 14:39:04.731253 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) May 19 14:39:04.733482 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects May 19 14:39:04.733482 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) May 19 14:39:04.733482 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 19 14:39:04.733482 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 19 14:39:04.733482 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 19 14:39:04.733482 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:39:04.733482 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 459/965] 10.222.0.57 () {60 vars in 1131 bytes} [Thu May 19 14:39:04 2022] POST /identity/v3/users => generated 160 bytes in 216 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 19 14:39:04.733482 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c39640-f226-468b-96a9-22c700dafbe5 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:04.766622 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:04.775954 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf52b8e-1d83-43e5-ad85-b7509adf2eb4 None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:04.812260 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:04.819734 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e4037f-c123-4c7c-a9ce-cb4214087a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:04.820917 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e4037f-c123-4c7c-a9ce-cb4214087a22 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:04.821552 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e4037f-c123-4c7c-a9ce-cb4214087a22 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:04.821962 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e4037f-c123-4c7c-a9ce-cb4214087a22 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:04.834057 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e4037f-c123-4c7c-a9ce-cb4214087a22 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:39:04.844059 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-38e4037f-c123-4c7c-a9ce-cb4214087a22 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 14:39:04.844059 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:39:04.844059 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:39:04.844059 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:39:04.844059 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:39:04.844059 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:39:04.844059 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:39:04.844059 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:39:04.844059 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:39:04.844059 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:39:04.844059 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:39:04.844059 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:39:04.844059 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:39:04.844059 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:39:04.844059 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:39:04.844059 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:39:04.845342 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:39:04.845342 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:39:04.845342 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:39:04.845342 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:39:04.845342 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:39:04.845342 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:39:04.845342 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:39:04.845342 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:39:04.845342 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:39:04.845342 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:39:04.845342 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:39:04.845342 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:39:04.845342 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:39:04.845342 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:39:04.845342 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:39:04.845342 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:39:04.846564 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:39:04.846564 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:39:04.846564 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:39:04.846564 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:39:04.846564 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:39:04.846564 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:39:04.846564 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:39:04.846564 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:39:04.846564 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:39:04.846564 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:39:04.846564 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 14:39:04.846564 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:39:04.848071 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 507/966] 10.222.0.57 () {56 vars in 1120 bytes} [Thu May 19 14:39:04 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:39:04.854325 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e4037f-c123-4c7c-a9ce-cb4214087a22 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:04.870502 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c39640-f226-468b-96a9-22c700dafbe5 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:04.921344 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32c7006-9ca9-4ff6-8e90-333237124ef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:04.922757 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32c7006-9ca9-4ff6-8e90-333237124ef9 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:04.923041 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32c7006-9ca9-4ff6-8e90-333237124ef9 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:04.923322 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32c7006-9ca9-4ff6-8e90-333237124ef9 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:04.932897 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32c7006-9ca9-4ff6-8e90-333237124ef9 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:04.934598 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32c7006-9ca9-4ff6-8e90-333237124ef9 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:04.949824 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-a32c7006-9ca9-4ff6-8e90-333237124ef9 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 14:39:04.949824 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:39:04.949824 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:39:04.949824 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:39:04.949824 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:39:04.949824 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:39:04.949824 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:39:04.949824 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:39:04.949824 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:39:04.949824 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:39:04.949824 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:39:04.949824 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:39:04.949824 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:39:04.949824 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:39:04.949824 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:39:04.949824 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:39:04.950939 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:39:04.950939 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:39:04.950939 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:39:04.950939 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:39:04.950939 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:39:04.950939 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:39:04.950939 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:39:04.950939 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:39:04.950939 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:39:04.950939 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:39:04.950939 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:39:04.950939 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:39:04.950939 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:39:04.950939 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:39:04.950939 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:39:04.950939 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:39:04.950939 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:39:04.950939 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:39:04.951983 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:39:04.951983 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:39:04.951983 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:39:04.951983 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:39:04.951983 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:39:04.951983 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:39:04.951983 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:39:04.951983 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:39:04.951983 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 14:39:04.951983 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:39:04.955943 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 460/967] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:39:04 2022] GET /identity/v3/users/cinder => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:39:04.956837 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32c7006-9ca9-4ff6-8e90-333237124ef9 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:04.978528 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e4037f-c123-4c7c-a9ce-cb4214087a22 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:05.017590 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3344708-b51f-4a2f-8af1-cd85d807a9a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:05.018426 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3344708-b51f-4a2f-8af1-cd85d807a9a6 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:05.018715 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3344708-b51f-4a2f-8af1-cd85d807a9a6 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:05.018994 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3344708-b51f-4a2f-8af1-cd85d807a9a6 None admin] PATH_INFO: `/v3/roles` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:05.019766 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3344708-b51f-4a2f-8af1-cd85d807a9a6 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:39:05.020074 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3344708-b51f-4a2f-8af1-cd85d807a9a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:05.021520 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3344708-b51f-4a2f-8af1-cd85d807a9a6 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:05.036353 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 508/968] 10.222.0.57 () {56 vars in 1115 bytes} [Thu May 19 14:39:04 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:39:05.037265 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3344708-b51f-4a2f-8af1-cd85d807a9a6 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:05.076715 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:05.094338 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32c7006-9ca9-4ff6-8e90-333237124ef9 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:05.114508 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:05.132973 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e414be3-0a76-4fbe-b969-424b6791de79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:05.133758 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e414be3-0a76-4fbe-b969-424b6791de79 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:05.134095 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e414be3-0a76-4fbe-b969-424b6791de79 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:05.134377 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e414be3-0a76-4fbe-b969-424b6791de79 None admin] PATH_INFO: `/v3/users` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:05.135216 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e414be3-0a76-4fbe-b969-424b6791de79 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:39:05.135969 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e414be3-0a76-4fbe-b969-424b6791de79 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:05.137133 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e414be3-0a76-4fbe-b969-424b6791de79 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:05.177643 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3344708-b51f-4a2f-8af1-cd85d807a9a6 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:05.179631 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 461/969] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:39:04 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:39:05.195005 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dd3486-9afb-4dbf-96d7-b48c398d6ece None None] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:05.195005 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dd3486-9afb-4dbf-96d7-b48c398d6ece None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:05.195359 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dd3486-9afb-4dbf-96d7-b48c398d6ece None None] PATH_INFO: `/` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:05.201479 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 462/970] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:39:05 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:39:05.203001 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dccca5-e223-43f6-9b4e-d224a2e4a2eb None None] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:05.203299 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dccca5-e223-43f6-9b4e-d224a2e4a2eb None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:05.203584 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dccca5-e223-43f6-9b4e-d224a2e4a2eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:05.222469 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.auth.core [None req-45dccca5-e223-43f6-9b4e-d224a2e4a2eb None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:39:05.224067 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-45dccca5-e223-43f6-9b4e-d224a2e4a2eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:39:05.224757 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173f5314-b64f-497c-bf32-7063b0dcb72e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:05.226309 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173f5314-b64f-497c-bf32-7063b0dcb72e None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:05.226717 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173f5314-b64f-497c-bf32-7063b0dcb72e None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:05.227449 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173f5314-b64f-497c-bf32-7063b0dcb72e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:05.243267 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173f5314-b64f-497c-bf32-7063b0dcb72e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:05.244738 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173f5314-b64f-497c-bf32-7063b0dcb72e None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:05.264301 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-173f5314-b64f-497c-bf32-7063b0dcb72e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:39:05.264301 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:39:05.264301 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:39:05.264301 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:39:05.264301 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:39:05.264301 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:39:05.264301 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:39:05.264301 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:39:05.264301 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:39:05.264301 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:39:05.264301 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:39:05.264301 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:39:05.264301 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:39:05.264301 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:39:05.264301 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:39:05.265582 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:39:05.265582 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:39:05.265582 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:39:05.265582 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:39:05.265582 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:39:05.265582 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:39:05.265582 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:39:05.265582 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:39:05.265582 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:39:05.265582 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:39:05.265582 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:39:05.265582 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:39:05.265582 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:39:05.265582 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:39:05.265582 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:39:05.265582 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:39:05.265582 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:39:05.266844 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:39:05.266844 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:39:05.266844 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:39:05.266844 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:39:05.266844 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 509/971] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:39:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:39:05.273626 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173f5314-b64f-497c-bf32-7063b0dcb72e None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:05.320535 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 463/972] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:39:05 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:39:05.344849 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dccca5-e223-43f6-9b4e-d224a2e4a2eb None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:05.427074 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173f5314-b64f-497c-bf32-7063b0dcb72e None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:05.431222 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:05.458269 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:05.477308 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd67023-8786-41af-8616-6bb3fc7052b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:05.478278 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd67023-8786-41af-8616-6bb3fc7052b3 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:05.478741 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd67023-8786-41af-8616-6bb3fc7052b3 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:05.479174 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd67023-8786-41af-8616-6bb3fc7052b3 None admin] PATH_INFO: `/v3/domains` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:05.480064 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd67023-8786-41af-8616-6bb3fc7052b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:39:05.481147 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd67023-8786-41af-8616-6bb3fc7052b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:05.482912 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd67023-8786-41af-8616-6bb3fc7052b3 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:05.497371 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 510/973] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:39:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:39:05.500026 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dccca5-e223-43f6-9b4e-d224a2e4a2eb None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:05.502550 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd67023-8786-41af-8616-6bb3fc7052b3 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:05.541267 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68e6af7-8831-442e-9f65-9eb8c70f5aeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:05.542396 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68e6af7-8831-442e-9f65-9eb8c70f5aeb None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:05.542863 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68e6af7-8831-442e-9f65-9eb8c70f5aeb None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:05.543296 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68e6af7-8831-442e-9f65-9eb8c70f5aeb None admin] PATH_INFO: `/v3/services/object-store` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:05.544107 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68e6af7-8831-442e-9f65-9eb8c70f5aeb None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:05.545652 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68e6af7-8831-442e-9f65-9eb8c70f5aeb None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:05.559446 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-d68e6af7-8831-442e-9f65-9eb8c70f5aeb None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 14:39:05.559446 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:39:05.559446 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:39:05.559446 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:39:05.559446 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:39:05.559446 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:39:05.559446 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:39:05.559446 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:39:05.559446 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 14:39:05.559446 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:39:05.559446 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:39:05.559446 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:39:05.559446 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:39:05.559446 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:39:05.559446 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:39:05.559446 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:39:05.559446 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:39:05.560606 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:39:05.560606 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:39:05.560606 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:39:05.560606 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:39:05.560606 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:39:05.560606 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:39:05.560606 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:39:05.560606 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:39:05.560606 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:39:05.560606 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:39:05.560606 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:39:05.560606 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:39:05.560606 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:39:05.560606 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:39:05.560606 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:39:05.560606 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:39:05.561543 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:39:05.561543 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:39:05.561543 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:39:05.561543 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:39:05.561543 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:39:05.561543 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:39:05.561543 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:39:05.561543 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:39:05.561543 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:39:05.561543 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:39:05.561543 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:39:05.561543 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:39:05.561543 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 14:39:05.561543 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:39:05.561543 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 464/974] 10.222.0.57 () {56 vars in 1126 bytes} [Thu May 19 14:39:05 2022] GET /identity/v3/services/object-store => generated 93 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:39:05.566899 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68e6af7-8831-442e-9f65-9eb8c70f5aeb None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:05.614311 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd67023-8786-41af-8616-6bb3fc7052b3 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:05.652870 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efe3f43-c180-4552-ab12-27281ba77607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:05.653686 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efe3f43-c180-4552-ab12-27281ba77607 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:05.654009 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efe3f43-c180-4552-ab12-27281ba77607 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:05.654289 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efe3f43-c180-4552-ab12-27281ba77607 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:05.662974 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efe3f43-c180-4552-ab12-27281ba77607 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:05.664641 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efe3f43-c180-4552-ab12-27281ba77607 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:05.675620 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68e6af7-8831-442e-9f65-9eb8c70f5aeb None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:05.681829 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-0efe3f43-c180-4552-ab12-27281ba77607 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 14:39:05.681829 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:39:05.681829 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:39:05.681829 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:39:05.681829 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:39:05.681829 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:39:05.681829 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:39:05.681829 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:39:05.681829 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:39:05.681829 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:39:05.681829 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:39:05.681829 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:39:05.681829 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:39:05.681829 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:39:05.681829 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:39:05.681829 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:39:05.683028 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:39:05.683028 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:39:05.683028 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:39:05.683028 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:39:05.683028 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:39:05.683028 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:39:05.683028 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:39:05.683028 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:39:05.683028 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:39:05.683028 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:39:05.683028 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:39:05.683028 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:39:05.683028 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:39:05.683028 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:39:05.683028 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:39:05.683028 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:39:05.683028 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:39:05.683028 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:39:05.684097 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:39:05.684097 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:39:05.684097 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:39:05.684097 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:39:05.684097 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:39:05.684097 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:39:05.684097 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:39:05.684097 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:39:05.684097 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 14:39:05.684097 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:39:05.688096 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 511/975] 10.222.0.57 () {56 vars in 1117 bytes} [Thu May 19 14:39:05 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:39:05.689075 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efe3f43-c180-4552-ab12-27281ba77607 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:05.716782 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:05.719001 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab92da6-1aca-4554-9df7-9f6a4bb83347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:05.719758 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab92da6-1aca-4554-9df7-9f6a4bb83347 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:05.720118 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab92da6-1aca-4554-9df7-9f6a4bb83347 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:05.720477 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab92da6-1aca-4554-9df7-9f6a4bb83347 None admin] PATH_INFO: `/v3/services` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:05.721267 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab92da6-1aca-4554-9df7-9f6a4bb83347 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:39:05.721653 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab92da6-1aca-4554-9df7-9f6a4bb83347 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:05.723143 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab92da6-1aca-4554-9df7-9f6a4bb83347 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:05.729396 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 465/976] 10.222.0.57 () {56 vars in 1122 bytes} [Thu May 19 14:39:05 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:39:05.734940 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab92da6-1aca-4554-9df7-9f6a4bb83347 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:05.737622 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:05.805627 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efe3f43-c180-4552-ab12-27281ba77607 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:05.842999 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3c560c-3b35-414e-a310-85fde59163cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:05.844029 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3c560c-3b35-414e-a310-85fde59163cc None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:05.844308 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3c560c-3b35-414e-a310-85fde59163cc None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:05.844572 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3c560c-3b35-414e-a310-85fde59163cc None admin] PATH_INFO: `/v3/users` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:05.845421 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3c560c-3b35-414e-a310-85fde59163cc None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:39:05.845738 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3c560c-3b35-414e-a310-85fde59163cc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:05.847227 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3c560c-3b35-414e-a310-85fde59163cc None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:05.849583 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab92da6-1aca-4554-9df7-9f6a4bb83347 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:05.866337 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 512/977] 10.222.0.57 () {56 vars in 1149 bytes} [Thu May 19 14:39:05 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:39:05.872090 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1746962d-0391-433a-a151-f91c58cc5678 None None] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:05.872381 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1746962d-0391-433a-a151-f91c58cc5678 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:05.872665 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1746962d-0391-433a-a151-f91c58cc5678 None None] PATH_INFO: `/` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:05.874017 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 513/978] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:39:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:39:05.874716 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1746962d-0391-433a-a151-f91c58cc5678 None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:05.890287 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeb3c25-33ee-42b0-9c8c-065ca4d3d707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:05.891034 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeb3c25-33ee-42b0-9c8c-065ca4d3d707 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:05.891325 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeb3c25-33ee-42b0-9c8c-065ca4d3d707 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:05.891606 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeb3c25-33ee-42b0-9c8c-065ca4d3d707 None admin] PATH_INFO: `/v3/services` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:05.892324 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceeb3c25-33ee-42b0-9c8c-065ca4d3d707 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:39:05.892635 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceeb3c25-33ee-42b0-9c8c-065ca4d3d707 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:05.894057 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceeb3c25-33ee-42b0-9c8c-065ca4d3d707 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:05.899691 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 466/979] 10.222.0.57 () {56 vars in 1122 bytes} [Thu May 19 14:39:05 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:39:05.900890 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240b5f2d-6ea6-43f5-9474-e47b0e5c455b None None] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:05.901184 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240b5f2d-6ea6-43f5-9474-e47b0e5c455b None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:05.901468 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240b5f2d-6ea6-43f5-9474-e47b0e5c455b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:05.919059 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.auth.core [None req-240b5f2d-6ea6-43f5-9474-e47b0e5c455b None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:39:05.920266 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-240b5f2d-6ea6-43f5-9474-e47b0e5c455b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:39:05.964538 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:05.980888 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:05.984535 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1746962d-0391-433a-a151-f91c58cc5678 None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:05.988705 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 467/980] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:39:05 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:39:05.989523 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240b5f2d-6ea6-43f5-9474-e47b0e5c455b None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:06.020446 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951958ee-9220-4afe-87c8-b608bab0285a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:06.021103 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951958ee-9220-4afe-87c8-b608bab0285a None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:06.021386 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951958ee-9220-4afe-87c8-b608bab0285a None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:06.021673 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951958ee-9220-4afe-87c8-b608bab0285a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:06.089936 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951958ee-9220-4afe-87c8-b608bab0285a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:06.090842 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240b5f2d-6ea6-43f5-9474-e47b0e5c455b None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:06.091312 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951958ee-9220-4afe-87c8-b608bab0285a None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:06.124937 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b04eb1b-17e7-4e89-a22d-9f2a1889d31f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:06.125588 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b04eb1b-17e7-4e89-a22d-9f2a1889d31f None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:06.125898 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b04eb1b-17e7-4e89-a22d-9f2a1889d31f None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:06.126171 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b04eb1b-17e7-4e89-a22d-9f2a1889d31f None admin] PATH_INFO: `/v3/endpoints` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:06.126954 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b04eb1b-17e7-4e89-a22d-9f2a1889d31f None admin] RBAC: Adding query filter params (interface=admin, service_id=ede24f9b42b4401dbca7f94c7c6fd5cb, region_id=RegionOne) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:39:06.127264 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b04eb1b-17e7-4e89-a22d-9f2a1889d31f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:06.128628 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b04eb1b-17e7-4e89-a22d-9f2a1889d31f None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:06.134425 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 468/981] 10.222.0.57 () {56 vars in 1249 bytes} [Thu May 19 14:39:05 2022] GET /identity/v3/endpoints?service_id=ede24f9b42b4401dbca7f94c7c6fd5cb&interface=admin®ion_id=RegionOne => generated 193 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:39:06.135195 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b04eb1b-17e7-4e89-a22d-9f2a1889d31f None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:06.182001 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:06.198024 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:06.202585 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 514/982] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:39:05 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:39:06.203434 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951958ee-9220-4afe-87c8-b608bab0285a None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:06.242052 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b04eb1b-17e7-4e89-a22d-9f2a1889d31f None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:06.279084 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0d8314-3a13-46c1-b58f-c9093b8bb85a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:06.279789 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0d8314-3a13-46c1-b58f-c9093b8bb85a None admin] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:06.280072 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0d8314-3a13-46c1-b58f-c9093b8bb85a None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:06.280471 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0d8314-3a13-46c1-b58f-c9093b8bb85a None admin] PATH_INFO: `/v3/services` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:06.281190 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0d8314-3a13-46c1-b58f-c9093b8bb85a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:06.282577 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0d8314-3a13-46c1-b58f-c9093b8bb85a None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:06.296866 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 469/983] 10.222.0.57 () {60 vars in 1141 bytes} [Thu May 19 14:39:06 2022] POST /identity/v3/services => generated 242 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:39:06.297821 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0d8314-3a13-46c1-b58f-c9093b8bb85a None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:06.306229 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951958ee-9220-4afe-87c8-b608bab0285a None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:06.355442 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afd674a-441e-405c-8b4b-75d3dc2411e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:06.356203 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afd674a-441e-405c-8b4b-75d3dc2411e0 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:06.356570 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afd674a-441e-405c-8b4b-75d3dc2411e0 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:06.356851 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afd674a-441e-405c-8b4b-75d3dc2411e0 None admin] PATH_INFO: `/v3/services` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:06.357557 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afd674a-441e-405c-8b4b-75d3dc2411e0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:06.359050 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afd674a-441e-405c-8b4b-75d3dc2411e0 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:06.367596 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 515/984] 10.222.0.57 () {56 vars in 1087 bytes} [Thu May 19 14:39:06 2022] GET /identity/v3/services => generated 1727 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 14:39:06.425179 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0d8314-3a13-46c1-b58f-c9093b8bb85a None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:06.451033 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:06.471223 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeb3baf-e5ad-4906-9d3b-03440e239079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:06.472150 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeb3baf-e5ad-4906-9d3b-03440e239079 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:06.472500 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeb3baf-e5ad-4906-9d3b-03440e239079 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:06.472841 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeb3baf-e5ad-4906-9d3b-03440e239079 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:06.485339 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beeb3baf-e5ad-4906-9d3b-03440e239079 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:06.486982 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beeb3baf-e5ad-4906-9d3b-03440e239079 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:06.508541 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-beeb3baf-e5ad-4906-9d3b-03440e239079 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:39:06.508541 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:39:06.508541 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:39:06.508541 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:39:06.508541 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:39:06.508541 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:39:06.508541 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:39:06.508541 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:39:06.508541 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:39:06.508541 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:39:06.508541 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:39:06.508541 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:39:06.508541 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:39:06.508541 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:39:06.508541 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:39:06.509807 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:39:06.509807 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:39:06.509807 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:39:06.509807 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:39:06.509807 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:39:06.509807 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:39:06.509807 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:39:06.509807 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:39:06.509807 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:39:06.509807 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:39:06.509807 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:39:06.509807 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:39:06.509807 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:39:06.509807 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:39:06.509807 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:39:06.509807 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:39:06.509807 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:39:06.511148 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:39:06.511148 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:39:06.511148 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:39:06.511148 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:39:06.511148 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 470/985] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:39:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:39:06.516208 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afd674a-441e-405c-8b4b-75d3dc2411e0 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:06.569162 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:06.631780 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afd674a-441e-405c-8b4b-75d3dc2411e0 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:06.667928 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfa5ae5-8159-45bb-b94b-7d9cfd43fb5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:06.668599 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfa5ae5-8159-45bb-b94b-7d9cfd43fb5d None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:06.668884 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfa5ae5-8159-45bb-b94b-7d9cfd43fb5d None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:06.669161 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfa5ae5-8159-45bb-b94b-7d9cfd43fb5d None admin] PATH_INFO: `/v3/domains` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:06.669912 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfa5ae5-8159-45bb-b94b-7d9cfd43fb5d None admin] RBAC: Adding query filter params (name=Default) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:39:06.670225 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfa5ae5-8159-45bb-b94b-7d9cfd43fb5d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:06.671614 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfa5ae5-8159-45bb-b94b-7d9cfd43fb5d None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:06.684195 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 516/986] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:39:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:39:06.690121 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeb3baf-e5ad-4906-9d3b-03440e239079 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:06.794970 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeb3baf-e5ad-4906-9d3b-03440e239079 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:06.831058 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8c0803-e264-4b4f-abd2-7004df112a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:06.831857 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8c0803-e264-4b4f-abd2-7004df112a33 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:06.832147 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8c0803-e264-4b4f-abd2-7004df112a33 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:06.832425 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8c0803-e264-4b4f-abd2-7004df112a33 None admin] PATH_INFO: `/v3/projects/service` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:06.840374 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8c0803-e264-4b4f-abd2-7004df112a33 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:06.841830 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8c0803-e264-4b4f-abd2-7004df112a33 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:06.856042 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-9c8c0803-e264-4b4f-abd2-7004df112a33 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:39:06.856042 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:39:06.856042 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:39:06.856042 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:39:06.856042 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:39:06.856042 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:39:06.856042 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:39:06.856042 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:39:06.856042 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:39:06.856042 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:39:06.856042 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:39:06.856042 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:39:06.856042 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:39:06.856042 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:39:06.856042 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:39:06.857063 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:39:06.857063 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:39:06.857063 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:39:06.857063 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:39:06.857063 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:39:06.857063 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:39:06.857063 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:39:06.857063 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:39:06.857063 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:39:06.857063 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:39:06.857063 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:39:06.857063 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:39:06.857063 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:39:06.857063 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:39:06.857063 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:39:06.857063 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:39:06.857063 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:39:06.858060 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:39:06.858060 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:39:06.858060 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:39:06.858060 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:39:06.858060 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:39:06.858060 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:39:06.858060 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:39:06.858060 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:39:06.858060 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 471/987] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:39:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:39:06.864623 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfa5ae5-8159-45bb-b94b-7d9cfd43fb5d None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:06.974507 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfa5ae5-8159-45bb-b94b-7d9cfd43fb5d None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:06.976315 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:07.011774 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153a0af3-f6f1-41e9-9697-6eab598b7267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:07.012445 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153a0af3-f6f1-41e9-9697-6eab598b7267 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:07.012732 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153a0af3-f6f1-41e9-9697-6eab598b7267 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:07.013011 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153a0af3-f6f1-41e9-9697-6eab598b7267 None admin] PATH_INFO: `/v3/projects` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:07.013802 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153a0af3-f6f1-41e9-9697-6eab598b7267 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:39:07.014144 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153a0af3-f6f1-41e9-9697-6eab598b7267 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:07.015757 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153a0af3-f6f1-41e9-9697-6eab598b7267 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:07.032865 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 517/988] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:39:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:39:07.034569 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8c0803-e264-4b4f-abd2-7004df112a33 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:07.069354 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:07.143858 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8c0803-e264-4b4f-abd2-7004df112a33 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:07.185547 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcc24b6-6ec2-4004-813b-e526a56024d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:07.186542 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcc24b6-6ec2-4004-813b-e526a56024d5 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:07.187026 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcc24b6-6ec2-4004-813b-e526a56024d5 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:07.187481 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcc24b6-6ec2-4004-813b-e526a56024d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:07.188475 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcc24b6-6ec2-4004-813b-e526a56024d5 None admin] RBAC: Adding query filter params (role.id=db02c08ef7c847c0ba25ed9ddc05974c, scope.project.id=42f35b96b9d4499abfb5e5f7cca49adc, user.id=2cbf483c6556421fa36179722100f8b9) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:39:07.188952 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcc24b6-6ec2-4004-813b-e526a56024d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:07.190615 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcc24b6-6ec2-4004-813b-e526a56024d5 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:07.198577 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 472/989] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:39:07 2022] GET /identity/v3/role_assignments?user.id=2cbf483c6556421fa36179722100f8b9&scope.project.id=42f35b96b9d4499abfb5e5f7cca49adc&role.id=db02c08ef7c847c0ba25ed9ddc05974c => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:39:08.633835 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c731de-f0b5-482a-a888-7f65bc06fa2f None None] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:08.634384 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c731de-f0b5-482a-a888-7f65bc06fa2f None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:08.634496 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c731de-f0b5-482a-a888-7f65bc06fa2f None None] PATH_INFO: `/` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:08.635965 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 518/990] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:39:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:39:08.641112 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4491e9b-c6a4-4594-b6d9-5f9f5d2a436a None None] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:08.641503 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4491e9b-c6a4-4594-b6d9-5f9f5d2a436a None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:08.641684 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4491e9b-c6a4-4594-b6d9-5f9f5d2a436a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:08.659463 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.auth.core [None req-e4491e9b-c6a4-4594-b6d9-5f9f5d2a436a None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:39:08.660663 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-e4491e9b-c6a4-4594-b6d9-5f9f5d2a436a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:39:08.674528 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bd0e3e-a8ad-46f6-b9e9-b759112b2f3f None None] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:08.674811 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bd0e3e-a8ad-46f6-b9e9-b759112b2f3f None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:08.675095 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bd0e3e-a8ad-46f6-b9e9-b759112b2f3f None None] PATH_INFO: `/` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:08.676467 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 519/991] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:39:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:39:08.681559 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dde20c-8acf-4089-8ad6-2e171b7fe826 None None] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:08.681882 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dde20c-8acf-4089-8ad6-2e171b7fe826 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:08.682168 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dde20c-8acf-4089-8ad6-2e171b7fe826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:08.699719 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.auth.core [None req-64dde20c-8acf-4089-8ad6-2e171b7fe826 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:39:08.700926 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.fernet_utils [None req-64dde20c-8acf-4089-8ad6-2e171b7fe826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:39:08.733010 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 473/992] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:39:08 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:39:08.756597 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4491e9b-c6a4-4594-b6d9-5f9f5d2a436a None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:08.770777 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 520/993] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:39:08 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:39:08.783897 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:08.794079 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dde20c-8acf-4089-8ad6-2e171b7fe826 None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:08.864952 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4491e9b-c6a4-4594-b6d9-5f9f5d2a436a None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:08.887293 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:08.897579 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dde20c-8acf-4089-8ad6-2e171b7fe826 None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:08.902052 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6910eddf-091b-43f4-81b5-7981d5a5d5ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:08.902858 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6910eddf-091b-43f4-81b5-7981d5a5d5ae None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:08.903141 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6910eddf-091b-43f4-81b5-7981d5a5d5ae None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:08.903418 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6910eddf-091b-43f4-81b5-7981d5a5d5ae None admin] PATH_INFO: `/v3/services/compute` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:08.904076 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6910eddf-091b-43f4-81b5-7981d5a5d5ae None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:08.905467 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6910eddf-091b-43f4-81b5-7981d5a5d5ae None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:08.918276 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-6910eddf-091b-43f4-81b5-7981d5a5d5ae None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 19 14:39:08.918276 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:39:08.918276 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:39:08.918276 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:39:08.918276 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:39:08.918276 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:39:08.918276 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:39:08.918276 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:39:08.918276 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 14:39:08.918276 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:39:08.918276 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:39:08.918276 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:39:08.918276 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:39:08.918276 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:39:08.918276 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:39:08.918276 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:39:08.918276 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:39:08.919373 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:39:08.919373 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:39:08.919373 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:39:08.919373 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:39:08.919373 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:39:08.919373 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:39:08.919373 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:39:08.919373 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:39:08.919373 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:39:08.919373 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:39:08.919373 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:39:08.919373 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:39:08.919373 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:39:08.919373 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:39:08.919373 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:39:08.919373 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:39:08.920299 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:39:08.920299 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:39:08.920299 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:39:08.920299 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:39:08.920299 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:39:08.920299 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:39:08.920299 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:39:08.920299 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:39:08.920299 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:39:08.920299 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:39:08.920299 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:39:08.920299 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:39:08.920299 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 14:39:08.920299 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:39:08.920299 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 474/994] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:39:08 2022] GET /identity/v3/services/compute => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:39:08.925416 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6910eddf-091b-43f4-81b5-7981d5a5d5ae None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:08.937102 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438e0aaf-e463-40fa-825e-467ce6ec717b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:08.937953 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438e0aaf-e463-40fa-825e-467ce6ec717b None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:08.938244 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438e0aaf-e463-40fa-825e-467ce6ec717b None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:08.938525 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438e0aaf-e463-40fa-825e-467ce6ec717b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:08.939192 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438e0aaf-e463-40fa-825e-467ce6ec717b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:08.940589 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438e0aaf-e463-40fa-825e-467ce6ec717b None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:08.953019 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-438e0aaf-e463-40fa-825e-467ce6ec717b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 14:39:08.953019 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:39:08.953019 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:39:08.953019 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:39:08.953019 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:39:08.953019 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:39:08.953019 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:39:08.953019 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:39:08.953019 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 14:39:08.953019 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:39:08.953019 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:39:08.953019 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:39:08.953019 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:39:08.953019 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:39:08.953019 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:39:08.953019 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:39:08.953019 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:39:08.954189 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:39:08.954189 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:39:08.954189 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:39:08.954189 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:39:08.954189 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:39:08.954189 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:39:08.954189 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:39:08.954189 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:39:08.954189 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:39:08.954189 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:39:08.954189 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:39:08.954189 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:39:08.954189 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:39:08.954189 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:39:08.954189 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:39:08.954189 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:39:08.955136 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:39:08.955136 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:39:08.955136 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:39:08.955136 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:39:08.955136 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:39:08.955136 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:39:08.955136 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:39:08.955136 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:39:08.955136 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:39:08.955136 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:39:08.955136 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:39:08.955136 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:39:08.955136 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 14:39:08.955136 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:39:08.959789 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 521/995] 10.222.0.57 () {56 vars in 1126 bytes} [Thu May 19 14:39:08 2022] GET /identity/v3/services/object-store => generated 93 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:39:08.960561 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438e0aaf-e463-40fa-825e-467ce6ec717b None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:09.026542 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:09.038074 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6910eddf-091b-43f4-81b5-7981d5a5d5ae None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:09.074325 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c583efa0-c57b-4cde-90ed-2a6bb7efb95b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:09.075000 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c583efa0-c57b-4cde-90ed-2a6bb7efb95b None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:09.075290 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c583efa0-c57b-4cde-90ed-2a6bb7efb95b None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:09.075571 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c583efa0-c57b-4cde-90ed-2a6bb7efb95b None admin] PATH_INFO: `/v3/services` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:09.076287 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c583efa0-c57b-4cde-90ed-2a6bb7efb95b None admin] RBAC: Adding query filter params (name=compute) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:39:09.076600 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c583efa0-c57b-4cde-90ed-2a6bb7efb95b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:09.077049 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438e0aaf-e463-40fa-825e-467ce6ec717b None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:09.078062 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c583efa0-c57b-4cde-90ed-2a6bb7efb95b None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:09.083140 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 475/996] 10.222.0.57 () {56 vars in 1112 bytes} [Thu May 19 14:39:08 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:39:09.088955 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c583efa0-c57b-4cde-90ed-2a6bb7efb95b None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:09.118408 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5f29a7-f2e6-4e18-b446-122c0a1676a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:09.119087 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5f29a7-f2e6-4e18-b446-122c0a1676a5 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:09.119434 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5f29a7-f2e6-4e18-b446-122c0a1676a5 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:09.119739 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5f29a7-f2e6-4e18-b446-122c0a1676a5 None admin] PATH_INFO: `/v3/services` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:09.120355 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5f29a7-f2e6-4e18-b446-122c0a1676a5 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:39:09.120665 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5f29a7-f2e6-4e18-b446-122c0a1676a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:09.122076 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5f29a7-f2e6-4e18-b446-122c0a1676a5 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:09.127669 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 522/997] 10.222.0.57 () {56 vars in 1122 bytes} [Thu May 19 14:39:08 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:39:09.133606 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5f29a7-f2e6-4e18-b446-122c0a1676a5 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:09.160481 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:09.204940 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c583efa0-c57b-4cde-90ed-2a6bb7efb95b None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:09.244274 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5031e056-c781-4c0b-a4de-59dc3cd30a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:09.244942 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5031e056-c781-4c0b-a4de-59dc3cd30a9a None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:09.245229 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5031e056-c781-4c0b-a4de-59dc3cd30a9a None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:09.245510 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5031e056-c781-4c0b-a4de-59dc3cd30a9a None admin] PATH_INFO: `/v3/services` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:09.246261 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5031e056-c781-4c0b-a4de-59dc3cd30a9a None admin] RBAC: Adding query filter params (type=compute) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:39:09.246573 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5031e056-c781-4c0b-a4de-59dc3cd30a9a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:09.247950 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5031e056-c781-4c0b-a4de-59dc3cd30a9a None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:09.253309 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 476/998] 10.222.0.57 () {56 vars in 1112 bytes} [Thu May 19 14:39:09 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:39:09.259041 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5031e056-c781-4c0b-a4de-59dc3cd30a9a None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:09.265957 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5f29a7-f2e6-4e18-b446-122c0a1676a5 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:09.282378 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:09.313363 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d89e09c-c937-4339-9552-03065b9c3a7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:09.313678 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d89e09c-c937-4339-9552-03065b9c3a7b None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:09.313996 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d89e09c-c937-4339-9552-03065b9c3a7b None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:09.314276 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d89e09c-c937-4339-9552-03065b9c3a7b None admin] PATH_INFO: `/v3/services` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:09.315005 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d89e09c-c937-4339-9552-03065b9c3a7b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:39:09.315317 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d89e09c-c937-4339-9552-03065b9c3a7b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:09.316810 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d89e09c-c937-4339-9552-03065b9c3a7b None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:09.328347 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 523/999] 10.222.0.57 () {56 vars in 1122 bytes} [Thu May 19 14:39:09 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:39:09.329825 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d89e09c-c937-4339-9552-03065b9c3a7b None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:09.377729 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5031e056-c781-4c0b-a4de-59dc3cd30a9a None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:09.415448 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9910cbf0-80bf-4ae8-88a8-8b1fc766a909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:09.416030 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9910cbf0-80bf-4ae8-88a8-8b1fc766a909 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:09.416326 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9910cbf0-80bf-4ae8-88a8-8b1fc766a909 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:09.416605 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9910cbf0-80bf-4ae8-88a8-8b1fc766a909 None admin] PATH_INFO: `/v3/endpoints` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:09.417403 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9910cbf0-80bf-4ae8-88a8-8b1fc766a909 None admin] RBAC: Adding query filter params (interface=public, service_id=99c0883d3fe04f2783b74f38c2307142, region_id=RegionOne) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:39:09.417717 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9910cbf0-80bf-4ae8-88a8-8b1fc766a909 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:09.419147 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9910cbf0-80bf-4ae8-88a8-8b1fc766a909 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:09.423268 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:09.425887 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 477/1000] 10.222.0.57 () {56 vars in 1251 bytes} [Thu May 19 14:39:09 2022] GET /identity/v3/endpoints?service_id=99c0883d3fe04f2783b74f38c2307142&interface=public®ion_id=RegionOne => generated 194 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:39:09.430989 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9910cbf0-80bf-4ae8-88a8-8b1fc766a909 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:09.445618 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d89e09c-c937-4339-9552-03065b9c3a7b None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:09.481809 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82155abd-a90b-48ec-9650-d5a70a9f3eef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:09.482496 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82155abd-a90b-48ec-9650-d5a70a9f3eef None admin] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:09.482784 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82155abd-a90b-48ec-9650-d5a70a9f3eef None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:09.483075 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82155abd-a90b-48ec-9650-d5a70a9f3eef None admin] PATH_INFO: `/v3/endpoints` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:09.483813 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82155abd-a90b-48ec-9650-d5a70a9f3eef None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:09.485183 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82155abd-a90b-48ec-9650-d5a70a9f3eef None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:09.501281 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 524/1001] 10.222.0.57 () {60 vars in 1144 bytes} [Thu May 19 14:39:09 2022] POST /identity/v3/endpoints => generated 325 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 14:39:09.502636 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72b68e1-9ae0-41a0-8893-34a1a43920df None None] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:09.503181 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72b68e1-9ae0-41a0-8893-34a1a43920df None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:09.503181 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72b68e1-9ae0-41a0-8893-34a1a43920df None None] PATH_INFO: `/` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:09.504429 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 525/1002] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:39:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:39:09.510750 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4884e9-6878-4dd8-8ba1-2e5db943ea6d None None] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:09.511041 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4884e9-6878-4dd8-8ba1-2e5db943ea6d None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:09.511322 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4884e9-6878-4dd8-8ba1-2e5db943ea6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:09.520603 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:09.531330 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.auth.core [None req-5e4884e9-6878-4dd8-8ba1-2e5db943ea6d None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:39:09.532576 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.fernet_utils [None req-5e4884e9-6878-4dd8-8ba1-2e5db943ea6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:39:09.543520 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9910cbf0-80bf-4ae8-88a8-8b1fc766a909 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:09.593391 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706ee12a-6587-4512-b695-97c2f5f80b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:09.594277 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706ee12a-6587-4512-b695-97c2f5f80b17 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:09.594562 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706ee12a-6587-4512-b695-97c2f5f80b17 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:09.594838 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706ee12a-6587-4512-b695-97c2f5f80b17 None admin] PATH_INFO: `/v3/services` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:09.595538 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706ee12a-6587-4512-b695-97c2f5f80b17 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:09.596971 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706ee12a-6587-4512-b695-97c2f5f80b17 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:09.605028 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 478/1003] 10.222.0.57 () {56 vars in 1087 bytes} [Thu May 19 14:39:09 2022] GET /identity/v3/services => generated 1727 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 14:39:09.625628 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 526/1004] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:39:09 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:39:09.650068 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706ee12a-6587-4512-b695-97c2f5f80b17 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:09.787772 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706ee12a-6587-4512-b695-97c2f5f80b17 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:09.829431 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b3058e-61fc-4df5-b05d-0e4d0702e6e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:09.830272 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b3058e-61fc-4df5-b05d-0e4d0702e6e3 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:09.830586 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b3058e-61fc-4df5-b05d-0e4d0702e6e3 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:09.830829 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b3058e-61fc-4df5-b05d-0e4d0702e6e3 None admin] PATH_INFO: `/v3/services/image` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:09.831498 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b3058e-61fc-4df5-b05d-0e4d0702e6e3 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:09.832895 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b3058e-61fc-4df5-b05d-0e4d0702e6e3 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:09.846782 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-52b3058e-61fc-4df5-b05d-0e4d0702e6e3 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 19 14:39:09.846782 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:39:09.846782 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:39:09.846782 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:39:09.846782 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:39:09.846782 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:39:09.846782 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:39:09.846782 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:39:09.846782 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 14:39:09.846782 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:39:09.846782 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:39:09.846782 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:39:09.846782 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:39:09.846782 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:39:09.846782 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:39:09.846782 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:39:09.846782 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:39:09.848152 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:39:09.848152 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:39:09.848152 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:39:09.848152 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:39:09.848152 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:39:09.848152 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:39:09.848152 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:39:09.848152 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:39:09.848152 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:39:09.848152 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:39:09.848152 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:39:09.848152 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:39:09.848152 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:39:09.848152 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:39:09.848152 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:39:09.848152 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:39:09.849221 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:39:09.849221 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:39:09.849221 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:39:09.849221 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:39:09.849221 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:39:09.849221 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:39:09.849221 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:39:09.849221 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:39:09.849221 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:39:09.849221 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:39:09.849221 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:39:09.849221 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:39:09.849221 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 14:39:09.849221 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:39:09.849221 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 479/1005] 10.222.0.57 () {56 vars in 1105 bytes} [Thu May 19 14:39:09 2022] GET /identity/v3/services/image => generated 86 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:39:09.854316 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4884e9-6878-4dd8-8ba1-2e5db943ea6d None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:09.933792 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:09.964356 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4884e9-6878-4dd8-8ba1-2e5db943ea6d None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:09.999940 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310cdf85-47ab-415e-b809-53a6963bd253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:10.000611 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310cdf85-47ab-415e-b809-53a6963bd253 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:10.000894 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310cdf85-47ab-415e-b809-53a6963bd253 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:10.001187 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310cdf85-47ab-415e-b809-53a6963bd253 None admin] PATH_INFO: `/v3/services` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:10.001910 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310cdf85-47ab-415e-b809-53a6963bd253 None admin] RBAC: Adding query filter params (name=image) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:39:10.002192 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310cdf85-47ab-415e-b809-53a6963bd253 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:10.003587 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310cdf85-47ab-415e-b809-53a6963bd253 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:10.009765 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 527/1006] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:39:09 2022] GET /identity/v3/services?name=image => generated 122 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:39:10.015244 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b3058e-61fc-4df5-b05d-0e4d0702e6e3 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:10.026327 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:10.136748 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b3058e-61fc-4df5-b05d-0e4d0702e6e3 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:10.182990 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92e596c-6c20-42e1-aad9-e77e87683e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:10.184143 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92e596c-6c20-42e1-aad9-e77e87683e06 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:10.184143 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92e596c-6c20-42e1-aad9-e77e87683e06 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:10.184333 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92e596c-6c20-42e1-aad9-e77e87683e06 None admin] PATH_INFO: `/v3/services` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:10.185022 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92e596c-6c20-42e1-aad9-e77e87683e06 None admin] RBAC: Adding query filter params (type=image) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:39:10.185337 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92e596c-6c20-42e1-aad9-e77e87683e06 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:10.188060 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92e596c-6c20-42e1-aad9-e77e87683e06 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:10.197930 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 480/1007] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:39:10 2022] GET /identity/v3/services?type=image => generated 122 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:39:11.898107 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e372fa7-4b70-43f8-9f68-afe319e2dfa5 None None] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:11.898679 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e372fa7-4b70-43f8-9f68-afe319e2dfa5 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:11.898767 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0bb371-9530-4fc1-b173-904f3163eac2 None None] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:11.899117 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e372fa7-4b70-43f8-9f68-afe319e2dfa5 None None] PATH_INFO: `/` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:11.899219 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0bb371-9530-4fc1-b173-904f3163eac2 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:11.899219 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0bb371-9530-4fc1-b173-904f3163eac2 None None] PATH_INFO: `/` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:11.900214 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 528/1008] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:39:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:39:11.900694 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 481/1009] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:39:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:39:11.905292 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c76f5dc-90dc-4e36-8794-2a36d1db5fec None None] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:11.905578 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c76f5dc-90dc-4e36-8794-2a36d1db5fec None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:11.905874 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c76f5dc-90dc-4e36-8794-2a36d1db5fec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:11.906126 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fcb9fc-9a10-4f9c-b10a-7522a2a632ba None None] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:11.906398 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fcb9fc-9a10-4f9c-b10a-7522a2a632ba None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:11.906676 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fcb9fc-9a10-4f9c-b10a-7522a2a632ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:11.924435 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.auth.core [None req-0c76f5dc-90dc-4e36-8794-2a36d1db5fec None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:39:11.924638 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.auth.core [None req-42fcb9fc-9a10-4f9c-b10a-7522a2a632ba None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:39:11.925626 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.fernet_utils [None req-0c76f5dc-90dc-4e36-8794-2a36d1db5fec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:39:11.925740 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-42fcb9fc-9a10-4f9c-b10a-7522a2a632ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:39:11.998797 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:12.005903 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 482/1010] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:39:11 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:39:12.006088 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 529/1011] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:39:11 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:39:12.027852 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fcb9fc-9a10-4f9c-b10a-7522a2a632ba None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:12.028081 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c76f5dc-90dc-4e36-8794-2a36d1db5fec None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:12.104969 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:12.135737 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fcb9fc-9a10-4f9c-b10a-7522a2a632ba None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:12.141926 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c76f5dc-90dc-4e36-8794-2a36d1db5fec None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:12.178726 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0984d79e-4560-4f76-acb1-3cb89c93aa8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:12.179538 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0984d79e-4560-4f76-acb1-3cb89c93aa8d None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:12.179825 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0984d79e-4560-4f76-acb1-3cb89c93aa8d None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:12.180106 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0984d79e-4560-4f76-acb1-3cb89c93aa8d None admin] PATH_INFO: `/v3/services/compute` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:12.180760 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0984d79e-4560-4f76-acb1-3cb89c93aa8d None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:12.182164 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0984d79e-4560-4f76-acb1-3cb89c93aa8d None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:12.185573 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16062f7c-542c-4e35-a4d9-71b4179a894b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:12.186405 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16062f7c-542c-4e35-a4d9-71b4179a894b None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:12.186695 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16062f7c-542c-4e35-a4d9-71b4179a894b None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:12.186971 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16062f7c-542c-4e35-a4d9-71b4179a894b None admin] PATH_INFO: `/v3/domains/default` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:12.188159 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16062f7c-542c-4e35-a4d9-71b4179a894b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:12.189591 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16062f7c-542c-4e35-a4d9-71b4179a894b None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:12.191247 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 530/1012] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:39:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:39:12.196953 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16062f7c-542c-4e35-a4d9-71b4179a894b None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:12.199576 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-0984d79e-4560-4f76-acb1-3cb89c93aa8d None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 19 14:39:12.199576 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:39:12.199576 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:39:12.199576 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:39:12.199576 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:39:12.199576 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:39:12.199576 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:39:12.199576 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:39:12.199576 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 14:39:12.199576 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:39:12.199576 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:39:12.199576 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:39:12.199576 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:39:12.199576 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:39:12.199576 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:39:12.199576 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:39:12.199576 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:39:12.200690 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:39:12.200690 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:39:12.200690 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:39:12.200690 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:39:12.200690 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:39:12.200690 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:39:12.200690 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:39:12.200690 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:39:12.200690 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:39:12.200690 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:39:12.200690 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:39:12.200690 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:39:12.200690 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:39:12.200690 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:39:12.200690 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:39:12.200690 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:39:12.201644 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:39:12.201644 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:39:12.201644 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:39:12.201644 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:39:12.201644 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:39:12.201644 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:39:12.201644 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:39:12.201644 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:39:12.201644 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:39:12.201644 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:39:12.201644 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:39:12.201644 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:39:12.201644 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 14:39:12.201644 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:39:12.201644 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 483/1013] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:39:12 2022] GET /identity/v3/services/compute => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:39:12.207133 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0984d79e-4560-4f76-acb1-3cb89c93aa8d None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:12.278059 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:12.316205 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16062f7c-542c-4e35-a4d9-71b4179a894b None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:12.324725 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0984d79e-4560-4f76-acb1-3cb89c93aa8d None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:12.354738 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4c6b34-97f6-4e6f-a201-97900fa0b277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:12.355409 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4c6b34-97f6-4e6f-a201-97900fa0b277 None admin] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:12.355695 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4c6b34-97f6-4e6f-a201-97900fa0b277 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:12.355981 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4c6b34-97f6-4e6f-a201-97900fa0b277 None admin] PATH_INFO: `/v3/projects` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:12.356723 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4c6b34-97f6-4e6f-a201-97900fa0b277 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:12.358196 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4c6b34-97f6-4e6f-a201-97900fa0b277 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:12.361247 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e58671-3d7a-413e-9065-26f0091faa9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:12.362062 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e58671-3d7a-413e-9065-26f0091faa9f None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:12.362547 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:12.363065 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e58671-3d7a-413e-9065-26f0091faa9f None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:12.363352 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e58671-3d7a-413e-9065-26f0091faa9f None admin] PATH_INFO: `/v3/services` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:12.364113 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e58671-3d7a-413e-9065-26f0091faa9f None admin] RBAC: Adding query filter params (name=compute) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:39:12.364380 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e58671-3d7a-413e-9065-26f0091faa9f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:12.366347 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e58671-3d7a-413e-9065-26f0091faa9f None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:12.376299 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 484/1014] 10.222.0.57 () {56 vars in 1112 bytes} [Thu May 19 14:39:12 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:39:12.377434 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e58671-3d7a-413e-9065-26f0091faa9f None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:12.391976 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 531/1015] 10.222.0.57 () {60 vars in 1141 bytes} [Thu May 19 14:39:12 2022] POST /identity/v3/projects => generated 311 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 14:39:12.465970 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42898355-bee7-45f0-a143-c6422f9184df None None] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:12.466174 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42898355-bee7-45f0-a143-c6422f9184df None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:12.466434 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42898355-bee7-45f0-a143-c6422f9184df None None] PATH_INFO: `/` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:12.468123 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 532/1016] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:39:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:39:12.474657 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7b2605-be11-4e42-a7b8-5b218681ee3e None None] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:12.475460 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7b2605-be11-4e42-a7b8-5b218681ee3e None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:12.475740 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7b2605-be11-4e42-a7b8-5b218681ee3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:12.493578 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.auth.core [None req-7e7b2605-be11-4e42-a7b8-5b218681ee3e None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:39:12.495158 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.fernet_utils [None req-7e7b2605-be11-4e42-a7b8-5b218681ee3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:39:12.507701 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e58671-3d7a-413e-9065-26f0091faa9f None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:12.537315 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:12.547175 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44a5860-ba72-474d-89fd-be6d807d583f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:12.547863 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44a5860-ba72-474d-89fd-be6d807d583f None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:12.548149 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44a5860-ba72-474d-89fd-be6d807d583f None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:12.548426 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44a5860-ba72-474d-89fd-be6d807d583f None admin] PATH_INFO: `/v3/services` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:12.549154 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44a5860-ba72-474d-89fd-be6d807d583f None admin] RBAC: Adding query filter params (type=compute) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:39:12.549461 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44a5860-ba72-474d-89fd-be6d807d583f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:12.550906 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44a5860-ba72-474d-89fd-be6d807d583f None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:12.556860 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 485/1017] 10.222.0.57 () {56 vars in 1112 bytes} [Thu May 19 14:39:12 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:39:12.562765 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44a5860-ba72-474d-89fd-be6d807d583f None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:12.572506 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 533/1018] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:39:12 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:39:12.596971 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7b2605-be11-4e42-a7b8-5b218681ee3e None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:12.679651 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44a5860-ba72-474d-89fd-be6d807d583f None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:12.711952 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7b2605-be11-4e42-a7b8-5b218681ee3e None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:12.719884 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db7a764-2571-4348-b8e3-62491b4dafee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:12.720335 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db7a764-2571-4348-b8e3-62491b4dafee None admin] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:12.720661 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db7a764-2571-4348-b8e3-62491b4dafee None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:12.720945 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db7a764-2571-4348-b8e3-62491b4dafee None admin] PATH_INFO: `/v3/endpoints` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:12.721696 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db7a764-2571-4348-b8e3-62491b4dafee None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:12.723110 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db7a764-2571-4348-b8e3-62491b4dafee None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:12.731631 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:12.745275 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 486/1019] 10.222.0.57 () {60 vars in 1144 bytes} [Thu May 19 14:39:12 2022] POST /identity/v3/endpoints => generated 334 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 14:39:12.759536 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b7e1c5-9d4f-4b29-9042-43b1280d92d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:12.760243 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b7e1c5-9d4f-4b29-9042-43b1280d92d3 None admin] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:12.760511 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b7e1c5-9d4f-4b29-9042-43b1280d92d3 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:12.760779 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b7e1c5-9d4f-4b29-9042-43b1280d92d3 None admin] PATH_INFO: `/v3/services` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:12.761521 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b7e1c5-9d4f-4b29-9042-43b1280d92d3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:12.763022 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b7e1c5-9d4f-4b29-9042-43b1280d92d3 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:12.789606 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 534/1020] 10.222.0.57 () {60 vars in 1141 bytes} [Thu May 19 14:39:12 2022] POST /identity/v3/services => generated 242 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 14:39:14.725366 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3cea09-3fc5-4521-bc55-dfa6072c220a None None] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:14.726218 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3cea09-3fc5-4521-bc55-dfa6072c220a None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:14.726218 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3cea09-3fc5-4521-bc55-dfa6072c220a None None] PATH_INFO: `/` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:14.727574 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 487/1021] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:39:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:39:14.732776 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8487a569-9c0b-458e-aa4b-73026c2ca29f None None] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:14.733181 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8487a569-9c0b-458e-aa4b-73026c2ca29f None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:14.733378 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8487a569-9c0b-458e-aa4b-73026c2ca29f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:14.750685 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.auth.core [None req-8487a569-9c0b-458e-aa4b-73026c2ca29f None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:39:14.751889 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.fernet_utils [None req-8487a569-9c0b-458e-aa4b-73026c2ca29f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:39:14.830597 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 535/1022] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:39:14 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:39:14.856430 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3cea09-3fc5-4521-bc55-dfa6072c220a None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:14.919443 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:14.986202 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3cea09-3fc5-4521-bc55-dfa6072c220a None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:15.023261 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bce99c-3e59-40ce-a303-0bd830a15a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:15.024348 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bce99c-3e59-40ce-a303-0bd830a15a5f None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:15.024792 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bce99c-3e59-40ce-a303-0bd830a15a5f None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:15.025225 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bce99c-3e59-40ce-a303-0bd830a15a5f None admin] PATH_INFO: `/v3/domains/default` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:15.026605 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bce99c-3e59-40ce-a303-0bd830a15a5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:15.028241 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bce99c-3e59-40ce-a303-0bd830a15a5f None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:15.030429 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 488/1023] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:39:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:39:15.035738 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8487a569-9c0b-458e-aa4b-73026c2ca29f None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:15.065636 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621b5557-70f5-46c9-a534-6e172f37d9a9 None None] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:15.065997 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621b5557-70f5-46c9-a534-6e172f37d9a9 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:15.066287 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621b5557-70f5-46c9-a534-6e172f37d9a9 None None] PATH_INFO: `/` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:15.067682 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 489/1024] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:39:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:39:15.073956 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7444ab-c5ad-4352-a68f-19d10633888d None None] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:15.074252 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7444ab-c5ad-4352-a68f-19d10633888d None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:15.074538 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7444ab-c5ad-4352-a68f-19d10633888d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:15.092987 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.auth.core [None req-3d7444ab-c5ad-4352-a68f-19d10633888d None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:39:15.094204 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-3d7444ab-c5ad-4352-a68f-19d10633888d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:39:15.146088 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:15.149347 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8487a569-9c0b-458e-aa4b-73026c2ca29f None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:15.169247 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 490/1025] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:39:15 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:39:15.170688 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ecfca2-414f-4646-9f64-0bf30508cb46 None None] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:15.170982 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ecfca2-414f-4646-9f64-0bf30508cb46 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:15.171257 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ecfca2-414f-4646-9f64-0bf30508cb46 None None] PATH_INFO: `/` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:15.172643 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 491/1026] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:39:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:39:15.178348 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41955a0e-c541-412c-8211-0ccf8c79c910 None None] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:15.178644 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41955a0e-c541-412c-8211-0ccf8c79c910 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:15.178914 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41955a0e-c541-412c-8211-0ccf8c79c910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:15.189743 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768727c2-cc9f-4d2b-bbeb-0fb54e96ec6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:15.190447 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768727c2-cc9f-4d2b-bbeb-0fb54e96ec6f None admin] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:15.190728 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768727c2-cc9f-4d2b-bbeb-0fb54e96ec6f None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:15.190999 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768727c2-cc9f-4d2b-bbeb-0fb54e96ec6f None admin] PATH_INFO: `/v3/users` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:15.191724 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768727c2-cc9f-4d2b-bbeb-0fb54e96ec6f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:15.193167 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768727c2-cc9f-4d2b-bbeb-0fb54e96ec6f None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:15.196901 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.auth.core [None req-41955a0e-c541-412c-8211-0ccf8c79c910 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:39:15.198138 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-41955a0e-c541-412c-8211-0ccf8c79c910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:39:15.217834 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 536/1027] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 19 14:39:15 2022] POST /identity/v3/users => generated 285 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:39:15.218691 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768727c2-cc9f-4d2b-bbeb-0fb54e96ec6f None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:15.252892 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:15.283507 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 492/1028] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:39:15 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:39:15.308841 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41955a0e-c541-412c-8211-0ccf8c79c910 None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:15.338672 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768727c2-cc9f-4d2b-bbeb-0fb54e96ec6f None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:15.381609 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab00b5f-c236-47bc-bbe6-968ce92db78f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:15.382572 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab00b5f-c236-47bc-bbe6-968ce92db78f None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:15.382855 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab00b5f-c236-47bc-bbe6-968ce92db78f None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:15.383250 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab00b5f-c236-47bc-bbe6-968ce92db78f None admin] PATH_INFO: `/v3/services/image` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:15.383930 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab00b5f-c236-47bc-bbe6-968ce92db78f None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:15.385358 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab00b5f-c236-47bc-bbe6-968ce92db78f None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:15.400850 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-cab00b5f-c236-47bc-bbe6-968ce92db78f None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 19 14:39:15.400850 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:39:15.400850 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:39:15.400850 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:39:15.400850 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:39:15.400850 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:39:15.400850 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:39:15.400850 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:39:15.400850 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 14:39:15.400850 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:39:15.400850 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:39:15.400850 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:39:15.400850 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:39:15.400850 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:39:15.400850 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:39:15.400850 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:39:15.400850 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:39:15.402099 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:39:15.402099 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:39:15.402099 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:39:15.402099 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:39:15.402099 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:39:15.402099 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:39:15.402099 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:39:15.402099 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:39:15.402099 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:39:15.402099 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:39:15.402099 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:39:15.402099 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:39:15.402099 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:39:15.402099 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:39:15.402099 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:39:15.402099 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:39:15.403141 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:39:15.403141 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:39:15.403141 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:39:15.403141 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:39:15.403141 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:39:15.403141 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:39:15.403141 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:39:15.403141 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:39:15.403141 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:39:15.403141 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:39:15.403141 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:39:15.403141 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:39:15.403141 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 14:39:15.403141 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:39:15.403141 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 537/1029] 10.222.0.57 () {56 vars in 1105 bytes} [Thu May 19 14:39:15 2022] GET /identity/v3/services/image => generated 86 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:39:15.408362 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab00b5f-c236-47bc-bbe6-968ce92db78f None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:15.425591 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41955a0e-c541-412c-8211-0ccf8c79c910 None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:15.434900 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:15.470077 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4843f44c-9f4a-4e37-9012-d4fadcb7ee68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:15.470901 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4843f44c-9f4a-4e37-9012-d4fadcb7ee68 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:15.471187 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4843f44c-9f4a-4e37-9012-d4fadcb7ee68 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:15.471461 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4843f44c-9f4a-4e37-9012-d4fadcb7ee68 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:15.486981 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4843f44c-9f4a-4e37-9012-d4fadcb7ee68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:39:15.495786 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-4843f44c-9f4a-4e37-9012-d4fadcb7ee68 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 14:39:15.495786 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:39:15.495786 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:39:15.495786 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:39:15.495786 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:39:15.495786 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:39:15.495786 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:39:15.495786 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:39:15.495786 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:39:15.495786 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:39:15.495786 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:39:15.495786 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:39:15.495786 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:39:15.495786 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:39:15.495786 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:39:15.495786 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:39:15.496848 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:39:15.496848 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:39:15.496848 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:39:15.496848 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:39:15.496848 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:39:15.496848 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:39:15.496848 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:39:15.496848 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:39:15.496848 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:39:15.496848 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:39:15.496848 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:39:15.496848 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:39:15.496848 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:39:15.496848 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:39:15.496848 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:39:15.496848 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:39:15.497796 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:39:15.497796 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:39:15.497796 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:39:15.497796 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:39:15.497796 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:39:15.497796 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:39:15.497796 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:39:15.497796 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:39:15.497796 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:39:15.497796 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:39:15.497796 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 14:39:15.497796 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:39:15.497796 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 493/1030] 10.222.0.57 () {56 vars in 1120 bytes} [Thu May 19 14:39:15 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:39:15.504571 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4843f44c-9f4a-4e37-9012-d4fadcb7ee68 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:15.530899 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab00b5f-c236-47bc-bbe6-968ce92db78f None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:15.545653 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:15.573774 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e741bd87-6ae5-4f72-ba0e-57cda846ffb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:15.574575 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e741bd87-6ae5-4f72-ba0e-57cda846ffb2 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:15.574860 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e741bd87-6ae5-4f72-ba0e-57cda846ffb2 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:15.575258 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e741bd87-6ae5-4f72-ba0e-57cda846ffb2 None admin] PATH_INFO: `/v3/services` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:15.575959 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e741bd87-6ae5-4f72-ba0e-57cda846ffb2 None admin] RBAC: Adding query filter params (name=image) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:39:15.576267 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e741bd87-6ae5-4f72-ba0e-57cda846ffb2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:15.577903 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e741bd87-6ae5-4f72-ba0e-57cda846ffb2 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:15.588955 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 538/1031] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:39:15 2022] GET /identity/v3/services?name=image => generated 122 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:39:15.589811 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e741bd87-6ae5-4f72-ba0e-57cda846ffb2 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:15.621161 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4843f44c-9f4a-4e37-9012-d4fadcb7ee68 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:15.656636 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e3e58e-98e2-4827-bcfc-684eaf6c7e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:15.657308 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e3e58e-98e2-4827-bcfc-684eaf6c7e0c None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:15.657595 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e3e58e-98e2-4827-bcfc-684eaf6c7e0c None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:15.657888 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e3e58e-98e2-4827-bcfc-684eaf6c7e0c None admin] PATH_INFO: `/v3/roles` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:15.658698 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e3e58e-98e2-4827-bcfc-684eaf6c7e0c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:39:15.658954 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e3e58e-98e2-4827-bcfc-684eaf6c7e0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:15.660337 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e3e58e-98e2-4827-bcfc-684eaf6c7e0c None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:15.674514 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 494/1032] 10.222.0.57 () {56 vars in 1115 bytes} [Thu May 19 14:39:15 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:39:15.675554 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e3e58e-98e2-4827-bcfc-684eaf6c7e0c None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:15.686029 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:15.703248 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e741bd87-6ae5-4f72-ba0e-57cda846ffb2 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:15.742522 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337d5c59-3b7b-4aee-a85f-560bc163ed43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:15.743431 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337d5c59-3b7b-4aee-a85f-560bc163ed43 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:15.743655 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337d5c59-3b7b-4aee-a85f-560bc163ed43 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:15.743900 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337d5c59-3b7b-4aee-a85f-560bc163ed43 None admin] PATH_INFO: `/v3/services` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:15.744615 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337d5c59-3b7b-4aee-a85f-560bc163ed43 None admin] RBAC: Adding query filter params (type=image) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:39:15.744926 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337d5c59-3b7b-4aee-a85f-560bc163ed43 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:15.746404 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337d5c59-3b7b-4aee-a85f-560bc163ed43 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:15.751460 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 539/1033] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:39:15 2022] GET /identity/v3/services?type=image => generated 350 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:39:15.757241 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337d5c59-3b7b-4aee-a85f-560bc163ed43 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:15.784066 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:15.789170 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e3e58e-98e2-4827-bcfc-684eaf6c7e0c None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:15.829710 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780a9dec-8bfe-44fc-8589-090936f4ff62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:15.830546 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780a9dec-8bfe-44fc-8589-090936f4ff62 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:15.830831 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780a9dec-8bfe-44fc-8589-090936f4ff62 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:15.831095 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780a9dec-8bfe-44fc-8589-090936f4ff62 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:15.842661 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780a9dec-8bfe-44fc-8589-090936f4ff62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:15.844178 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780a9dec-8bfe-44fc-8589-090936f4ff62 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:15.862342 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-780a9dec-8bfe-44fc-8589-090936f4ff62 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:39:15.862342 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:39:15.862342 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:39:15.862342 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:39:15.862342 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:39:15.862342 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:39:15.862342 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:39:15.862342 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:39:15.862342 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:39:15.862342 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:39:15.862342 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:39:15.862342 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:39:15.862342 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:39:15.862342 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:39:15.862342 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:39:15.863591 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:39:15.863591 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:39:15.863591 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:39:15.863591 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:39:15.863591 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:39:15.863591 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:39:15.863591 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:39:15.863591 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:39:15.863591 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:39:15.863591 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:39:15.863591 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:39:15.863591 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:39:15.863591 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:39:15.863591 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:39:15.863591 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:39:15.863591 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:39:15.863591 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:39:15.864766 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:39:15.864766 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:39:15.864766 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:39:15.864766 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:39:15.864766 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 495/1034] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:39:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:39:15.870335 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780a9dec-8bfe-44fc-8589-090936f4ff62 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:15.878331 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337d5c59-3b7b-4aee-a85f-560bc163ed43 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:15.942697 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0072d7-4307-4567-ab4d-27d843ef53cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:15.945718 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0072d7-4307-4567-ab4d-27d843ef53cd None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:15.946046 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0072d7-4307-4567-ab4d-27d843ef53cd None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:15.946331 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0072d7-4307-4567-ab4d-27d843ef53cd None admin] PATH_INFO: `/v3/endpoints` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:15.947418 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0072d7-4307-4567-ab4d-27d843ef53cd None admin] RBAC: Adding query filter params (interface=public, service_id=172aa94098c24846a11c8fca45012cbd, region_id=RegionOne) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:39:15.947733 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0072d7-4307-4567-ab4d-27d843ef53cd None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:15.949773 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0072d7-4307-4567-ab4d-27d843ef53cd None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:15.962063 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 540/1035] 10.222.0.57 () {56 vars in 1251 bytes} [Thu May 19 14:39:15 2022] GET /identity/v3/endpoints?service_id=172aa94098c24846a11c8fca45012cbd&interface=public®ion_id=RegionOne => generated 194 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:39:15.963088 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0072d7-4307-4567-ab4d-27d843ef53cd None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:15.979575 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:16.010717 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780a9dec-8bfe-44fc-8589-090936f4ff62 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:16.049357 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94afe6f5-ba4b-4caa-8735-ae3299fd4225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:16.050177 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94afe6f5-ba4b-4caa-8735-ae3299fd4225 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:16.050463 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94afe6f5-ba4b-4caa-8735-ae3299fd4225 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:16.050862 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94afe6f5-ba4b-4caa-8735-ae3299fd4225 None admin] PATH_INFO: `/v3/domains` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:16.051602 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94afe6f5-ba4b-4caa-8735-ae3299fd4225 None admin] RBAC: Adding query filter params (name=Default) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:39:16.051911 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94afe6f5-ba4b-4caa-8735-ae3299fd4225 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:16.053360 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94afe6f5-ba4b-4caa-8735-ae3299fd4225 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:16.066921 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 496/1036] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:39:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:39:16.071704 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0072d7-4307-4567-ab4d-27d843ef53cd None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:16.072704 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94afe6f5-ba4b-4caa-8735-ae3299fd4225 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:16.086347 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:16.109644 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddc356a-22e7-464a-b84f-9ec38e060582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:16.110343 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddc356a-22e7-464a-b84f-9ec38e060582 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:16.110625 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddc356a-22e7-464a-b84f-9ec38e060582 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:16.110896 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddc356a-22e7-464a-b84f-9ec38e060582 None admin] PATH_INFO: `/v3/services` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:16.111577 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddc356a-22e7-464a-b84f-9ec38e060582 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:16.112972 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddc356a-22e7-464a-b84f-9ec38e060582 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:16.119589 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 541/1037] 10.222.0.57 () {56 vars in 1087 bytes} [Thu May 19 14:39:15 2022] GET /identity/v3/services => generated 1957 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 14:39:16.200261 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94afe6f5-ba4b-4caa-8735-ae3299fd4225 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:16.239650 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f0ee7c-7559-4f81-9a8a-f8fb956b7ef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:16.240508 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f0ee7c-7559-4f81-9a8a-f8fb956b7ef8 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:16.240793 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f0ee7c-7559-4f81-9a8a-f8fb956b7ef8 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:16.241133 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f0ee7c-7559-4f81-9a8a-f8fb956b7ef8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:16.249247 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f0ee7c-7559-4f81-9a8a-f8fb956b7ef8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:16.250766 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f0ee7c-7559-4f81-9a8a-f8fb956b7ef8 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:16.267542 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-30f0ee7c-7559-4f81-9a8a-f8fb956b7ef8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 14:39:16.267542 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:39:16.267542 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:39:16.267542 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:39:16.267542 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:39:16.267542 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:39:16.267542 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:39:16.267542 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:39:16.267542 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:39:16.267542 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:39:16.267542 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:39:16.267542 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:39:16.267542 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:39:16.267542 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:39:16.267542 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:39:16.267542 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:39:16.268636 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:39:16.268636 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:39:16.268636 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:39:16.268636 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:39:16.268636 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:39:16.268636 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:39:16.268636 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:39:16.268636 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:39:16.268636 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:39:16.268636 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:39:16.268636 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:39:16.268636 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:39:16.268636 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:39:16.268636 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:39:16.268636 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:39:16.268636 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:39:16.268636 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:39:16.268636 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:39:16.269708 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:39:16.269708 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:39:16.269708 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:39:16.269708 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:39:16.269708 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:39:16.269708 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:39:16.269708 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:39:16.269708 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:39:16.269708 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 14:39:16.269708 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:39:16.269708 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 497/1038] 10.222.0.57 () {56 vars in 1093 bytes} [Thu May 19 14:39:16 2022] GET /identity/v3/users/nova => generated 82 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:39:16.275288 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddc356a-22e7-464a-b84f-9ec38e060582 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:16.363178 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:16.388997 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddc356a-22e7-464a-b84f-9ec38e060582 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:16.424748 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68920044-4c25-492c-ab36-033a2a84c41b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:16.425446 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68920044-4c25-492c-ab36-033a2a84c41b None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:16.425730 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68920044-4c25-492c-ab36-033a2a84c41b None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:16.426042 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68920044-4c25-492c-ab36-033a2a84c41b None admin] PATH_INFO: `/v3/users` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:16.426946 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68920044-4c25-492c-ab36-033a2a84c41b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:39:16.427263 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68920044-4c25-492c-ab36-033a2a84c41b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:16.429602 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68920044-4c25-492c-ab36-033a2a84c41b None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:16.449211 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 542/1039] 10.222.0.57 () {56 vars in 1133 bytes} [Thu May 19 14:39:16 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:39:16.455110 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f0ee7c-7559-4f81-9a8a-f8fb956b7ef8 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:16.528076 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:16.559973 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f0ee7c-7559-4f81-9a8a-f8fb956b7ef8 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:16.596126 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a282196d-7ada-4d48-82cd-c921e6efd2a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:16.596778 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a282196d-7ada-4d48-82cd-c921e6efd2a7 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:16.597055 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a282196d-7ada-4d48-82cd-c921e6efd2a7 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:16.597326 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a282196d-7ada-4d48-82cd-c921e6efd2a7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:16.669269 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a282196d-7ada-4d48-82cd-c921e6efd2a7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:16.670766 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a282196d-7ada-4d48-82cd-c921e6efd2a7 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:16.739519 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:16.773687 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 498/1040] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:39:16 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:39:16.779296 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68920044-4c25-492c-ab36-033a2a84c41b None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:16.889313 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68920044-4c25-492c-ab36-033a2a84c41b None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:16.925363 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7d8fb0-6215-4c73-a5e2-8c2956ac65d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:16.926342 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7d8fb0-6215-4c73-a5e2-8c2956ac65d9 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:16.926731 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7d8fb0-6215-4c73-a5e2-8c2956ac65d9 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:16.927110 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7d8fb0-6215-4c73-a5e2-8c2956ac65d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:16.929683 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:16.939834 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7d8fb0-6215-4c73-a5e2-8c2956ac65d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:16.941452 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7d8fb0-6215-4c73-a5e2-8c2956ac65d9 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:16.961823 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-9a7d8fb0-6215-4c73-a5e2-8c2956ac65d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:39:16.961823 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:39:16.961823 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:39:16.961823 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:39:16.961823 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:39:16.961823 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:39:16.961823 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:39:16.961823 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:39:16.961823 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:39:16.961823 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:39:16.961823 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:39:16.961823 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:39:16.961823 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:39:16.961823 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:39:16.961823 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:39:16.963133 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:39:16.963133 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:39:16.963133 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:39:16.963133 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:39:16.963133 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:39:16.963133 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:39:16.963133 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:39:16.963133 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:39:16.963133 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:39:16.963133 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:39:16.963133 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:39:16.963133 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:39:16.963133 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:39:16.963133 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:39:16.963133 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:39:16.963133 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:39:16.963133 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:39:16.964334 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:39:16.964334 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:39:16.964334 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:39:16.964334 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:39:16.965729 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 543/1041] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:39:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:39:16.971238 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a282196d-7ada-4d48-82cd-c921e6efd2a7 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:17.075146 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a282196d-7ada-4d48-82cd-c921e6efd2a7 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:17.110159 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6e7845-8771-4885-8d6f-163677f9efa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:17.110829 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6e7845-8771-4885-8d6f-163677f9efa1 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:17.111116 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6e7845-8771-4885-8d6f-163677f9efa1 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:17.111395 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6e7845-8771-4885-8d6f-163677f9efa1 None admin] PATH_INFO: `/v3/domains` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:17.112129 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6e7845-8771-4885-8d6f-163677f9efa1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:39:17.112496 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6e7845-8771-4885-8d6f-163677f9efa1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:17.113918 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6e7845-8771-4885-8d6f-163677f9efa1 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:17.130648 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 499/1042] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:39:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:39:17.132079 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7d8fb0-6215-4c73-a5e2-8c2956ac65d9 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:17.247609 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7d8fb0-6215-4c73-a5e2-8c2956ac65d9 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:17.287969 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fbfcc0-6d28-4ede-8829-4751f26d8f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:17.288774 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fbfcc0-6d28-4ede-8829-4751f26d8f9a None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:17.289060 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fbfcc0-6d28-4ede-8829-4751f26d8f9a None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:17.289338 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fbfcc0-6d28-4ede-8829-4751f26d8f9a None admin] PATH_INFO: `/v3/projects/service` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:17.296760 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fbfcc0-6d28-4ede-8829-4751f26d8f9a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:17.298427 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fbfcc0-6d28-4ede-8829-4751f26d8f9a None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:17.313532 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-22fbfcc0-6d28-4ede-8829-4751f26d8f9a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:39:17.313532 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:39:17.313532 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:39:17.313532 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:39:17.313532 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:39:17.313532 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:39:17.313532 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:39:17.313532 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:39:17.313532 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:39:17.313532 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:39:17.313532 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:39:17.313532 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:39:17.313532 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:39:17.313532 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:39:17.313532 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:39:17.314588 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:39:17.314588 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:39:17.314588 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:39:17.314588 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:39:17.314588 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:39:17.314588 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:39:17.314588 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:39:17.314588 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:39:17.314588 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:39:17.314588 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:39:17.314588 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:39:17.314588 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:39:17.314588 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:39:17.314588 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:39:17.314588 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:39:17.314588 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:39:17.314588 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:39:17.321404 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:39:17.321404 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:39:17.321404 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:39:17.321404 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:39:17.321404 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:39:17.321404 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:39:17.321404 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:39:17.321404 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:39:17.321404 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 544/1043] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:39:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:39:17.322010 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6e7845-8771-4885-8d6f-163677f9efa1 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:17.356412 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:17.434811 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6e7845-8771-4885-8d6f-163677f9efa1 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:17.470442 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15477ab-4781-4b49-bbda-2607320f0030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:17.471106 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15477ab-4781-4b49-bbda-2607320f0030 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:17.471390 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15477ab-4781-4b49-bbda-2607320f0030 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:17.471665 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15477ab-4781-4b49-bbda-2607320f0030 None admin] PATH_INFO: `/v3/projects` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:17.472453 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15477ab-4781-4b49-bbda-2607320f0030 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:39:17.472764 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15477ab-4781-4b49-bbda-2607320f0030 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:17.474217 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15477ab-4781-4b49-bbda-2607320f0030 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:17.487007 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 500/1044] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:39:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:39:17.494848 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fbfcc0-6d28-4ede-8829-4751f26d8f9a None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:17.542779 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:17.592038 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3275fb-7ecc-4091-ad51-b6ec38b6c37f None None] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:17.592363 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3275fb-7ecc-4091-ad51-b6ec38b6c37f None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:17.592619 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3275fb-7ecc-4091-ad51-b6ec38b6c37f None None] PATH_INFO: `/` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:17.594043 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 501/1045] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:39:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:39:17.598712 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fbfcc0-6d28-4ede-8829-4751f26d8f9a None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:17.599196 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62025b7-d19b-4a7d-9e7c-420a0e293368 None None] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:17.599559 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62025b7-d19b-4a7d-9e7c-420a0e293368 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:17.599869 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62025b7-d19b-4a7d-9e7c-420a0e293368 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:17.617234 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.auth.core [None req-f62025b7-d19b-4a7d-9e7c-420a0e293368 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:39:17.618408 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-f62025b7-d19b-4a7d-9e7c-420a0e293368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:39:17.636814 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af98fc1-ca82-4a1d-a85c-0a118fcff22c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:17.637481 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af98fc1-ca82-4a1d-a85c-0a118fcff22c None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:17.637770 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af98fc1-ca82-4a1d-a85c-0a118fcff22c None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:17.638096 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af98fc1-ca82-4a1d-a85c-0a118fcff22c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:17.638945 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af98fc1-ca82-4a1d-a85c-0a118fcff22c None admin] RBAC: Adding query filter params (role.id=db02c08ef7c847c0ba25ed9ddc05974c, scope.project.id=42f35b96b9d4499abfb5e5f7cca49adc, user.id=a9591727b2f54244baf90b11a8e2277f) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:39:17.639264 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af98fc1-ca82-4a1d-a85c-0a118fcff22c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:17.640712 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af98fc1-ca82-4a1d-a85c-0a118fcff22c None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:17.646537 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 545/1046] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:39:17 2022] GET /identity/v3/role_assignments?user.id=a9591727b2f54244baf90b11a8e2277f&scope.project.id=42f35b96b9d4499abfb5e5f7cca49adc&role.id=db02c08ef7c847c0ba25ed9ddc05974c => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:39:17.697437 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 502/1047] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:39:17 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:39:17.721696 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af98fc1-ca82-4a1d-a85c-0a118fcff22c None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:17.836984 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af98fc1-ca82-4a1d-a85c-0a118fcff22c None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:17.848425 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:17.877573 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dda35da-028c-45a5-9e25-986f6308271d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:17.878418 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dda35da-028c-45a5-9e25-986f6308271d None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:17.878702 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dda35da-028c-45a5-9e25-986f6308271d None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:17.878977 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dda35da-028c-45a5-9e25-986f6308271d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:17.890015 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dda35da-028c-45a5-9e25-986f6308271d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:39:17.897724 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-8dda35da-028c-45a5-9e25-986f6308271d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:39:17.897724 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:39:17.897724 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:39:17.897724 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:39:17.897724 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:39:17.897724 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:39:17.897724 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:39:17.897724 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:39:17.897724 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:39:17.897724 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:39:17.897724 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:39:17.897724 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:39:17.897724 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:39:17.897724 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:39:17.897724 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:39:17.897724 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:39:17.898822 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:39:17.898822 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:39:17.898822 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:39:17.898822 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:39:17.898822 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:39:17.898822 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:39:17.898822 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:39:17.898822 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:39:17.898822 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:39:17.898822 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:39:17.898822 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:39:17.898822 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:39:17.898822 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:39:17.898822 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:39:17.898822 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:39:17.898822 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:39:17.899736 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:39:17.899736 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:39:17.899736 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:39:17.899736 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:39:17.899736 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:39:17.899736 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:39:17.899736 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:39:17.899736 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:39:17.899736 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:39:17.899736 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:39:17.899736 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:39:17.899736 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:39:17.901912 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 546/1048] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:39:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:39:17.905053 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62025b7-d19b-4a7d-9e7c-420a0e293368 None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:17.926929 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:18.010214 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62025b7-d19b-4a7d-9e7c-420a0e293368 None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:18.045512 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6934ef61-929a-49f3-a280-414936c35e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:18.046222 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6934ef61-929a-49f3-a280-414936c35e7f None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:18.046512 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6934ef61-929a-49f3-a280-414936c35e7f None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:18.046789 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6934ef61-929a-49f3-a280-414936c35e7f None admin] PATH_INFO: `/v3/roles` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:18.047547 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6934ef61-929a-49f3-a280-414936c35e7f None admin] RBAC: Adding query filter params (name=admin) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:39:18.047858 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6934ef61-929a-49f3-a280-414936c35e7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:18.049235 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6934ef61-929a-49f3-a280-414936c35e7f None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:18.062859 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 503/1049] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:39:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:39:18.064131 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dda35da-028c-45a5-9e25-986f6308271d None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:18.185533 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dda35da-028c-45a5-9e25-986f6308271d None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:18.225471 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0324cd9-0ec2-4f5d-8dee-122d113e1d3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:18.226469 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0324cd9-0ec2-4f5d-8dee-122d113e1d3f None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:18.226756 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0324cd9-0ec2-4f5d-8dee-122d113e1d3f None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:18.227087 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0324cd9-0ec2-4f5d-8dee-122d113e1d3f None admin] PATH_INFO: `/v3/users/adc0083f4763459891e0473ecbbb070e` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:18.246633 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0324cd9-0ec2-4f5d-8dee-122d113e1d3f None admin] RBAC: Authorizing `identity:get_user(user_id=adc0083f4763459891e0473ecbbb070e)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:18.248125 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0324cd9-0ec2-4f5d-8dee-122d113e1d3f None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:18.250097 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 547/1050] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:39:18 2022] GET /identity/v3/users/adc0083f4763459891e0473ecbbb070e => generated 285 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:39:18.255633 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6934ef61-929a-49f3-a280-414936c35e7f None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:18.347511 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:18.368406 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6934ef61-929a-49f3-a280-414936c35e7f None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:18.404456 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e4e9df-b288-4af2-a771-c4d7150e84ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:18.405122 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e4e9df-b288-4af2-a771-c4d7150e84ae None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:18.405412 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e4e9df-b288-4af2-a771-c4d7150e84ae None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:18.405696 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e4e9df-b288-4af2-a771-c4d7150e84ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:18.418914 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dce41b7-a1f4-43d5-8aa5-9503e9388602 None None] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:18.419110 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dce41b7-a1f4-43d5-8aa5-9503e9388602 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:18.419387 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dce41b7-a1f4-43d5-8aa5-9503e9388602 None None] PATH_INFO: `/` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:18.420759 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 548/1051] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:39:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:39:18.427997 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a1d746-06ea-4bbb-9848-73019e76d1b3 None None] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:18.428315 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a1d746-06ea-4bbb-9848-73019e76d1b3 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:18.428568 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a1d746-06ea-4bbb-9848-73019e76d1b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:18.446083 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.auth.core [None req-58a1d746-06ea-4bbb-9848-73019e76d1b3 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:39:18.447321 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.fernet_utils [None req-58a1d746-06ea-4bbb-9848-73019e76d1b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:39:18.472882 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:18.486587 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e4e9df-b288-4af2-a771-c4d7150e84ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:18.488043 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e4e9df-b288-4af2-a771-c4d7150e84ae None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:18.525322 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 549/1052] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:39:18 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:39:18.547974 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a1d746-06ea-4bbb-9848-73019e76d1b3 None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:18.578449 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:18.602262 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 504/1053] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:39:18 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:39:18.608098 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e4e9df-b288-4af2-a771-c4d7150e84ae None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:18.658946 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a1d746-06ea-4bbb-9848-73019e76d1b3 None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:18.695539 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7733d6-f205-4011-95fb-572a73d6cd8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:18.696250 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7733d6-f205-4011-95fb-572a73d6cd8f None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:18.696534 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7733d6-f205-4011-95fb-572a73d6cd8f None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:18.696819 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7733d6-f205-4011-95fb-572a73d6cd8f None admin] PATH_INFO: `/v3/services/image` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:18.697479 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7733d6-f205-4011-95fb-572a73d6cd8f None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:18.698898 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7733d6-f205-4011-95fb-572a73d6cd8f None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:18.712294 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-4f7733d6-f205-4011-95fb-572a73d6cd8f None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 19 14:39:18.712294 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:39:18.712294 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:39:18.712294 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:39:18.712294 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:39:18.712294 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:39:18.712294 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:39:18.712294 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:39:18.712294 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 14:39:18.712294 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:39:18.712294 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:39:18.712294 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:39:18.712294 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:39:18.712294 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:39:18.712294 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:39:18.712294 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:39:18.712294 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:39:18.713698 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:39:18.713698 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:39:18.713698 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:39:18.713698 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:39:18.713698 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:39:18.713698 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:39:18.713698 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:39:18.713698 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:39:18.713698 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:39:18.713698 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:39:18.713698 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:39:18.713698 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:39:18.713698 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:39:18.713698 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:39:18.713698 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:39:18.713698 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:39:18.714754 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:39:18.714754 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:39:18.714754 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:39:18.714754 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:39:18.714754 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:39:18.714754 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:39:18.714754 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:39:18.714754 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:39:18.714754 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:39:18.714754 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:39:18.714754 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:39:18.714754 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:39:18.714754 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 14:39:18.714754 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:39:18.714754 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 550/1054] 10.222.0.57 () {56 vars in 1105 bytes} [Thu May 19 14:39:18 2022] GET /identity/v3/services/image => generated 86 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:39:18.719364 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7733d6-f205-4011-95fb-572a73d6cd8f None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:18.725882 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e4e9df-b288-4af2-a771-c4d7150e84ae None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:18.746362 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:18.764106 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778d22d1-5ab4-47c3-bbaa-d0914938db42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:18.764968 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778d22d1-5ab4-47c3-bbaa-d0914938db42 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:18.765248 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778d22d1-5ab4-47c3-bbaa-d0914938db42 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:18.765520 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778d22d1-5ab4-47c3-bbaa-d0914938db42 None admin] PATH_INFO: `/v3/projects/b65078fee1a04ca1b954acd01fd0f717` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:18.766761 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778d22d1-5ab4-47c3-bbaa-d0914938db42 None admin] RBAC: Authorizing `identity:get_project(project_id=b65078fee1a04ca1b954acd01fd0f717)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:18.768298 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778d22d1-5ab4-47c3-bbaa-d0914938db42 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:18.775756 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 505/1055] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:39:18 2022] GET /identity/v3/projects/b65078fee1a04ca1b954acd01fd0f717 => generated 311 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:39:18.776611 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778d22d1-5ab4-47c3-bbaa-d0914938db42 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:18.835751 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:18.837963 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7733d6-f205-4011-95fb-572a73d6cd8f None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:18.874833 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0ca4c1-8a94-4341-af7f-2747c62eea4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:18.875619 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0ca4c1-8a94-4341-af7f-2747c62eea4e None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:18.875893 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0ca4c1-8a94-4341-af7f-2747c62eea4e None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:18.876288 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0ca4c1-8a94-4341-af7f-2747c62eea4e None admin] PATH_INFO: `/v3/services` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:18.876988 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0ca4c1-8a94-4341-af7f-2747c62eea4e None admin] RBAC: Adding query filter params (name=image) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:39:18.877292 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0ca4c1-8a94-4341-af7f-2747c62eea4e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:18.878701 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0ca4c1-8a94-4341-af7f-2747c62eea4e None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:18.890051 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 551/1056] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:39:18 2022] GET /identity/v3/services?name=image => generated 122 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:39:18.890865 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0ca4c1-8a94-4341-af7f-2747c62eea4e None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:18.892524 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778d22d1-5ab4-47c3-bbaa-d0914938db42 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:18.931192 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3da7126-ea3f-468e-91a0-56a94eb6452f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:18.931855 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3da7126-ea3f-468e-91a0-56a94eb6452f None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:18.932143 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3da7126-ea3f-468e-91a0-56a94eb6452f None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:18.932425 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3da7126-ea3f-468e-91a0-56a94eb6452f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:18.933275 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3da7126-ea3f-468e-91a0-56a94eb6452f None admin] RBAC: Adding query filter params (role.id=c2935b1a25ca46fcba419fe3d1715330, scope.project.id=b65078fee1a04ca1b954acd01fd0f717, user.id=adc0083f4763459891e0473ecbbb070e) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:39:18.933597 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3da7126-ea3f-468e-91a0-56a94eb6452f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:18.935093 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3da7126-ea3f-468e-91a0-56a94eb6452f None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:18.940737 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 506/1057] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:39:18 2022] GET /identity/v3/role_assignments?user.id=adc0083f4763459891e0473ecbbb070e&scope.project.id=b65078fee1a04ca1b954acd01fd0f717&role.id=c2935b1a25ca46fcba419fe3d1715330 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:39:18.984924 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:19.003232 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0ca4c1-8a94-4341-af7f-2747c62eea4e None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:19.041399 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daf4358-f7ef-4934-a206-c599c24004af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:19.042453 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daf4358-f7ef-4934-a206-c599c24004af None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:19.042944 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daf4358-f7ef-4934-a206-c599c24004af None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:19.043418 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daf4358-f7ef-4934-a206-c599c24004af None admin] PATH_INFO: `/v3/services` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:19.044337 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3daf4358-f7ef-4934-a206-c599c24004af None admin] RBAC: Adding query filter params (type=image) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:39:19.044838 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3daf4358-f7ef-4934-a206-c599c24004af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:19.046552 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3daf4358-f7ef-4934-a206-c599c24004af None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:19.052362 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 552/1058] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:39:18 2022] GET /identity/v3/services?type=image => generated 350 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:39:19.059104 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3da7126-ea3f-468e-91a0-56a94eb6452f None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:19.170839 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3da7126-ea3f-468e-91a0-56a94eb6452f None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:19.185772 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:19.210056 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308c5bf8-14ba-4e9e-b8ac-0ef22ef72640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:19.210726 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308c5bf8-14ba-4e9e-b8ac-0ef22ef72640 None admin] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:19.211009 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308c5bf8-14ba-4e9e-b8ac-0ef22ef72640 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:19.211283 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308c5bf8-14ba-4e9e-b8ac-0ef22ef72640 None admin] PATH_INFO: `/v3/endpoints` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:19.212260 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308c5bf8-14ba-4e9e-b8ac-0ef22ef72640 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:19.213664 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308c5bf8-14ba-4e9e-b8ac-0ef22ef72640 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:19.239433 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 507/1059] 10.222.0.57 () {60 vars in 1144 bytes} [Thu May 19 14:39:19 2022] POST /identity/v3/endpoints => generated 327 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:39:19.958055 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6254c1b2-9c95-4fa0-8401-f994c0ffdca7 None None] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:19.958623 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6254c1b2-9c95-4fa0-8401-f994c0ffdca7 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:19.958623 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6254c1b2-9c95-4fa0-8401-f994c0ffdca7 None None] PATH_INFO: `/` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:19.965735 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 553/1060] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:39:19 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:39:19.972672 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664b20e8-a378-45f3-b59a-80e16db1e3b9 None None] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:19.972958 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664b20e8-a378-45f3-b59a-80e16db1e3b9 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:19.973243 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664b20e8-a378-45f3-b59a-80e16db1e3b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:19.991641 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.auth.core [None req-664b20e8-a378-45f3-b59a-80e16db1e3b9 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:39:19.992842 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-664b20e8-a378-45f3-b59a-80e16db1e3b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:39:20.073726 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 508/1061] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:39:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:39:20.100570 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6254c1b2-9c95-4fa0-8401-f994c0ffdca7 None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:20.221712 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6254c1b2-9c95-4fa0-8401-f994c0ffdca7 None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:20.260523 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1cfdc8-9061-45da-a3bd-b8c87579ac4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:20.261353 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1cfdc8-9061-45da-a3bd-b8c87579ac4d None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:20.261637 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1cfdc8-9061-45da-a3bd-b8c87579ac4d None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:20.261936 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1cfdc8-9061-45da-a3bd-b8c87579ac4d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:20.275521 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1cfdc8-9061-45da-a3bd-b8c87579ac4d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:39:20.283765 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-cb1cfdc8-9061-45da-a3bd-b8c87579ac4d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 14:39:20.283765 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:39:20.283765 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:39:20.283765 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:39:20.283765 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:39:20.283765 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:39:20.283765 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:39:20.283765 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:39:20.283765 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:39:20.283765 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:39:20.283765 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:39:20.283765 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:39:20.283765 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:39:20.283765 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:39:20.283765 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:39:20.283765 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:39:20.284823 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:39:20.284823 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:39:20.284823 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:39:20.284823 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:39:20.284823 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:39:20.284823 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:39:20.284823 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:39:20.284823 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:39:20.284823 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:39:20.284823 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:39:20.284823 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:39:20.284823 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:39:20.284823 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:39:20.284823 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:39:20.284823 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:39:20.284823 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:39:20.285797 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:39:20.285797 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:39:20.285797 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:39:20.285797 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:39:20.285797 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:39:20.285797 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:39:20.285797 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:39:20.285797 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:39:20.285797 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:39:20.285797 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:39:20.285797 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 14:39:20.285797 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:39:20.290279 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 554/1062] 10.222.0.57 () {56 vars in 1120 bytes} [Thu May 19 14:39:20 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:39:20.291910 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664b20e8-a378-45f3-b59a-80e16db1e3b9 None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:20.400037 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:20.406628 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664b20e8-a378-45f3-b59a-80e16db1e3b9 None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:20.449278 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68e7987-0d06-4b25-a233-32cff88564c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:20.449988 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68e7987-0d06-4b25-a233-32cff88564c3 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:20.450281 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68e7987-0d06-4b25-a233-32cff88564c3 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:20.450552 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68e7987-0d06-4b25-a233-32cff88564c3 None admin] PATH_INFO: `/v3/roles` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:20.451306 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68e7987-0d06-4b25-a233-32cff88564c3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:39:20.451618 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68e7987-0d06-4b25-a233-32cff88564c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:20.453220 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68e7987-0d06-4b25-a233-32cff88564c3 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:20.467660 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 509/1063] 10.222.0.57 () {56 vars in 1115 bytes} [Thu May 19 14:39:20 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:39:20.468628 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1cfdc8-9061-45da-a3bd-b8c87579ac4d None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:20.482101 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:20.580251 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1cfdc8-9061-45da-a3bd-b8c87579ac4d None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:20.619351 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75f42e2-7848-49b0-a676-98cab084c404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:20.620160 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75f42e2-7848-49b0-a676-98cab084c404 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:20.620439 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75f42e2-7848-49b0-a676-98cab084c404 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:20.620710 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75f42e2-7848-49b0-a676-98cab084c404 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:20.634634 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75f42e2-7848-49b0-a676-98cab084c404 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:20.636102 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75f42e2-7848-49b0-a676-98cab084c404 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:20.658381 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-d75f42e2-7848-49b0-a676-98cab084c404 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:39:20.658381 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:39:20.658381 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:39:20.658381 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:39:20.658381 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:39:20.658381 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:39:20.658381 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:39:20.658381 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:39:20.658381 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:39:20.658381 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:39:20.658381 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:39:20.658381 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:39:20.658381 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:39:20.658381 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:39:20.658381 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:39:20.659867 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:39:20.659867 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:39:20.659867 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:39:20.659867 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:39:20.659867 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:39:20.659867 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:39:20.659867 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:39:20.659867 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:39:20.659867 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:39:20.659867 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:39:20.659867 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:39:20.659867 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:39:20.659867 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:39:20.659867 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:39:20.659867 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:39:20.659867 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:39:20.659867 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:39:20.660890 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:39:20.660890 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:39:20.660890 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:39:20.660890 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:39:20.662270 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 555/1064] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:39:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:39:20.668375 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68e7987-0d06-4b25-a233-32cff88564c3 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:20.797263 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68e7987-0d06-4b25-a233-32cff88564c3 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:20.840138 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18af5f43-f484-4e41-af1a-ab722f1dfc4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:20.840800 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18af5f43-f484-4e41-af1a-ab722f1dfc4b None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:20.841083 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18af5f43-f484-4e41-af1a-ab722f1dfc4b None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:20.841360 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18af5f43-f484-4e41-af1a-ab722f1dfc4b None admin] PATH_INFO: `/v3/domains` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:20.842110 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18af5f43-f484-4e41-af1a-ab722f1dfc4b None admin] RBAC: Adding query filter params (name=Default) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:39:20.842421 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18af5f43-f484-4e41-af1a-ab722f1dfc4b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:20.843888 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18af5f43-f484-4e41-af1a-ab722f1dfc4b None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:20.857739 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 510/1065] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:39:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:39:20.863227 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75f42e2-7848-49b0-a676-98cab084c404 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:20.928663 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:20.969744 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75f42e2-7848-49b0-a676-98cab084c404 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:21.004736 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40f3790-20b5-4664-98fd-639e0125e476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:21.005534 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40f3790-20b5-4664-98fd-639e0125e476 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:21.005862 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40f3790-20b5-4664-98fd-639e0125e476 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:21.006143 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40f3790-20b5-4664-98fd-639e0125e476 None admin] PATH_INFO: `/v3/users/nova` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:21.014361 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40f3790-20b5-4664-98fd-639e0125e476 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:21.016054 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40f3790-20b5-4664-98fd-639e0125e476 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:21.030830 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-a40f3790-20b5-4664-98fd-639e0125e476 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 14:39:21.030830 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:39:21.030830 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:39:21.030830 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:39:21.030830 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:39:21.030830 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:39:21.030830 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:39:21.030830 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:39:21.030830 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:39:21.030830 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:39:21.030830 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:39:21.030830 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:39:21.030830 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:39:21.030830 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:39:21.030830 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:39:21.030830 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:39:21.031942 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:39:21.031942 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:39:21.031942 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:39:21.031942 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:39:21.031942 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:39:21.031942 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:39:21.031942 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:39:21.031942 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:39:21.031942 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:39:21.031942 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:39:21.031942 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:39:21.031942 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:39:21.031942 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:39:21.031942 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:39:21.031942 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:39:21.031942 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:39:21.031942 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:39:21.031942 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:39:21.034060 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:39:21.034060 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:39:21.034060 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:39:21.034060 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:39:21.034060 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:39:21.034060 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:39:21.034060 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:39:21.034060 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:39:21.034060 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 14:39:21.034060 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:39:21.037885 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 556/1066] 10.222.0.57 () {56 vars in 1093 bytes} [Thu May 19 14:39:20 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:39:21.038692 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18af5f43-f484-4e41-af1a-ab722f1dfc4b None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:21.059937 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:21.150888 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18af5f43-f484-4e41-af1a-ab722f1dfc4b None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:21.208470 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48adf4d-204b-4474-98a6-457ad651d8b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:21.209347 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48adf4d-204b-4474-98a6-457ad651d8b5 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:21.210125 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48adf4d-204b-4474-98a6-457ad651d8b5 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:21.210519 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48adf4d-204b-4474-98a6-457ad651d8b5 None admin] PATH_INFO: `/v3/users` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:21.211510 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48adf4d-204b-4474-98a6-457ad651d8b5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:39:21.211951 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48adf4d-204b-4474-98a6-457ad651d8b5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:21.213536 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48adf4d-204b-4474-98a6-457ad651d8b5 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:21.232091 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 511/1067] 10.222.0.57 () {56 vars in 1133 bytes} [Thu May 19 14:39:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:39:21.237588 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40f3790-20b5-4664-98fd-639e0125e476 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:21.348732 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9301a65-863f-45de-8463-50ee4014ce96 None None] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:21.349045 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9301a65-863f-45de-8463-50ee4014ce96 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:21.349717 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9301a65-863f-45de-8463-50ee4014ce96 None None] PATH_INFO: `/` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:21.351345 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 512/1068] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:39:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:39:21.352151 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40f3790-20b5-4664-98fd-639e0125e476 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:21.364674 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21905975-d389-4880-b82e-e8b9b47fecb7 None None] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:21.365770 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21905975-d389-4880-b82e-e8b9b47fecb7 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:21.365770 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21905975-d389-4880-b82e-e8b9b47fecb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:21.381760 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:21.383889 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.auth.core [None req-21905975-d389-4880-b82e-e8b9b47fecb7 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:39:21.385096 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-21905975-d389-4880-b82e-e8b9b47fecb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:39:21.397129 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07a45da-680c-4a0b-943f-7f5e528a1c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:21.397981 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07a45da-680c-4a0b-943f-7f5e528a1c6b None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:21.398269 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07a45da-680c-4a0b-943f-7f5e528a1c6b None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:21.398542 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07a45da-680c-4a0b-943f-7f5e528a1c6b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:21.411559 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07a45da-680c-4a0b-943f-7f5e528a1c6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:21.413287 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07a45da-680c-4a0b-943f-7f5e528a1c6b None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:21.431476 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-b07a45da-680c-4a0b-943f-7f5e528a1c6b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:39:21.431476 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:39:21.431476 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:39:21.431476 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:39:21.431476 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:39:21.431476 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:39:21.431476 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:39:21.431476 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:39:21.431476 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:39:21.431476 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:39:21.431476 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:39:21.431476 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:39:21.431476 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:39:21.431476 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:39:21.431476 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:39:21.432692 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:39:21.432692 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:39:21.432692 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:39:21.432692 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:39:21.432692 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:39:21.432692 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:39:21.432692 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:39:21.432692 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:39:21.432692 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:39:21.432692 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:39:21.432692 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:39:21.432692 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:39:21.432692 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:39:21.432692 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:39:21.432692 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:39:21.432692 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:39:21.432692 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:39:21.433725 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:39:21.433725 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:39:21.433725 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:39:21.433725 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:39:21.433725 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 557/1069] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:39:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:39:21.438535 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07a45da-680c-4a0b-943f-7f5e528a1c6b None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:21.464440 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 513/1070] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:39:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:39:21.487148 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21905975-d389-4880-b82e-e8b9b47fecb7 None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:21.496624 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:21.550644 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07a45da-680c-4a0b-943f-7f5e528a1c6b None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:21.586392 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa639b1d-ee24-4de4-b90d-08b2a974ba69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:21.587151 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa639b1d-ee24-4de4-b90d-08b2a974ba69 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:21.587429 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa639b1d-ee24-4de4-b90d-08b2a974ba69 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:21.587698 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa639b1d-ee24-4de4-b90d-08b2a974ba69 None admin] PATH_INFO: `/v3/domains` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:21.588408 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa639b1d-ee24-4de4-b90d-08b2a974ba69 None admin] RBAC: Adding query filter params (name=Default) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:39:21.588714 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa639b1d-ee24-4de4-b90d-08b2a974ba69 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:21.590098 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa639b1d-ee24-4de4-b90d-08b2a974ba69 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:21.592553 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21905975-d389-4880-b82e-e8b9b47fecb7 None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:21.602478 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 558/1071] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:39:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:39:21.603746 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73ed5ab-51b8-444c-85c7-af082d81086c None None] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:21.604304 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73ed5ab-51b8-444c-85c7-af082d81086c None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:21.604304 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73ed5ab-51b8-444c-85c7-af082d81086c None None] PATH_INFO: `/` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:21.605653 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 559/1072] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:39:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:39:21.608633 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73ed5ab-51b8-444c-85c7-af082d81086c None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:21.629495 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6ef78c-506c-44be-a52c-6a286acfc5e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:21.630336 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6ef78c-506c-44be-a52c-6a286acfc5e6 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:21.630703 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6ef78c-506c-44be-a52c-6a286acfc5e6 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:21.631027 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6ef78c-506c-44be-a52c-6a286acfc5e6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:21.641445 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6ef78c-506c-44be-a52c-6a286acfc5e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:39:21.643831 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:21.649372 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-2c6ef78c-506c-44be-a52c-6a286acfc5e6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:39:21.649372 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:39:21.649372 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:39:21.649372 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:39:21.649372 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:39:21.649372 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:39:21.649372 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:39:21.649372 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:39:21.649372 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:39:21.649372 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:39:21.649372 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:39:21.649372 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:39:21.649372 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:39:21.649372 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:39:21.649372 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:39:21.649372 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:39:21.653911 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:39:21.653911 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:39:21.653911 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:39:21.653911 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:39:21.653911 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:39:21.653911 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:39:21.653911 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:39:21.653911 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:39:21.653911 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:39:21.653911 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:39:21.653911 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:39:21.653911 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:39:21.653911 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:39:21.653911 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:39:21.653911 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:39:21.653911 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:39:21.655203 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:39:21.655203 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:39:21.655203 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:39:21.655203 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:39:21.655203 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:39:21.655203 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:39:21.655203 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:39:21.655203 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:39:21.655203 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:39:21.655203 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:39:21.655203 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:39:21.655203 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:39:21.655203 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 514/1073] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:39:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:39:21.655203 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca030d88-5f0e-4290-afb2-a1621889d1b8 None None] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:21.656245 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca030d88-5f0e-4290-afb2-a1621889d1b8 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:21.656245 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca030d88-5f0e-4290-afb2-a1621889d1b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:21.671167 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.auth.core [None req-ca030d88-5f0e-4290-afb2-a1621889d1b8 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:39:21.672873 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-ca030d88-5f0e-4290-afb2-a1621889d1b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:39:21.720146 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73ed5ab-51b8-444c-85c7-af082d81086c None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:21.734442 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:21.742892 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 515/1074] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:39:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:39:21.743716 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca030d88-5f0e-4290-afb2-a1621889d1b8 None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:21.756963 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfec4a6-e13d-422d-869e-db153a30e3b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:21.757937 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfec4a6-e13d-422d-869e-db153a30e3b3 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:21.758329 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfec4a6-e13d-422d-869e-db153a30e3b3 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:21.758710 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfec4a6-e13d-422d-869e-db153a30e3b3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:21.766000 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfec4a6-e13d-422d-869e-db153a30e3b3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:21.767474 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfec4a6-e13d-422d-869e-db153a30e3b3 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:21.782100 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-3dfec4a6-e13d-422d-869e-db153a30e3b3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:39:21.782100 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:39:21.782100 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:39:21.782100 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:39:21.782100 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:39:21.782100 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:39:21.782100 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:39:21.782100 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:39:21.782100 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:39:21.782100 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:39:21.782100 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:39:21.782100 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:39:21.782100 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:39:21.782100 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:39:21.782100 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:39:21.783296 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:39:21.783296 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:39:21.783296 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:39:21.783296 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:39:21.783296 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:39:21.783296 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:39:21.783296 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:39:21.783296 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:39:21.783296 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:39:21.783296 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:39:21.783296 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:39:21.783296 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:39:21.783296 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:39:21.783296 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:39:21.783296 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:39:21.783296 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:39:21.783296 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:39:21.784531 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:39:21.784531 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:39:21.784531 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:39:21.784531 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:39:21.784531 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:39:21.784531 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:39:21.784531 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:39:21.784531 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:39:21.784531 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 560/1075] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:39:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:39:21.784531 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfec4a6-e13d-422d-869e-db153a30e3b3 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:21.847028 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca030d88-5f0e-4290-afb2-a1621889d1b8 None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:21.880853 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:21.882022 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26cfa64-1a2e-4998-a5ea-4867e6e4bed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:21.882829 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26cfa64-1a2e-4998-a5ea-4867e6e4bed6 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:21.883116 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26cfa64-1a2e-4998-a5ea-4867e6e4bed6 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:21.883491 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26cfa64-1a2e-4998-a5ea-4867e6e4bed6 None admin] PATH_INFO: `/v3/roles` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:21.884144 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26cfa64-1a2e-4998-a5ea-4867e6e4bed6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:39:21.884453 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26cfa64-1a2e-4998-a5ea-4867e6e4bed6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:21.885862 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26cfa64-1a2e-4998-a5ea-4867e6e4bed6 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:21.888392 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfec4a6-e13d-422d-869e-db153a30e3b3 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:21.894089 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 516/1076] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:39:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:39:21.894899 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26cfa64-1a2e-4998-a5ea-4867e6e4bed6 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:21.924929 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df85d91-ff25-4273-b41d-0e54127dcf86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:21.925586 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df85d91-ff25-4273-b41d-0e54127dcf86 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:21.925905 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df85d91-ff25-4273-b41d-0e54127dcf86 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:21.926193 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df85d91-ff25-4273-b41d-0e54127dcf86 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:21.958256 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:21.996383 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df85d91-ff25-4273-b41d-0e54127dcf86 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:21.997863 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df85d91-ff25-4273-b41d-0e54127dcf86 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:22.001930 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26cfa64-1a2e-4998-a5ea-4867e6e4bed6 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:22.036769 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3bc5cb-459f-4c37-a909-0a25d8f67dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:22.038659 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3bc5cb-459f-4c37-a909-0a25d8f67dcd None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:22.038944 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3bc5cb-459f-4c37-a909-0a25d8f67dcd None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:22.039215 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3bc5cb-459f-4c37-a909-0a25d8f67dcd None admin] PATH_INFO: `/v3/projects` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:22.040033 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3bc5cb-459f-4c37-a909-0a25d8f67dcd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:39:22.040339 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3bc5cb-459f-4c37-a909-0a25d8f67dcd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:22.041775 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3bc5cb-459f-4c37-a909-0a25d8f67dcd None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:22.054250 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 517/1077] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:39:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:39:22.055060 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3bc5cb-459f-4c37-a909-0a25d8f67dcd None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:22.092849 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:22.102499 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 561/1078] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:39:21 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:39:22.103126 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df85d91-ff25-4273-b41d-0e54127dcf86 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:22.163695 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3bc5cb-459f-4c37-a909-0a25d8f67dcd None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:22.182340 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:22.200480 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5797d67b-b577-4102-9aba-34a8d728a2fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:22.201358 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5797d67b-b577-4102-9aba-34a8d728a2fd None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:22.201645 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5797d67b-b577-4102-9aba-34a8d728a2fd None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:22.201957 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5797d67b-b577-4102-9aba-34a8d728a2fd None admin] PATH_INFO: `/v3/users/adc0083f4763459891e0473ecbbb070e` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:22.203188 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5797d67b-b577-4102-9aba-34a8d728a2fd None admin] RBAC: Authorizing `identity:get_user(user_id=adc0083f4763459891e0473ecbbb070e)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:22.204633 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5797d67b-b577-4102-9aba-34a8d728a2fd None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:22.206329 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 518/1079] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:39:22 2022] GET /identity/v3/users/adc0083f4763459891e0473ecbbb070e => generated 285 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:39:22.207102 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5797d67b-b577-4102-9aba-34a8d728a2fd None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:22.208529 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df85d91-ff25-4273-b41d-0e54127dcf86 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:22.245479 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11979228-2939-4eda-a70a-e95aa431190f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:22.246395 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11979228-2939-4eda-a70a-e95aa431190f None admin] REQUEST_METHOD: `PUT` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:22.246675 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11979228-2939-4eda-a70a-e95aa431190f None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:22.246944 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11979228-2939-4eda-a70a-e95aa431190f None admin] PATH_INFO: `/v3/projects/42f35b96b9d4499abfb5e5f7cca49adc/users/a9591727b2f54244baf90b11a8e2277f/roles/db02c08ef7c847c0ba25ed9ddc05974c` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:22.249364 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11979228-2939-4eda-a70a-e95aa431190f None admin] RBAC: Authorizing `identity:create_grant(project_id=42f35b96b9d4499abfb5e5f7cca49adc, user_id=a9591727b2f54244baf90b11a8e2277f, role_id=db02c08ef7c847c0ba25ed9ddc05974c)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:22.250786 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11979228-2939-4eda-a70a-e95aa431190f None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:22.264453 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 562/1080] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 14:39:22 2022] PUT /identity/v3/projects/42f35b96b9d4499abfb5e5f7cca49adc/users/a9591727b2f54244baf90b11a8e2277f/roles/db02c08ef7c847c0ba25ed9ddc05974c => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:39:22.265389 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11979228-2939-4eda-a70a-e95aa431190f None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:22.329757 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:22.348559 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5797d67b-b577-4102-9aba-34a8d728a2fd None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:22.387571 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7db34a1-fb43-416b-a7c9-5db50711558a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:22.388392 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7db34a1-fb43-416b-a7c9-5db50711558a None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:22.388682 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7db34a1-fb43-416b-a7c9-5db50711558a None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:22.388963 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7db34a1-fb43-416b-a7c9-5db50711558a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:22.390667 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11979228-2939-4eda-a70a-e95aa431190f None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:22.400844 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7db34a1-fb43-416b-a7c9-5db50711558a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:22.402373 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7db34a1-fb43-416b-a7c9-5db50711558a None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:22.425132 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-e7db34a1-fb43-416b-a7c9-5db50711558a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:39:22.425132 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:39:22.425132 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:39:22.425132 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:39:22.425132 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:39:22.425132 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:39:22.425132 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:39:22.425132 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:39:22.425132 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:39:22.425132 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:39:22.425132 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:39:22.425132 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:39:22.425132 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:39:22.425132 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:39:22.425132 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:39:22.426244 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:39:22.426244 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:39:22.426244 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:39:22.426244 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:39:22.426244 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:39:22.426244 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:39:22.426244 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:39:22.426244 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:39:22.426244 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:39:22.426244 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:39:22.426244 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:39:22.426244 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:39:22.426244 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:39:22.426244 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:39:22.426244 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:39:22.426244 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:39:22.426244 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:39:22.427444 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:39:22.427444 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:39:22.427444 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:39:22.427444 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:39:22.427444 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 519/1081] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:39:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:39:22.433277 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7db34a1-fb43-416b-a7c9-5db50711558a None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:22.436134 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec95b79d-85a6-41b7-902c-f16ce73a20f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:22.438228 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec95b79d-85a6-41b7-902c-f16ce73a20f3 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:22.438714 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec95b79d-85a6-41b7-902c-f16ce73a20f3 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:22.439164 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec95b79d-85a6-41b7-902c-f16ce73a20f3 None admin] PATH_INFO: `/v3/projects/b65078fee1a04ca1b954acd01fd0f717` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:22.440689 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec95b79d-85a6-41b7-902c-f16ce73a20f3 None admin] RBAC: Authorizing `identity:get_project(project_id=b65078fee1a04ca1b954acd01fd0f717)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:22.443113 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec95b79d-85a6-41b7-902c-f16ce73a20f3 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:22.445794 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 563/1082] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:39:22 2022] GET /identity/v3/projects/b65078fee1a04ca1b954acd01fd0f717 => generated 311 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:39:22.451738 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec95b79d-85a6-41b7-902c-f16ce73a20f3 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:22.480069 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:22.546352 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7db34a1-fb43-416b-a7c9-5db50711558a None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:22.563818 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec95b79d-85a6-41b7-902c-f16ce73a20f3 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:22.585094 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3156ab2b-9cb5-4579-ae27-a05ed03e74f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:22.585759 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3156ab2b-9cb5-4579-ae27-a05ed03e74f5 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:22.586075 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3156ab2b-9cb5-4579-ae27-a05ed03e74f5 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:22.586788 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3156ab2b-9cb5-4579-ae27-a05ed03e74f5 None admin] PATH_INFO: `/v3/domains` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:22.587539 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3156ab2b-9cb5-4579-ae27-a05ed03e74f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:39:22.587996 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3156ab2b-9cb5-4579-ae27-a05ed03e74f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:22.589281 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:22.589496 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3156ab2b-9cb5-4579-ae27-a05ed03e74f5 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:22.609235 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 520/1083] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:39:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:39:22.609456 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d2bba8-71d7-46b8-aa94-c28919a315f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:22.609456 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d2bba8-71d7-46b8-aa94-c28919a315f2 None admin] REQUEST_METHOD: `PUT` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:22.609456 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d2bba8-71d7-46b8-aa94-c28919a315f2 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:22.609456 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d2bba8-71d7-46b8-aa94-c28919a315f2 None admin] PATH_INFO: `/v3/projects/b65078fee1a04ca1b954acd01fd0f717/users/adc0083f4763459891e0473ecbbb070e/roles/c2935b1a25ca46fcba419fe3d1715330` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:22.609867 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d2bba8-71d7-46b8-aa94-c28919a315f2 None admin] RBAC: Authorizing `identity:create_grant(project_id=b65078fee1a04ca1b954acd01fd0f717, user_id=adc0083f4763459891e0473ecbbb070e, role_id=c2935b1a25ca46fcba419fe3d1715330)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:22.609867 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d2bba8-71d7-46b8-aa94-c28919a315f2 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:22.626317 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 564/1084] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 14:39:22 2022] PUT /identity/v3/projects/b65078fee1a04ca1b954acd01fd0f717/users/adc0083f4763459891e0473ecbbb070e/roles/c2935b1a25ca46fcba419fe3d1715330 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 14:39:24.521963 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687bf8f4-f67d-4da0-a74a-b8f4c8124cf7 None None] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:24.522565 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687bf8f4-f67d-4da0-a74a-b8f4c8124cf7 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:24.522686 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687bf8f4-f67d-4da0-a74a-b8f4c8124cf7 None None] PATH_INFO: `/` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:24.524199 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 521/1085] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:39:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:39:24.529384 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1018bb7-6275-4c9b-9deb-474b5d2048a3 None None] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:24.529818 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1018bb7-6275-4c9b-9deb-474b5d2048a3 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:24.530105 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1018bb7-6275-4c9b-9deb-474b5d2048a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:24.547399 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.auth.core [None req-c1018bb7-6275-4c9b-9deb-474b5d2048a3 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:39:24.548599 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.fernet_utils [None req-c1018bb7-6275-4c9b-9deb-474b5d2048a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:39:24.616063 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 565/1086] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:39:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:39:24.640125 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687bf8f4-f67d-4da0-a74a-b8f4c8124cf7 None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:24.742886 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:24.748352 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687bf8f4-f67d-4da0-a74a-b8f4c8124cf7 None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:24.782633 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ca3006-f2ca-4b52-9459-8189ba54fa48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:24.783356 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ca3006-f2ca-4b52-9459-8189ba54fa48 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:24.783645 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ca3006-f2ca-4b52-9459-8189ba54fa48 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:24.783920 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ca3006-f2ca-4b52-9459-8189ba54fa48 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:24.796558 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ca3006-f2ca-4b52-9459-8189ba54fa48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:39:24.805220 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-a5ca3006-f2ca-4b52-9459-8189ba54fa48 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 14:39:24.805220 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:39:24.805220 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:39:24.805220 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:39:24.805220 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:39:24.805220 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:39:24.805220 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:39:24.805220 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:39:24.805220 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:39:24.805220 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:39:24.805220 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:39:24.805220 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:39:24.805220 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:39:24.805220 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:39:24.805220 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:39:24.805220 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:39:24.806470 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:39:24.806470 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:39:24.806470 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:39:24.806470 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:39:24.806470 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:39:24.806470 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:39:24.806470 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:39:24.806470 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:39:24.806470 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:39:24.806470 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:39:24.806470 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:39:24.806470 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:39:24.806470 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:39:24.806470 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:39:24.806470 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:39:24.806470 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:39:24.807454 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:39:24.807454 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:39:24.807454 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:39:24.807454 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:39:24.807454 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:39:24.807454 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:39:24.807454 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:39:24.807454 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:39:24.807454 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:39:24.807454 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:39:24.807454 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 14:39:24.807454 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:39:24.807454 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 522/1087] 10.222.0.57 () {56 vars in 1120 bytes} [Thu May 19 14:39:24 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:39:24.812444 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1018bb7-6275-4c9b-9deb-474b5d2048a3 None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:24.888380 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8114976b-3e4e-4ac0-8b9f-6e4f08f0d8aa None None] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:24.888673 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8114976b-3e4e-4ac0-8b9f-6e4f08f0d8aa None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:24.888958 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8114976b-3e4e-4ac0-8b9f-6e4f08f0d8aa None None] PATH_INFO: `/` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:24.890557 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 523/1088] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:39:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:39:24.895439 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeef3a8-e553-4d45-a065-fc4f8ebc298f None None] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:24.895724 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeef3a8-e553-4d45-a065-fc4f8ebc298f None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:24.895997 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeef3a8-e553-4d45-a065-fc4f8ebc298f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:24.913031 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.auth.core [None req-8aeef3a8-e553-4d45-a065-fc4f8ebc298f None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:39:24.913590 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1018bb7-6275-4c9b-9deb-474b5d2048a3 None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:24.914254 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-8aeef3a8-e553-4d45-a065-fc4f8ebc298f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:39:24.926780 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:24.950416 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a10f7b-9385-471e-ad16-0ab18d18125d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:24.951100 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a10f7b-9385-471e-ad16-0ab18d18125d None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:24.951385 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a10f7b-9385-471e-ad16-0ab18d18125d None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:24.951663 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a10f7b-9385-471e-ad16-0ab18d18125d None admin] PATH_INFO: `/v3/roles` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:24.952412 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a10f7b-9385-471e-ad16-0ab18d18125d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:39:24.952721 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a10f7b-9385-471e-ad16-0ab18d18125d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:24.954143 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a10f7b-9385-471e-ad16-0ab18d18125d None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:24.962669 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 566/1089] 10.222.0.57 () {56 vars in 1115 bytes} [Thu May 19 14:39:24 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:39:24.968380 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a10f7b-9385-471e-ad16-0ab18d18125d None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:24.981434 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 524/1090] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:39:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:39:25.005194 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeef3a8-e553-4d45-a065-fc4f8ebc298f None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:25.069285 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a10f7b-9385-471e-ad16-0ab18d18125d None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:25.093092 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:25.103904 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594e71f8-5836-403f-8c02-c336c8b250fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:25.104778 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594e71f8-5836-403f-8c02-c336c8b250fe None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:25.105068 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594e71f8-5836-403f-8c02-c336c8b250fe None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:25.105342 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594e71f8-5836-403f-8c02-c336c8b250fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:25.105829 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeef3a8-e553-4d45-a065-fc4f8ebc298f None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:25.116671 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594e71f8-5836-403f-8c02-c336c8b250fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:25.118160 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594e71f8-5836-403f-8c02-c336c8b250fe None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:25.135891 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-594e71f8-5836-403f-8c02-c336c8b250fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:39:25.135891 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:39:25.135891 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:39:25.135891 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:39:25.135891 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:39:25.135891 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:39:25.135891 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:39:25.135891 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:39:25.135891 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:39:25.135891 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:39:25.135891 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:39:25.135891 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:39:25.135891 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:39:25.135891 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:39:25.135891 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:39:25.136957 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:39:25.136957 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:39:25.136957 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:39:25.136957 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:39:25.136957 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:39:25.136957 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:39:25.136957 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:39:25.136957 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:39:25.136957 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:39:25.136957 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:39:25.136957 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:39:25.136957 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:39:25.136957 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:39:25.136957 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:39:25.136957 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:39:25.136957 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:39:25.136957 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:39:25.138527 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:39:25.138527 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:39:25.138527 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:39:25.138527 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:39:25.138527 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 567/1091] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:39:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:39:25.142459 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71e51bd-e3c6-4930-8107-aeda0d107930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:25.142767 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594e71f8-5836-403f-8c02-c336c8b250fe None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:25.143318 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71e51bd-e3c6-4930-8107-aeda0d107930 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:25.143612 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71e51bd-e3c6-4930-8107-aeda0d107930 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:25.143893 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71e51bd-e3c6-4930-8107-aeda0d107930 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:25.154659 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71e51bd-e3c6-4930-8107-aeda0d107930 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:39:25.162654 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-e71e51bd-e3c6-4930-8107-aeda0d107930 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:39:25.162654 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:39:25.162654 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:39:25.162654 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:39:25.162654 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:39:25.162654 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:39:25.162654 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:39:25.162654 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:39:25.162654 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:39:25.162654 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:39:25.162654 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:39:25.162654 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:39:25.162654 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:39:25.162654 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:39:25.162654 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:39:25.162654 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:39:25.163676 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:39:25.163676 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:39:25.163676 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:39:25.163676 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:39:25.163676 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:39:25.163676 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:39:25.163676 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:39:25.163676 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:39:25.163676 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:39:25.163676 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:39:25.163676 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:39:25.163676 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:39:25.163676 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:39:25.163676 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:39:25.163676 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:39:25.163676 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:39:25.164635 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:39:25.164635 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:39:25.164635 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:39:25.164635 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:39:25.164635 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:39:25.164635 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:39:25.164635 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:39:25.164635 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:39:25.164635 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:39:25.164635 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:39:25.164635 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:39:25.164635 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:39:25.164635 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 525/1092] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:39:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:39:25.169752 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71e51bd-e3c6-4930-8107-aeda0d107930 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:25.175719 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:25.251713 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594e71f8-5836-403f-8c02-c336c8b250fe None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:25.274170 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71e51bd-e3c6-4930-8107-aeda0d107930 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:25.285830 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3982b0b7-61ae-4c36-b09d-de26dbf0c1dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:25.286520 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3982b0b7-61ae-4c36-b09d-de26dbf0c1dc None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:25.287287 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3982b0b7-61ae-4c36-b09d-de26dbf0c1dc None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:25.287567 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3982b0b7-61ae-4c36-b09d-de26dbf0c1dc None admin] PATH_INFO: `/v3/domains` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:25.288392 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3982b0b7-61ae-4c36-b09d-de26dbf0c1dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:39:25.288701 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3982b0b7-61ae-4c36-b09d-de26dbf0c1dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:25.290099 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3982b0b7-61ae-4c36-b09d-de26dbf0c1dc None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:25.301704 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 568/1093] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:39:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:39:25.307985 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3982b0b7-61ae-4c36-b09d-de26dbf0c1dc None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:25.309815 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3e2950-eedc-452b-b5c0-cee1c33cb91b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:25.310518 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3e2950-eedc-452b-b5c0-cee1c33cb91b None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:25.310805 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3e2950-eedc-452b-b5c0-cee1c33cb91b None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:25.311082 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3e2950-eedc-452b-b5c0-cee1c33cb91b None admin] PATH_INFO: `/v3/roles` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:25.311841 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3e2950-eedc-452b-b5c0-cee1c33cb91b None admin] RBAC: Adding query filter params (name=admin) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:39:25.312154 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3e2950-eedc-452b-b5c0-cee1c33cb91b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:25.313562 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3e2950-eedc-452b-b5c0-cee1c33cb91b None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:25.322008 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 526/1094] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:39:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:39:25.327702 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3e2950-eedc-452b-b5c0-cee1c33cb91b None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:25.340445 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:25.396231 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:25.412445 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3982b0b7-61ae-4c36-b09d-de26dbf0c1dc None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:25.430669 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3e2950-eedc-452b-b5c0-cee1c33cb91b None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:25.447675 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c52ed85-9003-4689-ba60-c7beef3b6fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:25.448508 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c52ed85-9003-4689-ba60-c7beef3b6fe6 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:25.448781 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c52ed85-9003-4689-ba60-c7beef3b6fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:25.449059 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c52ed85-9003-4689-ba60-c7beef3b6fe6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:25.456208 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c52ed85-9003-4689-ba60-c7beef3b6fe6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:25.457662 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c52ed85-9003-4689-ba60-c7beef3b6fe6 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:25.464684 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c0d36c-8310-4a62-badb-b2584f0da0ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:25.465552 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c0d36c-8310-4a62-badb-b2584f0da0ff None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:25.465834 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c0d36c-8310-4a62-badb-b2584f0da0ff None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:25.466146 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c0d36c-8310-4a62-badb-b2584f0da0ff None admin] PATH_INFO: `/v3/users/adc0083f4763459891e0473ecbbb070e` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:25.467350 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c0d36c-8310-4a62-badb-b2584f0da0ff None admin] RBAC: Authorizing `identity:get_user(user_id=adc0083f4763459891e0473ecbbb070e)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:25.468776 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c0d36c-8310-4a62-badb-b2584f0da0ff None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:25.470447 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 527/1095] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:39:25 2022] GET /identity/v3/users/adc0083f4763459891e0473ecbbb070e => generated 285 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:39:25.472528 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-5c52ed85-9003-4689-ba60-c7beef3b6fe6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 14:39:25.472528 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:39:25.472528 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:39:25.472528 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:39:25.472528 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:39:25.472528 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:39:25.472528 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:39:25.472528 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:39:25.472528 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:39:25.472528 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:39:25.472528 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:39:25.472528 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:39:25.472528 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:39:25.472528 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:39:25.472528 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:39:25.472528 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:39:25.473799 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:39:25.473799 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:39:25.473799 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:39:25.473799 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:39:25.473799 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:39:25.473799 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:39:25.473799 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:39:25.473799 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:39:25.473799 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:39:25.473799 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:39:25.473799 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:39:25.473799 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:39:25.473799 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:39:25.473799 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:39:25.473799 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:39:25.473799 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:39:25.473799 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:39:25.473799 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:39:25.476326 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:39:25.476326 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:39:25.476326 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:39:25.476326 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:39:25.476326 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:39:25.476326 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:39:25.476326 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:39:25.476326 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:39:25.476326 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 14:39:25.476326 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:39:25.476326 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 569/1096] 10.222.0.57 () {56 vars in 1093 bytes} [Thu May 19 14:39:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:39:25.477220 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c0d36c-8310-4a62-badb-b2584f0da0ff None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:25.479865 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c52ed85-9003-4689-ba60-c7beef3b6fe6 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:25.556529 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:25.578707 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c0d36c-8310-4a62-badb-b2584f0da0ff None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:25.581989 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c52ed85-9003-4689-ba60-c7beef3b6fe6 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:25.613476 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6911d4ff-8104-4a90-923e-3f1916b3efc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:25.614164 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6911d4ff-8104-4a90-923e-3f1916b3efc8 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:25.614455 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6911d4ff-8104-4a90-923e-3f1916b3efc8 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:25.614735 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6911d4ff-8104-4a90-923e-3f1916b3efc8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:25.620460 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da388cc-88f4-408d-9bab-e3dc4120ed01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:25.621130 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da388cc-88f4-408d-9bab-e3dc4120ed01 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:25.621412 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da388cc-88f4-408d-9bab-e3dc4120ed01 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:25.621687 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da388cc-88f4-408d-9bab-e3dc4120ed01 None admin] PATH_INFO: `/v3/users` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:25.622559 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da388cc-88f4-408d-9bab-e3dc4120ed01 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:39:25.622870 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da388cc-88f4-408d-9bab-e3dc4120ed01 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:25.624305 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da388cc-88f4-408d-9bab-e3dc4120ed01 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:25.641597 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 570/1097] 10.222.0.57 () {56 vars in 1133 bytes} [Thu May 19 14:39:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:39:25.647978 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da388cc-88f4-408d-9bab-e3dc4120ed01 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:25.651530 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:25.683408 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6911d4ff-8104-4a90-923e-3f1916b3efc8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:25.684859 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6911d4ff-8104-4a90-923e-3f1916b3efc8 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:25.752450 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da388cc-88f4-408d-9bab-e3dc4120ed01 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:25.762989 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:25.785575 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 528/1098] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:39:25 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 19 14:39:25.786482 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2c9c95-426a-4d17-8fe2-cb582b8b6645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:25.787897 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2c9c95-426a-4d17-8fe2-cb582b8b6645 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:25.788263 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2c9c95-426a-4d17-8fe2-cb582b8b6645 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:25.788541 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2c9c95-426a-4d17-8fe2-cb582b8b6645 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:25.792073 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6911d4ff-8104-4a90-923e-3f1916b3efc8 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:25.857684 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2c9c95-426a-4d17-8fe2-cb582b8b6645 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:25.859121 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2c9c95-426a-4d17-8fe2-cb582b8b6645 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:25.861859 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:25.893055 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6911d4ff-8104-4a90-923e-3f1916b3efc8 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:25.926556 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c698d42-7eab-4e29-a2d9-9156a907626b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:25.927914 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c698d42-7eab-4e29-a2d9-9156a907626b None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:25.928210 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c698d42-7eab-4e29-a2d9-9156a907626b None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:25.928565 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c698d42-7eab-4e29-a2d9-9156a907626b None admin] PATH_INFO: `/v3/projects/b65078fee1a04ca1b954acd01fd0f717` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:25.929865 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c698d42-7eab-4e29-a2d9-9156a907626b None admin] RBAC: Authorizing `identity:get_project(project_id=b65078fee1a04ca1b954acd01fd0f717)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:25.931565 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c698d42-7eab-4e29-a2d9-9156a907626b None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:25.934287 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 529/1099] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:39:25 2022] GET /identity/v3/projects/b65078fee1a04ca1b954acd01fd0f717 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:39:25.940131 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c698d42-7eab-4e29-a2d9-9156a907626b None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:25.964135 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 571/1100] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:39:25 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 19 14:39:25.969697 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2c9c95-426a-4d17-8fe2-cb582b8b6645 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:25.989353 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:26.042870 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c698d42-7eab-4e29-a2d9-9156a907626b None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:26.069107 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2c9c95-426a-4d17-8fe2-cb582b8b6645 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:26.071561 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:26.076977 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afff1d94-8c21-4607-84c2-12df520a7fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:26.077626 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afff1d94-8c21-4607-84c2-12df520a7fae None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:26.077941 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afff1d94-8c21-4607-84c2-12df520a7fae None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:26.078222 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afff1d94-8c21-4607-84c2-12df520a7fae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:26.079085 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afff1d94-8c21-4607-84c2-12df520a7fae None admin] RBAC: Adding query filter params (role.id=c2935b1a25ca46fcba419fe3d1715330, scope.project.id=b65078fee1a04ca1b954acd01fd0f717, user.id=adc0083f4763459891e0473ecbbb070e) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:39:26.079397 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afff1d94-8c21-4607-84c2-12df520a7fae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:26.080832 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afff1d94-8c21-4607-84c2-12df520a7fae None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:26.087732 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 530/1101] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:39:25 2022] GET /identity/v3/role_assignments?user.id=adc0083f4763459891e0473ecbbb070e&scope.project.id=b65078fee1a04ca1b954acd01fd0f717&role.id=c2935b1a25ca46fcba419fe3d1715330 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:39:26.104517 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7239f1-1cb5-477d-8102-910cf8cdc7e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:26.105341 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7239f1-1cb5-477d-8102-910cf8cdc7e0 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:26.105625 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7239f1-1cb5-477d-8102-910cf8cdc7e0 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:26.105931 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7239f1-1cb5-477d-8102-910cf8cdc7e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:26.118319 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7239f1-1cb5-477d-8102-910cf8cdc7e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:26.119735 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7239f1-1cb5-477d-8102-910cf8cdc7e0 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:26.138830 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-aa7239f1-1cb5-477d-8102-910cf8cdc7e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:39:26.138830 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:39:26.138830 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:39:26.138830 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:39:26.138830 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:39:26.138830 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:39:26.138830 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:39:26.138830 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:39:26.138830 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:39:26.138830 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:39:26.138830 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:39:26.138830 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:39:26.138830 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:39:26.138830 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:39:26.138830 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:39:26.140113 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:39:26.140113 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:39:26.140113 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:39:26.140113 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:39:26.140113 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:39:26.140113 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:39:26.140113 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:39:26.140113 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:39:26.140113 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:39:26.140113 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:39:26.140113 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:39:26.140113 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:39:26.140113 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:39:26.140113 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:39:26.140113 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:39:26.140113 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:39:26.140113 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:39:26.141522 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:39:26.141522 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:39:26.141522 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:39:26.141522 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:39:26.141522 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 572/1102] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:39:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:39:26.145963 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afff1d94-8c21-4607-84c2-12df520a7fae None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:26.258227 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afff1d94-8c21-4607-84c2-12df520a7fae None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:26.270198 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:26.299202 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9337f5-a57e-4783-8c8c-95f80b9d290d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:26.299867 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9337f5-a57e-4783-8c8c-95f80b9d290d None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:26.300152 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9337f5-a57e-4783-8c8c-95f80b9d290d None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:26.300426 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9337f5-a57e-4783-8c8c-95f80b9d290d None admin] PATH_INFO: `/v3/domains` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:26.301152 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9337f5-a57e-4783-8c8c-95f80b9d290d None admin] RBAC: Adding query filter params (name=Default) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:39:26.301462 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9337f5-a57e-4783-8c8c-95f80b9d290d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:26.303053 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9337f5-a57e-4783-8c8c-95f80b9d290d None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:26.314507 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 531/1103] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:39:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:39:26.320916 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7239f1-1cb5-477d-8102-910cf8cdc7e0 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:26.420765 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7239f1-1cb5-477d-8102-910cf8cdc7e0 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:26.455170 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cde90b-b15e-4299-a246-e69629214ae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:26.455961 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cde90b-b15e-4299-a246-e69629214ae7 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:26.456240 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cde90b-b15e-4299-a246-e69629214ae7 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:26.456512 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cde90b-b15e-4299-a246-e69629214ae7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:26.463485 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cde90b-b15e-4299-a246-e69629214ae7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:26.464932 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cde90b-b15e-4299-a246-e69629214ae7 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:26.478617 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-45cde90b-b15e-4299-a246-e69629214ae7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:39:26.478617 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:39:26.478617 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:39:26.478617 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:39:26.478617 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:39:26.478617 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:39:26.478617 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:39:26.478617 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:39:26.478617 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:39:26.478617 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:39:26.478617 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:39:26.478617 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:39:26.478617 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:39:26.478617 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:39:26.478617 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:39:26.479812 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:39:26.479812 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:39:26.479812 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:39:26.479812 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:39:26.479812 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:39:26.479812 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:39:26.479812 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:39:26.479812 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:39:26.479812 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:39:26.479812 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:39:26.479812 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:39:26.479812 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:39:26.479812 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:39:26.479812 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:39:26.479812 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:39:26.479812 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:39:26.479812 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:39:26.481039 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:39:26.481039 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:39:26.481039 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:39:26.481039 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:39:26.481039 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:39:26.481039 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:39:26.481039 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:39:26.481039 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:39:26.482011 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 573/1104] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:39:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:39:26.485552 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9337f5-a57e-4783-8c8c-95f80b9d290d None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:26.586459 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9337f5-a57e-4783-8c8c-95f80b9d290d None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:26.620219 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e95dd72-d8b4-4bd4-879e-e9d5050115b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:26.620878 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e95dd72-d8b4-4bd4-879e-e9d5050115b6 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:26.621165 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e95dd72-d8b4-4bd4-879e-e9d5050115b6 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:26.621445 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e95dd72-d8b4-4bd4-879e-e9d5050115b6 None admin] PATH_INFO: `/v3/projects` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:26.622272 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e95dd72-d8b4-4bd4-879e-e9d5050115b6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:39:26.622587 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e95dd72-d8b4-4bd4-879e-e9d5050115b6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:26.624027 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e95dd72-d8b4-4bd4-879e-e9d5050115b6 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:26.637120 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 532/1105] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:39:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:39:26.643944 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cde90b-b15e-4299-a246-e69629214ae7 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:26.663500 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:26.744980 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cde90b-b15e-4299-a246-e69629214ae7 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:26.778199 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4c98fb-4c44-40bb-a3a2-08b97bc4cb36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:26.778844 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4c98fb-4c44-40bb-a3a2-08b97bc4cb36 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:26.779125 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4c98fb-4c44-40bb-a3a2-08b97bc4cb36 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:26.779396 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4c98fb-4c44-40bb-a3a2-08b97bc4cb36 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:26.780224 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4c98fb-4c44-40bb-a3a2-08b97bc4cb36 None admin] RBAC: Adding query filter params (role.id=db02c08ef7c847c0ba25ed9ddc05974c, scope.project.id=42f35b96b9d4499abfb5e5f7cca49adc, user.id=a9591727b2f54244baf90b11a8e2277f) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:39:26.780533 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4c98fb-4c44-40bb-a3a2-08b97bc4cb36 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:26.781981 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4c98fb-4c44-40bb-a3a2-08b97bc4cb36 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:26.788575 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 574/1106] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:39:26 2022] GET /identity/v3/role_assignments?user.id=a9591727b2f54244baf90b11a8e2277f&scope.project.id=42f35b96b9d4499abfb5e5f7cca49adc&role.id=db02c08ef7c847c0ba25ed9ddc05974c => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:39:28.386271 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7853dbfa-5496-4834-b47b-6435f901da6a None None] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:28.386862 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7853dbfa-5496-4834-b47b-6435f901da6a None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:28.386862 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7853dbfa-5496-4834-b47b-6435f901da6a None None] PATH_INFO: `/` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:28.388223 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 533/1107] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:39:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:39:28.393625 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefb7c33-8504-4f69-aab4-7dcb9f40a25e None None] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:28.394188 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefb7c33-8504-4f69-aab4-7dcb9f40a25e None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:28.394188 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefb7c33-8504-4f69-aab4-7dcb9f40a25e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:28.411350 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.auth.core [None req-eefb7c33-8504-4f69-aab4-7dcb9f40a25e None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:39:28.413047 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.fernet_utils [None req-eefb7c33-8504-4f69-aab4-7dcb9f40a25e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:39:28.479367 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 575/1108] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:39:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:39:28.504474 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7853dbfa-5496-4834-b47b-6435f901da6a None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:28.534730 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:28.607569 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7853dbfa-5496-4834-b47b-6435f901da6a None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:28.642521 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d2f4a9-1160-4b9e-a48a-a4a14b85348a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:28.643299 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d2f4a9-1160-4b9e-a48a-a4a14b85348a None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:28.643579 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d2f4a9-1160-4b9e-a48a-a4a14b85348a None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:28.643850 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d2f4a9-1160-4b9e-a48a-a4a14b85348a None admin] PATH_INFO: `/v3/domains/default` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:28.645007 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d2f4a9-1160-4b9e-a48a-a4a14b85348a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:28.647574 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d2f4a9-1160-4b9e-a48a-a4a14b85348a None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:28.649192 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 534/1109] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:39:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:39:28.654924 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefb7c33-8504-4f69-aab4-7dcb9f40a25e None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:28.667109 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:28.751490 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefb7c33-8504-4f69-aab4-7dcb9f40a25e None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:28.784633 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3857f1d2-06a0-40cc-a428-b7b1e8eea10e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:28.785296 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3857f1d2-06a0-40cc-a428-b7b1e8eea10e None admin] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:28.785576 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3857f1d2-06a0-40cc-a428-b7b1e8eea10e None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:28.785898 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3857f1d2-06a0-40cc-a428-b7b1e8eea10e None admin] PATH_INFO: `/v3/users` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:28.786594 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3857f1d2-06a0-40cc-a428-b7b1e8eea10e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:28.788807 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3857f1d2-06a0-40cc-a428-b7b1e8eea10e None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:28.808156 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 576/1110] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 19 14:39:28 2022] POST /identity/v3/users => generated 286 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:39:31.104207 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954be3bf-dc79-48ea-85b9-cd2268786314 None None] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:31.104806 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954be3bf-dc79-48ea-85b9-cd2268786314 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:31.104806 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954be3bf-dc79-48ea-85b9-cd2268786314 None None] PATH_INFO: `/` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:31.106195 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 535/1111] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:39:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:39:31.111445 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3ad7ea-72c8-49fd-9dcc-238f9a280442 None None] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:31.111860 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3ad7ea-72c8-49fd-9dcc-238f9a280442 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:31.111999 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3ad7ea-72c8-49fd-9dcc-238f9a280442 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:31.129548 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.auth.core [None req-3a3ad7ea-72c8-49fd-9dcc-238f9a280442 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:39:31.131912 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.fernet_utils [None req-3a3ad7ea-72c8-49fd-9dcc-238f9a280442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:39:31.191650 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:31.200871 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 577/1112] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:39:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:39:31.225476 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954be3bf-dc79-48ea-85b9-cd2268786314 None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:31.320288 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:31.325507 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954be3bf-dc79-48ea-85b9-cd2268786314 None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:31.359764 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df164021-db0a-4186-83b4-d13004077fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:31.360543 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df164021-db0a-4186-83b4-d13004077fa3 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:31.360829 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df164021-db0a-4186-83b4-d13004077fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:31.361111 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df164021-db0a-4186-83b4-d13004077fa3 None admin] PATH_INFO: `/v3/roles/b894e00b17e34f0cbde0ab3d760e2433` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:31.362627 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df164021-db0a-4186-83b4-d13004077fa3 None admin] RBAC: Authorizing `identity:get_role(role_id=b894e00b17e34f0cbde0ab3d760e2433)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:31.364003 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df164021-db0a-4186-83b4-d13004077fa3 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:31.366238 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 536/1113] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:39:31 2022] GET /identity/v3/roles/b894e00b17e34f0cbde0ab3d760e2433 => generated 223 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:39:31.372035 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3ad7ea-72c8-49fd-9dcc-238f9a280442 None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:31.469438 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3ad7ea-72c8-49fd-9dcc-238f9a280442 None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:31.503131 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eaf6b3-87b2-4c75-9baa-f503798ce2f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:31.503993 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eaf6b3-87b2-4c75-9baa-f503798ce2f4 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:31.504276 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eaf6b3-87b2-4c75-9baa-f503798ce2f4 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:31.504551 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eaf6b3-87b2-4c75-9baa-f503798ce2f4 None admin] PATH_INFO: `/v3/users/e59c3e65543843d8a69d6cddc49a9bf7` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:31.517325 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79eaf6b3-87b2-4c75-9baa-f503798ce2f4 None admin] RBAC: Authorizing `identity:get_user(user_id=e59c3e65543843d8a69d6cddc49a9bf7)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:31.518800 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79eaf6b3-87b2-4c75-9baa-f503798ce2f4 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:31.520451 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 578/1114] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:39:31 2022] GET /identity/v3/users/e59c3e65543843d8a69d6cddc49a9bf7 => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:39:31.526308 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df164021-db0a-4186-83b4-d13004077fa3 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:31.624480 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df164021-db0a-4186-83b4-d13004077fa3 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:31.657633 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2c51f1-c222-4fcb-97e6-1dbcd29cf5ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:31.658308 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2c51f1-c222-4fcb-97e6-1dbcd29cf5ac None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:31.658596 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2c51f1-c222-4fcb-97e6-1dbcd29cf5ac None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:31.658874 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2c51f1-c222-4fcb-97e6-1dbcd29cf5ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:31.692012 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:31.727262 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2c51f1-c222-4fcb-97e6-1dbcd29cf5ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:31.728713 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2c51f1-c222-4fcb-97e6-1dbcd29cf5ac None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:31.828467 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 537/1115] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:39:31 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:39:31.834651 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eaf6b3-87b2-4c75-9baa-f503798ce2f4 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:31.889688 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:31.933458 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eaf6b3-87b2-4c75-9baa-f503798ce2f4 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:31.968477 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56b67db-fd78-46bc-8f6f-dcf154b49842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:31.969330 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56b67db-fd78-46bc-8f6f-dcf154b49842 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:31.969612 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56b67db-fd78-46bc-8f6f-dcf154b49842 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:31.969915 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56b67db-fd78-46bc-8f6f-dcf154b49842 None admin] PATH_INFO: `/v3/projects/b65078fee1a04ca1b954acd01fd0f717` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:31.971103 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56b67db-fd78-46bc-8f6f-dcf154b49842 None admin] RBAC: Authorizing `identity:get_project(project_id=b65078fee1a04ca1b954acd01fd0f717)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:31.972541 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56b67db-fd78-46bc-8f6f-dcf154b49842 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:31.975086 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 579/1116] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:39:31 2022] GET /identity/v3/projects/b65078fee1a04ca1b954acd01fd0f717 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:39:31.980822 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2c51f1-c222-4fcb-97e6-1dbcd29cf5ac None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:32.045672 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:32.081962 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2c51f1-c222-4fcb-97e6-1dbcd29cf5ac None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:32.116180 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19eb4928-0377-4f22-b52c-ac1bdc5f3475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:32.116825 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eb4928-0377-4f22-b52c-ac1bdc5f3475 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:32.117108 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eb4928-0377-4f22-b52c-ac1bdc5f3475 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:32.117383 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eb4928-0377-4f22-b52c-ac1bdc5f3475 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:32.118241 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19eb4928-0377-4f22-b52c-ac1bdc5f3475 None admin] RBAC: Adding query filter params (role.id=b894e00b17e34f0cbde0ab3d760e2433, scope.project.id=b65078fee1a04ca1b954acd01fd0f717, user.id=e59c3e65543843d8a69d6cddc49a9bf7) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:39:32.118558 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19eb4928-0377-4f22-b52c-ac1bdc5f3475 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:32.119997 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19eb4928-0377-4f22-b52c-ac1bdc5f3475 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:32.125403 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 538/1117] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:39:31 2022] GET /identity/v3/role_assignments?user.id=e59c3e65543843d8a69d6cddc49a9bf7&scope.project.id=b65078fee1a04ca1b954acd01fd0f717&role.id=b894e00b17e34f0cbde0ab3d760e2433 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:39:34.346156 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d200b2a-285d-4662-93ca-1c57ab632cc2 None None] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:34.346699 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d200b2a-285d-4662-93ca-1c57ab632cc2 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:34.346699 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d200b2a-285d-4662-93ca-1c57ab632cc2 None None] PATH_INFO: `/` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:34.348065 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 580/1118] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:39:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:39:34.354182 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b81e67-c55d-410c-b3ab-05a9b99988d7 None None] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:34.354578 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b81e67-c55d-410c-b3ab-05a9b99988d7 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:34.354749 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b81e67-c55d-410c-b3ab-05a9b99988d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:34.372093 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.auth.core [None req-94b81e67-c55d-410c-b3ab-05a9b99988d7 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:39:34.373264 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-94b81e67-c55d-410c-b3ab-05a9b99988d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:39:34.439671 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 539/1119] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:39:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:39:34.462632 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d200b2a-285d-4662-93ca-1c57ab632cc2 None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:34.562166 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d200b2a-285d-4662-93ca-1c57ab632cc2 None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:34.584683 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:34.596300 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc6aa51-be5a-4a22-b0dc-14411bf29680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:34.597138 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc6aa51-be5a-4a22-b0dc-14411bf29680 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:34.597418 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc6aa51-be5a-4a22-b0dc-14411bf29680 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:34.597690 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc6aa51-be5a-4a22-b0dc-14411bf29680 None admin] PATH_INFO: `/v3/roles/b894e00b17e34f0cbde0ab3d760e2433` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:34.599741 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc6aa51-be5a-4a22-b0dc-14411bf29680 None admin] RBAC: Authorizing `identity:get_role(role_id=b894e00b17e34f0cbde0ab3d760e2433)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:34.601146 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc6aa51-be5a-4a22-b0dc-14411bf29680 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:34.602533 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 581/1120] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:39:34 2022] GET /identity/v3/roles/b894e00b17e34f0cbde0ab3d760e2433 => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:39:34.608454 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b81e67-c55d-410c-b3ab-05a9b99988d7 None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:34.667835 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:34.710150 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b81e67-c55d-410c-b3ab-05a9b99988d7 None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:34.744145 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d680e7-0b46-4f30-97ff-0cf3268536d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:34.745009 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d680e7-0b46-4f30-97ff-0cf3268536d7 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:34.745295 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d680e7-0b46-4f30-97ff-0cf3268536d7 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:34.745571 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d680e7-0b46-4f30-97ff-0cf3268536d7 None admin] PATH_INFO: `/v3/users/e59c3e65543843d8a69d6cddc49a9bf7` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:34.746788 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d680e7-0b46-4f30-97ff-0cf3268536d7 None admin] RBAC: Authorizing `identity:get_user(user_id=e59c3e65543843d8a69d6cddc49a9bf7)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:34.748226 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d680e7-0b46-4f30-97ff-0cf3268536d7 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:34.749890 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 540/1121] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:39:34 2022] GET /identity/v3/users/e59c3e65543843d8a69d6cddc49a9bf7 => generated 286 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:39:34.755769 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc6aa51-be5a-4a22-b0dc-14411bf29680 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:34.854744 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc6aa51-be5a-4a22-b0dc-14411bf29680 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:34.887408 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d86fa7-b85d-4c7c-9b67-9f56b502cbba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:34.888245 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d86fa7-b85d-4c7c-9b67-9f56b502cbba None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:34.888520 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d86fa7-b85d-4c7c-9b67-9f56b502cbba None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:34.888787 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d86fa7-b85d-4c7c-9b67-9f56b502cbba None admin] PATH_INFO: `/v3/projects/b65078fee1a04ca1b954acd01fd0f717` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:34.890223 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d86fa7-b85d-4c7c-9b67-9f56b502cbba None admin] RBAC: Authorizing `identity:get_project(project_id=b65078fee1a04ca1b954acd01fd0f717)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:34.891659 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d86fa7-b85d-4c7c-9b67-9f56b502cbba None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:34.893334 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 582/1122] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:39:34 2022] GET /identity/v3/projects/b65078fee1a04ca1b954acd01fd0f717 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:39:34.899911 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d680e7-0b46-4f30-97ff-0cf3268536d7 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:34.997665 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d680e7-0b46-4f30-97ff-0cf3268536d7 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:35.024431 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:35.032615 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272fbd62-54b4-42e0-9b4a-afaacf04a407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:35.033496 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272fbd62-54b4-42e0-9b4a-afaacf04a407 None admin] REQUEST_METHOD: `PUT` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:35.033778 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272fbd62-54b4-42e0-9b4a-afaacf04a407 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:35.034084 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272fbd62-54b4-42e0-9b4a-afaacf04a407 None admin] PATH_INFO: `/v3/projects/b65078fee1a04ca1b954acd01fd0f717/users/e59c3e65543843d8a69d6cddc49a9bf7/roles/b894e00b17e34f0cbde0ab3d760e2433` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:35.036321 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272fbd62-54b4-42e0-9b4a-afaacf04a407 None admin] RBAC: Authorizing `identity:create_grant(project_id=b65078fee1a04ca1b954acd01fd0f717, user_id=e59c3e65543843d8a69d6cddc49a9bf7, role_id=b894e00b17e34f0cbde0ab3d760e2433)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:35.038003 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272fbd62-54b4-42e0-9b4a-afaacf04a407 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:35.050634 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 541/1123] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 14:39:34 2022] PUT /identity/v3/projects/b65078fee1a04ca1b954acd01fd0f717/users/e59c3e65543843d8a69d6cddc49a9bf7/roles/b894e00b17e34f0cbde0ab3d760e2433 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:39:37.270438 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62ac52b-34cf-4419-865b-b7a9ef77dec3 None None] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:37.271695 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62ac52b-34cf-4419-865b-b7a9ef77dec3 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:37.271695 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62ac52b-34cf-4419-865b-b7a9ef77dec3 None None] PATH_INFO: `/` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:37.273263 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 583/1124] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:39:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:39:37.278648 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dd05d0-78cc-4192-9cb7-47a59ea685cc None None] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:37.279045 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dd05d0-78cc-4192-9cb7-47a59ea685cc None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:37.279200 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dd05d0-78cc-4192-9cb7-47a59ea685cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:37.295671 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.auth.core [None req-72dd05d0-78cc-4192-9cb7-47a59ea685cc None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:39:37.296850 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-72dd05d0-78cc-4192-9cb7-47a59ea685cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:39:37.362608 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 542/1125] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:39:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:39:37.386513 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62ac52b-34cf-4419-865b-b7a9ef77dec3 None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:37.446631 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:37.488790 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62ac52b-34cf-4419-865b-b7a9ef77dec3 None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:37.522382 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dbb119-4420-4997-95d9-515498e322f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:37.523225 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dbb119-4420-4997-95d9-515498e322f0 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:37.523507 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dbb119-4420-4997-95d9-515498e322f0 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:37.523778 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dbb119-4420-4997-95d9-515498e322f0 None admin] PATH_INFO: `/v3/roles/b894e00b17e34f0cbde0ab3d760e2433` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:37.525235 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dbb119-4420-4997-95d9-515498e322f0 None admin] RBAC: Authorizing `identity:get_role(role_id=b894e00b17e34f0cbde0ab3d760e2433)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:37.526631 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dbb119-4420-4997-95d9-515498e322f0 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:37.528028 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 584/1126] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:39:37 2022] GET /identity/v3/roles/b894e00b17e34f0cbde0ab3d760e2433 => generated 223 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:39:37.533931 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dd05d0-78cc-4192-9cb7-47a59ea685cc None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:37.631955 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dd05d0-78cc-4192-9cb7-47a59ea685cc None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:37.648879 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:37.666402 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735e05e8-e9d0-4dd7-9fd9-2617269e2099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:37.667289 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735e05e8-e9d0-4dd7-9fd9-2617269e2099 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:37.667575 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735e05e8-e9d0-4dd7-9fd9-2617269e2099 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:37.667853 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735e05e8-e9d0-4dd7-9fd9-2617269e2099 None admin] PATH_INFO: `/v3/users/e59c3e65543843d8a69d6cddc49a9bf7` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:37.669064 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735e05e8-e9d0-4dd7-9fd9-2617269e2099 None admin] RBAC: Authorizing `identity:get_user(user_id=e59c3e65543843d8a69d6cddc49a9bf7)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:37.670549 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735e05e8-e9d0-4dd7-9fd9-2617269e2099 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:37.672206 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 543/1127] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:39:37 2022] GET /identity/v3/users/e59c3e65543843d8a69d6cddc49a9bf7 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:39:37.678558 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dbb119-4420-4997-95d9-515498e322f0 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:37.777195 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dbb119-4420-4997-95d9-515498e322f0 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:37.803592 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:37.812606 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7a0fde-8005-4446-a41e-558104cad0c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:37.813258 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7a0fde-8005-4446-a41e-558104cad0c2 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:37.813546 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7a0fde-8005-4446-a41e-558104cad0c2 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:37.813823 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7a0fde-8005-4446-a41e-558104cad0c2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:37.881854 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7a0fde-8005-4446-a41e-558104cad0c2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:37.883332 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7a0fde-8005-4446-a41e-558104cad0c2 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:37.980076 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 585/1128] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:39:37 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:39:37.986108 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735e05e8-e9d0-4dd7-9fd9-2617269e2099 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:38.085323 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735e05e8-e9d0-4dd7-9fd9-2617269e2099 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:38.119366 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc1a6cf-3c30-468d-9cf9-0bd56721a66c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:38.120225 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc1a6cf-3c30-468d-9cf9-0bd56721a66c None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:38.120508 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc1a6cf-3c30-468d-9cf9-0bd56721a66c None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:38.120782 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc1a6cf-3c30-468d-9cf9-0bd56721a66c None admin] PATH_INFO: `/v3/projects/b65078fee1a04ca1b954acd01fd0f717` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:38.121992 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc1a6cf-3c30-468d-9cf9-0bd56721a66c None admin] RBAC: Authorizing `identity:get_project(project_id=b65078fee1a04ca1b954acd01fd0f717)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:38.123439 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc1a6cf-3c30-468d-9cf9-0bd56721a66c None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:38.125157 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 544/1129] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:39:37 2022] GET /identity/v3/projects/b65078fee1a04ca1b954acd01fd0f717 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:39:38.131884 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7a0fde-8005-4446-a41e-558104cad0c2 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:38.158808 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:38.230967 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7a0fde-8005-4446-a41e-558104cad0c2 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:38.264334 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0c2603-fdf3-41da-8ed1-1fdf5bf475ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:38.264986 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0c2603-fdf3-41da-8ed1-1fdf5bf475ab None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:38.265268 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0c2603-fdf3-41da-8ed1-1fdf5bf475ab None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:38.265539 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0c2603-fdf3-41da-8ed1-1fdf5bf475ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:38.266393 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0c2603-fdf3-41da-8ed1-1fdf5bf475ab None admin] RBAC: Adding query filter params (role.id=b894e00b17e34f0cbde0ab3d760e2433, scope.project.id=b65078fee1a04ca1b954acd01fd0f717, user.id=e59c3e65543843d8a69d6cddc49a9bf7) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:39:38.266703 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0c2603-fdf3-41da-8ed1-1fdf5bf475ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:38.268128 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0c2603-fdf3-41da-8ed1-1fdf5bf475ab None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:38.274839 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 586/1130] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:39:38 2022] GET /identity/v3/role_assignments?user.id=e59c3e65543843d8a69d6cddc49a9bf7&scope.project.id=b65078fee1a04ca1b954acd01fd0f717&role.id=b894e00b17e34f0cbde0ab3d760e2433 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:39:40.513606 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e7d714-34a2-4117-886c-29a07b15b464 None None] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:40.514249 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e7d714-34a2-4117-886c-29a07b15b464 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:40.514249 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e7d714-34a2-4117-886c-29a07b15b464 None None] PATH_INFO: `/` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:40.515495 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 545/1131] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:39:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:39:40.520774 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f3570d-3b49-4bb6-80e8-46712bca3736 None None] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:40.521182 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f3570d-3b49-4bb6-80e8-46712bca3736 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:40.521891 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f3570d-3b49-4bb6-80e8-46712bca3736 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:40.538608 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.auth.core [None req-e1f3570d-3b49-4bb6-80e8-46712bca3736 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:39:40.539801 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.fernet_utils [None req-e1f3570d-3b49-4bb6-80e8-46712bca3736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:39:40.606419 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 587/1132] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:39:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:39:40.631338 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e7d714-34a2-4117-886c-29a07b15b464 None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:40.686604 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:40.731815 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e7d714-34a2-4117-886c-29a07b15b464 None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:40.766105 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c9d17f-9874-42a6-8889-39174675347f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:40.766830 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c9d17f-9874-42a6-8889-39174675347f None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:40.767114 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c9d17f-9874-42a6-8889-39174675347f None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:40.767390 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c9d17f-9874-42a6-8889-39174675347f None admin] PATH_INFO: `/v3/domains/default` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:40.768573 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c9d17f-9874-42a6-8889-39174675347f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:40.770061 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c9d17f-9874-42a6-8889-39174675347f None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:40.771715 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 546/1133] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:39:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:39:40.777468 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f3570d-3b49-4bb6-80e8-46712bca3736 None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:40.798729 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:40.876306 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f3570d-3b49-4bb6-80e8-46712bca3736 None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:40.910684 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99532de0-f9f8-465c-aaa6-c0b951c0cb3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:40.911345 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99532de0-f9f8-465c-aaa6-c0b951c0cb3a None admin] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:40.911731 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99532de0-f9f8-465c-aaa6-c0b951c0cb3a None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:40.912017 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99532de0-f9f8-465c-aaa6-c0b951c0cb3a None admin] PATH_INFO: `/v3/projects` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:40.912731 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99532de0-f9f8-465c-aaa6-c0b951c0cb3a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:40.914206 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99532de0-f9f8-465c-aaa6-c0b951c0cb3a None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:40.938705 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 588/1134] 10.222.0.57 () {60 vars in 1141 bytes} [Thu May 19 14:39:40 2022] POST /identity/v3/projects => generated 311 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:39:43.204481 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0be6bbb-7595-4926-a71a-a8db48ce8bf1 None None] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:43.205025 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0be6bbb-7595-4926-a71a-a8db48ce8bf1 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:43.205025 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0be6bbb-7595-4926-a71a-a8db48ce8bf1 None None] PATH_INFO: `/` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:43.206445 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 547/1135] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:39:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:39:43.213011 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985503fc-a28d-4866-a78a-61bbcde18a47 None None] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:43.213412 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985503fc-a28d-4866-a78a-61bbcde18a47 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:43.213579 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985503fc-a28d-4866-a78a-61bbcde18a47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:43.231152 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.auth.core [None req-985503fc-a28d-4866-a78a-61bbcde18a47 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:39:43.232343 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.fernet_utils [None req-985503fc-a28d-4866-a78a-61bbcde18a47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:39:43.298818 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 589/1136] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:39:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:39:43.325259 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0be6bbb-7595-4926-a71a-a8db48ce8bf1 None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:43.425451 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0be6bbb-7595-4926-a71a-a8db48ce8bf1 None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:43.448458 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:43.462336 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019513c5-92e8-4558-9024-97899d0d47b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:43.463123 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019513c5-92e8-4558-9024-97899d0d47b5 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:43.463404 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019513c5-92e8-4558-9024-97899d0d47b5 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:43.463679 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019513c5-92e8-4558-9024-97899d0d47b5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:43.464857 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019513c5-92e8-4558-9024-97899d0d47b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:43.466343 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019513c5-92e8-4558-9024-97899d0d47b5 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:43.467977 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 548/1137] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:39:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:39:43.473745 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985503fc-a28d-4866-a78a-61bbcde18a47 None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:43.476667 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:43.573907 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985503fc-a28d-4866-a78a-61bbcde18a47 None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:43.608142 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f2cad8-3ddb-43ee-bc2b-0e35781a3d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:43.608808 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f2cad8-3ddb-43ee-bc2b-0e35781a3d65 None admin] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:43.609095 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f2cad8-3ddb-43ee-bc2b-0e35781a3d65 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:43.609375 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f2cad8-3ddb-43ee-bc2b-0e35781a3d65 None admin] PATH_INFO: `/v3/users` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:43.610110 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f2cad8-3ddb-43ee-bc2b-0e35781a3d65 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:43.611554 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f2cad8-3ddb-43ee-bc2b-0e35781a3d65 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:43.631664 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 590/1138] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 19 14:39:43 2022] POST /identity/v3/users => generated 286 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:39:45.898306 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f8a91d-f28e-451d-92dc-80559c9afc70 None None] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:45.898861 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f8a91d-f28e-451d-92dc-80559c9afc70 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:45.898861 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f8a91d-f28e-451d-92dc-80559c9afc70 None None] PATH_INFO: `/` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:45.900222 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 549/1139] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:39:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:39:45.905424 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f03ad1-15f3-4e40-aa19-3fc376f90f9d None None] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:45.905871 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f03ad1-15f3-4e40-aa19-3fc376f90f9d None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:45.906022 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f03ad1-15f3-4e40-aa19-3fc376f90f9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:45.923622 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.auth.core [None req-64f03ad1-15f3-4e40-aa19-3fc376f90f9d None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:39:45.924822 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.fernet_utils [None req-64f03ad1-15f3-4e40-aa19-3fc376f90f9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:39:45.977957 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:45.992498 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 591/1140] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:39:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:39:46.016751 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f8a91d-f28e-451d-92dc-80559c9afc70 None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:46.117505 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f8a91d-f28e-451d-92dc-80559c9afc70 None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:46.150543 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17c4cc8-62c9-4790-9bb9-619c645fd409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:46.151249 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17c4cc8-62c9-4790-9bb9-619c645fd409 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:46.151527 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17c4cc8-62c9-4790-9bb9-619c645fd409 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:46.151793 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17c4cc8-62c9-4790-9bb9-619c645fd409 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:46.162680 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17c4cc8-62c9-4790-9bb9-619c645fd409 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:39:46.170306 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-e17c4cc8-62c9-4790-9bb9-619c645fd409 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:39:46.170306 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:39:46.170306 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:39:46.170306 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:39:46.170306 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:39:46.170306 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:39:46.170306 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:39:46.170306 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:39:46.170306 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:39:46.170306 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:39:46.170306 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:39:46.170306 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:39:46.170306 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:39:46.170306 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:39:46.170306 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:39:46.170306 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:39:46.171509 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:39:46.171509 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:39:46.171509 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:39:46.171509 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:39:46.171509 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:39:46.171509 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:39:46.171509 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:39:46.171509 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:39:46.171509 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:39:46.171509 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:39:46.171509 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:39:46.171509 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:39:46.171509 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:39:46.171509 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:39:46.171509 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:39:46.171509 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:39:46.172479 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:39:46.172479 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:39:46.172479 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:39:46.172479 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:39:46.172479 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:39:46.172479 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:39:46.172479 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:39:46.172479 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:39:46.172479 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:39:46.172479 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:39:46.172479 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:39:46.172479 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:39:46.172479 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 550/1141] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:39:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:39:46.177541 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f03ad1-15f3-4e40-aa19-3fc376f90f9d None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:46.278623 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f03ad1-15f3-4e40-aa19-3fc376f90f9d None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:46.311355 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81f409c-fbeb-4c50-9808-c732878130c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:46.312021 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81f409c-fbeb-4c50-9808-c732878130c9 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:46.312304 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81f409c-fbeb-4c50-9808-c732878130c9 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:46.312584 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81f409c-fbeb-4c50-9808-c732878130c9 None admin] PATH_INFO: `/v3/roles` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:46.313321 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81f409c-fbeb-4c50-9808-c732878130c9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:39:46.313630 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81f409c-fbeb-4c50-9808-c732878130c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:46.315039 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81f409c-fbeb-4c50-9808-c732878130c9 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:46.323591 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 592/1142] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:39:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:39:46.329463 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17c4cc8-62c9-4790-9bb9-619c645fd409 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:46.379343 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:46.430053 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17c4cc8-62c9-4790-9bb9-619c645fd409 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:46.464164 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4acfcd-f0d1-4136-b25e-69347006acec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:46.465040 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4acfcd-f0d1-4136-b25e-69347006acec None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:46.465333 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4acfcd-f0d1-4136-b25e-69347006acec None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:46.465606 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4acfcd-f0d1-4136-b25e-69347006acec None admin] PATH_INFO: `/v3/users/0d473c45cf914b73be27945bdac7323c` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:46.479155 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4acfcd-f0d1-4136-b25e-69347006acec None admin] RBAC: Authorizing `identity:get_user(user_id=0d473c45cf914b73be27945bdac7323c)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:46.480601 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4acfcd-f0d1-4136-b25e-69347006acec None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:46.482337 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 551/1143] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:39:46 2022] GET /identity/v3/users/0d473c45cf914b73be27945bdac7323c => generated 286 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:39:46.488094 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81f409c-fbeb-4c50-9808-c732878130c9 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:46.538125 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:46.587889 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81f409c-fbeb-4c50-9808-c732878130c9 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:46.620734 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dd2a4d-b8a3-45c3-84cd-209477b81c0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:46.621376 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dd2a4d-b8a3-45c3-84cd-209477b81c0e None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:46.621660 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dd2a4d-b8a3-45c3-84cd-209477b81c0e None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:46.621966 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dd2a4d-b8a3-45c3-84cd-209477b81c0e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:46.691383 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dd2a4d-b8a3-45c3-84cd-209477b81c0e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:46.693342 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dd2a4d-b8a3-45c3-84cd-209477b81c0e None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:46.739374 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:46.789261 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 593/1144] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:39:46 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:39:46.795175 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4acfcd-f0d1-4136-b25e-69347006acec None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:46.893449 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4acfcd-f0d1-4136-b25e-69347006acec None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:46.902575 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:46.928853 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93929227-5787-4d9a-b203-b8146143c050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:46.929715 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93929227-5787-4d9a-b203-b8146143c050 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:46.930034 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93929227-5787-4d9a-b203-b8146143c050 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:46.930317 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93929227-5787-4d9a-b203-b8146143c050 None admin] PATH_INFO: `/v3/projects/0dc25a13ff504482b6a0594307fdcc2f` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:46.931504 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93929227-5787-4d9a-b203-b8146143c050 None admin] RBAC: Authorizing `identity:get_project(project_id=0dc25a13ff504482b6a0594307fdcc2f)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:46.932945 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93929227-5787-4d9a-b203-b8146143c050 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:46.934700 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 552/1145] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:39:46 2022] GET /identity/v3/projects/0dc25a13ff504482b6a0594307fdcc2f => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:39:46.940776 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dd2a4d-b8a3-45c3-84cd-209477b81c0e None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:47.040244 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dd2a4d-b8a3-45c3-84cd-209477b81c0e None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:47.073882 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58db4890-e963-4733-99f6-0a8c37747af6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:47.074530 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58db4890-e963-4733-99f6-0a8c37747af6 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:47.074808 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58db4890-e963-4733-99f6-0a8c37747af6 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:47.075080 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58db4890-e963-4733-99f6-0a8c37747af6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:47.075899 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58db4890-e963-4733-99f6-0a8c37747af6 None admin] RBAC: Adding query filter params (role.id=c2935b1a25ca46fcba419fe3d1715330, scope.project.id=0dc25a13ff504482b6a0594307fdcc2f, user.id=0d473c45cf914b73be27945bdac7323c) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:39:47.076207 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58db4890-e963-4733-99f6-0a8c37747af6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:47.077635 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58db4890-e963-4733-99f6-0a8c37747af6 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:47.082963 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 594/1146] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:39:46 2022] GET /identity/v3/role_assignments?user.id=0d473c45cf914b73be27945bdac7323c&scope.project.id=0dc25a13ff504482b6a0594307fdcc2f&role.id=c2935b1a25ca46fcba419fe3d1715330 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:39:49.282995 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbed53f-500a-45ee-b8c9-11208deca099 None None] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:49.283600 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbed53f-500a-45ee-b8c9-11208deca099 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:49.283600 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbed53f-500a-45ee-b8c9-11208deca099 None None] PATH_INFO: `/` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:49.284925 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 553/1147] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:39:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:39:49.290742 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c086d51f-d259-43bf-aeab-21b047c26d1b None None] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:49.291181 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c086d51f-d259-43bf-aeab-21b047c26d1b None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:49.291288 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c086d51f-d259-43bf-aeab-21b047c26d1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:49.308913 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.auth.core [None req-c086d51f-d259-43bf-aeab-21b047c26d1b None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:39:49.310143 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.fernet_utils [None req-c086d51f-d259-43bf-aeab-21b047c26d1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:39:49.319448 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:49.377918 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 595/1148] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:39:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:39:49.400833 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbed53f-500a-45ee-b8c9-11208deca099 None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:49.503324 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbed53f-500a-45ee-b8c9-11208deca099 None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:49.537473 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec4a2ee-9f3d-4da0-947d-0ff57bf8cf51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:49.538288 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec4a2ee-9f3d-4da0-947d-0ff57bf8cf51 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:49.538577 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec4a2ee-9f3d-4da0-947d-0ff57bf8cf51 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:49.538859 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec4a2ee-9f3d-4da0-947d-0ff57bf8cf51 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:49.548934 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec4a2ee-9f3d-4da0-947d-0ff57bf8cf51 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:39:49.557259 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-eec4a2ee-9f3d-4da0-947d-0ff57bf8cf51 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:39:49.557259 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:39:49.557259 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:39:49.557259 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:39:49.557259 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:39:49.557259 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:39:49.557259 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:39:49.557259 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:39:49.557259 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:39:49.557259 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:39:49.557259 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:39:49.557259 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:39:49.557259 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:39:49.557259 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:39:49.557259 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:39:49.557259 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:39:49.558575 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:39:49.558575 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:39:49.558575 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:39:49.558575 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:39:49.558575 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:39:49.558575 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:39:49.558575 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:39:49.558575 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:39:49.558575 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:39:49.558575 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:39:49.558575 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:39:49.558575 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:39:49.558575 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:39:49.558575 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:39:49.558575 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:39:49.558575 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:39:49.559787 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:39:49.559787 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:39:49.559787 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:39:49.559787 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:39:49.559787 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:39:49.559787 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:39:49.559787 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:39:49.559787 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:39:49.559787 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:39:49.559787 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:39:49.559787 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:39:49.559787 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:39:49.559787 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 554/1149] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:39:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:39:49.564556 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c086d51f-d259-43bf-aeab-21b047c26d1b None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:49.663612 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c086d51f-d259-43bf-aeab-21b047c26d1b None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:49.697478 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfed60f-a33f-4b2b-9fea-61b455dc0ad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:49.698188 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfed60f-a33f-4b2b-9fea-61b455dc0ad4 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:49.698476 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfed60f-a33f-4b2b-9fea-61b455dc0ad4 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:49.698753 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfed60f-a33f-4b2b-9fea-61b455dc0ad4 None admin] PATH_INFO: `/v3/roles` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:49.699498 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfed60f-a33f-4b2b-9fea-61b455dc0ad4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:39:49.699806 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfed60f-a33f-4b2b-9fea-61b455dc0ad4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:49.701177 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfed60f-a33f-4b2b-9fea-61b455dc0ad4 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:49.710129 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 596/1150] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:39:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:39:49.716080 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec4a2ee-9f3d-4da0-947d-0ff57bf8cf51 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:49.751456 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:49.816417 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec4a2ee-9f3d-4da0-947d-0ff57bf8cf51 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:49.849549 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50af1f32-dda2-48e5-92c2-3fd99f640f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:49.850435 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50af1f32-dda2-48e5-92c2-3fd99f640f78 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:49.850718 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50af1f32-dda2-48e5-92c2-3fd99f640f78 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:49.850991 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50af1f32-dda2-48e5-92c2-3fd99f640f78 None admin] PATH_INFO: `/v3/users/0d473c45cf914b73be27945bdac7323c` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:49.852169 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50af1f32-dda2-48e5-92c2-3fd99f640f78 None admin] RBAC: Authorizing `identity:get_user(user_id=0d473c45cf914b73be27945bdac7323c)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:49.853607 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50af1f32-dda2-48e5-92c2-3fd99f640f78 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:49.855251 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 555/1151] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:39:49 2022] GET /identity/v3/users/0d473c45cf914b73be27945bdac7323c => generated 286 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:39:49.861006 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfed60f-a33f-4b2b-9fea-61b455dc0ad4 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:49.870607 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:49.962277 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfed60f-a33f-4b2b-9fea-61b455dc0ad4 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:49.995926 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703bc492-a837-46a7-bfe1-92b73f8dedda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:49.996771 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703bc492-a837-46a7-bfe1-92b73f8dedda None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:49.997052 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703bc492-a837-46a7-bfe1-92b73f8dedda None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:49.997326 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703bc492-a837-46a7-bfe1-92b73f8dedda None admin] PATH_INFO: `/v3/projects/0dc25a13ff504482b6a0594307fdcc2f` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:49.998501 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703bc492-a837-46a7-bfe1-92b73f8dedda None admin] RBAC: Authorizing `identity:get_project(project_id=0dc25a13ff504482b6a0594307fdcc2f)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:49.999928 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703bc492-a837-46a7-bfe1-92b73f8dedda None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:50.001603 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 597/1152] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:39:49 2022] GET /identity/v3/projects/0dc25a13ff504482b6a0594307fdcc2f => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:39:50.008317 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50af1f32-dda2-48e5-92c2-3fd99f640f78 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:50.106698 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50af1f32-dda2-48e5-92c2-3fd99f640f78 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:50.108923 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:50.139874 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e224bfb7-9f41-448f-855e-33e078d0d911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:50.140754 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e224bfb7-9f41-448f-855e-33e078d0d911 None admin] REQUEST_METHOD: `PUT` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:50.141035 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e224bfb7-9f41-448f-855e-33e078d0d911 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:50.141310 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e224bfb7-9f41-448f-855e-33e078d0d911 None admin] PATH_INFO: `/v3/projects/0dc25a13ff504482b6a0594307fdcc2f/users/0d473c45cf914b73be27945bdac7323c/roles/c2935b1a25ca46fcba419fe3d1715330` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:50.143297 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e224bfb7-9f41-448f-855e-33e078d0d911 None admin] RBAC: Authorizing `identity:create_grant(project_id=0dc25a13ff504482b6a0594307fdcc2f, user_id=0d473c45cf914b73be27945bdac7323c, role_id=c2935b1a25ca46fcba419fe3d1715330)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:50.144733 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e224bfb7-9f41-448f-855e-33e078d0d911 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:50.157399 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 556/1153] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 14:39:50 2022] PUT /identity/v3/projects/0dc25a13ff504482b6a0594307fdcc2f/users/0d473c45cf914b73be27945bdac7323c/roles/c2935b1a25ca46fcba419fe3d1715330 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 14:39:52.355008 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e43d49-2f08-45b4-bdbd-3ad378ba55f5 None None] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:52.355571 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e43d49-2f08-45b4-bdbd-3ad378ba55f5 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:52.355571 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e43d49-2f08-45b4-bdbd-3ad378ba55f5 None None] PATH_INFO: `/` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:52.356952 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 598/1154] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:39:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:39:52.362407 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280565b2-4fa4-4faa-9e82-515df3de37cf None None] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:52.362813 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280565b2-4fa4-4faa-9e82-515df3de37cf None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:52.362965 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280565b2-4fa4-4faa-9e82-515df3de37cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:52.380469 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.auth.core [None req-280565b2-4fa4-4faa-9e82-515df3de37cf None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:39:52.381588 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-280565b2-4fa4-4faa-9e82-515df3de37cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:39:52.447010 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 557/1155] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:39:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:39:52.470574 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e43d49-2f08-45b4-bdbd-3ad378ba55f5 None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:52.547850 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:52.573788 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e43d49-2f08-45b4-bdbd-3ad378ba55f5 None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:52.607107 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285293b9-57f8-4f33-9175-701fbee9568d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:52.607896 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285293b9-57f8-4f33-9175-701fbee9568d None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:52.608181 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285293b9-57f8-4f33-9175-701fbee9568d None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:52.608458 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285293b9-57f8-4f33-9175-701fbee9568d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:52.619509 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285293b9-57f8-4f33-9175-701fbee9568d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:39:52.627135 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-285293b9-57f8-4f33-9175-701fbee9568d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:39:52.627135 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:39:52.627135 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:39:52.627135 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:39:52.627135 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:39:52.627135 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:39:52.627135 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:39:52.627135 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:39:52.627135 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:39:52.627135 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:39:52.627135 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:39:52.627135 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:39:52.627135 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:39:52.627135 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:39:52.627135 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:39:52.627135 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:39:52.628343 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:39:52.628343 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:39:52.628343 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:39:52.628343 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:39:52.628343 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:39:52.628343 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:39:52.628343 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:39:52.628343 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:39:52.628343 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:39:52.628343 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:39:52.628343 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:39:52.628343 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:39:52.628343 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:39:52.628343 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:39:52.628343 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:39:52.628343 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:39:52.629334 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:39:52.629334 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:39:52.629334 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:39:52.629334 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:39:52.629334 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:39:52.629334 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:39:52.629334 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:39:52.629334 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:39:52.629334 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:39:52.629334 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:39:52.629334 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:39:52.629334 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:39:52.629334 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 599/1156] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:39:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:39:52.634540 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280565b2-4fa4-4faa-9e82-515df3de37cf None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:52.726605 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:52.734124 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280565b2-4fa4-4faa-9e82-515df3de37cf None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:52.768152 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130be6f2-e91a-4af8-b25d-1267c936d99c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:52.768813 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130be6f2-e91a-4af8-b25d-1267c936d99c None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:52.769100 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130be6f2-e91a-4af8-b25d-1267c936d99c None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:52.769376 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130be6f2-e91a-4af8-b25d-1267c936d99c None admin] PATH_INFO: `/v3/roles` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:52.770154 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130be6f2-e91a-4af8-b25d-1267c936d99c None admin] RBAC: Adding query filter params (name=admin) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:39:52.770465 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130be6f2-e91a-4af8-b25d-1267c936d99c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:52.772025 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130be6f2-e91a-4af8-b25d-1267c936d99c None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:52.779936 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 558/1157] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:39:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:39:52.785709 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285293b9-57f8-4f33-9175-701fbee9568d None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:52.884647 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285293b9-57f8-4f33-9175-701fbee9568d None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:52.917237 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9176bc86-4e48-49f4-a6d2-cd0d9dd1da21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:52.918128 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9176bc86-4e48-49f4-a6d2-cd0d9dd1da21 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:52.918412 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9176bc86-4e48-49f4-a6d2-cd0d9dd1da21 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:52.918687 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9176bc86-4e48-49f4-a6d2-cd0d9dd1da21 None admin] PATH_INFO: `/v3/users/0d473c45cf914b73be27945bdac7323c` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:52.919881 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9176bc86-4e48-49f4-a6d2-cd0d9dd1da21 None admin] RBAC: Authorizing `identity:get_user(user_id=0d473c45cf914b73be27945bdac7323c)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:52.921324 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9176bc86-4e48-49f4-a6d2-cd0d9dd1da21 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:52.923171 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 600/1158] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:39:52 2022] GET /identity/v3/users/0d473c45cf914b73be27945bdac7323c => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:39:52.929249 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130be6f2-e91a-4af8-b25d-1267c936d99c None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:53.027686 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130be6f2-e91a-4af8-b25d-1267c936d99c None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:53.060864 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2eef40-adf2-4999-ba39-6b32b5a051b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:53.061500 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2eef40-adf2-4999-ba39-6b32b5a051b9 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:53.061783 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2eef40-adf2-4999-ba39-6b32b5a051b9 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:53.062083 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2eef40-adf2-4999-ba39-6b32b5a051b9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:53.086122 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:53.128583 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2eef40-adf2-4999-ba39-6b32b5a051b9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:53.130127 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2eef40-adf2-4999-ba39-6b32b5a051b9 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:53.226913 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 559/1159] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:39:52 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:39:53.232791 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9176bc86-4e48-49f4-a6d2-cd0d9dd1da21 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:53.235690 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:53.332142 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9176bc86-4e48-49f4-a6d2-cd0d9dd1da21 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:53.366031 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed0d873-46e5-493e-bbb2-fa55dfe47347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:53.366885 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed0d873-46e5-493e-bbb2-fa55dfe47347 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:53.367168 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed0d873-46e5-493e-bbb2-fa55dfe47347 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:53.367445 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed0d873-46e5-493e-bbb2-fa55dfe47347 None admin] PATH_INFO: `/v3/projects/0dc25a13ff504482b6a0594307fdcc2f` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:53.368628 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed0d873-46e5-493e-bbb2-fa55dfe47347 None admin] RBAC: Authorizing `identity:get_project(project_id=0dc25a13ff504482b6a0594307fdcc2f)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:53.370097 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed0d873-46e5-493e-bbb2-fa55dfe47347 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:53.371799 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 601/1160] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:39:53 2022] GET /identity/v3/projects/0dc25a13ff504482b6a0594307fdcc2f => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:39:53.377962 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2eef40-adf2-4999-ba39-6b32b5a051b9 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:53.432777 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:53.476769 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2eef40-adf2-4999-ba39-6b32b5a051b9 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:53.509716 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d79e6e-b18d-4282-b070-3b7588229cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:53.510380 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d79e6e-b18d-4282-b070-3b7588229cdc None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:53.510661 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d79e6e-b18d-4282-b070-3b7588229cdc None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:53.511050 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d79e6e-b18d-4282-b070-3b7588229cdc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:53.517229 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d79e6e-b18d-4282-b070-3b7588229cdc None admin] RBAC: Adding query filter params (role.id=c2935b1a25ca46fcba419fe3d1715330, scope.project.id=0dc25a13ff504482b6a0594307fdcc2f, user.id=0d473c45cf914b73be27945bdac7323c) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:39:53.517555 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d79e6e-b18d-4282-b070-3b7588229cdc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:53.519152 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d79e6e-b18d-4282-b070-3b7588229cdc None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:53.525732 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 560/1161] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:39:53 2022] GET /identity/v3/role_assignments?user.id=0d473c45cf914b73be27945bdac7323c&scope.project.id=0dc25a13ff504482b6a0594307fdcc2f&role.id=c2935b1a25ca46fcba419fe3d1715330 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:39:55.778995 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c120fd4-84fe-4f84-a436-bd93d16d8c54 None None] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:55.779585 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c120fd4-84fe-4f84-a436-bd93d16d8c54 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:55.779585 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c120fd4-84fe-4f84-a436-bd93d16d8c54 None None] PATH_INFO: `/` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:55.780924 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 602/1162] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:39:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:39:55.786238 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ffa7d4-b471-4583-8c5a-b2af866e47e0 None None] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:55.786833 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ffa7d4-b471-4583-8c5a-b2af866e47e0 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:55.786946 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ffa7d4-b471-4583-8c5a-b2af866e47e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:55.806116 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.auth.core [None req-05ffa7d4-b471-4583-8c5a-b2af866e47e0 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:39:55.807322 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-05ffa7d4-b471-4583-8c5a-b2af866e47e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:39:55.872980 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 561/1163] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:39:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:39:55.896277 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c120fd4-84fe-4f84-a436-bd93d16d8c54 None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:55.966995 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:55.999000 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c120fd4-84fe-4f84-a436-bd93d16d8c54 None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:56.033952 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b36fbd-2e05-4940-b7cc-71827f90c506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:56.034540 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b36fbd-2e05-4940-b7cc-71827f90c506 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:56.034827 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b36fbd-2e05-4940-b7cc-71827f90c506 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:56.035108 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b36fbd-2e05-4940-b7cc-71827f90c506 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:56.109771 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b36fbd-2e05-4940-b7cc-71827f90c506 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:56.111273 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b36fbd-2e05-4940-b7cc-71827f90c506 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:56.179794 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:56.211428 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 603/1164] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:39:55 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:39:56.217580 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ffa7d4-b471-4583-8c5a-b2af866e47e0 None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:56.266425 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:56.315615 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ffa7d4-b471-4583-8c5a-b2af866e47e0 None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:56.349203 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1520bd-0925-4e4b-9a68-ad89ac26f083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:56.350086 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1520bd-0925-4e4b-9a68-ad89ac26f083 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:56.350374 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1520bd-0925-4e4b-9a68-ad89ac26f083 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:56.350651 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1520bd-0925-4e4b-9a68-ad89ac26f083 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:56.357245 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1520bd-0925-4e4b-9a68-ad89ac26f083 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:56.358721 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1520bd-0925-4e4b-9a68-ad89ac26f083 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:56.373404 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-cd1520bd-0925-4e4b-9a68-ad89ac26f083 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. May 19 14:39:56.373404 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:39:56.373404 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 19 14:39:56.373404 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 19 14:39:56.373404 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:39:56.373404 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:39:56.373404 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:39:56.373404 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:39:56.373404 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. May 19 14:39:56.373404 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:39:56.373404 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:39:56.373404 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:39:56.373404 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:39:56.373404 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:39:56.373404 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:39:56.373404 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:39:56.373404 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:39:56.374823 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:39:56.374823 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:39:56.374823 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:39:56.374823 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:39:56.374823 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:39:56.374823 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:39:56.374823 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:39:56.374823 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:39:56.374823 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:39:56.374823 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:39:56.374823 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:39:56.374823 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:39:56.374823 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:39:56.374823 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:39:56.374823 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:39:56.374823 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:39:56.375983 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:39:56.375983 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:39:56.375983 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:39:56.375983 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:39:56.375983 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:39:56.375983 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:39:56.375983 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:39:56.375983 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:39:56.375983 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:39:56.375983 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:39:56.375983 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 19 14:39:56.375983 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:39:56.375983 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. May 19 14:39:56.375983 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:39:56.375983 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 562/1165] 10.222.0.57 () {56 vars in 1117 bytes} [Thu May 19 14:39:56 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:39:56.380798 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b36fbd-2e05-4940-b7cc-71827f90c506 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:56.480159 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b36fbd-2e05-4940-b7cc-71827f90c506 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:56.513071 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc645bc-0b62-4816-a9e8-dc9ce08c7998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:56.513730 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc645bc-0b62-4816-a9e8-dc9ce08c7998 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:56.514048 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc645bc-0b62-4816-a9e8-dc9ce08c7998 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:56.514327 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc645bc-0b62-4816-a9e8-dc9ce08c7998 None admin] PATH_INFO: `/v3/domains` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:56.515052 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc645bc-0b62-4816-a9e8-dc9ce08c7998 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:39:56.515361 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc645bc-0b62-4816-a9e8-dc9ce08c7998 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:56.516746 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc645bc-0b62-4816-a9e8-dc9ce08c7998 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:56.524795 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 604/1166] 10.222.0.57 () {56 vars in 1115 bytes} [Thu May 19 14:39:56 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:39:58.742988 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dd3789-8252-4e35-a9b0-0703690f79e2 None None] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:58.743562 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dd3789-8252-4e35-a9b0-0703690f79e2 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:58.743562 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dd3789-8252-4e35-a9b0-0703690f79e2 None None] PATH_INFO: `/` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:58.744911 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 563/1167] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:39:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:39:58.750205 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5d696a-dfd0-4915-9d6a-1fe2b12de388 None None] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:58.750654 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5d696a-dfd0-4915-9d6a-1fe2b12de388 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:58.750762 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5d696a-dfd0-4915-9d6a-1fe2b12de388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:58.768122 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.auth.core [None req-3f5d696a-dfd0-4915-9d6a-1fe2b12de388 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:39:58.769314 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.fernet_utils [None req-3f5d696a-dfd0-4915-9d6a-1fe2b12de388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:39:58.795674 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:58.837622 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 605/1168] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:39:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:39:58.861110 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dd3789-8252-4e35-a9b0-0703690f79e2 None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:58.962238 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dd3789-8252-4e35-a9b0-0703690f79e2 None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:58.972270 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:58.996292 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc8646b-1d80-45e9-a98d-6d0893fb609b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:58.996876 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc8646b-1d80-45e9-a98d-6d0893fb609b None admin] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:58.997156 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc8646b-1d80-45e9-a98d-6d0893fb609b None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:58.997428 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc8646b-1d80-45e9-a98d-6d0893fb609b None admin] PATH_INFO: `/v3/domains` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:58.998383 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc8646b-1d80-45e9-a98d-6d0893fb609b None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:58.999646 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc8646b-1d80-45e9-a98d-6d0893fb609b None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:59.013892 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 564/1169] 10.222.0.57 () {60 vars in 1138 bytes} [Thu May 19 14:39:58 2022] POST /identity/v3/domains => generated 267 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:40:01.261232 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62f417d-e838-46cf-845e-b9beeb591924 None None] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:40:01.261786 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62f417d-e838-46cf-845e-b9beeb591924 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:40:01.261786 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62f417d-e838-46cf-845e-b9beeb591924 None None] PATH_INFO: `/` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:40:01.263208 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 606/1170] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:40:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:40:01.268406 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcbd02a-1b77-4903-90c9-838f85bd2501 None None] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:40:01.268857 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcbd02a-1b77-4903-90c9-838f85bd2501 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:40:01.268966 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcbd02a-1b77-4903-90c9-838f85bd2501 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:40:01.286374 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.auth.core [None req-ffcbd02a-1b77-4903-90c9-838f85bd2501 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:40:01.288307 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-ffcbd02a-1b77-4903-90c9-838f85bd2501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:40:01.354014 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 565/1171] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:40:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:40:01.377478 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62f417d-e838-46cf-845e-b9beeb591924 None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:40:01.476776 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62f417d-e838-46cf-845e-b9beeb591924 None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:40:01.509707 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a47a66-d732-4d44-8fd9-eb5f7701903a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:40:01.510580 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a47a66-d732-4d44-8fd9-eb5f7701903a None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:40:01.510857 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a47a66-d732-4d44-8fd9-eb5f7701903a None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:40:01.511123 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a47a66-d732-4d44-8fd9-eb5f7701903a None admin] PATH_INFO: `/v3/domains/514fb4affab747f694a684eeb22557f8` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:40:01.521390 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a47a66-d732-4d44-8fd9-eb5f7701903a None admin] RBAC: Authorizing `identity:get_domain(domain_id=514fb4affab747f694a684eeb22557f8)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:40:01.523480 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a47a66-d732-4d44-8fd9-eb5f7701903a None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:40:01.525093 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 607/1172] 10.222.0.57 () {56 vars in 1183 bytes} [Thu May 19 14:40:01 2022] GET /identity/v3/domains/514fb4affab747f694a684eeb22557f8 => generated 267 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:40:01.530978 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcbd02a-1b77-4903-90c9-838f85bd2501 None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:40:01.626430 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:40:01.629470 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcbd02a-1b77-4903-90c9-838f85bd2501 None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:40:01.664499 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d888b3-5b1c-4f9b-9947-6147e1e59e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:40:01.665153 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d888b3-5b1c-4f9b-9947-6147e1e59e5e None admin] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:40:01.665433 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d888b3-5b1c-4f9b-9947-6147e1e59e5e None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:40:01.665709 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d888b3-5b1c-4f9b-9947-6147e1e59e5e None admin] PATH_INFO: `/v3/projects` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:40:01.666457 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d888b3-5b1c-4f9b-9947-6147e1e59e5e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:40:01.667881 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d888b3-5b1c-4f9b-9947-6147e1e59e5e None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:40:01.692926 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 566/1173] 10.222.0.57 () {60 vars in 1141 bytes} [Thu May 19 14:40:01 2022] POST /identity/v3/projects => generated 361 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:40:03.958005 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e38a3f-c218-4b41-b282-68172f3fcc05 None None] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:40:03.958559 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e38a3f-c218-4b41-b282-68172f3fcc05 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:40:03.958559 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e38a3f-c218-4b41-b282-68172f3fcc05 None None] PATH_INFO: `/` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:40:03.960875 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 608/1174] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:40:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:40:03.966388 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d881424-c277-43f2-a1f3-c57712c7758d None None] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:40:03.966823 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d881424-c277-43f2-a1f3-c57712c7758d None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:40:03.966948 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d881424-c277-43f2-a1f3-c57712c7758d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:40:03.984564 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.auth.core [None req-5d881424-c277-43f2-a1f3-c57712c7758d None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:40:03.985743 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-5d881424-c277-43f2-a1f3-c57712c7758d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:40:04.050740 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 567/1175] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:40:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:40:04.075856 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e38a3f-c218-4b41-b282-68172f3fcc05 None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:40:04.102301 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:40:04.180470 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e38a3f-c218-4b41-b282-68172f3fcc05 None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:40:04.213801 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba0d96a-4c93-442e-beea-a3ace3ad5c1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:40:04.214633 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba0d96a-4c93-442e-beea-a3ace3ad5c1e None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:40:04.214915 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba0d96a-4c93-442e-beea-a3ace3ad5c1e None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:40:04.215187 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba0d96a-4c93-442e-beea-a3ace3ad5c1e None admin] PATH_INFO: `/v3/domains/514fb4affab747f694a684eeb22557f8` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:40:04.216482 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba0d96a-4c93-442e-beea-a3ace3ad5c1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=514fb4affab747f694a684eeb22557f8)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:40:04.217942 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba0d96a-4c93-442e-beea-a3ace3ad5c1e None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:40:04.219561 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 609/1176] 10.222.0.57 () {56 vars in 1183 bytes} [Thu May 19 14:40:04 2022] GET /identity/v3/domains/514fb4affab747f694a684eeb22557f8 => generated 267 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:40:04.226077 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d881424-c277-43f2-a1f3-c57712c7758d None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:40:04.304662 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:40:04.324011 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d881424-c277-43f2-a1f3-c57712c7758d None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:40:04.357269 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b473af1-ced5-4a48-bf58-8bcdb2da6f3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:40:04.357953 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b473af1-ced5-4a48-bf58-8bcdb2da6f3f None admin] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:40:04.358239 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b473af1-ced5-4a48-bf58-8bcdb2da6f3f None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:40:04.358515 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b473af1-ced5-4a48-bf58-8bcdb2da6f3f None admin] PATH_INFO: `/v3/users` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:40:04.359232 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b473af1-ced5-4a48-bf58-8bcdb2da6f3f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:40:04.360678 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b473af1-ced5-4a48-bf58-8bcdb2da6f3f None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:40:04.381602 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 568/1177] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 19 14:40:04 2022] POST /identity/v3/users => generated 311 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:40:06.659419 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a282aa8e-c5c5-4213-9931-f541274edb5e None None] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:40:06.659976 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a282aa8e-c5c5-4213-9931-f541274edb5e None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:40:06.659976 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a282aa8e-c5c5-4213-9931-f541274edb5e None None] PATH_INFO: `/` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:40:06.661356 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 610/1178] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:40:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:40:06.667348 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c7b1bd-23d9-4b71-bcb8-1a533fd15a1b None None] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:40:06.667795 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c7b1bd-23d9-4b71-bcb8-1a533fd15a1b None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:40:06.667902 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c7b1bd-23d9-4b71-bcb8-1a533fd15a1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:40:06.685279 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.auth.core [None req-f5c7b1bd-23d9-4b71-bcb8-1a533fd15a1b None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:40:06.686501 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-f5c7b1bd-23d9-4b71-bcb8-1a533fd15a1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:40:06.752040 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 569/1179] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:40:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:40:06.776787 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a282aa8e-c5c5-4213-9931-f541274edb5e None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:40:06.869917 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:40:06.879734 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a282aa8e-c5c5-4213-9931-f541274edb5e None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:40:06.912821 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaa2730-279d-41c6-a753-539f18b1ad91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:40:06.913538 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaa2730-279d-41c6-a753-539f18b1ad91 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:40:06.913822 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaa2730-279d-41c6-a753-539f18b1ad91 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:40:06.914122 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaa2730-279d-41c6-a753-539f18b1ad91 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:40:06.924381 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaa2730-279d-41c6-a753-539f18b1ad91 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:40:06.932792 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-ceaa2730-279d-41c6-a753-539f18b1ad91 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:40:06.932792 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:40:06.932792 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:40:06.932792 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:40:06.932792 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:40:06.932792 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:40:06.932792 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:40:06.932792 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:40:06.932792 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:40:06.932792 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:40:06.932792 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:40:06.932792 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:40:06.932792 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:40:06.932792 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:40:06.932792 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:40:06.932792 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:40:06.934027 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:40:06.934027 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:40:06.934027 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:40:06.934027 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:40:06.934027 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:40:06.934027 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:40:06.934027 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:40:06.934027 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:40:06.934027 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:40:06.934027 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:40:06.934027 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:40:06.934027 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:40:06.934027 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:40:06.934027 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:40:06.934027 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:40:06.934027 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:40:06.935329 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:40:06.935329 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:40:06.935329 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:40:06.935329 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:40:06.935329 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:40:06.935329 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:40:06.935329 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:40:06.935329 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:40:06.935329 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:40:06.935329 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:40:06.935329 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:40:06.935329 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:40:06.935329 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 611/1180] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:40:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:40:06.939974 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c7b1bd-23d9-4b71-bcb8-1a533fd15a1b None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:40:07.005270 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:40:07.040357 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c7b1bd-23d9-4b71-bcb8-1a533fd15a1b None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:40:07.074700 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a716b8c-449a-46d1-84fa-64fc21958ae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:40:07.075365 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a716b8c-449a-46d1-84fa-64fc21958ae2 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:40:07.075650 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a716b8c-449a-46d1-84fa-64fc21958ae2 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:40:07.075922 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a716b8c-449a-46d1-84fa-64fc21958ae2 None admin] PATH_INFO: `/v3/roles` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:40:07.076663 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a716b8c-449a-46d1-84fa-64fc21958ae2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:40:07.076969 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a716b8c-449a-46d1-84fa-64fc21958ae2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:40:07.078380 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a716b8c-449a-46d1-84fa-64fc21958ae2 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:40:07.086692 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 570/1181] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:40:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:40:07.092630 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaa2730-279d-41c6-a753-539f18b1ad91 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:40:07.196965 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaa2730-279d-41c6-a753-539f18b1ad91 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:40:07.230536 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa3c9a8-683a-4d16-8a64-dd5d065bd0e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:40:07.231399 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa3c9a8-683a-4d16-8a64-dd5d065bd0e8 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:40:07.231681 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa3c9a8-683a-4d16-8a64-dd5d065bd0e8 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:40:07.232031 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa3c9a8-683a-4d16-8a64-dd5d065bd0e8 None admin] PATH_INFO: `/v3/users/90656db3c9604956ace38d7d4e714d5b` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:40:07.245388 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa3c9a8-683a-4d16-8a64-dd5d065bd0e8 None admin] RBAC: Authorizing `identity:get_user(user_id=90656db3c9604956ace38d7d4e714d5b)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:40:07.246864 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa3c9a8-683a-4d16-8a64-dd5d065bd0e8 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:40:07.248506 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 612/1182] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:40:07 2022] GET /identity/v3/users/90656db3c9604956ace38d7d4e714d5b => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:40:07.254367 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a716b8c-449a-46d1-84fa-64fc21958ae2 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:40:07.353708 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a716b8c-449a-46d1-84fa-64fc21958ae2 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:40:07.381653 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:40:07.388012 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abac2e06-2b0b-43f6-a261-486ab504f6b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:40:07.388663 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abac2e06-2b0b-43f6-a261-486ab504f6b3 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:40:07.388944 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abac2e06-2b0b-43f6-a261-486ab504f6b3 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:40:07.389218 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abac2e06-2b0b-43f6-a261-486ab504f6b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:40:07.454522 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abac2e06-2b0b-43f6-a261-486ab504f6b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:40:07.455899 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abac2e06-2b0b-43f6-a261-486ab504f6b3 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:40:07.552407 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 571/1183] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:40:07 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:40:07.559046 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa3c9a8-683a-4d16-8a64-dd5d065bd0e8 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:40:07.571436 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:40:07.660051 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa3c9a8-683a-4d16-8a64-dd5d065bd0e8 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:40:07.694204 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6862f8fc-8346-4152-a201-ad465ff50b6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:40:07.695060 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6862f8fc-8346-4152-a201-ad465ff50b6f None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:40:07.695346 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6862f8fc-8346-4152-a201-ad465ff50b6f None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:40:07.695621 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6862f8fc-8346-4152-a201-ad465ff50b6f None admin] PATH_INFO: `/v3/projects/fff2431c625f47bbb7758e3e9b2b3cfa` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:40:07.696799 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6862f8fc-8346-4152-a201-ad465ff50b6f None admin] RBAC: Authorizing `identity:get_project(project_id=fff2431c625f47bbb7758e3e9b2b3cfa)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:40:07.698343 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6862f8fc-8346-4152-a201-ad465ff50b6f None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:40:07.700049 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 613/1184] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:40:07 2022] GET /identity/v3/projects/fff2431c625f47bbb7758e3e9b2b3cfa => generated 361 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:40:07.706288 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abac2e06-2b0b-43f6-a261-486ab504f6b3 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:40:07.736637 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:40:07.805516 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abac2e06-2b0b-43f6-a261-486ab504f6b3 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:40:07.838956 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829c2cc6-a0e3-4574-9da9-548657451bcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:40:07.839593 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829c2cc6-a0e3-4574-9da9-548657451bcd None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:40:07.839876 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829c2cc6-a0e3-4574-9da9-548657451bcd None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:40:07.840151 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829c2cc6-a0e3-4574-9da9-548657451bcd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:40:07.840972 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829c2cc6-a0e3-4574-9da9-548657451bcd None admin] RBAC: Adding query filter params (role.id=c2935b1a25ca46fcba419fe3d1715330, scope.project.id=fff2431c625f47bbb7758e3e9b2b3cfa, user.id=90656db3c9604956ace38d7d4e714d5b) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:40:07.841286 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829c2cc6-a0e3-4574-9da9-548657451bcd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:40:07.842740 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829c2cc6-a0e3-4574-9da9-548657451bcd None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:40:07.848061 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 572/1185] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:40:07 2022] GET /identity/v3/role_assignments?user.id=90656db3c9604956ace38d7d4e714d5b&scope.project.id=fff2431c625f47bbb7758e3e9b2b3cfa&role.id=c2935b1a25ca46fcba419fe3d1715330 => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:40:10.054691 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdee7d83-60ab-4413-b8b6-2d92f9c3fb97 None None] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:40:10.055274 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdee7d83-60ab-4413-b8b6-2d92f9c3fb97 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:40:10.055274 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdee7d83-60ab-4413-b8b6-2d92f9c3fb97 None None] PATH_INFO: `/` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:40:10.056601 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 614/1186] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:40:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:40:10.061974 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a47fa1-fa4e-4d1c-b427-eaa3ef44e8fd None None] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:40:10.062425 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a47fa1-fa4e-4d1c-b427-eaa3ef44e8fd None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:40:10.062542 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a47fa1-fa4e-4d1c-b427-eaa3ef44e8fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:40:10.079670 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.auth.core [None req-32a47fa1-fa4e-4d1c-b427-eaa3ef44e8fd None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:40:10.080851 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-32a47fa1-fa4e-4d1c-b427-eaa3ef44e8fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:40:10.153168 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 573/1187] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:40:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:40:10.175801 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdee7d83-60ab-4413-b8b6-2d92f9c3fb97 None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:40:10.256941 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:40:10.279984 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdee7d83-60ab-4413-b8b6-2d92f9c3fb97 None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:40:10.314228 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8877ee6f-fbdc-4724-86e7-00811db04900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:40:10.315076 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8877ee6f-fbdc-4724-86e7-00811db04900 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:40:10.315356 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8877ee6f-fbdc-4724-86e7-00811db04900 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:40:10.315626 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8877ee6f-fbdc-4724-86e7-00811db04900 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:40:10.325505 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8877ee6f-fbdc-4724-86e7-00811db04900 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:40:10.333215 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-8877ee6f-fbdc-4724-86e7-00811db04900 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:40:10.333215 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:40:10.333215 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:40:10.333215 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:40:10.333215 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:40:10.333215 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:40:10.333215 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:40:10.333215 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:40:10.333215 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:40:10.333215 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:40:10.333215 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:40:10.333215 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:40:10.333215 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:40:10.333215 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:40:10.333215 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:40:10.333215 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:40:10.334886 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:40:10.334886 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:40:10.334886 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:40:10.334886 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:40:10.334886 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:40:10.334886 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:40:10.334886 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:40:10.334886 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:40:10.334886 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:40:10.334886 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:40:10.334886 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:40:10.334886 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:40:10.334886 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:40:10.334886 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:40:10.334886 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:40:10.334886 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:40:10.336204 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:40:10.336204 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:40:10.336204 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:40:10.336204 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:40:10.336204 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:40:10.336204 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:40:10.336204 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:40:10.336204 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:40:10.336204 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:40:10.336204 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:40:10.336204 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:40:10.336204 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:40:10.336204 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 615/1188] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:40:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:40:10.341034 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a47fa1-fa4e-4d1c-b427-eaa3ef44e8fd None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:40:10.366673 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:40:10.441258 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a47fa1-fa4e-4d1c-b427-eaa3ef44e8fd None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:40:10.475153 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291c2760-d913-4e2b-b561-7e054c5aa4c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:40:10.475803 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291c2760-d913-4e2b-b561-7e054c5aa4c5 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:40:10.476084 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291c2760-d913-4e2b-b561-7e054c5aa4c5 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:40:10.476352 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291c2760-d913-4e2b-b561-7e054c5aa4c5 None admin] PATH_INFO: `/v3/roles` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:40:10.477084 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291c2760-d913-4e2b-b561-7e054c5aa4c5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:40:10.477387 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291c2760-d913-4e2b-b561-7e054c5aa4c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:40:10.478773 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291c2760-d913-4e2b-b561-7e054c5aa4c5 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:40:10.486740 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 574/1189] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:40:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:40:10.493489 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8877ee6f-fbdc-4724-86e7-00811db04900 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:40:10.594825 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8877ee6f-fbdc-4724-86e7-00811db04900 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:40:10.629801 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf35ff73-9097-415a-b089-29f4b8fd3ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:40:10.630748 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf35ff73-9097-415a-b089-29f4b8fd3ffb None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:40:10.631039 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf35ff73-9097-415a-b089-29f4b8fd3ffb None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:40:10.631466 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf35ff73-9097-415a-b089-29f4b8fd3ffb None admin] PATH_INFO: `/v3/users/90656db3c9604956ace38d7d4e714d5b` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:40:10.632710 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf35ff73-9097-415a-b089-29f4b8fd3ffb None admin] RBAC: Authorizing `identity:get_user(user_id=90656db3c9604956ace38d7d4e714d5b)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:40:10.634201 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf35ff73-9097-415a-b089-29f4b8fd3ffb None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:40:10.635856 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 616/1190] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:40:10 2022] GET /identity/v3/users/90656db3c9604956ace38d7d4e714d5b => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:40:10.641770 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291c2760-d913-4e2b-b561-7e054c5aa4c5 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:40:10.733497 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:40:10.750873 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291c2760-d913-4e2b-b561-7e054c5aa4c5 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:40:10.793895 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66e264c-d0db-4d9e-a574-4e40b5209099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:40:10.794756 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66e264c-d0db-4d9e-a574-4e40b5209099 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:40:10.795038 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66e264c-d0db-4d9e-a574-4e40b5209099 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:40:10.795311 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66e264c-d0db-4d9e-a574-4e40b5209099 None admin] PATH_INFO: `/v3/projects/fff2431c625f47bbb7758e3e9b2b3cfa` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:40:10.796529 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66e264c-d0db-4d9e-a574-4e40b5209099 None admin] RBAC: Authorizing `identity:get_project(project_id=fff2431c625f47bbb7758e3e9b2b3cfa)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:40:10.798004 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66e264c-d0db-4d9e-a574-4e40b5209099 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:40:10.799744 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 575/1191] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:40:10 2022] GET /identity/v3/projects/fff2431c625f47bbb7758e3e9b2b3cfa => generated 361 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:40:10.807018 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf35ff73-9097-415a-b089-29f4b8fd3ffb None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:40:10.816581 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:40:10.914479 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf35ff73-9097-415a-b089-29f4b8fd3ffb None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:40:10.951869 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87164cb-67b1-41f0-9c90-01dda9dd1a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:40:10.952765 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87164cb-67b1-41f0-9c90-01dda9dd1a93 None admin] REQUEST_METHOD: `PUT` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:40:10.953050 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87164cb-67b1-41f0-9c90-01dda9dd1a93 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:40:10.953331 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87164cb-67b1-41f0-9c90-01dda9dd1a93 None admin] PATH_INFO: `/v3/projects/fff2431c625f47bbb7758e3e9b2b3cfa/users/90656db3c9604956ace38d7d4e714d5b/roles/c2935b1a25ca46fcba419fe3d1715330` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:40:10.955407 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87164cb-67b1-41f0-9c90-01dda9dd1a93 None admin] RBAC: Authorizing `identity:create_grant(project_id=fff2431c625f47bbb7758e3e9b2b3cfa, user_id=90656db3c9604956ace38d7d4e714d5b, role_id=c2935b1a25ca46fcba419fe3d1715330)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:40:10.956859 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87164cb-67b1-41f0-9c90-01dda9dd1a93 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:40:10.971464 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 617/1192] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 14:40:10 2022] PUT /identity/v3/projects/fff2431c625f47bbb7758e3e9b2b3cfa/users/90656db3c9604956ace38d7d4e714d5b/roles/c2935b1a25ca46fcba419fe3d1715330 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:40:13.189136 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916fdab2-ba88-471e-83e3-c65dd865ac51 None None] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:40:13.189636 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916fdab2-ba88-471e-83e3-c65dd865ac51 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:40:13.189762 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916fdab2-ba88-471e-83e3-c65dd865ac51 None None] PATH_INFO: `/` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:40:13.191108 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 576/1193] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:40:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:40:13.196721 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5414a438-2ecc-4e55-9cbc-06efcef480dc None None] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:40:13.197159 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5414a438-2ecc-4e55-9cbc-06efcef480dc None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:40:13.197276 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5414a438-2ecc-4e55-9cbc-06efcef480dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:40:13.214897 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.auth.core [None req-5414a438-2ecc-4e55-9cbc-06efcef480dc None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:40:13.216084 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.fernet_utils [None req-5414a438-2ecc-4e55-9cbc-06efcef480dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:40:13.275309 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:40:13.283695 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 618/1194] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:40:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:40:13.307346 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916fdab2-ba88-471e-83e3-c65dd865ac51 None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:40:13.407644 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916fdab2-ba88-471e-83e3-c65dd865ac51 None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:40:13.440546 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87085745-f512-4af4-8e71-618206d66b6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:40:13.441261 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87085745-f512-4af4-8e71-618206d66b6d None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:40:13.441544 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87085745-f512-4af4-8e71-618206d66b6d None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:40:13.441818 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87085745-f512-4af4-8e71-618206d66b6d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:40:13.452365 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87085745-f512-4af4-8e71-618206d66b6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:40:13.460375 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-87085745-f512-4af4-8e71-618206d66b6d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:40:13.460375 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:40:13.460375 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:40:13.460375 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:40:13.460375 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:40:13.460375 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:40:13.460375 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:40:13.460375 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:40:13.460375 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:40:13.460375 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:40:13.460375 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:40:13.460375 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:40:13.460375 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:40:13.460375 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:40:13.460375 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:40:13.460375 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:40:13.461605 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:40:13.461605 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:40:13.461605 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:40:13.461605 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:40:13.461605 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:40:13.461605 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:40:13.461605 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:40:13.461605 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:40:13.461605 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:40:13.461605 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:40:13.461605 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:40:13.461605 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:40:13.461605 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:40:13.461605 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:40:13.461605 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:40:13.461605 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:40:13.462663 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:40:13.462663 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:40:13.462663 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:40:13.462663 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:40:13.462663 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:40:13.462663 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:40:13.462663 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:40:13.462663 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:40:13.462663 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:40:13.462663 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:40:13.462663 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:40:13.462663 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:40:13.462663 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 577/1195] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:40:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:40:13.467598 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5414a438-2ecc-4e55-9cbc-06efcef480dc None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:40:13.566022 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5414a438-2ecc-4e55-9cbc-06efcef480dc None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:40:13.601696 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5569c6a9-9483-4b95-9daa-63dfac7d76d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:40:13.602400 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5569c6a9-9483-4b95-9daa-63dfac7d76d6 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:40:13.602688 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5569c6a9-9483-4b95-9daa-63dfac7d76d6 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:40:13.602967 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5569c6a9-9483-4b95-9daa-63dfac7d76d6 None admin] PATH_INFO: `/v3/roles` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:40:13.603720 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5569c6a9-9483-4b95-9daa-63dfac7d76d6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:40:13.604032 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5569c6a9-9483-4b95-9daa-63dfac7d76d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:40:13.605427 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5569c6a9-9483-4b95-9daa-63dfac7d76d6 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:40:13.613703 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 619/1196] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:40:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:40:13.619861 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87085745-f512-4af4-8e71-618206d66b6d None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:40:13.635537 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:40:13.720301 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87085745-f512-4af4-8e71-618206d66b6d None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:40:13.753547 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54282a96-3f4a-4496-a89e-e7a608ab018e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:40:13.754437 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54282a96-3f4a-4496-a89e-e7a608ab018e None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:40:13.754718 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54282a96-3f4a-4496-a89e-e7a608ab018e None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:40:13.754992 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54282a96-3f4a-4496-a89e-e7a608ab018e None admin] PATH_INFO: `/v3/users/90656db3c9604956ace38d7d4e714d5b` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:40:13.756226 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54282a96-3f4a-4496-a89e-e7a608ab018e None admin] RBAC: Authorizing `identity:get_user(user_id=90656db3c9604956ace38d7d4e714d5b)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:40:13.757656 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54282a96-3f4a-4496-a89e-e7a608ab018e None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:40:13.759318 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 578/1197] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:40:13 2022] GET /identity/v3/users/90656db3c9604956ace38d7d4e714d5b => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:40:13.765101 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5569c6a9-9483-4b95-9daa-63dfac7d76d6 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:40:13.824848 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:40:13.866565 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5569c6a9-9483-4b95-9daa-63dfac7d76d6 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:40:13.899487 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed01f3f9-4cc2-4ec4-9f5a-4bd0705bb98a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:40:13.900130 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed01f3f9-4cc2-4ec4-9f5a-4bd0705bb98a None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:40:13.900414 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed01f3f9-4cc2-4ec4-9f5a-4bd0705bb98a None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:40:13.900690 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed01f3f9-4cc2-4ec4-9f5a-4bd0705bb98a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:40:13.967843 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed01f3f9-4cc2-4ec4-9f5a-4bd0705bb98a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:40:13.969293 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed01f3f9-4cc2-4ec4-9f5a-4bd0705bb98a None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:40:14.024937 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:40:14.067547 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 620/1198] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:40:13 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:40:14.073614 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54282a96-3f4a-4496-a89e-e7a608ab018e None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:40:14.153411 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:40:14.172301 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54282a96-3f4a-4496-a89e-e7a608ab018e None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:40:14.205622 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0f52b3-2288-4090-a739-bb12197f47c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:40:14.206511 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0f52b3-2288-4090-a739-bb12197f47c1 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:40:14.206792 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0f52b3-2288-4090-a739-bb12197f47c1 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:40:14.207064 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0f52b3-2288-4090-a739-bb12197f47c1 None admin] PATH_INFO: `/v3/projects/fff2431c625f47bbb7758e3e9b2b3cfa` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:40:14.208247 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0f52b3-2288-4090-a739-bb12197f47c1 None admin] RBAC: Authorizing `identity:get_project(project_id=fff2431c625f47bbb7758e3e9b2b3cfa)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:40:14.210204 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0f52b3-2288-4090-a739-bb12197f47c1 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:40:14.211912 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 579/1199] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:40:14 2022] GET /identity/v3/projects/fff2431c625f47bbb7758e3e9b2b3cfa => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:40:14.218181 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed01f3f9-4cc2-4ec4-9f5a-4bd0705bb98a None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:40:14.319115 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed01f3f9-4cc2-4ec4-9f5a-4bd0705bb98a None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:40:14.353214 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfa99e2-0dc6-4c25-b816-d56a9677f972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:40:14.353903 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfa99e2-0dc6-4c25-b816-d56a9677f972 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:40:14.354187 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfa99e2-0dc6-4c25-b816-d56a9677f972 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:40:14.354465 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfa99e2-0dc6-4c25-b816-d56a9677f972 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:40:14.355289 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfa99e2-0dc6-4c25-b816-d56a9677f972 None admin] RBAC: Adding query filter params (role.id=c2935b1a25ca46fcba419fe3d1715330, scope.project.id=fff2431c625f47bbb7758e3e9b2b3cfa, user.id=90656db3c9604956ace38d7d4e714d5b) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:40:14.355602 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfa99e2-0dc6-4c25-b816-d56a9677f972 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:40:14.357067 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfa99e2-0dc6-4c25-b816-d56a9677f972 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:40:14.362907 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 621/1200] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:40:14 2022] GET /identity/v3/role_assignments?user.id=90656db3c9604956ace38d7d4e714d5b&scope.project.id=fff2431c625f47bbb7758e3e9b2b3cfa&role.id=c2935b1a25ca46fcba419fe3d1715330 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:40:24.201427 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=25 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:40:24.360179 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=89 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:41:13.034892 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d54229-684f-4481-8a7f-c8a493712ab8 None None] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:41:13.035499 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d54229-684f-4481-8a7f-c8a493712ab8 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:41:13.035499 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d54229-684f-4481-8a7f-c8a493712ab8 None None] PATH_INFO: `/` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:41:13.036870 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 580/1201] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:41:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:41:13.045977 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00db36e5-de81-46a0-9018-52df8359ad23 None None] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:41:13.046541 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00db36e5-de81-46a0-9018-52df8359ad23 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:41:13.046541 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00db36e5-de81-46a0-9018-52df8359ad23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:41:13.069035 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.auth.core [None req-00db36e5-de81-46a0-9018-52df8359ad23 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:41:13.071086 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.fernet_utils [None req-00db36e5-de81-46a0-9018-52df8359ad23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:41:13.153148 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 622/1202] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:41:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:41:13.180225 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d54229-684f-4481-8a7f-c8a493712ab8 None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:41:13.310125 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d54229-684f-4481-8a7f-c8a493712ab8 None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:41:13.355046 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2642e908-d3ff-440e-a47e-5c89f959c05e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:41:13.355860 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2642e908-d3ff-440e-a47e-5c89f959c05e None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:41:13.356144 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2642e908-d3ff-440e-a47e-5c89f959c05e None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:41:13.356419 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2642e908-d3ff-440e-a47e-5c89f959c05e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:41:13.373374 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2642e908-d3ff-440e-a47e-5c89f959c05e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:41:13.374940 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2642e908-d3ff-440e-a47e-5c89f959c05e None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:41:13.396050 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-2642e908-d3ff-440e-a47e-5c89f959c05e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:41:13.396050 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:41:13.396050 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:41:13.396050 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:41:13.396050 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:41:13.396050 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:41:13.396050 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:41:13.396050 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:41:13.396050 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:41:13.396050 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:41:13.396050 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:41:13.396050 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:41:13.396050 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:41:13.396050 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:41:13.396050 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:41:13.397699 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:41:13.397699 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:41:13.397699 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:41:13.397699 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:41:13.397699 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:41:13.397699 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:41:13.397699 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:41:13.397699 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:41:13.397699 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:41:13.397699 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:41:13.397699 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:41:13.397699 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:41:13.397699 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:41:13.397699 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:41:13.397699 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:41:13.397699 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:41:13.397699 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:41:13.407043 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:41:13.407043 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:41:13.407043 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:41:13.407043 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:41:13.407043 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 581/1203] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:41:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:41:13.407399 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00db36e5-de81-46a0-9018-52df8359ad23 None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:41:13.530499 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00db36e5-de81-46a0-9018-52df8359ad23 None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:41:13.565531 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:41:13.572404 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ee55ab-ffff-45c4-9429-b4488e38043c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:41:13.573097 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ee55ab-ffff-45c4-9429-b4488e38043c None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:41:13.573369 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ee55ab-ffff-45c4-9429-b4488e38043c None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:41:13.573654 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ee55ab-ffff-45c4-9429-b4488e38043c None admin] PATH_INFO: `/v3/domains` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:41:13.574481 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ee55ab-ffff-45c4-9429-b4488e38043c None admin] RBAC: Adding query filter params (name=Default) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:41:13.574735 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ee55ab-ffff-45c4-9429-b4488e38043c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:41:13.576117 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ee55ab-ffff-45c4-9429-b4488e38043c None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:41:13.589566 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 623/1204] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:41:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:41:13.595403 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2642e908-d3ff-440e-a47e-5c89f959c05e None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:41:13.657682 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:41:13.712825 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2642e908-d3ff-440e-a47e-5c89f959c05e None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:41:13.767684 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8badc95-1b75-4efb-a040-844e3b95a2bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:41:13.768483 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8badc95-1b75-4efb-a040-844e3b95a2bf None admin] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:41:13.768784 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8badc95-1b75-4efb-a040-844e3b95a2bf None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:41:13.769145 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8badc95-1b75-4efb-a040-844e3b95a2bf None admin] PATH_INFO: `/v3/users` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:41:13.769884 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8badc95-1b75-4efb-a040-844e3b95a2bf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:41:13.771856 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8badc95-1b75-4efb-a040-844e3b95a2bf None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:41:13.800506 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 582/1205] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 19 14:41:13 2022] POST /identity/v3/users => generated 251 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:41:16.408070 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee63cd42-7767-4e00-94f5-4223f9ea09fe None None] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:41:16.408634 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee63cd42-7767-4e00-94f5-4223f9ea09fe None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:41:16.408634 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee63cd42-7767-4e00-94f5-4223f9ea09fe None None] PATH_INFO: `/` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:41:16.410300 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 624/1206] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:41:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:41:16.417297 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17828db-a120-4edc-b281-9ff5fe26af7d None None] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:41:16.417752 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17828db-a120-4edc-b281-9ff5fe26af7d None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:41:16.417875 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17828db-a120-4edc-b281-9ff5fe26af7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:41:16.437623 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.auth.core [None req-b17828db-a120-4edc-b281-9ff5fe26af7d None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:41:16.439101 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-b17828db-a120-4edc-b281-9ff5fe26af7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:41:16.523200 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 583/1207] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:41:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:41:16.551807 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee63cd42-7767-4e00-94f5-4223f9ea09fe None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:41:16.687103 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee63cd42-7767-4e00-94f5-4223f9ea09fe None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:41:16.730422 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7cbfe8-3582-42b4-8866-20eb5fdc1ee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:41:16.731597 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7cbfe8-3582-42b4-8866-20eb5fdc1ee8 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:41:16.731885 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7cbfe8-3582-42b4-8866-20eb5fdc1ee8 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:41:16.732184 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7cbfe8-3582-42b4-8866-20eb5fdc1ee8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:41:16.748078 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7cbfe8-3582-42b4-8866-20eb5fdc1ee8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:41:16.756611 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-aa7cbfe8-3582-42b4-8866-20eb5fdc1ee8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:41:16.756611 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:41:16.756611 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:41:16.756611 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:41:16.756611 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:41:16.756611 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:41:16.756611 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:41:16.756611 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:41:16.756611 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:41:16.756611 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:41:16.756611 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:41:16.756611 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:41:16.756611 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:41:16.756611 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:41:16.756611 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:41:16.756611 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:41:16.758246 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:41:16.758246 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:41:16.758246 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:41:16.758246 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:41:16.758246 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:41:16.758246 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:41:16.758246 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:41:16.758246 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:41:16.758246 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:41:16.758246 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:41:16.758246 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:41:16.758246 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:41:16.758246 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:41:16.758246 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:41:16.758246 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:41:16.758246 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:41:16.759221 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:41:16.759221 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:41:16.759221 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:41:16.759221 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:41:16.759221 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:41:16.759221 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:41:16.759221 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:41:16.759221 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:41:16.759221 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:41:16.759221 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:41:16.759221 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:41:16.759221 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:41:16.759221 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 625/1208] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 14:41:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:41:16.765448 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17828db-a120-4edc-b281-9ff5fe26af7d None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:41:16.817629 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:41:16.913207 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17828db-a120-4edc-b281-9ff5fe26af7d None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:41:16.957630 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd572e8-5f8f-4e14-8b5b-1a943c1490dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:41:16.958593 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd572e8-5f8f-4e14-8b5b-1a943c1490dc None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:41:16.958978 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd572e8-5f8f-4e14-8b5b-1a943c1490dc None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:41:16.959377 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd572e8-5f8f-4e14-8b5b-1a943c1490dc None admin] PATH_INFO: `/v3/roles` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:41:16.960345 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd572e8-5f8f-4e14-8b5b-1a943c1490dc None admin] RBAC: Adding query filter params (name=service) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:41:16.960683 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd572e8-5f8f-4e14-8b5b-1a943c1490dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:41:16.962242 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd572e8-5f8f-4e14-8b5b-1a943c1490dc None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:41:16.972009 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 584/1209] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 19 14:41:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:41:16.978737 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7cbfe8-3582-42b4-8866-20eb5fdc1ee8 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:41:17.038095 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:41:17.098874 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7cbfe8-3582-42b4-8866-20eb5fdc1ee8 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:41:17.138067 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b015ede6-72e5-43d6-a767-a57a6960037c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:41:17.139117 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b015ede6-72e5-43d6-a767-a57a6960037c None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:41:17.139541 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b015ede6-72e5-43d6-a767-a57a6960037c None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:41:17.139951 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b015ede6-72e5-43d6-a767-a57a6960037c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:41:17.152175 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b015ede6-72e5-43d6-a767-a57a6960037c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:41:17.153797 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b015ede6-72e5-43d6-a767-a57a6960037c None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:41:17.173609 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-b015ede6-72e5-43d6-a767-a57a6960037c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:41:17.173609 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:41:17.173609 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:41:17.173609 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:41:17.173609 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:41:17.173609 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:41:17.173609 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:41:17.173609 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:41:17.173609 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:41:17.173609 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:41:17.173609 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:41:17.173609 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:41:17.173609 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:41:17.173609 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:41:17.173609 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:41:17.174855 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:41:17.174855 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:41:17.174855 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:41:17.174855 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:41:17.174855 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:41:17.174855 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:41:17.174855 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:41:17.174855 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:41:17.174855 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:41:17.174855 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:41:17.174855 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:41:17.174855 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:41:17.174855 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:41:17.174855 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:41:17.174855 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:41:17.174855 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:41:17.174855 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:41:17.175857 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:41:17.175857 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:41:17.175857 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:41:17.175857 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:41:17.175857 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 626/1210] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:41:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:41:17.183429 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd572e8-5f8f-4e14-8b5b-1a943c1490dc None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:41:17.312392 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:41:17.315781 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd572e8-5f8f-4e14-8b5b-1a943c1490dc None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:41:17.366411 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1dee70-03ff-4e71-81e8-29fd9f77d222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:41:17.367129 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1dee70-03ff-4e71-81e8-29fd9f77d222 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:41:17.367419 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1dee70-03ff-4e71-81e8-29fd9f77d222 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:41:17.367724 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1dee70-03ff-4e71-81e8-29fd9f77d222 None admin] PATH_INFO: `/v3/domains` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:41:17.368587 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1dee70-03ff-4e71-81e8-29fd9f77d222 None admin] RBAC: Adding query filter params (name=Default) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:41:17.368892 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1dee70-03ff-4e71-81e8-29fd9f77d222 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:41:17.370471 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1dee70-03ff-4e71-81e8-29fd9f77d222 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:41:17.384375 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 585/1211] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:41:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:41:17.390559 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b015ede6-72e5-43d6-a767-a57a6960037c None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:41:17.503296 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b015ede6-72e5-43d6-a767-a57a6960037c None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:41:17.504854 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:41:17.546092 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ac5189-1a34-4d9b-bc4c-db620e080813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:41:17.546962 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ac5189-1a34-4d9b-bc4c-db620e080813 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:41:17.547238 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ac5189-1a34-4d9b-bc4c-db620e080813 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:41:17.547531 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ac5189-1a34-4d9b-bc4c-db620e080813 None admin] PATH_INFO: `/v3/users/placement` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:41:17.556527 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ac5189-1a34-4d9b-bc4c-db620e080813 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:41:17.558056 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ac5189-1a34-4d9b-bc4c-db620e080813 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:41:17.577567 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-66ac5189-1a34-4d9b-bc4c-db620e080813 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 14:41:17.577567 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:41:17.577567 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:41:17.577567 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:41:17.577567 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:41:17.577567 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:41:17.577567 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:41:17.577567 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:41:17.577567 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:41:17.577567 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:41:17.577567 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:41:17.577567 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:41:17.577567 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:41:17.577567 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:41:17.577567 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:41:17.577567 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:41:17.578996 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:41:17.578996 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:41:17.578996 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:41:17.578996 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:41:17.578996 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:41:17.578996 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:41:17.578996 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:41:17.578996 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:41:17.578996 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:41:17.578996 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:41:17.578996 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:41:17.578996 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:41:17.578996 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:41:17.578996 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:41:17.578996 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:41:17.578996 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:41:17.578996 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:41:17.578996 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:41:17.580344 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:41:17.580344 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:41:17.580344 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:41:17.580344 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:41:17.580344 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:41:17.580344 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:41:17.580344 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:41:17.580344 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:41:17.580344 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 14:41:17.580344 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:41:17.582048 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 627/1212] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:41:17 2022] GET /identity/v3/users/placement => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:41:17.585348 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1dee70-03ff-4e71-81e8-29fd9f77d222 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:41:17.700640 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1dee70-03ff-4e71-81e8-29fd9f77d222 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:41:17.738426 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed14dda-847a-45ea-a7d6-8c8533c3480b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:41:17.739110 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed14dda-847a-45ea-a7d6-8c8533c3480b None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:41:17.739393 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed14dda-847a-45ea-a7d6-8c8533c3480b None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:41:17.739725 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed14dda-847a-45ea-a7d6-8c8533c3480b None admin] PATH_INFO: `/v3/users` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:41:17.740778 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed14dda-847a-45ea-a7d6-8c8533c3480b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:41:17.741079 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed14dda-847a-45ea-a7d6-8c8533c3480b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:41:17.742616 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed14dda-847a-45ea-a7d6-8c8533c3480b None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:41:17.763632 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 586/1213] 10.222.0.57 () {56 vars in 1143 bytes} [Thu May 19 14:41:17 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:41:17.769556 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ac5189-1a34-4d9b-bc4c-db620e080813 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:41:17.889962 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ac5189-1a34-4d9b-bc4c-db620e080813 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:41:17.936521 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bc5705-5deb-436f-8b5f-65b6374592b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:41:17.937155 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bc5705-5deb-436f-8b5f-65b6374592b8 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:41:17.937439 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bc5705-5deb-436f-8b5f-65b6374592b8 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:41:17.937713 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bc5705-5deb-436f-8b5f-65b6374592b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:41:17.976708 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:41:18.030843 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bc5705-5deb-436f-8b5f-65b6374592b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:41:18.032568 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bc5705-5deb-436f-8b5f-65b6374592b8 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:41:18.173923 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 628/1214] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:41:17 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:41:18.177645 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed14dda-847a-45ea-a7d6-8c8533c3480b None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:41:18.200620 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:41:18.304810 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed14dda-847a-45ea-a7d6-8c8533c3480b None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:41:18.343733 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aaeb81-b345-4331-b4c5-fae97345d1b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:41:18.344591 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aaeb81-b345-4331-b4c5-fae97345d1b7 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:41:18.344919 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aaeb81-b345-4331-b4c5-fae97345d1b7 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:41:18.345327 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aaeb81-b345-4331-b4c5-fae97345d1b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:41:18.357399 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0aaeb81-b345-4331-b4c5-fae97345d1b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:41:18.358961 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0aaeb81-b345-4331-b4c5-fae97345d1b7 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:41:18.379816 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-f0aaeb81-b345-4331-b4c5-fae97345d1b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:41:18.379816 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:41:18.379816 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:41:18.379816 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:41:18.379816 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:41:18.379816 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:41:18.379816 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:41:18.379816 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:41:18.379816 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:41:18.379816 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:41:18.379816 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:41:18.379816 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:41:18.379816 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:41:18.379816 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:41:18.379816 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:41:18.381724 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:41:18.381724 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:41:18.381724 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:41:18.381724 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:41:18.381724 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:41:18.381724 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:41:18.381724 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:41:18.381724 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:41:18.381724 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:41:18.381724 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:41:18.381724 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:41:18.381724 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:41:18.381724 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:41:18.381724 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:41:18.381724 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:41:18.381724 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:41:18.381724 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:41:18.383084 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:41:18.383084 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:41:18.383084 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:41:18.383084 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:41:18.383084 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 587/1215] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:41:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:41:18.388201 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bc5705-5deb-436f-8b5f-65b6374592b8 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:41:18.465403 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:41:18.512337 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bc5705-5deb-436f-8b5f-65b6374592b8 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:41:18.553665 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905c4d1c-a720-4d32-81ab-049fa7032295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:41:18.554524 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905c4d1c-a720-4d32-81ab-049fa7032295 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:41:18.554859 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905c4d1c-a720-4d32-81ab-049fa7032295 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:41:18.555206 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905c4d1c-a720-4d32-81ab-049fa7032295 None admin] PATH_INFO: `/v3/domains` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:41:18.556016 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905c4d1c-a720-4d32-81ab-049fa7032295 None admin] RBAC: Adding query filter params (name=Default) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:41:18.556404 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905c4d1c-a720-4d32-81ab-049fa7032295 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:41:18.557862 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905c4d1c-a720-4d32-81ab-049fa7032295 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:41:18.572756 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 629/1216] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:41:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:41:18.578663 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aaeb81-b345-4331-b4c5-fae97345d1b7 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:41:18.662107 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:41:18.701042 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aaeb81-b345-4331-b4c5-fae97345d1b7 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:41:18.740522 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee6ff62-2f3e-49af-854c-6b80a8cd543e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:41:18.741432 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee6ff62-2f3e-49af-854c-6b80a8cd543e None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:41:18.741820 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee6ff62-2f3e-49af-854c-6b80a8cd543e None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:41:18.742225 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee6ff62-2f3e-49af-854c-6b80a8cd543e None admin] PATH_INFO: `/v3/projects/service` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:41:18.749787 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee6ff62-2f3e-49af-854c-6b80a8cd543e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:41:18.752762 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee6ff62-2f3e-49af-854c-6b80a8cd543e None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:41:18.767053 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-9ee6ff62-2f3e-49af-854c-6b80a8cd543e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:41:18.767053 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:41:18.767053 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:41:18.767053 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:41:18.767053 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:41:18.767053 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:41:18.767053 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:41:18.767053 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:41:18.767053 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:41:18.767053 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:41:18.767053 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:41:18.767053 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:41:18.767053 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:41:18.767053 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:41:18.767053 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:41:18.768395 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:41:18.768395 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:41:18.768395 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:41:18.768395 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:41:18.768395 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:41:18.768395 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:41:18.768395 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:41:18.768395 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:41:18.768395 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:41:18.768395 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:41:18.768395 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:41:18.768395 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:41:18.768395 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:41:18.768395 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:41:18.768395 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:41:18.768395 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:41:18.768395 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:41:18.769676 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:41:18.769676 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:41:18.769676 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:41:18.769676 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:41:18.769676 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:41:18.769676 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:41:18.769676 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:41:18.769676 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:41:18.771335 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 588/1217] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:41:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:41:18.777122 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905c4d1c-a720-4d32-81ab-049fa7032295 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:41:18.905966 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905c4d1c-a720-4d32-81ab-049fa7032295 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:41:18.934988 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:41:18.950087 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602d14b4-c3b0-4664-aeda-9e6466b52c31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:41:18.950967 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602d14b4-c3b0-4664-aeda-9e6466b52c31 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:41:18.951406 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602d14b4-c3b0-4664-aeda-9e6466b52c31 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:41:18.951712 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602d14b4-c3b0-4664-aeda-9e6466b52c31 None admin] PATH_INFO: `/v3/projects` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:41:18.952601 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602d14b4-c3b0-4664-aeda-9e6466b52c31 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:41:18.952918 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602d14b4-c3b0-4664-aeda-9e6466b52c31 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:41:18.954555 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602d14b4-c3b0-4664-aeda-9e6466b52c31 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:41:18.969678 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 630/1218] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:41:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:41:18.976682 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee6ff62-2f3e-49af-854c-6b80a8cd543e None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:41:19.096005 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee6ff62-2f3e-49af-854c-6b80a8cd543e None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:41:19.136869 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:41:19.143817 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58023649-1dce-4d94-af88-5dacc3cd0654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:41:19.144534 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58023649-1dce-4d94-af88-5dacc3cd0654 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:41:19.144857 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58023649-1dce-4d94-af88-5dacc3cd0654 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:41:19.145101 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58023649-1dce-4d94-af88-5dacc3cd0654 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:41:19.146076 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58023649-1dce-4d94-af88-5dacc3cd0654 None admin] RBAC: Adding query filter params (role.id=ffb8da90af53491980816087ee621ccb, scope.project.id=42f35b96b9d4499abfb5e5f7cca49adc, user.id=b230516fafae4ac092c036f3b9aeb047) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:41:19.146374 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58023649-1dce-4d94-af88-5dacc3cd0654 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:41:19.147960 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58023649-1dce-4d94-af88-5dacc3cd0654 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:41:19.154497 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 589/1219] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:41:18 2022] GET /identity/v3/role_assignments?user.id=b230516fafae4ac092c036f3b9aeb047&scope.project.id=42f35b96b9d4499abfb5e5f7cca49adc&role.id=ffb8da90af53491980816087ee621ccb => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:41:21.505480 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3adb94-1501-409d-8cdd-684456529063 None None] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:41:21.506220 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3adb94-1501-409d-8cdd-684456529063 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:41:21.506220 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3adb94-1501-409d-8cdd-684456529063 None None] PATH_INFO: `/` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:41:21.507764 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 631/1220] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:41:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:41:21.513204 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ec13e0-e2d3-482b-93ca-65c47459f255 None None] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:41:21.513732 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ec13e0-e2d3-482b-93ca-65c47459f255 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:41:21.513855 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ec13e0-e2d3-482b-93ca-65c47459f255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:41:21.532417 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.auth.core [None req-39ec13e0-e2d3-482b-93ca-65c47459f255 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:41:21.533671 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-39ec13e0-e2d3-482b-93ca-65c47459f255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:41:21.611466 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 590/1221] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:41:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:41:21.634977 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3adb94-1501-409d-8cdd-684456529063 None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:41:21.754326 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3adb94-1501-409d-8cdd-684456529063 None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:41:21.794755 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282b9487-f698-40b8-8e3a-0805d44f8c52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:41:21.795492 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282b9487-f698-40b8-8e3a-0805d44f8c52 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:41:21.795774 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282b9487-f698-40b8-8e3a-0805d44f8c52 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:41:21.796049 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282b9487-f698-40b8-8e3a-0805d44f8c52 None admin] PATH_INFO: `/v3/roles/service` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:41:21.807421 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282b9487-f698-40b8-8e3a-0805d44f8c52 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:41:21.815398 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-282b9487-f698-40b8-8e3a-0805d44f8c52 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:41:21.815398 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:41:21.815398 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:41:21.815398 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:41:21.815398 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:41:21.815398 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:41:21.815398 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:41:21.815398 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:41:21.815398 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:41:21.815398 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:41:21.815398 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:41:21.815398 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:41:21.815398 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:41:21.815398 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:41:21.815398 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:41:21.815398 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:41:21.818229 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:41:21.818229 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:41:21.818229 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:41:21.818229 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:41:21.818229 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:41:21.818229 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:41:21.818229 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:41:21.818229 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:41:21.818229 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:41:21.818229 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:41:21.818229 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:41:21.818229 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:41:21.818229 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:41:21.818229 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:41:21.818229 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:41:21.818229 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:41:21.819828 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:41:21.819828 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:41:21.819828 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:41:21.819828 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:41:21.819828 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:41:21.819828 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:41:21.819828 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:41:21.819828 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:41:21.819828 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:41:21.819828 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:41:21.819828 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:41:21.819828 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:41:21.819828 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 632/1222] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 14:41:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:41:21.823846 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ec13e0-e2d3-482b-93ca-65c47459f255 None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:41:21.941470 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ec13e0-e2d3-482b-93ca-65c47459f255 None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:41:21.966962 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:41:21.980472 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22eb1f95-e7f0-42ea-ab2f-4ce199a5b2d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:41:21.981435 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eb1f95-e7f0-42ea-ab2f-4ce199a5b2d1 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:41:21.982054 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eb1f95-e7f0-42ea-ab2f-4ce199a5b2d1 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:41:21.982428 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eb1f95-e7f0-42ea-ab2f-4ce199a5b2d1 None admin] PATH_INFO: `/v3/roles` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:41:21.983507 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22eb1f95-e7f0-42ea-ab2f-4ce199a5b2d1 None admin] RBAC: Adding query filter params (name=service) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:41:21.983971 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22eb1f95-e7f0-42ea-ab2f-4ce199a5b2d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:41:21.986058 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22eb1f95-e7f0-42ea-ab2f-4ce199a5b2d1 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:41:21.998409 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 591/1223] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 19 14:41:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:41:22.004364 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282b9487-f698-40b8-8e3a-0805d44f8c52 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:41:22.059498 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:41:22.122421 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282b9487-f698-40b8-8e3a-0805d44f8c52 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:41:22.162991 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9958f0a4-2b07-4b9e-80b8-2903a03d4d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:41:22.163802 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9958f0a4-2b07-4b9e-80b8-2903a03d4d53 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:41:22.164094 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9958f0a4-2b07-4b9e-80b8-2903a03d4d53 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:41:22.164375 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9958f0a4-2b07-4b9e-80b8-2903a03d4d53 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:41:22.175835 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9958f0a4-2b07-4b9e-80b8-2903a03d4d53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:41:22.177872 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9958f0a4-2b07-4b9e-80b8-2903a03d4d53 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:41:22.196105 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-9958f0a4-2b07-4b9e-80b8-2903a03d4d53 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:41:22.196105 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:41:22.196105 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:41:22.196105 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:41:22.196105 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:41:22.196105 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:41:22.196105 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:41:22.196105 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:41:22.196105 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:41:22.196105 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:41:22.196105 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:41:22.196105 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:41:22.196105 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:41:22.196105 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:41:22.196105 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:41:22.202185 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:41:22.202185 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:41:22.202185 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:41:22.202185 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:41:22.202185 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:41:22.202185 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:41:22.202185 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:41:22.202185 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:41:22.202185 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:41:22.202185 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:41:22.202185 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:41:22.202185 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:41:22.202185 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:41:22.202185 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:41:22.202185 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:41:22.202185 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:41:22.202185 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:41:22.203424 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:41:22.203424 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:41:22.203424 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:41:22.203424 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:41:22.203424 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 633/1224] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:41:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:41:22.203792 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22eb1f95-e7f0-42ea-ab2f-4ce199a5b2d1 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:41:22.311975 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22eb1f95-e7f0-42ea-ab2f-4ce199a5b2d1 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:41:22.351030 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dadfc4a-a4c8-43f4-9c49-949f7c7edd3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:41:22.351701 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dadfc4a-a4c8-43f4-9c49-949f7c7edd3c None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:41:22.351986 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dadfc4a-a4c8-43f4-9c49-949f7c7edd3c None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:41:22.352266 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dadfc4a-a4c8-43f4-9c49-949f7c7edd3c None admin] PATH_INFO: `/v3/domains` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:41:22.353001 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dadfc4a-a4c8-43f4-9c49-949f7c7edd3c None admin] RBAC: Adding query filter params (name=Default) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:41:22.353311 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dadfc4a-a4c8-43f4-9c49-949f7c7edd3c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:41:22.354733 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dadfc4a-a4c8-43f4-9c49-949f7c7edd3c None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:41:22.367720 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 592/1225] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:41:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:41:22.373728 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9958f0a4-2b07-4b9e-80b8-2903a03d4d53 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:41:22.479031 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:41:22.487544 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9958f0a4-2b07-4b9e-80b8-2903a03d4d53 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:41:22.523439 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2c47cc-e7b5-48fe-986b-470f069cdf0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:41:22.524242 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2c47cc-e7b5-48fe-986b-470f069cdf0c None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:41:22.524524 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2c47cc-e7b5-48fe-986b-470f069cdf0c None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:41:22.524799 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2c47cc-e7b5-48fe-986b-470f069cdf0c None admin] PATH_INFO: `/v3/users/placement` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:41:22.532324 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2c47cc-e7b5-48fe-986b-470f069cdf0c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:41:22.533768 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2c47cc-e7b5-48fe-986b-470f069cdf0c None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:41:22.549104 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-1e2c47cc-e7b5-48fe-986b-470f069cdf0c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 14:41:22.549104 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:41:22.549104 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:41:22.549104 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:41:22.549104 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:41:22.549104 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:41:22.549104 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:41:22.549104 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:41:22.549104 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:41:22.549104 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:41:22.549104 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:41:22.549104 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:41:22.549104 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:41:22.549104 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:41:22.549104 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:41:22.549104 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:41:22.550447 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:41:22.550447 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:41:22.550447 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:41:22.550447 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:41:22.550447 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:41:22.550447 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:41:22.550447 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:41:22.550447 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:41:22.550447 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:41:22.550447 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:41:22.550447 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:41:22.550447 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:41:22.550447 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:41:22.550447 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:41:22.550447 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:41:22.550447 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:41:22.550447 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:41:22.550447 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:41:22.551825 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:41:22.551825 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:41:22.551825 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:41:22.551825 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:41:22.551825 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:41:22.551825 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:41:22.551825 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:41:22.551825 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:41:22.551825 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 14:41:22.551825 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:41:22.555954 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 634/1226] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:41:22 2022] GET /identity/v3/users/placement => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:41:22.557277 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dadfc4a-a4c8-43f4-9c49-949f7c7edd3c None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:41:22.607214 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:41:22.667689 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dadfc4a-a4c8-43f4-9c49-949f7c7edd3c None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:41:22.708785 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec55a5f-d524-4786-97b7-93ab48ed51ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:41:22.709457 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec55a5f-d524-4786-97b7-93ab48ed51ef None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:41:22.709739 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec55a5f-d524-4786-97b7-93ab48ed51ef None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:41:22.710049 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec55a5f-d524-4786-97b7-93ab48ed51ef None admin] PATH_INFO: `/v3/users` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:41:22.710894 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec55a5f-d524-4786-97b7-93ab48ed51ef None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:41:22.711209 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec55a5f-d524-4786-97b7-93ab48ed51ef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:41:22.712644 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec55a5f-d524-4786-97b7-93ab48ed51ef None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:41:22.732003 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 593/1227] 10.222.0.57 () {56 vars in 1143 bytes} [Thu May 19 14:41:22 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:41:22.737687 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2c47cc-e7b5-48fe-986b-470f069cdf0c None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:41:22.861360 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2c47cc-e7b5-48fe-986b-470f069cdf0c None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:41:22.895359 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6f25e4-e252-4127-8155-c2ba9cb5cc02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:41:22.896691 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6f25e4-e252-4127-8155-c2ba9cb5cc02 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:41:22.896976 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6f25e4-e252-4127-8155-c2ba9cb5cc02 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:41:22.897250 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6f25e4-e252-4127-8155-c2ba9cb5cc02 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:41:22.908018 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6f25e4-e252-4127-8155-c2ba9cb5cc02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:41:22.909510 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6f25e4-e252-4127-8155-c2ba9cb5cc02 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:41:22.930371 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-6d6f25e4-e252-4127-8155-c2ba9cb5cc02 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:41:22.930371 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:41:22.930371 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:41:22.930371 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:41:22.930371 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:41:22.930371 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:41:22.930371 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:41:22.930371 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:41:22.930371 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:41:22.930371 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:41:22.930371 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:41:22.930371 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:41:22.930371 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:41:22.930371 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:41:22.930371 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:41:22.931632 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:41:22.931632 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:41:22.931632 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:41:22.931632 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:41:22.931632 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:41:22.931632 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:41:22.931632 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:41:22.931632 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:41:22.931632 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:41:22.931632 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:41:22.931632 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:41:22.931632 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:41:22.931632 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:41:22.931632 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:41:22.931632 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:41:22.931632 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:41:22.931632 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:41:22.933189 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:41:22.933189 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:41:22.933189 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:41:22.933189 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:41:22.933189 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 635/1228] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:41:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:41:22.937389 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec55a5f-d524-4786-97b7-93ab48ed51ef None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:41:23.045500 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec55a5f-d524-4786-97b7-93ab48ed51ef None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:41:23.048161 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:41:23.085662 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc95854-5218-44c5-bd52-b8f018da723d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:41:23.086425 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc95854-5218-44c5-bd52-b8f018da723d None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:41:23.086702 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc95854-5218-44c5-bd52-b8f018da723d None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:41:23.087028 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc95854-5218-44c5-bd52-b8f018da723d None admin] PATH_INFO: `/v3/domains` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:41:23.087796 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc95854-5218-44c5-bd52-b8f018da723d None admin] RBAC: Adding query filter params (name=Default) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:41:23.088100 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc95854-5218-44c5-bd52-b8f018da723d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:41:23.089485 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc95854-5218-44c5-bd52-b8f018da723d None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:41:23.102902 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 594/1229] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:41:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:41:23.108522 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6f25e4-e252-4127-8155-c2ba9cb5cc02 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:41:23.118890 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:41:23.228606 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6f25e4-e252-4127-8155-c2ba9cb5cc02 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:41:23.267406 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fc2c47-65c5-4f79-8833-6a850d84f7f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:41:23.268320 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fc2c47-65c5-4f79-8833-6a850d84f7f4 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:41:23.268615 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fc2c47-65c5-4f79-8833-6a850d84f7f4 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:41:23.269015 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fc2c47-65c5-4f79-8833-6a850d84f7f4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:41:23.278972 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fc2c47-65c5-4f79-8833-6a850d84f7f4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:41:23.279652 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fc2c47-65c5-4f79-8833-6a850d84f7f4 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:41:23.297991 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-79fc2c47-65c5-4f79-8833-6a850d84f7f4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:41:23.297991 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:41:23.297991 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:41:23.297991 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:41:23.297991 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:41:23.297991 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:41:23.297991 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:41:23.297991 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:41:23.297991 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:41:23.297991 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:41:23.297991 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:41:23.297991 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:41:23.297991 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:41:23.297991 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:41:23.297991 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:41:23.299050 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:41:23.299050 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:41:23.299050 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:41:23.299050 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:41:23.299050 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:41:23.299050 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:41:23.299050 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:41:23.299050 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:41:23.299050 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:41:23.299050 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:41:23.299050 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:41:23.299050 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:41:23.299050 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:41:23.299050 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:41:23.299050 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:41:23.299050 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:41:23.299050 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:41:23.300341 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:41:23.300341 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:41:23.300341 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:41:23.300341 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:41:23.300341 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:41:23.300341 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:41:23.300341 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:41:23.300341 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:41:23.300341 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 636/1230] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:41:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:41:23.305816 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc95854-5218-44c5-bd52-b8f018da723d None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:41:23.419060 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc95854-5218-44c5-bd52-b8f018da723d None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:41:23.455979 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e333bf-f87e-4f0a-85d9-99e8542c2d3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:41:23.456640 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e333bf-f87e-4f0a-85d9-99e8542c2d3f None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:41:23.456926 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e333bf-f87e-4f0a-85d9-99e8542c2d3f None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:41:23.457202 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e333bf-f87e-4f0a-85d9-99e8542c2d3f None admin] PATH_INFO: `/v3/projects` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:41:23.458051 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e333bf-f87e-4f0a-85d9-99e8542c2d3f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:41:23.458905 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e333bf-f87e-4f0a-85d9-99e8542c2d3f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:41:23.460335 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e333bf-f87e-4f0a-85d9-99e8542c2d3f None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:41:23.473318 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 595/1231] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:41:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:41:23.479973 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fc2c47-65c5-4f79-8833-6a850d84f7f4 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:41:23.555319 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:41:23.594590 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fc2c47-65c5-4f79-8833-6a850d84f7f4 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:41:23.637464 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f8d291-3ec5-4331-a669-c309e7801e15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:41:23.638424 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f8d291-3ec5-4331-a669-c309e7801e15 None admin] REQUEST_METHOD: `PUT` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:41:23.638711 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f8d291-3ec5-4331-a669-c309e7801e15 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:41:23.638990 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f8d291-3ec5-4331-a669-c309e7801e15 None admin] PATH_INFO: `/v3/projects/42f35b96b9d4499abfb5e5f7cca49adc/users/b230516fafae4ac092c036f3b9aeb047/roles/ffb8da90af53491980816087ee621ccb` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:41:23.655181 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f8d291-3ec5-4331-a669-c309e7801e15 None admin] RBAC: Authorizing `identity:create_grant(project_id=42f35b96b9d4499abfb5e5f7cca49adc, user_id=b230516fafae4ac092c036f3b9aeb047, role_id=ffb8da90af53491980816087ee621ccb)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:41:23.656657 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f8d291-3ec5-4331-a669-c309e7801e15 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:41:23.670848 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 637/1232] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 14:41:23 2022] PUT /identity/v3/projects/42f35b96b9d4499abfb5e5f7cca49adc/users/b230516fafae4ac092c036f3b9aeb047/roles/ffb8da90af53491980816087ee621ccb => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:41:26.116531 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9e825c-1aed-457d-abb3-91704160c4a6 None None] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:41:26.117104 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9e825c-1aed-457d-abb3-91704160c4a6 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:41:26.117104 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9e825c-1aed-457d-abb3-91704160c4a6 None None] PATH_INFO: `/` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:41:26.118501 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 596/1233] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:41:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:41:26.124766 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8850251-94d7-4740-b500-00d54ce01af8 None None] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:41:26.125273 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8850251-94d7-4740-b500-00d54ce01af8 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:41:26.125399 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8850251-94d7-4740-b500-00d54ce01af8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:41:26.142313 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.auth.core [None req-b8850251-94d7-4740-b500-00d54ce01af8 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:41:26.143498 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.fernet_utils [None req-b8850251-94d7-4740-b500-00d54ce01af8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:41:26.219923 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 638/1234] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:41:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:41:26.244066 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9e825c-1aed-457d-abb3-91704160c4a6 None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:41:26.272138 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:41:26.357566 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9e825c-1aed-457d-abb3-91704160c4a6 None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:41:26.394656 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fde28d-3f0c-42d0-b433-5752b521b734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:41:26.395457 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fde28d-3f0c-42d0-b433-5752b521b734 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:41:26.395894 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fde28d-3f0c-42d0-b433-5752b521b734 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:41:26.396116 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fde28d-3f0c-42d0-b433-5752b521b734 None admin] PATH_INFO: `/v3/roles/service` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:41:26.406851 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fde28d-3f0c-42d0-b433-5752b521b734 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:41:26.414922 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-34fde28d-3f0c-42d0-b433-5752b521b734 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:41:26.414922 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:41:26.414922 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:41:26.414922 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:41:26.414922 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:41:26.414922 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:41:26.414922 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:41:26.414922 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:41:26.414922 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:41:26.414922 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:41:26.414922 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:41:26.414922 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:41:26.414922 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:41:26.414922 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:41:26.414922 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:41:26.414922 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:41:26.421732 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:41:26.421732 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:41:26.421732 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:41:26.421732 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:41:26.421732 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:41:26.421732 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:41:26.421732 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:41:26.421732 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:41:26.421732 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:41:26.421732 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:41:26.421732 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:41:26.421732 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:41:26.421732 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:41:26.421732 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:41:26.421732 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:41:26.421732 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:41:26.423032 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:41:26.423032 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:41:26.423032 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:41:26.423032 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:41:26.423032 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:41:26.423032 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:41:26.423032 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:41:26.423032 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:41:26.423032 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:41:26.423032 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:41:26.423032 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:41:26.423032 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:41:26.423032 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 597/1235] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 14:41:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:41:26.424095 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8850251-94d7-4740-b500-00d54ce01af8 None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:41:26.475875 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:41:26.539342 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8850251-94d7-4740-b500-00d54ce01af8 None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:41:26.581110 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b303c5c-583d-491f-b08e-f4152c6abab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:41:26.581792 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b303c5c-583d-491f-b08e-f4152c6abab6 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:41:26.582153 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b303c5c-583d-491f-b08e-f4152c6abab6 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:41:26.582436 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b303c5c-583d-491f-b08e-f4152c6abab6 None admin] PATH_INFO: `/v3/roles` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:41:26.583605 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b303c5c-583d-491f-b08e-f4152c6abab6 None admin] RBAC: Adding query filter params (name=service) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:41:26.583922 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b303c5c-583d-491f-b08e-f4152c6abab6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:41:26.585312 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b303c5c-583d-491f-b08e-f4152c6abab6 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:41:26.594209 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 639/1236] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 19 14:41:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:41:26.600356 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fde28d-3f0c-42d0-b433-5752b521b734 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:41:26.694382 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:41:26.712147 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fde28d-3f0c-42d0-b433-5752b521b734 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:41:26.749896 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d837e90-e540-44e8-94bb-8e84e052cbc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:41:26.750764 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d837e90-e540-44e8-94bb-8e84e052cbc4 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:41:26.751057 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d837e90-e540-44e8-94bb-8e84e052cbc4 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:41:26.751336 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d837e90-e540-44e8-94bb-8e84e052cbc4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:41:26.763587 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d837e90-e540-44e8-94bb-8e84e052cbc4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:41:26.765101 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d837e90-e540-44e8-94bb-8e84e052cbc4 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:41:26.783001 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-5d837e90-e540-44e8-94bb-8e84e052cbc4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:41:26.783001 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:41:26.783001 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:41:26.783001 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:41:26.783001 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:41:26.783001 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:41:26.783001 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:41:26.783001 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:41:26.783001 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:41:26.783001 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:41:26.783001 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:41:26.783001 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:41:26.783001 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:41:26.783001 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:41:26.783001 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:41:26.784084 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:41:26.784084 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:41:26.784084 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:41:26.784084 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:41:26.784084 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:41:26.784084 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:41:26.784084 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:41:26.784084 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:41:26.784084 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:41:26.784084 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:41:26.784084 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:41:26.784084 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:41:26.784084 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:41:26.784084 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:41:26.784084 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:41:26.784084 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:41:26.784084 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:41:26.785126 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:41:26.785126 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:41:26.785126 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:41:26.785126 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:41:26.785126 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 598/1237] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:41:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:41:26.791087 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b303c5c-583d-491f-b08e-f4152c6abab6 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:41:26.909920 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b303c5c-583d-491f-b08e-f4152c6abab6 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:41:26.921419 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:41:26.955111 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c21f610-2a5b-489d-af5b-6c0a25965294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:41:26.955707 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c21f610-2a5b-489d-af5b-6c0a25965294 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:41:26.956004 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c21f610-2a5b-489d-af5b-6c0a25965294 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:41:26.956284 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c21f610-2a5b-489d-af5b-6c0a25965294 None admin] PATH_INFO: `/v3/domains` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:41:26.957074 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c21f610-2a5b-489d-af5b-6c0a25965294 None admin] RBAC: Adding query filter params (name=Default) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:41:26.957415 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c21f610-2a5b-489d-af5b-6c0a25965294 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:41:26.959790 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c21f610-2a5b-489d-af5b-6c0a25965294 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:41:26.971989 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 640/1238] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:41:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:41:26.978300 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d837e90-e540-44e8-94bb-8e84e052cbc4 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:41:27.094895 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d837e90-e540-44e8-94bb-8e84e052cbc4 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:41:27.135208 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07696385-508d-4a88-86e1-67691f2e9d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:41:27.136069 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07696385-508d-4a88-86e1-67691f2e9d34 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:41:27.136308 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07696385-508d-4a88-86e1-67691f2e9d34 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:41:27.136617 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07696385-508d-4a88-86e1-67691f2e9d34 None admin] PATH_INFO: `/v3/users/placement` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:41:27.140328 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:41:27.146359 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07696385-508d-4a88-86e1-67691f2e9d34 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:41:27.147857 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07696385-508d-4a88-86e1-67691f2e9d34 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:41:27.166487 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-07696385-508d-4a88-86e1-67691f2e9d34 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 14:41:27.166487 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:41:27.166487 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:41:27.166487 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:41:27.166487 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:41:27.166487 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:41:27.166487 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:41:27.166487 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:41:27.166487 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:41:27.166487 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:41:27.166487 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:41:27.166487 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:41:27.166487 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:41:27.166487 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:41:27.166487 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:41:27.166487 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:41:27.167701 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:41:27.167701 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:41:27.167701 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:41:27.167701 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:41:27.167701 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:41:27.167701 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:41:27.167701 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:41:27.167701 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:41:27.167701 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:41:27.167701 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:41:27.167701 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:41:27.167701 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:41:27.167701 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:41:27.167701 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:41:27.167701 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:41:27.167701 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:41:27.167701 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:41:27.167701 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:41:27.169288 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:41:27.169288 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:41:27.169288 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:41:27.169288 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:41:27.169288 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:41:27.169288 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:41:27.169288 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:41:27.169288 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:41:27.169288 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 14:41:27.169288 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:41:27.169288 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 599/1239] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:41:26 2022] GET /identity/v3/users/placement => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:41:27.173609 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c21f610-2a5b-489d-af5b-6c0a25965294 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:41:27.285792 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c21f610-2a5b-489d-af5b-6c0a25965294 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:41:27.324246 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99407927-fe44-49a9-b49b-11e469857a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:41:27.325002 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99407927-fe44-49a9-b49b-11e469857a69 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:41:27.325340 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99407927-fe44-49a9-b49b-11e469857a69 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:41:27.325623 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99407927-fe44-49a9-b49b-11e469857a69 None admin] PATH_INFO: `/v3/users` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:41:27.326522 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99407927-fe44-49a9-b49b-11e469857a69 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:41:27.326871 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99407927-fe44-49a9-b49b-11e469857a69 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:41:27.328517 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99407927-fe44-49a9-b49b-11e469857a69 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:41:27.350462 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 641/1240] 10.222.0.57 () {56 vars in 1143 bytes} [Thu May 19 14:41:27 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:41:27.356740 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07696385-508d-4a88-86e1-67691f2e9d34 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:41:27.479698 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07696385-508d-4a88-86e1-67691f2e9d34 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:41:27.516617 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8143dd-4da9-47c3-9ba2-2a8fe9ad83c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:41:27.517268 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8143dd-4da9-47c3-9ba2-2a8fe9ad83c4 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:41:27.517554 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8143dd-4da9-47c3-9ba2-2a8fe9ad83c4 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:41:27.517832 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8143dd-4da9-47c3-9ba2-2a8fe9ad83c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:41:27.606118 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:41:27.614794 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8143dd-4da9-47c3-9ba2-2a8fe9ad83c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:41:27.616415 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8143dd-4da9-47c3-9ba2-2a8fe9ad83c4 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:41:27.724033 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 600/1241] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:41:27 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:41:27.730104 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99407927-fe44-49a9-b49b-11e469857a69 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:41:27.760647 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:41:27.841377 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99407927-fe44-49a9-b49b-11e469857a69 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:41:27.881896 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904ed5e6-9f4a-45d9-adc3-7447189f3f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:41:27.882868 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904ed5e6-9f4a-45d9-adc3-7447189f3f0f None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:41:27.883136 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904ed5e6-9f4a-45d9-adc3-7447189f3f0f None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:41:27.883452 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904ed5e6-9f4a-45d9-adc3-7447189f3f0f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:41:27.896383 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904ed5e6-9f4a-45d9-adc3-7447189f3f0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:41:27.897922 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904ed5e6-9f4a-45d9-adc3-7447189f3f0f None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:41:27.918470 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-904ed5e6-9f4a-45d9-adc3-7447189f3f0f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:41:27.918470 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:41:27.918470 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:41:27.918470 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:41:27.918470 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:41:27.918470 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:41:27.918470 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:41:27.918470 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:41:27.918470 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:41:27.918470 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:41:27.918470 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:41:27.918470 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:41:27.918470 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:41:27.918470 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:41:27.918470 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:41:27.919796 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:41:27.919796 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:41:27.919796 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:41:27.919796 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:41:27.919796 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:41:27.919796 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:41:27.919796 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:41:27.919796 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:41:27.919796 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:41:27.919796 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:41:27.919796 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:41:27.919796 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:41:27.919796 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:41:27.919796 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:41:27.919796 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:41:27.919796 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:41:27.919796 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:41:27.921013 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:41:27.921013 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:41:27.921013 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:41:27.921013 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:41:27.921013 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 642/1242] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:41:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:41:27.926191 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8143dd-4da9-47c3-9ba2-2a8fe9ad83c4 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:41:28.041909 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:41:28.049320 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8143dd-4da9-47c3-9ba2-2a8fe9ad83c4 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:41:28.086432 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf7434e-5802-4990-8fe9-4cfe71329e5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:41:28.087102 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf7434e-5802-4990-8fe9-4cfe71329e5b None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:41:28.087391 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf7434e-5802-4990-8fe9-4cfe71329e5b None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:41:28.087673 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf7434e-5802-4990-8fe9-4cfe71329e5b None admin] PATH_INFO: `/v3/domains` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:41:28.088402 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf7434e-5802-4990-8fe9-4cfe71329e5b None admin] RBAC: Adding query filter params (name=Default) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:41:28.088713 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf7434e-5802-4990-8fe9-4cfe71329e5b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:41:28.090126 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf7434e-5802-4990-8fe9-4cfe71329e5b None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:41:28.102597 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 601/1243] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:41:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:41:28.108330 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904ed5e6-9f4a-45d9-adc3-7447189f3f0f None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:41:28.193682 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:41:28.222285 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904ed5e6-9f4a-45d9-adc3-7447189f3f0f None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:41:28.270264 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f8d07c-be27-4983-bace-a5d0a1b0f6ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:41:28.271189 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f8d07c-be27-4983-bace-a5d0a1b0f6ec None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:41:28.271535 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f8d07c-be27-4983-bace-a5d0a1b0f6ec None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:41:28.271836 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f8d07c-be27-4983-bace-a5d0a1b0f6ec None admin] PATH_INFO: `/v3/projects/service` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:41:28.279719 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f8d07c-be27-4983-bace-a5d0a1b0f6ec None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:41:28.281171 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f8d07c-be27-4983-bace-a5d0a1b0f6ec None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:41:28.295863 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-04f8d07c-be27-4983-bace-a5d0a1b0f6ec None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:41:28.295863 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:41:28.295863 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:41:28.295863 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:41:28.295863 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:41:28.295863 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:41:28.295863 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:41:28.295863 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:41:28.295863 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:41:28.295863 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:41:28.295863 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:41:28.295863 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:41:28.295863 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:41:28.295863 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:41:28.295863 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:41:28.297356 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:41:28.297356 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:41:28.297356 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:41:28.297356 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:41:28.297356 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:41:28.297356 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:41:28.297356 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:41:28.297356 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:41:28.297356 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:41:28.297356 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:41:28.297356 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:41:28.297356 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:41:28.297356 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:41:28.297356 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:41:28.297356 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:41:28.297356 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:41:28.297356 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:41:28.298717 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:41:28.298717 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:41:28.298717 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:41:28.298717 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:41:28.298717 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:41:28.298717 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:41:28.298717 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:41:28.298717 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:41:28.298717 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 643/1244] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:41:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:41:28.303657 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf7434e-5802-4990-8fe9-4cfe71329e5b None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:41:28.415618 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf7434e-5802-4990-8fe9-4cfe71329e5b None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:41:28.455192 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8b60c4-d4cf-4538-a883-d4afd889e5d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:41:28.455915 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8b60c4-d4cf-4538-a883-d4afd889e5d3 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:41:28.456311 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8b60c4-d4cf-4538-a883-d4afd889e5d3 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:41:28.456623 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8b60c4-d4cf-4538-a883-d4afd889e5d3 None admin] PATH_INFO: `/v3/projects` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:41:28.457421 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8b60c4-d4cf-4538-a883-d4afd889e5d3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:41:28.457826 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8b60c4-d4cf-4538-a883-d4afd889e5d3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:41:28.460457 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8b60c4-d4cf-4538-a883-d4afd889e5d3 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:41:28.474756 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 602/1245] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:41:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:41:28.480972 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f8d07c-be27-4983-bace-a5d0a1b0f6ec None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:41:28.595707 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f8d07c-be27-4983-bace-a5d0a1b0f6ec None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:41:28.633884 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aa3fa4-a980-4d76-bd60-8ae24eec6f4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:41:28.634572 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aa3fa4-a980-4d76-bd60-8ae24eec6f4e None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:41:28.634854 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aa3fa4-a980-4d76-bd60-8ae24eec6f4e None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:41:28.635129 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aa3fa4-a980-4d76-bd60-8ae24eec6f4e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:41:28.635968 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8aa3fa4-a980-4d76-bd60-8ae24eec6f4e None admin] RBAC: Adding query filter params (role.id=ffb8da90af53491980816087ee621ccb, scope.project.id=42f35b96b9d4499abfb5e5f7cca49adc, user.id=b230516fafae4ac092c036f3b9aeb047) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:41:28.636327 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8aa3fa4-a980-4d76-bd60-8ae24eec6f4e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:41:28.637774 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8aa3fa4-a980-4d76-bd60-8ae24eec6f4e None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:41:28.643887 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 644/1246] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:41:28 2022] GET /identity/v3/role_assignments?user.id=b230516fafae4ac092c036f3b9aeb047&scope.project.id=42f35b96b9d4499abfb5e5f7cca49adc&role.id=ffb8da90af53491980816087ee621ccb => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:41:31.173250 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8acc1a-0ff3-4ea2-aa80-7cfa8a999554 None None] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:41:31.173826 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8acc1a-0ff3-4ea2-aa80-7cfa8a999554 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:41:31.173963 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8acc1a-0ff3-4ea2-aa80-7cfa8a999554 None None] PATH_INFO: `/` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:41:31.175734 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 603/1247] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:41:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:41:31.181905 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7716e91-e267-40e7-832e-8ec8d361f98b None None] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:41:31.182398 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7716e91-e267-40e7-832e-8ec8d361f98b None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:41:31.182494 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7716e91-e267-40e7-832e-8ec8d361f98b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:41:31.188514 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:41:31.203474 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.auth.core [None req-a7716e91-e267-40e7-832e-8ec8d361f98b None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:41:31.204809 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.fernet_utils [None req-a7716e91-e267-40e7-832e-8ec8d361f98b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:41:31.282345 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 645/1248] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:41:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:41:31.307690 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8acc1a-0ff3-4ea2-aa80-7cfa8a999554 None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:41:31.328018 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:41:31.426180 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8acc1a-0ff3-4ea2-aa80-7cfa8a999554 None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:41:31.470576 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11872f94-6ba3-437d-b55a-74e355e1de37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:41:31.471358 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11872f94-6ba3-437d-b55a-74e355e1de37 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:41:31.471640 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11872f94-6ba3-437d-b55a-74e355e1de37 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:41:31.471918 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11872f94-6ba3-437d-b55a-74e355e1de37 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:41:31.483409 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11872f94-6ba3-437d-b55a-74e355e1de37 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:41:31.491594 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-11872f94-6ba3-437d-b55a-74e355e1de37 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:41:31.491594 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:41:31.491594 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:41:31.491594 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:41:31.491594 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:41:31.491594 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:41:31.491594 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:41:31.491594 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:41:31.491594 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:41:31.491594 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:41:31.491594 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:41:31.491594 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:41:31.491594 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:41:31.491594 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:41:31.491594 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:41:31.491594 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:41:31.492930 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:41:31.492930 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:41:31.492930 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:41:31.492930 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:41:31.492930 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:41:31.492930 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:41:31.492930 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:41:31.492930 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:41:31.492930 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:41:31.492930 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:41:31.492930 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:41:31.492930 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:41:31.492930 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:41:31.492930 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:41:31.492930 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:41:31.492930 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:41:31.494343 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:41:31.494343 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:41:31.494343 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:41:31.494343 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:41:31.494343 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:41:31.494343 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:41:31.494343 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:41:31.494343 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:41:31.494343 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:41:31.494343 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:41:31.494343 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:41:31.494343 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:41:31.494343 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 604/1249] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:41:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:41:31.499233 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7716e91-e267-40e7-832e-8ec8d361f98b None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:41:31.610952 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7716e91-e267-40e7-832e-8ec8d361f98b None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:41:31.646979 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:41:31.653030 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1279b2a0-9152-4214-ab01-5c16b4e8ce88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:41:31.653757 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1279b2a0-9152-4214-ab01-5c16b4e8ce88 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:41:31.654088 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1279b2a0-9152-4214-ab01-5c16b4e8ce88 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:41:31.654396 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1279b2a0-9152-4214-ab01-5c16b4e8ce88 None admin] PATH_INFO: `/v3/roles` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:41:31.655210 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1279b2a0-9152-4214-ab01-5c16b4e8ce88 None admin] RBAC: Adding query filter params (name=admin) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:41:31.655515 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1279b2a0-9152-4214-ab01-5c16b4e8ce88 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:41:31.657048 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1279b2a0-9152-4214-ab01-5c16b4e8ce88 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:41:31.666930 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 646/1250] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:41:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:41:31.673426 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11872f94-6ba3-437d-b55a-74e355e1de37 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:41:31.761709 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:41:31.787040 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11872f94-6ba3-437d-b55a-74e355e1de37 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:41:31.825862 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b5e07a-d3e1-4bcd-b191-87f56aea20a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:41:31.826869 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b5e07a-d3e1-4bcd-b191-87f56aea20a3 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:41:31.827216 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b5e07a-d3e1-4bcd-b191-87f56aea20a3 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:41:31.827514 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b5e07a-d3e1-4bcd-b191-87f56aea20a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:41:31.842588 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b5e07a-d3e1-4bcd-b191-87f56aea20a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:41:31.844275 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b5e07a-d3e1-4bcd-b191-87f56aea20a3 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:41:31.867830 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-c3b5e07a-d3e1-4bcd-b191-87f56aea20a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:41:31.867830 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:41:31.867830 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:41:31.867830 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:41:31.867830 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:41:31.867830 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:41:31.867830 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:41:31.867830 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:41:31.867830 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:41:31.867830 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:41:31.867830 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:41:31.867830 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:41:31.867830 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:41:31.867830 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:41:31.867830 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:41:31.869159 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:41:31.869159 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:41:31.869159 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:41:31.869159 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:41:31.869159 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:41:31.869159 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:41:31.869159 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:41:31.869159 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:41:31.869159 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:41:31.869159 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:41:31.869159 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:41:31.869159 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:41:31.869159 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:41:31.869159 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:41:31.869159 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:41:31.869159 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:41:31.869159 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:41:31.870398 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:41:31.870398 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:41:31.870398 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:41:31.870398 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:41:31.870398 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 605/1251] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:41:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:41:31.876246 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1279b2a0-9152-4214-ab01-5c16b4e8ce88 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:41:32.001007 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1279b2a0-9152-4214-ab01-5c16b4e8ce88 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:41:32.035350 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f7094a-e8a1-4697-b40c-adcaec94d180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:41:32.036471 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f7094a-e8a1-4697-b40c-adcaec94d180 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:41:32.036868 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f7094a-e8a1-4697-b40c-adcaec94d180 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:41:32.037151 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f7094a-e8a1-4697-b40c-adcaec94d180 None admin] PATH_INFO: `/v3/domains` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:41:32.037905 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f7094a-e8a1-4697-b40c-adcaec94d180 None admin] RBAC: Adding query filter params (name=Default) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:41:32.038216 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f7094a-e8a1-4697-b40c-adcaec94d180 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:41:32.039647 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f7094a-e8a1-4697-b40c-adcaec94d180 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:41:32.053883 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 647/1252] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:41:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:41:32.059531 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b5e07a-d3e1-4bcd-b191-87f56aea20a3 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:41:32.185344 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b5e07a-d3e1-4bcd-b191-87f56aea20a3 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:41:32.215259 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:41:32.225966 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2efbc6-cd1a-4b41-9373-34174a844df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:41:32.226779 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2efbc6-cd1a-4b41-9373-34174a844df3 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:41:32.227063 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2efbc6-cd1a-4b41-9373-34174a844df3 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:41:32.227340 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2efbc6-cd1a-4b41-9373-34174a844df3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:41:32.236882 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2efbc6-cd1a-4b41-9373-34174a844df3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:41:32.238639 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2efbc6-cd1a-4b41-9373-34174a844df3 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:41:32.256009 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-6b2efbc6-cd1a-4b41-9373-34174a844df3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 14:41:32.256009 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:41:32.256009 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:41:32.256009 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:41:32.256009 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:41:32.256009 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:41:32.256009 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:41:32.256009 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:41:32.256009 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:41:32.256009 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:41:32.256009 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:41:32.256009 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:41:32.256009 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:41:32.256009 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:41:32.256009 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:41:32.256009 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:41:32.257122 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:41:32.257122 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:41:32.257122 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:41:32.257122 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:41:32.257122 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:41:32.257122 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:41:32.257122 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:41:32.257122 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:41:32.257122 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:41:32.257122 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:41:32.257122 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:41:32.257122 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:41:32.257122 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:41:32.257122 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:41:32.257122 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:41:32.257122 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:41:32.257122 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:41:32.257122 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:41:32.258325 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:41:32.258325 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:41:32.258325 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:41:32.258325 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:41:32.258325 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:41:32.258325 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:41:32.258325 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:41:32.258325 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:41:32.258325 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 14:41:32.258325 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:41:32.258325 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 606/1253] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:41:32 2022] GET /identity/v3/users/placement => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:41:32.264006 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f7094a-e8a1-4697-b40c-adcaec94d180 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:41:32.328157 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:41:32.379578 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f7094a-e8a1-4697-b40c-adcaec94d180 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:41:32.417379 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745955ff-d2e9-4554-8c29-cb167a6dae74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:41:32.418412 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745955ff-d2e9-4554-8c29-cb167a6dae74 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:41:32.418412 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745955ff-d2e9-4554-8c29-cb167a6dae74 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:41:32.418652 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745955ff-d2e9-4554-8c29-cb167a6dae74 None admin] PATH_INFO: `/v3/users` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:41:32.419456 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745955ff-d2e9-4554-8c29-cb167a6dae74 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:41:32.419758 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745955ff-d2e9-4554-8c29-cb167a6dae74 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:41:32.421188 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745955ff-d2e9-4554-8c29-cb167a6dae74 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:41:32.441952 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 648/1254] 10.222.0.57 () {56 vars in 1143 bytes} [Thu May 19 14:41:32 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:41:32.448420 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2efbc6-cd1a-4b41-9373-34174a844df3 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:41:32.560128 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2efbc6-cd1a-4b41-9373-34174a844df3 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:41:32.599195 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99feae6b-e419-4cd8-834c-eac9ee835202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:41:32.599921 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99feae6b-e419-4cd8-834c-eac9ee835202 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:41:32.600329 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99feae6b-e419-4cd8-834c-eac9ee835202 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:41:32.600596 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99feae6b-e419-4cd8-834c-eac9ee835202 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:41:32.666234 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:41:32.682897 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99feae6b-e419-4cd8-834c-eac9ee835202 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:41:32.684546 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99feae6b-e419-4cd8-834c-eac9ee835202 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:41:32.794919 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 607/1255] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:41:32 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:41:32.800716 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745955ff-d2e9-4554-8c29-cb167a6dae74 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:41:32.913503 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745955ff-d2e9-4554-8c29-cb167a6dae74 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:41:32.927634 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:41:32.951737 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c676d976-5a82-46e9-9304-ea8babb34615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:41:32.952531 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c676d976-5a82-46e9-9304-ea8babb34615 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:41:32.952808 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c676d976-5a82-46e9-9304-ea8babb34615 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:41:32.953106 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c676d976-5a82-46e9-9304-ea8babb34615 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:41:32.963960 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c676d976-5a82-46e9-9304-ea8babb34615 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:41:32.965506 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c676d976-5a82-46e9-9304-ea8babb34615 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:41:32.985517 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-c676d976-5a82-46e9-9304-ea8babb34615 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:41:32.985517 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:41:32.985517 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:41:32.985517 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:41:32.985517 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:41:32.985517 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:41:32.985517 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:41:32.985517 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:41:32.985517 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:41:32.985517 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:41:32.985517 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:41:32.985517 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:41:32.985517 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:41:32.985517 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:41:32.985517 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:41:32.986954 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:41:32.986954 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:41:32.986954 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:41:32.986954 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:41:32.986954 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:41:32.986954 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:41:32.986954 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:41:32.986954 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:41:32.986954 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:41:32.986954 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:41:32.986954 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:41:32.986954 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:41:32.986954 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:41:32.986954 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:41:32.986954 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:41:32.986954 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:41:32.986954 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:41:32.988196 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:41:32.988196 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:41:32.988196 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:41:32.988196 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:41:32.988446 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 649/1256] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:41:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:41:32.994467 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99feae6b-e419-4cd8-834c-eac9ee835202 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:41:33.091225 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:41:33.106434 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99feae6b-e419-4cd8-834c-eac9ee835202 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:41:33.150892 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad895600-1cd4-4f45-99ce-3214f059b7a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:41:33.151570 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad895600-1cd4-4f45-99ce-3214f059b7a8 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:41:33.151852 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad895600-1cd4-4f45-99ce-3214f059b7a8 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:41:33.152124 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad895600-1cd4-4f45-99ce-3214f059b7a8 None admin] PATH_INFO: `/v3/domains` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:41:33.152859 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad895600-1cd4-4f45-99ce-3214f059b7a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:41:33.153162 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad895600-1cd4-4f45-99ce-3214f059b7a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:41:33.154629 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad895600-1cd4-4f45-99ce-3214f059b7a8 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:41:33.171463 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 608/1257] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:41:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:41:33.178784 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c676d976-5a82-46e9-9304-ea8babb34615 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:41:33.291093 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c676d976-5a82-46e9-9304-ea8babb34615 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:41:33.329488 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ffb931-8f24-4e7d-b5e9-9385eb67926a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:41:33.330345 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ffb931-8f24-4e7d-b5e9-9385eb67926a None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:41:33.330630 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ffb931-8f24-4e7d-b5e9-9385eb67926a None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:41:33.330906 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ffb931-8f24-4e7d-b5e9-9385eb67926a None admin] PATH_INFO: `/v3/projects/service` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:41:33.338975 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ffb931-8f24-4e7d-b5e9-9385eb67926a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:41:33.340475 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ffb931-8f24-4e7d-b5e9-9385eb67926a None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:41:33.355747 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-74ffb931-8f24-4e7d-b5e9-9385eb67926a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:41:33.355747 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:41:33.355747 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:41:33.355747 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:41:33.355747 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:41:33.355747 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:41:33.355747 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:41:33.355747 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:41:33.355747 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:41:33.355747 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:41:33.355747 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:41:33.355747 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:41:33.355747 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:41:33.355747 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:41:33.355747 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:41:33.357003 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:41:33.357003 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:41:33.357003 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:41:33.357003 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:41:33.357003 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:41:33.357003 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:41:33.357003 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:41:33.357003 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:41:33.357003 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:41:33.357003 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:41:33.357003 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:41:33.357003 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:41:33.357003 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:41:33.357003 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:41:33.357003 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:41:33.357003 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:41:33.357003 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:41:33.358111 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:41:33.358111 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:41:33.358111 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:41:33.358111 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:41:33.358111 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:41:33.358111 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:41:33.358111 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:41:33.358111 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:41:33.358111 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 650/1258] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:41:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:41:33.363314 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad895600-1cd4-4f45-99ce-3214f059b7a8 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:41:33.471123 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad895600-1cd4-4f45-99ce-3214f059b7a8 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:41:33.506926 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9105458e-9c13-4a5f-9760-9702f32b7a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:41:33.507632 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9105458e-9c13-4a5f-9760-9702f32b7a9a None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:41:33.507915 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9105458e-9c13-4a5f-9760-9702f32b7a9a None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:41:33.508188 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9105458e-9c13-4a5f-9760-9702f32b7a9a None admin] PATH_INFO: `/v3/projects` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:41:33.508988 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9105458e-9c13-4a5f-9760-9702f32b7a9a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:41:33.509301 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9105458e-9c13-4a5f-9760-9702f32b7a9a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:41:33.510804 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9105458e-9c13-4a5f-9760-9702f32b7a9a None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:41:33.524477 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 609/1259] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:41:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:41:33.530752 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ffb931-8f24-4e7d-b5e9-9385eb67926a None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:41:33.559181 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:41:33.637386 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ffb931-8f24-4e7d-b5e9-9385eb67926a None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:41:33.673568 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffd3e82-112d-44e4-99d3-c115bff64b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:41:33.674263 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffd3e82-112d-44e4-99d3-c115bff64b22 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:41:33.674549 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffd3e82-112d-44e4-99d3-c115bff64b22 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:41:33.674854 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffd3e82-112d-44e4-99d3-c115bff64b22 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:41:33.675672 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffd3e82-112d-44e4-99d3-c115bff64b22 None admin] RBAC: Adding query filter params (role.id=c2935b1a25ca46fcba419fe3d1715330, scope.project.id=42f35b96b9d4499abfb5e5f7cca49adc, user.id=b230516fafae4ac092c036f3b9aeb047) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:41:33.675982 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffd3e82-112d-44e4-99d3-c115bff64b22 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:41:33.677585 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffd3e82-112d-44e4-99d3-c115bff64b22 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:41:33.683184 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 651/1260] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:41:33 2022] GET /identity/v3/role_assignments?user.id=b230516fafae4ac092c036f3b9aeb047&scope.project.id=42f35b96b9d4499abfb5e5f7cca49adc&role.id=c2935b1a25ca46fcba419fe3d1715330 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:41:35.916728 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849dfce6-8d4c-4053-ac26-3768b8192e6b None None] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:41:35.917306 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849dfce6-8d4c-4053-ac26-3768b8192e6b None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:41:35.917429 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849dfce6-8d4c-4053-ac26-3768b8192e6b None None] PATH_INFO: `/` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:41:35.918769 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 610/1261] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:41:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:41:35.924115 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d0c580-8c72-491e-b77b-fa8b6ae2d6a8 None None] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:41:35.924522 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d0c580-8c72-491e-b77b-fa8b6ae2d6a8 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:41:35.924666 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d0c580-8c72-491e-b77b-fa8b6ae2d6a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:41:35.942012 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.auth.core [None req-f3d0c580-8c72-491e-b77b-fa8b6ae2d6a8 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:41:35.943904 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.fernet_utils [None req-f3d0c580-8c72-491e-b77b-fa8b6ae2d6a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:41:36.011250 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 652/1262] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:41:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:41:36.033741 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849dfce6-8d4c-4053-ac26-3768b8192e6b None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:41:36.044642 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:41:36.136987 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849dfce6-8d4c-4053-ac26-3768b8192e6b None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:41:36.171033 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed9d972-7aca-4835-9449-16a1c686daac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:41:36.171764 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed9d972-7aca-4835-9449-16a1c686daac None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:41:36.172046 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed9d972-7aca-4835-9449-16a1c686daac None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:41:36.172320 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed9d972-7aca-4835-9449-16a1c686daac None admin] PATH_INFO: `/v3/roles/admin` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:41:36.183066 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed9d972-7aca-4835-9449-16a1c686daac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:41:36.190597 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-4ed9d972-7aca-4835-9449-16a1c686daac None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:41:36.190597 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:41:36.190597 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:41:36.190597 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:41:36.190597 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:41:36.190597 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:41:36.190597 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:41:36.190597 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:41:36.190597 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:41:36.190597 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:41:36.190597 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:41:36.190597 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:41:36.190597 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:41:36.190597 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:41:36.190597 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:41:36.190597 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:41:36.191820 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:41:36.191820 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:41:36.191820 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:41:36.191820 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:41:36.191820 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:41:36.191820 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:41:36.191820 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:41:36.191820 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:41:36.191820 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:41:36.191820 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:41:36.191820 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:41:36.191820 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:41:36.191820 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:41:36.191820 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:41:36.191820 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:41:36.191820 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:41:36.193012 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:41:36.193012 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:41:36.193012 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:41:36.193012 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:41:36.193012 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:41:36.193012 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:41:36.193012 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:41:36.193012 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:41:36.193012 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:41:36.193012 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:41:36.193012 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:41:36.193012 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:41:36.193012 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 611/1263] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:41:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:41:36.197860 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d0c580-8c72-491e-b77b-fa8b6ae2d6a8 None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:41:36.215981 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:41:36.294514 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d0c580-8c72-491e-b77b-fa8b6ae2d6a8 None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:41:36.328177 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dc48ce-5bd4-4551-803a-0d0ef2f440f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:41:36.328828 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dc48ce-5bd4-4551-803a-0d0ef2f440f9 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:41:36.329108 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dc48ce-5bd4-4551-803a-0d0ef2f440f9 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:41:36.329378 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dc48ce-5bd4-4551-803a-0d0ef2f440f9 None admin] PATH_INFO: `/v3/roles` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:41:36.330140 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dc48ce-5bd4-4551-803a-0d0ef2f440f9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:41:36.330445 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dc48ce-5bd4-4551-803a-0d0ef2f440f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:41:36.331824 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dc48ce-5bd4-4551-803a-0d0ef2f440f9 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:41:36.340419 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 653/1264] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:41:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:41:36.346321 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed9d972-7aca-4835-9449-16a1c686daac None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:41:36.412572 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:41:36.446538 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed9d972-7aca-4835-9449-16a1c686daac None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:41:36.479508 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1de8006-fd2b-4482-aed1-c942d1c61400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:41:36.480278 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1de8006-fd2b-4482-aed1-c942d1c61400 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:41:36.480562 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1de8006-fd2b-4482-aed1-c942d1c61400 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:41:36.480837 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1de8006-fd2b-4482-aed1-c942d1c61400 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:41:36.491839 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1de8006-fd2b-4482-aed1-c942d1c61400 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:41:36.493285 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1de8006-fd2b-4482-aed1-c942d1c61400 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:41:36.510875 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-d1de8006-fd2b-4482-aed1-c942d1c61400 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:41:36.510875 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:41:36.510875 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:41:36.510875 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:41:36.510875 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:41:36.510875 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:41:36.510875 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:41:36.510875 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:41:36.510875 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:41:36.510875 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:41:36.510875 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:41:36.510875 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:41:36.510875 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:41:36.510875 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:41:36.510875 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:41:36.512336 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:41:36.512336 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:41:36.512336 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:41:36.512336 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:41:36.512336 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:41:36.512336 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:41:36.512336 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:41:36.512336 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:41:36.512336 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:41:36.512336 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:41:36.512336 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:41:36.512336 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:41:36.512336 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:41:36.512336 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:41:36.512336 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:41:36.512336 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:41:36.512336 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:41:36.513642 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:41:36.513642 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:41:36.513642 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:41:36.513642 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:41:36.513642 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 612/1265] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:41:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:41:36.517905 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dc48ce-5bd4-4551-803a-0d0ef2f440f9 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:41:36.598707 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:41:36.618207 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dc48ce-5bd4-4551-803a-0d0ef2f440f9 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:41:36.650999 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9532405c-e747-4b65-b513-9eea16d7ce4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:41:36.651725 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9532405c-e747-4b65-b513-9eea16d7ce4c None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:41:36.652008 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9532405c-e747-4b65-b513-9eea16d7ce4c None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:41:36.652282 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9532405c-e747-4b65-b513-9eea16d7ce4c None admin] PATH_INFO: `/v3/domains` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:41:36.652992 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9532405c-e747-4b65-b513-9eea16d7ce4c None admin] RBAC: Adding query filter params (name=Default) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:41:36.653293 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9532405c-e747-4b65-b513-9eea16d7ce4c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:41:36.654685 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9532405c-e747-4b65-b513-9eea16d7ce4c None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:41:36.666243 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 654/1266] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:41:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:41:36.671941 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1de8006-fd2b-4482-aed1-c942d1c61400 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:41:36.772976 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1de8006-fd2b-4482-aed1-c942d1c61400 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:41:36.792245 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:41:36.807064 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fde2ce8-1444-462f-b9c6-358a7811ed7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:41:36.807864 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fde2ce8-1444-462f-b9c6-358a7811ed7d None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:41:36.808153 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fde2ce8-1444-462f-b9c6-358a7811ed7d None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:41:36.808433 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fde2ce8-1444-462f-b9c6-358a7811ed7d None admin] PATH_INFO: `/v3/users/placement` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:41:36.817209 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fde2ce8-1444-462f-b9c6-358a7811ed7d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:41:36.819264 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fde2ce8-1444-462f-b9c6-358a7811ed7d None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:41:36.835239 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-6fde2ce8-1444-462f-b9c6-358a7811ed7d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 14:41:36.835239 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:41:36.835239 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:41:36.835239 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:41:36.835239 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:41:36.835239 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:41:36.835239 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:41:36.835239 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:41:36.835239 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:41:36.835239 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:41:36.835239 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:41:36.835239 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:41:36.835239 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:41:36.835239 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:41:36.835239 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:41:36.835239 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:41:36.836586 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:41:36.836586 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:41:36.836586 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:41:36.836586 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:41:36.836586 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:41:36.836586 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:41:36.836586 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:41:36.836586 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:41:36.836586 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:41:36.836586 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:41:36.836586 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:41:36.836586 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:41:36.836586 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:41:36.836586 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:41:36.836586 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:41:36.836586 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:41:36.836586 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:41:36.836586 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:41:36.837894 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:41:36.837894 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:41:36.837894 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:41:36.837894 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:41:36.837894 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:41:36.837894 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:41:36.837894 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:41:36.837894 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:41:36.837894 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 14:41:36.837894 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:41:36.837894 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 613/1267] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:41:36 2022] GET /identity/v3/users/placement => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:41:36.845082 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9532405c-e747-4b65-b513-9eea16d7ce4c None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:41:36.950418 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9532405c-e747-4b65-b513-9eea16d7ce4c None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:41:36.984345 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5062eac5-17fc-4194-bbe2-18df527941a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:41:36.985021 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5062eac5-17fc-4194-bbe2-18df527941a1 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:41:36.985304 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5062eac5-17fc-4194-bbe2-18df527941a1 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:41:36.985585 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5062eac5-17fc-4194-bbe2-18df527941a1 None admin] PATH_INFO: `/v3/users` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:41:36.986452 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5062eac5-17fc-4194-bbe2-18df527941a1 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:41:36.986764 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5062eac5-17fc-4194-bbe2-18df527941a1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:41:36.988199 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5062eac5-17fc-4194-bbe2-18df527941a1 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:41:36.998022 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:41:37.006891 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 655/1268] 10.222.0.57 () {56 vars in 1143 bytes} [Thu May 19 14:41:36 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:41:37.012623 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fde2ce8-1444-462f-b9c6-358a7811ed7d None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:41:37.116046 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fde2ce8-1444-462f-b9c6-358a7811ed7d None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:41:37.148676 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233cd250-0380-4202-9861-6032136ce75b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:41:37.149457 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233cd250-0380-4202-9861-6032136ce75b None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:41:37.149748 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233cd250-0380-4202-9861-6032136ce75b None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:41:37.150066 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233cd250-0380-4202-9861-6032136ce75b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:41:37.160644 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233cd250-0380-4202-9861-6032136ce75b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:41:37.162759 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233cd250-0380-4202-9861-6032136ce75b None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:41:37.180572 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-233cd250-0380-4202-9861-6032136ce75b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:41:37.180572 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:41:37.180572 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:41:37.180572 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:41:37.180572 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:41:37.180572 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:41:37.180572 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:41:37.180572 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:41:37.180572 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:41:37.180572 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:41:37.180572 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:41:37.180572 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:41:37.180572 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:41:37.180572 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:41:37.180572 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:41:37.181779 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:41:37.181779 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:41:37.181779 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:41:37.181779 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:41:37.181779 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:41:37.181779 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:41:37.181779 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:41:37.181779 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:41:37.181779 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:41:37.181779 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:41:37.181779 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:41:37.181779 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:41:37.181779 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:41:37.181779 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:41:37.181779 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:41:37.181779 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:41:37.181779 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:41:37.183225 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:41:37.183225 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:41:37.183225 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:41:37.183225 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:41:37.183225 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 614/1269] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:41:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:41:37.187553 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5062eac5-17fc-4194-bbe2-18df527941a1 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:41:37.289202 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5062eac5-17fc-4194-bbe2-18df527941a1 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:41:37.324089 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617b8e9d-37cb-4438-9cfa-761e98718f54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:41:37.324744 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617b8e9d-37cb-4438-9cfa-761e98718f54 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:41:37.325028 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617b8e9d-37cb-4438-9cfa-761e98718f54 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:41:37.325303 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617b8e9d-37cb-4438-9cfa-761e98718f54 None admin] PATH_INFO: `/v3/domains` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:41:37.326051 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617b8e9d-37cb-4438-9cfa-761e98718f54 None admin] RBAC: Adding query filter params (name=Default) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:41:37.326360 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617b8e9d-37cb-4438-9cfa-761e98718f54 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:41:37.327751 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617b8e9d-37cb-4438-9cfa-761e98718f54 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:41:37.339051 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 656/1270] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:41:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:41:37.344917 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233cd250-0380-4202-9861-6032136ce75b None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:41:37.377493 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:41:37.446634 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233cd250-0380-4202-9861-6032136ce75b None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:41:37.479123 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c995ab70-dbdb-4b4e-bfd7-9e6d38713659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:41:37.479956 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c995ab70-dbdb-4b4e-bfd7-9e6d38713659 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:41:37.480190 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c995ab70-dbdb-4b4e-bfd7-9e6d38713659 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:41:37.480464 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c995ab70-dbdb-4b4e-bfd7-9e6d38713659 None admin] PATH_INFO: `/v3/projects/service` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:41:37.487055 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c995ab70-dbdb-4b4e-bfd7-9e6d38713659 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:41:37.488502 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c995ab70-dbdb-4b4e-bfd7-9e6d38713659 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:41:37.503402 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-c995ab70-dbdb-4b4e-bfd7-9e6d38713659 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:41:37.503402 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:41:37.503402 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:41:37.503402 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:41:37.503402 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:41:37.503402 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:41:37.503402 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:41:37.503402 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:41:37.503402 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:41:37.503402 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:41:37.503402 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:41:37.503402 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:41:37.503402 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:41:37.503402 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:41:37.503402 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:41:37.504603 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:41:37.504603 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:41:37.504603 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:41:37.504603 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:41:37.504603 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:41:37.504603 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:41:37.504603 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:41:37.504603 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:41:37.504603 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:41:37.504603 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:41:37.504603 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:41:37.504603 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:41:37.504603 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:41:37.504603 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:41:37.504603 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:41:37.504603 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:41:37.504603 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:41:37.506545 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:41:37.506545 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:41:37.506545 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:41:37.506545 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:41:37.506545 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:41:37.506545 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:41:37.506545 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:41:37.506545 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:41:37.506545 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 615/1271] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:41:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:41:37.511050 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617b8e9d-37cb-4438-9cfa-761e98718f54 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:41:37.569026 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:41:37.610429 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617b8e9d-37cb-4438-9cfa-761e98718f54 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:41:37.643838 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e657c77b-0fd0-4f2f-a5d8-3e5e7d5e5fc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:41:37.644427 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e657c77b-0fd0-4f2f-a5d8-3e5e7d5e5fc9 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:41:37.644708 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e657c77b-0fd0-4f2f-a5d8-3e5e7d5e5fc9 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:41:37.644983 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e657c77b-0fd0-4f2f-a5d8-3e5e7d5e5fc9 None admin] PATH_INFO: `/v3/projects` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:41:37.645815 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e657c77b-0fd0-4f2f-a5d8-3e5e7d5e5fc9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:41:37.646153 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e657c77b-0fd0-4f2f-a5d8-3e5e7d5e5fc9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:41:37.647585 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e657c77b-0fd0-4f2f-a5d8-3e5e7d5e5fc9 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:41:37.659314 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 657/1272] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:41:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:41:37.665891 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c995ab70-dbdb-4b4e-bfd7-9e6d38713659 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:41:37.755847 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:41:37.764904 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c995ab70-dbdb-4b4e-bfd7-9e6d38713659 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:41:37.798327 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc354fe3-e8bc-49be-81cb-21a6b2e91a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:41:37.799204 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc354fe3-e8bc-49be-81cb-21a6b2e91a0f None admin] REQUEST_METHOD: `PUT` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:41:37.799491 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc354fe3-e8bc-49be-81cb-21a6b2e91a0f None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:41:37.799769 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc354fe3-e8bc-49be-81cb-21a6b2e91a0f None admin] PATH_INFO: `/v3/projects/42f35b96b9d4499abfb5e5f7cca49adc/users/b230516fafae4ac092c036f3b9aeb047/roles/c2935b1a25ca46fcba419fe3d1715330` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:41:37.802443 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc354fe3-e8bc-49be-81cb-21a6b2e91a0f None admin] RBAC: Authorizing `identity:create_grant(project_id=42f35b96b9d4499abfb5e5f7cca49adc, user_id=b230516fafae4ac092c036f3b9aeb047, role_id=c2935b1a25ca46fcba419fe3d1715330)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:41:37.803880 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc354fe3-e8bc-49be-81cb-21a6b2e91a0f None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:41:37.817509 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 616/1273] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 14:41:37 2022] PUT /identity/v3/projects/42f35b96b9d4499abfb5e5f7cca49adc/users/b230516fafae4ac092c036f3b9aeb047/roles/c2935b1a25ca46fcba419fe3d1715330 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:41:40.030142 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ced1eb-7312-40f1-ab7e-5978342a42d3 None None] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:41:40.030715 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ced1eb-7312-40f1-ab7e-5978342a42d3 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:41:40.030715 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ced1eb-7312-40f1-ab7e-5978342a42d3 None None] PATH_INFO: `/` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:41:40.032063 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 658/1274] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:41:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:41:40.037775 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9b8a4e-0102-4bc7-8c67-ca3a69f00895 None None] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:41:40.038249 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9b8a4e-0102-4bc7-8c67-ca3a69f00895 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:41:40.038368 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9b8a4e-0102-4bc7-8c67-ca3a69f00895 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:41:40.055520 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.auth.core [None req-7b9b8a4e-0102-4bc7-8c67-ca3a69f00895 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:41:40.056708 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-7b9b8a4e-0102-4bc7-8c67-ca3a69f00895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:41:40.131282 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 617/1275] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:41:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:41:40.155557 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ced1eb-7312-40f1-ab7e-5978342a42d3 None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:41:40.257731 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ced1eb-7312-40f1-ab7e-5978342a42d3 None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:41:40.274713 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:41:40.291338 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4059ee-759b-4575-a496-5923925d7bb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:41:40.292060 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4059ee-759b-4575-a496-5923925d7bb0 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:41:40.292343 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4059ee-759b-4575-a496-5923925d7bb0 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:41:40.292616 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4059ee-759b-4575-a496-5923925d7bb0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:41:40.303143 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4059ee-759b-4575-a496-5923925d7bb0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:41:40.310769 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-0c4059ee-759b-4575-a496-5923925d7bb0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:41:40.310769 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:41:40.310769 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:41:40.310769 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:41:40.310769 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:41:40.310769 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:41:40.310769 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:41:40.310769 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:41:40.310769 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:41:40.310769 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:41:40.310769 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:41:40.310769 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:41:40.310769 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:41:40.310769 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:41:40.310769 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:41:40.310769 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:41:40.312029 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:41:40.312029 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:41:40.312029 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:41:40.312029 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:41:40.312029 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:41:40.312029 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:41:40.312029 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:41:40.312029 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:41:40.312029 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:41:40.312029 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:41:40.312029 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:41:40.312029 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:41:40.312029 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:41:40.312029 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:41:40.312029 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:41:40.312029 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:41:40.313222 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:41:40.313222 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:41:40.313222 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:41:40.313222 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:41:40.313222 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:41:40.313222 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:41:40.313222 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:41:40.313222 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:41:40.313222 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:41:40.313222 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:41:40.313222 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:41:40.313222 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:41:40.313222 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 659/1276] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:41:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:41:40.317981 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9b8a4e-0102-4bc7-8c67-ca3a69f00895 None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:41:40.397744 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:41:40.416118 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9b8a4e-0102-4bc7-8c67-ca3a69f00895 None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:41:40.450798 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4677e0d7-0750-477e-9b5f-715bf1f9345f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:41:40.451453 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4677e0d7-0750-477e-9b5f-715bf1f9345f None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:41:40.451733 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4677e0d7-0750-477e-9b5f-715bf1f9345f None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:41:40.452005 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4677e0d7-0750-477e-9b5f-715bf1f9345f None admin] PATH_INFO: `/v3/roles` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:41:40.452742 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4677e0d7-0750-477e-9b5f-715bf1f9345f None admin] RBAC: Adding query filter params (name=admin) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:41:40.453042 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4677e0d7-0750-477e-9b5f-715bf1f9345f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:41:40.454441 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4677e0d7-0750-477e-9b5f-715bf1f9345f None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:41:40.462565 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 618/1277] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:41:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:41:40.468291 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4059ee-759b-4575-a496-5923925d7bb0 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:41:40.568914 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4059ee-759b-4575-a496-5923925d7bb0 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:41:40.602591 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07d0541-279b-46f1-8113-909b738526bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:41:40.603367 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07d0541-279b-46f1-8113-909b738526bb None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:41:40.603647 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07d0541-279b-46f1-8113-909b738526bb None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:41:40.603918 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07d0541-279b-46f1-8113-909b738526bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:41:40.614697 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07d0541-279b-46f1-8113-909b738526bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:41:40.616726 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07d0541-279b-46f1-8113-909b738526bb None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:41:40.633367 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-c07d0541-279b-46f1-8113-909b738526bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:41:40.633367 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:41:40.633367 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:41:40.633367 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:41:40.633367 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:41:40.633367 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:41:40.633367 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:41:40.633367 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:41:40.633367 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:41:40.633367 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:41:40.633367 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:41:40.633367 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:41:40.633367 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:41:40.633367 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:41:40.633367 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:41:40.634675 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:41:40.634675 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:41:40.634675 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:41:40.634675 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:41:40.634675 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:41:40.634675 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:41:40.634675 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:41:40.634675 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:41:40.634675 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:41:40.634675 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:41:40.634675 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:41:40.634675 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:41:40.634675 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:41:40.634675 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:41:40.634675 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:41:40.634675 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:41:40.634675 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:41:40.635975 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:41:40.635975 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:41:40.635975 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:41:40.635975 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:41:40.635975 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 660/1278] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:41:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:41:40.640315 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4677e0d7-0750-477e-9b5f-715bf1f9345f None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:41:40.739828 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4677e0d7-0750-477e-9b5f-715bf1f9345f None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:41:40.773088 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b23bd14-37cb-435c-894a-b5e156923ba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:41:40.773739 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b23bd14-37cb-435c-894a-b5e156923ba8 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:41:40.774060 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b23bd14-37cb-435c-894a-b5e156923ba8 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:41:40.774339 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b23bd14-37cb-435c-894a-b5e156923ba8 None admin] PATH_INFO: `/v3/domains` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:41:40.775058 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b23bd14-37cb-435c-894a-b5e156923ba8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:41:40.775365 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b23bd14-37cb-435c-894a-b5e156923ba8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:41:40.776736 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b23bd14-37cb-435c-894a-b5e156923ba8 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:41:40.788728 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 619/1279] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:41:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:41:40.794494 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07d0541-279b-46f1-8113-909b738526bb None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:41:40.833616 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:41:40.918940 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07d0541-279b-46f1-8113-909b738526bb None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:41:40.952341 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680d51b7-6ede-4dd3-be47-c371e5460dc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:41:40.953137 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680d51b7-6ede-4dd3-be47-c371e5460dc3 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:41:40.953422 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680d51b7-6ede-4dd3-be47-c371e5460dc3 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:41:40.953699 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680d51b7-6ede-4dd3-be47-c371e5460dc3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:41:40.961833 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680d51b7-6ede-4dd3-be47-c371e5460dc3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:41:40.963296 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680d51b7-6ede-4dd3-be47-c371e5460dc3 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:41:40.978589 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-680d51b7-6ede-4dd3-be47-c371e5460dc3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 14:41:40.978589 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:41:40.978589 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:41:40.978589 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:41:40.978589 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:41:40.978589 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:41:40.978589 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:41:40.978589 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:41:40.978589 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:41:40.978589 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:41:40.978589 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:41:40.978589 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:41:40.978589 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:41:40.978589 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:41:40.978589 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:41:40.978589 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:41:40.979897 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:41:40.979897 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:41:40.979897 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:41:40.979897 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:41:40.979897 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:41:40.979897 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:41:40.979897 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:41:40.979897 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:41:40.979897 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:41:40.979897 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:41:40.979897 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:41:40.979897 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:41:40.979897 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:41:40.979897 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:41:40.979897 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:41:40.979897 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:41:40.979897 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:41:40.979897 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:41:40.981019 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:41:40.981019 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:41:40.981019 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:41:40.981019 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:41:40.981019 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:41:40.981019 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:41:40.981019 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:41:40.981019 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:41:40.981019 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 14:41:40.981019 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:41:40.981019 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 661/1280] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:41:40 2022] GET /identity/v3/users/placement => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:41:40.985596 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b23bd14-37cb-435c-894a-b5e156923ba8 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:41:40.989373 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:41:41.088384 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b23bd14-37cb-435c-894a-b5e156923ba8 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:41:41.121921 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89b5059-ddcb-49cd-90e5-bb5fc634bceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:41:41.122585 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89b5059-ddcb-49cd-90e5-bb5fc634bceb None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:41:41.122871 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89b5059-ddcb-49cd-90e5-bb5fc634bceb None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:41:41.123147 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89b5059-ddcb-49cd-90e5-bb5fc634bceb None admin] PATH_INFO: `/v3/users` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:41:41.123981 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89b5059-ddcb-49cd-90e5-bb5fc634bceb None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:41:41.124293 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89b5059-ddcb-49cd-90e5-bb5fc634bceb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:41:41.125729 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89b5059-ddcb-49cd-90e5-bb5fc634bceb None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:41:41.142265 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 620/1281] 10.222.0.57 () {56 vars in 1143 bytes} [Thu May 19 14:41:40 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:41:41.148661 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680d51b7-6ede-4dd3-be47-c371e5460dc3 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:41:41.240907 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:41:41.250873 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680d51b7-6ede-4dd3-be47-c371e5460dc3 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:41:41.284534 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea4a4da-6754-4a12-86ee-cbb4decba3a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:41:41.285182 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea4a4da-6754-4a12-86ee-cbb4decba3a8 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:41:41.285469 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea4a4da-6754-4a12-86ee-cbb4decba3a8 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:41:41.285750 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea4a4da-6754-4a12-86ee-cbb4decba3a8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:41:41.352762 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea4a4da-6754-4a12-86ee-cbb4decba3a8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:41:41.354185 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea4a4da-6754-4a12-86ee-cbb4decba3a8 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:41:41.441028 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:41:41.452320 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 662/1282] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:41:41 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:41:41.458367 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89b5059-ddcb-49cd-90e5-bb5fc634bceb None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:41:41.520388 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:41:41.560009 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89b5059-ddcb-49cd-90e5-bb5fc634bceb None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:41:41.594868 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95782828-ea8b-491e-a64f-733702c44964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:41:41.595661 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95782828-ea8b-491e-a64f-733702c44964 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:41:41.595946 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95782828-ea8b-491e-a64f-733702c44964 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:41:41.596224 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95782828-ea8b-491e-a64f-733702c44964 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:41:41.606734 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95782828-ea8b-491e-a64f-733702c44964 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:41:41.608118 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95782828-ea8b-491e-a64f-733702c44964 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:41:41.625519 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-95782828-ea8b-491e-a64f-733702c44964 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:41:41.625519 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:41:41.625519 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:41:41.625519 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:41:41.625519 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:41:41.625519 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:41:41.625519 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:41:41.625519 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:41:41.625519 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:41:41.625519 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:41:41.625519 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:41:41.625519 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:41:41.625519 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:41:41.625519 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:41:41.625519 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:41:41.626732 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:41:41.626732 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:41:41.626732 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:41:41.626732 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:41:41.626732 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:41:41.626732 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:41:41.626732 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:41:41.626732 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:41:41.626732 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:41:41.626732 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:41:41.626732 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:41:41.626732 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:41:41.626732 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:41:41.626732 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:41:41.626732 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:41:41.626732 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:41:41.626732 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:41:41.628059 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:41:41.628059 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:41:41.628059 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:41:41.628059 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:41:41.628059 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 621/1283] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:41:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:41:41.632937 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea4a4da-6754-4a12-86ee-cbb4decba3a8 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:41:41.731187 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea4a4da-6754-4a12-86ee-cbb4decba3a8 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:41:41.765616 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7440b9fb-14d2-438a-9a27-5ed0255d972b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:41:41.766301 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7440b9fb-14d2-438a-9a27-5ed0255d972b None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:41:41.766590 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7440b9fb-14d2-438a-9a27-5ed0255d972b None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:41:41.766871 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7440b9fb-14d2-438a-9a27-5ed0255d972b None admin] PATH_INFO: `/v3/domains` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:41:41.767591 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7440b9fb-14d2-438a-9a27-5ed0255d972b None admin] RBAC: Adding query filter params (name=Default) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:41:41.767900 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7440b9fb-14d2-438a-9a27-5ed0255d972b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:41:41.769285 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7440b9fb-14d2-438a-9a27-5ed0255d972b None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:41:41.780357 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 663/1284] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:41:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:41:41.786007 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95782828-ea8b-491e-a64f-733702c44964 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:41:41.884245 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95782828-ea8b-491e-a64f-733702c44964 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:41:41.898298 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:41:41.918421 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410e9d17-f05c-4f25-a25b-05f2536ab9d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:41:41.919205 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410e9d17-f05c-4f25-a25b-05f2536ab9d9 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:41:41.919490 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410e9d17-f05c-4f25-a25b-05f2536ab9d9 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:41:41.919768 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410e9d17-f05c-4f25-a25b-05f2536ab9d9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:41:41.926325 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410e9d17-f05c-4f25-a25b-05f2536ab9d9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:41:41.928397 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410e9d17-f05c-4f25-a25b-05f2536ab9d9 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:41:41.941892 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application [None req-410e9d17-f05c-4f25-a25b-05f2536ab9d9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:41:41.941892 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:41:41.941892 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:41:41.941892 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:41:41.941892 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:41:41.941892 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:41:41.941892 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:41:41.941892 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:41:41.941892 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:41:41.941892 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:41:41.941892 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:41:41.941892 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:41:41.941892 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:41:41.941892 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:41:41.941892 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:41:41.942866 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:41:41.942866 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:41:41.942866 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:41:41.942866 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:41:41.942866 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:41:41.942866 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:41:41.942866 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:41:41.942866 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:41:41.942866 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application with Lock( May 19 14:41:41.942866 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:41:41.942866 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._enter() May 19 14:41:41.942866 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:41:41.942866 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:41:41.942866 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:41:41.942866 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.creator() May 19 14:41:41.942866 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:41:41.942866 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application created_value = creator( May 19 14:41:41.943944 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:41:41.943944 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:41:41.943944 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:41:41.943944 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:41:41.943944 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:41:41.943944 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:41:41.943944 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:41:41.943944 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: ERROR keystone.server.flask.application May 19 14:41:41.943944 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 622/1285] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:41:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:41:41.949608 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7440b9fb-14d2-438a-9a27-5ed0255d972b None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:41:41.999739 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:41:42.050401 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7440b9fb-14d2-438a-9a27-5ed0255d972b None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:41:42.085620 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d827a2-4f63-4e79-8432-18eaec34b3e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:41:42.086308 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d827a2-4f63-4e79-8432-18eaec34b3e4 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:41:42.086593 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d827a2-4f63-4e79-8432-18eaec34b3e4 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:41:42.086868 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d827a2-4f63-4e79-8432-18eaec34b3e4 None admin] PATH_INFO: `/v3/projects` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:41:42.087649 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d827a2-4f63-4e79-8432-18eaec34b3e4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:41:42.087961 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d827a2-4f63-4e79-8432-18eaec34b3e4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:41:42.089394 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d827a2-4f63-4e79-8432-18eaec34b3e4 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:41:42.101810 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 664/1286] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:41:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:41:42.107949 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410e9d17-f05c-4f25-a25b-05f2536ab9d9 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:41:42.211009 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410e9d17-f05c-4f25-a25b-05f2536ab9d9 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:41:42.243284 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ba0e3c-07ae-43f9-b50c-eef575b218a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:41:42.243930 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ba0e3c-07ae-43f9-b50c-eef575b218a9 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:41:42.244216 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ba0e3c-07ae-43f9-b50c-eef575b218a9 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:41:42.244492 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ba0e3c-07ae-43f9-b50c-eef575b218a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:41:42.245315 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ba0e3c-07ae-43f9-b50c-eef575b218a9 None admin] RBAC: Adding query filter params (role.id=c2935b1a25ca46fcba419fe3d1715330, scope.project.id=42f35b96b9d4499abfb5e5f7cca49adc, user.id=b230516fafae4ac092c036f3b9aeb047) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:41:42.245626 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ba0e3c-07ae-43f9-b50c-eef575b218a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:41:42.247083 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ba0e3c-07ae-43f9-b50c-eef575b218a9 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:41:42.252819 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 623/1287] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:41:42 2022] GET /identity/v3/role_assignments?user.id=b230516fafae4ac092c036f3b9aeb047&scope.project.id=42f35b96b9d4499abfb5e5f7cca49adc&role.id=c2935b1a25ca46fcba419fe3d1715330 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:41:44.504691 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d980fd-daa2-4b4f-9cc3-031d7eb4bed4 None None] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:41:44.505237 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d980fd-daa2-4b4f-9cc3-031d7eb4bed4 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:41:44.505237 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d980fd-daa2-4b4f-9cc3-031d7eb4bed4 None None] PATH_INFO: `/` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:41:44.507104 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 665/1288] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:41:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:41:44.512592 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873d5cc6-8d8d-4079-a8b3-e3781dab9936 None None] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:41:44.513015 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873d5cc6-8d8d-4079-a8b3-e3781dab9936 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:41:44.513160 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873d5cc6-8d8d-4079-a8b3-e3781dab9936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:41:44.530519 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.auth.core [None req-873d5cc6-8d8d-4079-a8b3-e3781dab9936 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:41:44.531715 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-873d5cc6-8d8d-4079-a8b3-e3781dab9936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:41:44.561339 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:41:44.600258 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 624/1289] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:41:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:41:44.622737 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d980fd-daa2-4b4f-9cc3-031d7eb4bed4 None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:41:44.723729 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d980fd-daa2-4b4f-9cc3-031d7eb4bed4 None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:41:44.734688 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:41:44.759285 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381258b2-9415-43f2-9dfb-4c2d6b0f287d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:41:44.760020 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381258b2-9415-43f2-9dfb-4c2d6b0f287d None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:41:44.760305 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381258b2-9415-43f2-9dfb-4c2d6b0f287d None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:41:44.760585 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381258b2-9415-43f2-9dfb-4c2d6b0f287d None admin] PATH_INFO: `/v3/services/placement` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:41:44.761260 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381258b2-9415-43f2-9dfb-4c2d6b0f287d None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:41:44.762664 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381258b2-9415-43f2-9dfb-4c2d6b0f287d None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:41:44.774634 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-381258b2-9415-43f2-9dfb-4c2d6b0f287d None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 19 14:41:44.774634 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:41:44.774634 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:41:44.774634 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:41:44.774634 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:41:44.774634 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:41:44.774634 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:41:44.774634 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:41:44.774634 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 14:41:44.774634 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:41:44.774634 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:41:44.774634 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:41:44.774634 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:41:44.774634 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:41:44.774634 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:41:44.774634 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:41:44.774634 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:41:44.776032 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:41:44.776032 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:41:44.776032 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:41:44.776032 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:41:44.776032 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:41:44.776032 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:41:44.776032 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:41:44.776032 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:41:44.776032 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:41:44.776032 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:41:44.776032 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:41:44.776032 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:41:44.776032 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:41:44.776032 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:41:44.776032 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:41:44.776032 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:41:44.777105 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:41:44.777105 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:41:44.777105 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:41:44.777105 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:41:44.777105 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:41:44.777105 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:41:44.777105 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:41:44.777105 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:41:44.777105 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:41:44.777105 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:41:44.777105 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:41:44.777105 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:41:44.777105 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 14:41:44.777105 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:41:44.777105 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 666/1290] 10.222.0.57 () {56 vars in 1117 bytes} [Thu May 19 14:41:44 2022] GET /identity/v3/services/placement => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:41:44.781972 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873d5cc6-8d8d-4079-a8b3-e3781dab9936 None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:41:44.901479 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873d5cc6-8d8d-4079-a8b3-e3781dab9936 None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:41:44.935536 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013d7302-edf3-4251-8f23-d606cc45479d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:41:44.936194 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013d7302-edf3-4251-8f23-d606cc45479d None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:41:44.936481 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013d7302-edf3-4251-8f23-d606cc45479d None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:41:44.936758 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013d7302-edf3-4251-8f23-d606cc45479d None admin] PATH_INFO: `/v3/services` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:41:44.937459 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013d7302-edf3-4251-8f23-d606cc45479d None admin] RBAC: Adding query filter params (name=placement) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:41:44.937769 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013d7302-edf3-4251-8f23-d606cc45479d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:41:44.939181 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013d7302-edf3-4251-8f23-d606cc45479d None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:41:44.944852 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 625/1291] 10.222.0.57 () {56 vars in 1116 bytes} [Thu May 19 14:41:44 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:41:44.950670 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381258b2-9415-43f2-9dfb-4c2d6b0f287d None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:41:45.050880 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381258b2-9415-43f2-9dfb-4c2d6b0f287d None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:41:45.085786 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b547b2-e4fe-4413-91ef-b7550e98fc6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:41:45.086476 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b547b2-e4fe-4413-91ef-b7550e98fc6d None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:41:45.086759 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b547b2-e4fe-4413-91ef-b7550e98fc6d None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:41:45.087036 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b547b2-e4fe-4413-91ef-b7550e98fc6d None admin] PATH_INFO: `/v3/services` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:41:45.087735 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b547b2-e4fe-4413-91ef-b7550e98fc6d None admin] RBAC: Adding query filter params (type=placement) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:41:45.088039 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b547b2-e4fe-4413-91ef-b7550e98fc6d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:41:45.089409 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b547b2-e4fe-4413-91ef-b7550e98fc6d None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:41:45.093813 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 667/1292] 10.222.0.57 () {56 vars in 1116 bytes} [Thu May 19 14:41:44 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:41:47.315954 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407675b0-6321-422d-ac3d-30f643d2b480 None None] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:41:47.316521 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407675b0-6321-422d-ac3d-30f643d2b480 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:41:47.316521 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407675b0-6321-422d-ac3d-30f643d2b480 None None] PATH_INFO: `/` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:41:47.317909 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 626/1293] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:41:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:41:47.323784 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519f4cf7-0d66-46fb-91e2-d629e18f4391 None None] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:41:47.324210 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519f4cf7-0d66-46fb-91e2-d629e18f4391 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:41:47.324318 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519f4cf7-0d66-46fb-91e2-d629e18f4391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:41:47.341877 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.auth.core [None req-519f4cf7-0d66-46fb-91e2-d629e18f4391 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:41:47.343062 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.fernet_utils [None req-519f4cf7-0d66-46fb-91e2-d629e18f4391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:41:47.371668 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:41:47.408859 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 668/1294] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:41:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:41:47.431942 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407675b0-6321-422d-ac3d-30f643d2b480 None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:41:47.463481 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:41:47.532442 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407675b0-6321-422d-ac3d-30f643d2b480 None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:41:47.565517 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deeee124-8e75-4724-9a97-102b80474749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:41:47.566214 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeee124-8e75-4724-9a97-102b80474749 None admin] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:41:47.566505 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeee124-8e75-4724-9a97-102b80474749 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:41:47.566787 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeee124-8e75-4724-9a97-102b80474749 None admin] PATH_INFO: `/v3/services` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:41:47.567509 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deeee124-8e75-4724-9a97-102b80474749 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:41:47.569380 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deeee124-8e75-4724-9a97-102b80474749 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:41:47.582204 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 627/1295] 10.222.0.57 () {60 vars in 1141 bytes} [Thu May 19 14:41:47 2022] POST /identity/v3/services => generated 246 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:41:49.838100 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9c662a-e749-4a73-a6a5-bcbba84afe25 None None] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:41:49.838654 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9c662a-e749-4a73-a6a5-bcbba84afe25 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:41:49.838654 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9c662a-e749-4a73-a6a5-bcbba84afe25 None None] PATH_INFO: `/` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:41:49.840210 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 669/1296] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:41:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:41:49.845971 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f2ad1c-e1d2-4d94-a89b-c6eac841292c None None] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:41:49.846590 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f2ad1c-e1d2-4d94-a89b-c6eac841292c None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:41:49.846590 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f2ad1c-e1d2-4d94-a89b-c6eac841292c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:41:49.865165 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.auth.core [None req-e1f2ad1c-e1d2-4d94-a89b-c6eac841292c None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:41:49.866574 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-e1f2ad1c-e1d2-4d94-a89b-c6eac841292c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:41:49.940582 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 628/1297] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:41:49 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:41:49.964325 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9c662a-e749-4a73-a6a5-bcbba84afe25 None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:41:50.063611 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9c662a-e749-4a73-a6a5-bcbba84afe25 None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:41:50.098645 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b001ed-bbae-4a68-8c8f-4f1ca510ae90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:41:50.099405 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b001ed-bbae-4a68-8c8f-4f1ca510ae90 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:41:50.100324 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b001ed-bbae-4a68-8c8f-4f1ca510ae90 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:41:50.100636 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b001ed-bbae-4a68-8c8f-4f1ca510ae90 None admin] PATH_INFO: `/v3/services/placement` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:41:50.101250 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b001ed-bbae-4a68-8c8f-4f1ca510ae90 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:41:50.102662 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b001ed-bbae-4a68-8c8f-4f1ca510ae90 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:41:50.114554 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-b3b001ed-bbae-4a68-8c8f-4f1ca510ae90 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 19 14:41:50.114554 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:41:50.114554 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:41:50.114554 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:41:50.114554 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:41:50.114554 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:41:50.114554 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:41:50.114554 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:41:50.114554 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 14:41:50.114554 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:41:50.114554 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:41:50.114554 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:41:50.114554 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:41:50.114554 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:41:50.114554 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:41:50.114554 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:41:50.114554 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:41:50.116759 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:41:50.116759 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:41:50.116759 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:41:50.116759 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:41:50.116759 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:41:50.116759 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:41:50.116759 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:41:50.116759 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:41:50.116759 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:41:50.116759 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:41:50.116759 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:41:50.116759 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:41:50.116759 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:41:50.116759 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:41:50.116759 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:41:50.116759 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:41:50.117944 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:41:50.117944 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:41:50.117944 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:41:50.117944 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:41:50.117944 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:41:50.117944 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:41:50.117944 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:41:50.117944 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:41:50.117944 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:41:50.117944 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:41:50.117944 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:41:50.117944 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:41:50.117944 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 14:41:50.117944 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:41:50.117944 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 670/1298] 10.222.0.57 () {56 vars in 1117 bytes} [Thu May 19 14:41:49 2022] GET /identity/v3/services/placement => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:41:50.122782 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f2ad1c-e1d2-4d94-a89b-c6eac841292c None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:41:50.141520 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:41:50.223576 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f2ad1c-e1d2-4d94-a89b-c6eac841292c None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:41:50.257461 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b5a9fa-44e4-456c-ab68-908d7607e413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:41:50.258145 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b5a9fa-44e4-456c-ab68-908d7607e413 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:41:50.258436 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b5a9fa-44e4-456c-ab68-908d7607e413 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:41:50.258717 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b5a9fa-44e4-456c-ab68-908d7607e413 None admin] PATH_INFO: `/v3/services` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:41:50.259419 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b5a9fa-44e4-456c-ab68-908d7607e413 None admin] RBAC: Adding query filter params (name=placement) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:41:50.259729 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b5a9fa-44e4-456c-ab68-908d7607e413 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:41:50.261105 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b5a9fa-44e4-456c-ab68-908d7607e413 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:41:50.266222 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 629/1299] 10.222.0.57 () {56 vars in 1116 bytes} [Thu May 19 14:41:50 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:41:50.272245 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b001ed-bbae-4a68-8c8f-4f1ca510ae90 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:41:50.304667 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:41:50.373023 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b001ed-bbae-4a68-8c8f-4f1ca510ae90 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:41:50.405495 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2d4c34-deee-4531-aa34-56913a2cd1ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:41:50.406172 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2d4c34-deee-4531-aa34-56913a2cd1ee None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:41:50.406459 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2d4c34-deee-4531-aa34-56913a2cd1ee None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:41:50.406739 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2d4c34-deee-4531-aa34-56913a2cd1ee None admin] PATH_INFO: `/v3/endpoints` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:41:50.407527 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2d4c34-deee-4531-aa34-56913a2cd1ee None admin] RBAC: Adding query filter params (interface=public, service_id=f6e77377498a4c10a66c3e32f43673d6, region_id=RegionOne) {{(pid=218625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:41:50.407840 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2d4c34-deee-4531-aa34-56913a2cd1ee None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:41:50.409214 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2d4c34-deee-4531-aa34-56913a2cd1ee None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:41:50.414968 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 671/1300] 10.222.0.57 () {56 vars in 1251 bytes} [Thu May 19 14:41:50 2022] GET /identity/v3/endpoints?service_id=f6e77377498a4c10a66c3e32f43673d6&interface=public®ion_id=RegionOne => generated 194 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:41:50.420603 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b5a9fa-44e4-456c-ab68-908d7607e413 None admin] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:41:50.501747 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:41:50.519778 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b5a9fa-44e4-456c-ab68-908d7607e413 None admin] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:41:50.553712 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d8f362-2667-4bcf-a374-89a861d39eb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:41:50.554390 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d8f362-2667-4bcf-a374-89a861d39eb3 None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:41:50.554677 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d8f362-2667-4bcf-a374-89a861d39eb3 None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:41:50.554952 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d8f362-2667-4bcf-a374-89a861d39eb3 None admin] PATH_INFO: `/v3/services` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:41:50.555629 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d8f362-2667-4bcf-a374-89a861d39eb3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:41:50.557018 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d8f362-2667-4bcf-a374-89a861d39eb3 None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:41:50.563751 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 630/1301] 10.222.0.57 () {56 vars in 1087 bytes} [Thu May 19 14:41:50 2022] GET /identity/v3/services => generated 2191 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 14:41:52.778329 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a751681-f3dc-4301-899f-0aa8f58e64e3 None None] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:41:52.778940 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a751681-f3dc-4301-899f-0aa8f58e64e3 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:41:52.778940 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a751681-f3dc-4301-899f-0aa8f58e64e3 None None] PATH_INFO: `/` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:41:52.780248 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 672/1302] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:41:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:41:52.785575 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03134c49-cbef-4aee-857b-91476310a0c9 None None] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:41:52.786029 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03134c49-cbef-4aee-857b-91476310a0c9 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:41:52.786172 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03134c49-cbef-4aee-857b-91476310a0c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:41:52.802759 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.auth.core [None req-03134c49-cbef-4aee-857b-91476310a0c9 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:41:52.803958 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-03134c49-cbef-4aee-857b-91476310a0c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:41:52.872138 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 631/1303] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:41:52 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:41:52.895306 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a751681-f3dc-4301-899f-0aa8f58e64e3 None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:41:52.991050 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:41:52.998043 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a751681-f3dc-4301-899f-0aa8f58e64e3 None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:41:53.031352 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6a1202-ef7b-42bf-9f85-8fe931f844f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:41:53.032076 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6a1202-ef7b-42bf-9f85-8fe931f844f4 None admin] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:41:53.032360 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6a1202-ef7b-42bf-9f85-8fe931f844f4 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:41:53.032632 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6a1202-ef7b-42bf-9f85-8fe931f844f4 None admin] PATH_INFO: `/v3/services/placement` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:41:53.033280 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6a1202-ef7b-42bf-9f85-8fe931f844f4 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:41:53.034684 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6a1202-ef7b-42bf-9f85-8fe931f844f4 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:41:53.046386 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application [None req-9e6a1202-ef7b-42bf-9f85-8fe931f844f4 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 19 14:41:53.046386 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:41:53.046386 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:41:53.046386 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:41:53.046386 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:41:53.046386 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:41:53.046386 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:41:53.046386 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:41:53.046386 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 14:41:53.046386 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:41:53.046386 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:41:53.046386 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:41:53.046386 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:41:53.046386 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:41:53.046386 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:41:53.046386 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:41:53.046386 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:41:53.047760 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:41:53.047760 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:41:53.047760 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:41:53.047760 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:41:53.047760 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:41:53.047760 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:41:53.047760 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:41:53.047760 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:41:53.047760 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:41:53.047760 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:41:53.047760 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:41:53.047760 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:41:53.047760 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:41:53.047760 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:41:53.047760 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:41:53.047760 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:41:53.049098 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:41:53.049098 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application with Lock( May 19 14:41:53.049098 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:41:53.049098 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self._enter() May 19 14:41:53.049098 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:41:53.049098 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:41:53.049098 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:41:53.049098 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application return self.creator() May 19 14:41:53.049098 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:41:53.049098 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application created_value = creator( May 19 14:41:53.049098 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:41:53.049098 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:41:53.049098 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 14:41:53.049098 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: ERROR keystone.server.flask.application May 19 14:41:53.049098 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 673/1304] 10.222.0.57 () {56 vars in 1117 bytes} [Thu May 19 14:41:52 2022] GET /identity/v3/services/placement => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:41:53.054568 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03134c49-cbef-4aee-857b-91476310a0c9 None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:41:53.129228 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:41:53.152919 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03134c49-cbef-4aee-857b-91476310a0c9 None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:41:53.187227 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb73750-02ed-4576-9079-caab9846450e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:41:53.187888 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb73750-02ed-4576-9079-caab9846450e None admin] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:41:53.188171 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb73750-02ed-4576-9079-caab9846450e None admin] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:41:53.188447 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb73750-02ed-4576-9079-caab9846450e None admin] PATH_INFO: `/v3/services` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:41:53.189148 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb73750-02ed-4576-9079-caab9846450e None admin] RBAC: Adding query filter params (name=placement) {{(pid=218624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:41:53.189456 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb73750-02ed-4576-9079-caab9846450e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:41:53.190860 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb73750-02ed-4576-9079-caab9846450e None admin] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:41:53.196684 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 632/1305] 10.222.0.57 () {56 vars in 1116 bytes} [Thu May 19 14:41:53 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:41:53.202505 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6a1202-ef7b-42bf-9f85-8fe931f844f4 None admin] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:41:53.300177 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6a1202-ef7b-42bf-9f85-8fe931f844f4 None admin] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:41:53.334171 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d646e04f-949d-4e25-8a2e-6ed004593c04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8062aba2d374babbc9d68178a9546e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:41:53.334827 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d646e04f-949d-4e25-8a2e-6ed004593c04 None admin] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:41:53.335113 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d646e04f-949d-4e25-8a2e-6ed004593c04 None admin] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:41:53.335392 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d646e04f-949d-4e25-8a2e-6ed004593c04 None admin] PATH_INFO: `/v3/endpoints` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:41:53.336143 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d646e04f-949d-4e25-8a2e-6ed004593c04 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:41:53.337522 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d646e04f-949d-4e25-8a2e-6ed004593c04 None admin] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:41:53.356393 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 674/1306] 10.222.0.57 () {60 vars in 1144 bytes} [Thu May 19 14:41:53 2022] POST /identity/v3/endpoints => generated 331 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:42:03.193828 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=30 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:42:03.351993 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:42:59.282346 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6118ab54-6eff-4225-ba40-5f2339f67b4c None None] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:42:59.283438 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6118ab54-6eff-4225-ba40-5f2339f67b4c None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:42:59.283959 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6118ab54-6eff-4225-ba40-5f2339f67b4c None None] PATH_INFO: `/` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:42:59.289230 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 633/1307] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:42:59 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:42:59.295190 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7308f9-c3d3-4520-b2d8-cb9df34fd47b None None] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:42:59.296255 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7308f9-c3d3-4520-b2d8-cb9df34fd47b None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:42:59.296773 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7308f9-c3d3-4520-b2d8-cb9df34fd47b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:42:59.461031 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.auth.core [None req-0e7308f9-c3d3-4520-b2d8-cb9df34fd47b None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:42:59.462686 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.fernet_utils [None req-0e7308f9-c3d3-4520-b2d8-cb9df34fd47b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:42:59.630237 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 675/1308] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:42:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 336 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:42:59.700357 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7752fd5e-d904-4423-8cd8-a3924ac2bda9 None None] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:42:59.701061 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7752fd5e-d904-4423-8cd8-a3924ac2bda9 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:42:59.701569 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7752fd5e-d904-4423-8cd8-a3924ac2bda9 None None] PATH_INFO: `/` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:42:59.703218 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 634/1309] 10.222.0.57 () {54 vars in 964 bytes} [Thu May 19 14:42:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:42:59.713502 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f8cf8d-be3a-4f5d-b28c-1addc2602987 None None] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:42:59.714198 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f8cf8d-be3a-4f5d-b28c-1addc2602987 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:42:59.714933 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f8cf8d-be3a-4f5d-b28c-1addc2602987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:42:59.852357 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.auth.core [None req-42f8cf8d-be3a-4f5d-b28c-1addc2602987 None None] MFA Rules not processed for user `c7cd031eea90473aaaa0f1375f04dc66`. Rule list: `[]` (Enabled: `True`). {{(pid=218625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:42:59.853991 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.fernet_utils [None req-42f8cf8d-be3a-4f5d-b28c-1addc2602987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:42:59.904003 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:43:00.006737 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 676/1310] 10.222.0.57 () {58 vars in 1061 bytes} [Thu May 19 14:42:59 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:43:00.017085 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7752fd5e-d904-4423-8cd8-a3924ac2bda9 None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:43:00.019703 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-7752fd5e-d904-4423-8cd8-a3924ac2bda9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:43:00.220301 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7752fd5e-d904-4423-8cd8-a3924ac2bda9 None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:43:00.337270 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c848d578-97f1-48c8-97da-b08d5765921e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7cd031eea90473aaaa0f1375f04dc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42f35b96b9d4499abfb5e5f7cca49adc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:43:00.337950 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c848d578-97f1-48c8-97da-b08d5765921e service neutron] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:43:00.338568 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c848d578-97f1-48c8-97da-b08d5765921e service neutron] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:43:00.339199 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c848d578-97f1-48c8-97da-b08d5765921e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:43:00.341160 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-c848d578-97f1-48c8-97da-b08d5765921e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:43:00.400190 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:43:00.477169 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c848d578-97f1-48c8-97da-b08d5765921e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:43:00.483054 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c848d578-97f1-48c8-97da-b08d5765921e service neutron] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:43:00.512426 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 635/1311] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 14:43:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 496 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:43:03.138408 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17700538-0f3d-42c0-b180-a9a6fe6b81c0 None None] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:43:03.138976 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17700538-0f3d-42c0-b180-a9a6fe6b81c0 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:43:03.138976 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17700538-0f3d-42c0-b180-a9a6fe6b81c0 None None] PATH_INFO: `/` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:43:03.140370 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 677/1312] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:43:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:43:03.146440 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02a9794-c64e-427c-b861-46b320578ccf None None] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:43:03.146887 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02a9794-c64e-427c-b861-46b320578ccf None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:43:03.146995 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02a9794-c64e-427c-b861-46b320578ccf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:43:03.273600 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.auth.core [None req-a02a9794-c64e-427c-b861-46b320578ccf None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:43:03.275636 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-a02a9794-c64e-427c-b861-46b320578ccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:43:03.399141 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 636/1313] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:43:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:43:03.439636 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17700538-0f3d-42c0-b180-a9a6fe6b81c0 None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:43:03.442006 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.fernet_utils [None req-17700538-0f3d-42c0-b180-a9a6fe6b81c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:43:03.566925 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17700538-0f3d-42c0-b180-a9a6fe6b81c0 None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:43:03.650988 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:43:03.654061 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b69bd53-666a-4687-857a-32612f0903ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7cd031eea90473aaaa0f1375f04dc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42f35b96b9d4499abfb5e5f7cca49adc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:43:03.656047 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b69bd53-666a-4687-857a-32612f0903ea service neutron] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:43:03.656047 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b69bd53-666a-4687-857a-32612f0903ea service neutron] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:43:03.656047 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b69bd53-666a-4687-857a-32612f0903ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:43:03.657610 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.fernet_utils [None req-9b69bd53-666a-4687-857a-32612f0903ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:43:03.786901 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b69bd53-666a-4687-857a-32612f0903ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:43:03.788646 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b69bd53-666a-4687-857a-32612f0903ea service neutron] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:43:03.810047 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 678/1314] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 14:43:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:43:06.449339 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aabb1a-9eb5-475e-abc9-90068ef13486 None None] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:43:06.449929 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aabb1a-9eb5-475e-abc9-90068ef13486 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:43:06.449929 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aabb1a-9eb5-475e-abc9-90068ef13486 None None] PATH_INFO: `/` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:43:06.451359 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 637/1315] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:43:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:43:06.457308 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf878373-fb1a-45f1-86eb-f40f9e2a25dc None None] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:43:06.457720 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf878373-fb1a-45f1-86eb-f40f9e2a25dc None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:43:06.457903 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf878373-fb1a-45f1-86eb-f40f9e2a25dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:43:06.522608 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.auth.core [None req-cf878373-fb1a-45f1-86eb-f40f9e2a25dc None None] MFA Rules not processed for user `d15c701d832345a08a0951afb6b6d914`. Rule list: `[]` (Enabled: `True`). {{(pid=218625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:43:06.523835 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.fernet_utils [None req-cf878373-fb1a-45f1-86eb-f40f9e2a25dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:43:06.576450 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:43:06.622908 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 679/1316] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:43:06 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:43:06.663597 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aabb1a-9eb5-475e-abc9-90068ef13486 None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:43:06.665230 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-b5aabb1a-9eb5-475e-abc9-90068ef13486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:43:06.674979 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:43:06.771748 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aabb1a-9eb5-475e-abc9-90068ef13486 None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:43:06.845793 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e91802-c69d-4e5d-9b61-1f979b7718a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7cd031eea90473aaaa0f1375f04dc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42f35b96b9d4499abfb5e5f7cca49adc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:43:06.846466 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e91802-c69d-4e5d-9b61-1f979b7718a4 service neutron] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:43:06.846772 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e91802-c69d-4e5d-9b61-1f979b7718a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:43:06.847563 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e91802-c69d-4e5d-9b61-1f979b7718a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:43:06.849272 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-a9e91802-c69d-4e5d-9b61-1f979b7718a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:43:06.914248 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:43:06.962671 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e91802-c69d-4e5d-9b61-1f979b7718a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:43:06.964107 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e91802-c69d-4e5d-9b61-1f979b7718a4 service neutron] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:43:06.982023 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 638/1317] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 14:43:06 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:43:09.803536 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0d80dd-54f4-4da2-83f7-a0f649a14fa4 None None] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:43:09.804095 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0d80dd-54f4-4da2-83f7-a0f649a14fa4 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:43:09.804095 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0d80dd-54f4-4da2-83f7-a0f649a14fa4 None None] PATH_INFO: `/` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:43:09.805478 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 680/1318] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:43:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:43:09.810829 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74252408-4d5e-4bb2-99ac-3220c049f37d None None] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:43:09.811221 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74252408-4d5e-4bb2-99ac-3220c049f37d None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:43:09.811395 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74252408-4d5e-4bb2-99ac-3220c049f37d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:43:09.881004 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.auth.core [None req-74252408-4d5e-4bb2-99ac-3220c049f37d None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:43:09.882287 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-74252408-4d5e-4bb2-99ac-3220c049f37d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:43:09.979520 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 639/1319] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:43:09 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:43:10.018310 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0d80dd-54f4-4da2-83f7-a0f649a14fa4 None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:43:10.020013 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.fernet_utils [None req-2e0d80dd-54f4-4da2-83f7-a0f649a14fa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:43:10.123577 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0d80dd-54f4-4da2-83f7-a0f649a14fa4 None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:43:10.196444 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9e5b35-4b3f-4cae-a6f8-e5ba5fbfd411 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7cd031eea90473aaaa0f1375f04dc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42f35b96b9d4499abfb5e5f7cca49adc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:43:10.197104 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9e5b35-4b3f-4cae-a6f8-e5ba5fbfd411 service neutron] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:43:10.197385 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9e5b35-4b3f-4cae-a6f8-e5ba5fbfd411 service neutron] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:43:10.197662 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9e5b35-4b3f-4cae-a6f8-e5ba5fbfd411 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:43:10.199343 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.fernet_utils [None req-2c9e5b35-4b3f-4cae-a6f8-e5ba5fbfd411 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:43:10.207866 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:43:10.313075 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9e5b35-4b3f-4cae-a6f8-e5ba5fbfd411 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:43:10.314498 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9e5b35-4b3f-4cae-a6f8-e5ba5fbfd411 service neutron] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:43:10.333050 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 681/1320] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 14:43:10 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:43:13.269150 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752d4ec0-5f6c-4b13-93fa-1b9860fbd973 None None] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:43:13.269675 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752d4ec0-5f6c-4b13-93fa-1b9860fbd973 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:43:13.269675 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752d4ec0-5f6c-4b13-93fa-1b9860fbd973 None None] PATH_INFO: `/` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:43:13.271229 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 640/1321] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:43:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:43:13.276412 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f4a873-d5ba-4552-8254-a6b53f1ccf2b None None] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:43:13.276823 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f4a873-d5ba-4552-8254-a6b53f1ccf2b None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:43:13.277101 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f4a873-d5ba-4552-8254-a6b53f1ccf2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:43:13.342908 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.auth.core [None req-04f4a873-d5ba-4552-8254-a6b53f1ccf2b None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:43:13.344121 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.fernet_utils [None req-04f4a873-d5ba-4552-8254-a6b53f1ccf2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:43:13.404433 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:43:13.446199 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 682/1322] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:43:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:43:13.482924 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752d4ec0-5f6c-4b13-93fa-1b9860fbd973 None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:43:13.484460 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-752d4ec0-5f6c-4b13-93fa-1b9860fbd973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:43:13.504276 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:43:13.592653 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752d4ec0-5f6c-4b13-93fa-1b9860fbd973 None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:43:13.668677 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31698e8-cebb-4b1d-8861-ee3a0bfd32d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7cd031eea90473aaaa0f1375f04dc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42f35b96b9d4499abfb5e5f7cca49adc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:43:13.669330 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31698e8-cebb-4b1d-8861-ee3a0bfd32d3 service neutron] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:43:13.669615 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31698e8-cebb-4b1d-8861-ee3a0bfd32d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:43:13.669921 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31698e8-cebb-4b1d-8861-ee3a0bfd32d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:43:13.671617 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-c31698e8-cebb-4b1d-8861-ee3a0bfd32d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:43:13.745596 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:43:13.785187 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31698e8-cebb-4b1d-8861-ee3a0bfd32d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:43:13.786676 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31698e8-cebb-4b1d-8861-ee3a0bfd32d3 service neutron] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:43:13.806177 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 641/1323] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 14:43:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:43:16.144301 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc06645-6366-4b87-b87e-7dd49d6aa2a6 None None] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:43:16.145021 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc06645-6366-4b87-b87e-7dd49d6aa2a6 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:43:16.145021 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc06645-6366-4b87-b87e-7dd49d6aa2a6 None None] PATH_INFO: `/` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:43:16.146240 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 683/1324] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:43:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:43:16.152633 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6908921f-743a-46ba-adc2-32fd3404213a None None] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:43:16.153115 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6908921f-743a-46ba-adc2-32fd3404213a None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:43:16.153353 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6908921f-743a-46ba-adc2-32fd3404213a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:43:16.223912 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.auth.core [None req-6908921f-743a-46ba-adc2-32fd3404213a None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:43:16.225133 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-6908921f-743a-46ba-adc2-32fd3404213a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:43:16.327519 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 642/1325] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:43:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:43:16.363664 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc06645-6366-4b87-b87e-7dd49d6aa2a6 None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:43:16.365254 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.fernet_utils [None req-dcc06645-6366-4b87-b87e-7dd49d6aa2a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:43:16.474169 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc06645-6366-4b87-b87e-7dd49d6aa2a6 None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:43:16.547521 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c87f028-2175-4731-8e2a-1203848d8fca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7cd031eea90473aaaa0f1375f04dc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42f35b96b9d4499abfb5e5f7cca49adc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:43:16.548181 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c87f028-2175-4731-8e2a-1203848d8fca service neutron] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:43:16.548467 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c87f028-2175-4731-8e2a-1203848d8fca service neutron] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:43:16.548753 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c87f028-2175-4731-8e2a-1203848d8fca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:43:16.550565 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.fernet_utils [None req-7c87f028-2175-4731-8e2a-1203848d8fca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:43:16.568485 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:43:16.665103 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c87f028-2175-4731-8e2a-1203848d8fca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:43:16.666604 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c87f028-2175-4731-8e2a-1203848d8fca service neutron] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:43:16.684608 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 684/1326] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 14:43:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:43:18.984342 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fb4fbd-c5a0-4d24-9513-4031fe9823d3 None None] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:43:18.984914 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fb4fbd-c5a0-4d24-9513-4031fe9823d3 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:43:18.984914 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fb4fbd-c5a0-4d24-9513-4031fe9823d3 None None] PATH_INFO: `/` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:43:18.986372 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 643/1327] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:43:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:43:18.992424 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb9d3ce-28e4-4b3c-89e3-5af042c5a762 None None] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:43:18.992890 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb9d3ce-28e4-4b3c-89e3-5af042c5a762 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:43:18.992992 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb9d3ce-28e4-4b3c-89e3-5af042c5a762 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:43:19.058438 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.auth.core [None req-ceb9d3ce-28e4-4b3c-89e3-5af042c5a762 None None] MFA Rules not processed for user `d15c701d832345a08a0951afb6b6d914`. Rule list: `[]` (Enabled: `True`). {{(pid=218625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:43:19.059663 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.fernet_utils [None req-ceb9d3ce-28e4-4b3c-89e3-5af042c5a762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:43:19.124910 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:43:19.157457 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 685/1328] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:43:18 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:43:19.209295 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ab0130-364a-4c0c-ac84-242599f95240 None None] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:43:19.209690 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ab0130-364a-4c0c-ac84-242599f95240 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:43:19.210011 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ab0130-364a-4c0c-ac84-242599f95240 None None] PATH_INFO: `/` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:43:19.211401 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 644/1329] 10.222.0.57 () {54 vars in 964 bytes} [Thu May 19 14:43:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:43:19.217138 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426a63ff-4f6c-454c-9768-a4495f6607aa None None] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:43:19.217426 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426a63ff-4f6c-454c-9768-a4495f6607aa None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:43:19.217701 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426a63ff-4f6c-454c-9768-a4495f6607aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:43:19.297825 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.auth.core [None req-426a63ff-4f6c-454c-9768-a4495f6607aa None None] MFA Rules not processed for user `c7cd031eea90473aaaa0f1375f04dc66`. Rule list: `[]` (Enabled: `True`). {{(pid=218625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:43:19.298997 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.fernet_utils [None req-426a63ff-4f6c-454c-9768-a4495f6607aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:43:19.375860 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 686/1330] 10.222.0.57 () {58 vars in 1061 bytes} [Thu May 19 14:43:19 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:43:19.384945 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ab0130-364a-4c0c-ac84-242599f95240 None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:43:19.386553 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-b5ab0130-364a-4c0c-ac84-242599f95240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:43:19.418794 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:43:19.491809 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ab0130-364a-4c0c-ac84-242599f95240 None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:43:19.563750 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f575ab37-9205-47ba-9a58-e0d14e3e3bcb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7cd031eea90473aaaa0f1375f04dc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42f35b96b9d4499abfb5e5f7cca49adc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:43:19.564307 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f575ab37-9205-47ba-9a58-e0d14e3e3bcb service neutron] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:43:19.564588 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f575ab37-9205-47ba-9a58-e0d14e3e3bcb service neutron] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:43:19.564859 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f575ab37-9205-47ba-9a58-e0d14e3e3bcb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:43:19.567142 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-f575ab37-9205-47ba-9a58-e0d14e3e3bcb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:43:19.642797 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:43:19.675456 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f575ab37-9205-47ba-9a58-e0d14e3e3bcb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:43:19.676882 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f575ab37-9205-47ba-9a58-e0d14e3e3bcb service neutron] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:43:19.695264 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 645/1331] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 14:43:19 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:43:22.602849 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783e08a6-88e5-40a4-b41c-d2cb8b741c2d None None] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:43:22.603374 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783e08a6-88e5-40a4-b41c-d2cb8b741c2d None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:43:22.603374 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783e08a6-88e5-40a4-b41c-d2cb8b741c2d None None] PATH_INFO: `/` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:43:22.604719 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 687/1332] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:43:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:43:22.610777 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe15de1f-4a02-45e7-8af4-58db6e9b6ec7 None None] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:43:22.611259 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe15de1f-4a02-45e7-8af4-58db6e9b6ec7 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:43:22.611360 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe15de1f-4a02-45e7-8af4-58db6e9b6ec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:43:22.682034 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.auth.core [None req-fe15de1f-4a02-45e7-8af4-58db6e9b6ec7 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:43:22.683286 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-fe15de1f-4a02-45e7-8af4-58db6e9b6ec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:43:22.785190 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 646/1333] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:43:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:43:22.822317 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783e08a6-88e5-40a4-b41c-d2cb8b741c2d None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:43:22.823861 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.fernet_utils [None req-783e08a6-88e5-40a4-b41c-d2cb8b741c2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:43:22.897074 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:43:22.932304 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783e08a6-88e5-40a4-b41c-d2cb8b741c2d None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:43:23.006612 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d448bd8-bfb7-4cd4-b1c4-4886cf67b727 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7cd031eea90473aaaa0f1375f04dc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42f35b96b9d4499abfb5e5f7cca49adc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:43:23.007267 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d448bd8-bfb7-4cd4-b1c4-4886cf67b727 service neutron] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:43:23.007549 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d448bd8-bfb7-4cd4-b1c4-4886cf67b727 service neutron] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:43:23.007824 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d448bd8-bfb7-4cd4-b1c4-4886cf67b727 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:43:23.009468 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.fernet_utils [None req-9d448bd8-bfb7-4cd4-b1c4-4886cf67b727 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:43:23.120374 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d448bd8-bfb7-4cd4-b1c4-4886cf67b727 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:43:23.121833 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d448bd8-bfb7-4cd4-b1c4-4886cf67b727 service neutron] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:43:23.132528 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:43:23.140786 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 688/1334] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 14:43:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:43:25.422623 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6332104-5961-4872-88a9-490ca2caf3ca None None] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:43:25.423204 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6332104-5961-4872-88a9-490ca2caf3ca None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:43:25.423204 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6332104-5961-4872-88a9-490ca2caf3ca None None] PATH_INFO: `/` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:43:25.425020 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 647/1335] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:43:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:43:25.430709 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcf3e9d-f469-4bf7-9c28-a8b47bcc8612 None None] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:43:25.431175 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcf3e9d-f469-4bf7-9c28-a8b47bcc8612 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:43:25.431282 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcf3e9d-f469-4bf7-9c28-a8b47bcc8612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:43:25.503814 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.auth.core [None req-3fcf3e9d-f469-4bf7-9c28-a8b47bcc8612 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:43:25.505163 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.fernet_utils [None req-3fcf3e9d-f469-4bf7-9c28-a8b47bcc8612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:43:25.601407 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 689/1336] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:43:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:43:25.641580 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6332104-5961-4872-88a9-490ca2caf3ca None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:43:25.643163 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-b6332104-5961-4872-88a9-490ca2caf3ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:43:25.684235 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:43:25.749939 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6332104-5961-4872-88a9-490ca2caf3ca None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:43:25.821612 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3408e59-aa00-4409-9e43-0cd921f68d4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7cd031eea90473aaaa0f1375f04dc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42f35b96b9d4499abfb5e5f7cca49adc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:43:25.822283 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3408e59-aa00-4409-9e43-0cd921f68d4e service neutron] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:43:25.822569 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3408e59-aa00-4409-9e43-0cd921f68d4e service neutron] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:43:25.822847 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3408e59-aa00-4409-9e43-0cd921f68d4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:43:25.824494 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-f3408e59-aa00-4409-9e43-0cd921f68d4e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:43:25.911579 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:43:25.938763 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3408e59-aa00-4409-9e43-0cd921f68d4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:43:25.940147 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3408e59-aa00-4409-9e43-0cd921f68d4e service neutron] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:43:25.959065 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 648/1337] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 14:43:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:43:28.676142 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8049a8d9-5d60-45a3-bf2b-03df52b5afea None None] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:43:28.676711 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8049a8d9-5d60-45a3-bf2b-03df52b5afea None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:43:28.676711 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8049a8d9-5d60-45a3-bf2b-03df52b5afea None None] PATH_INFO: `/` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:43:28.678112 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 690/1338] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:43:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:43:28.683612 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699c63af-3648-48c6-9c20-50042a6dff6a None None] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:43:28.684073 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699c63af-3648-48c6-9c20-50042a6dff6a None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:43:28.684177 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699c63af-3648-48c6-9c20-50042a6dff6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:43:28.748898 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.auth.core [None req-699c63af-3648-48c6-9c20-50042a6dff6a None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:43:28.750139 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-699c63af-3648-48c6-9c20-50042a6dff6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:43:28.848882 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 649/1339] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:43:28 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:43:28.885762 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8049a8d9-5d60-45a3-bf2b-03df52b5afea None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:43:28.887344 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.fernet_utils [None req-8049a8d9-5d60-45a3-bf2b-03df52b5afea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:43:28.967702 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:43:28.997477 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8049a8d9-5d60-45a3-bf2b-03df52b5afea None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:43:29.071140 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59401775-5e60-47bf-875f-ddbec5ad6bfb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7cd031eea90473aaaa0f1375f04dc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42f35b96b9d4499abfb5e5f7cca49adc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:43:29.071788 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59401775-5e60-47bf-875f-ddbec5ad6bfb service neutron] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:43:29.072073 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59401775-5e60-47bf-875f-ddbec5ad6bfb service neutron] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:43:29.072346 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59401775-5e60-47bf-875f-ddbec5ad6bfb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:43:29.074022 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.fernet_utils [None req-59401775-5e60-47bf-875f-ddbec5ad6bfb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:43:29.184318 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59401775-5e60-47bf-875f-ddbec5ad6bfb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:43:29.185764 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59401775-5e60-47bf-875f-ddbec5ad6bfb service neutron] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:43:29.204343 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 691/1340] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 14:43:28 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:43:33.584211 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503a5bcb-234c-4326-bf5c-9f267aa98d93 None None] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:43:33.584786 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503a5bcb-234c-4326-bf5c-9f267aa98d93 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:43:33.584786 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503a5bcb-234c-4326-bf5c-9f267aa98d93 None None] PATH_INFO: `/` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:43:33.586389 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 650/1341] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:43:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:43:33.591825 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafdbc3f-aa13-4188-b3fe-396b50c714c5 None None] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:43:33.592246 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafdbc3f-aa13-4188-b3fe-396b50c714c5 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:43:33.592357 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafdbc3f-aa13-4188-b3fe-396b50c714c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:43:33.598942 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=100 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:43:33.666936 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.auth.core [None req-bafdbc3f-aa13-4188-b3fe-396b50c714c5 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:43:33.668135 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.fernet_utils [None req-bafdbc3f-aa13-4188-b3fe-396b50c714c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:43:33.776375 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 692/1342] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:43:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:43:33.815974 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503a5bcb-234c-4326-bf5c-9f267aa98d93 None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:43:33.818303 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-503a5bcb-234c-4326-bf5c-9f267aa98d93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:43:33.871045 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:43:33.916832 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503a5bcb-234c-4326-bf5c-9f267aa98d93 None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:43:33.920472 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0f0288-ee04-4f21-82ff-9e9f14d0b6a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7cd031eea90473aaaa0f1375f04dc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42f35b96b9d4499abfb5e5f7cca49adc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:43:33.921113 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0f0288-ee04-4f21-82ff-9e9f14d0b6a3 service neutron] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:43:33.921395 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0f0288-ee04-4f21-82ff-9e9f14d0b6a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:43:33.921666 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0f0288-ee04-4f21-82ff-9e9f14d0b6a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:43:33.931601 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0f0288-ee04-4f21-82ff-9e9f14d0b6a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:43:33.933031 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0f0288-ee04-4f21-82ff-9e9f14d0b6a3 service neutron] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:43:33.942878 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 651/1343] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 14:43:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:43:36.822270 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001e5d44-aa2a-47ba-b8a3-78e228e1b0f7 None None] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:43:36.822803 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001e5d44-aa2a-47ba-b8a3-78e228e1b0f7 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:43:36.822803 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001e5d44-aa2a-47ba-b8a3-78e228e1b0f7 None None] PATH_INFO: `/` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:43:36.824161 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 693/1344] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:43:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:43:36.829527 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b649cb96-10c9-4f87-ae0e-9108ce192107 None None] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:43:36.830020 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b649cb96-10c9-4f87-ae0e-9108ce192107 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:43:36.830124 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b649cb96-10c9-4f87-ae0e-9108ce192107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:43:36.862753 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.auth.core [None req-b649cb96-10c9-4f87-ae0e-9108ce192107 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:43:36.864146 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-b649cb96-10c9-4f87-ae0e-9108ce192107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:43:36.944342 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 652/1345] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:43:36 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:43:36.981568 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001e5d44-aa2a-47ba-b8a3-78e228e1b0f7 None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:43:36.993242 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001e5d44-aa2a-47ba-b8a3-78e228e1b0f7 None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:43:36.996868 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5395c2a0-13de-4e61-a91a-dca066fdb4e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7cd031eea90473aaaa0f1375f04dc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42f35b96b9d4499abfb5e5f7cca49adc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:43:36.997510 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5395c2a0-13de-4e61-a91a-dca066fdb4e2 service neutron] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:43:36.997929 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5395c2a0-13de-4e61-a91a-dca066fdb4e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:43:36.998210 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5395c2a0-13de-4e61-a91a-dca066fdb4e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:43:37.007036 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5395c2a0-13de-4e61-a91a-dca066fdb4e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:43:37.008499 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5395c2a0-13de-4e61-a91a-dca066fdb4e2 service neutron] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:43:37.018692 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 694/1346] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 14:43:36 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:43:41.347270 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2834e2d8-c479-4b11-abf1-7bcc5373e57b None None] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:43:41.347790 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2834e2d8-c479-4b11-abf1-7bcc5373e57b None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:43:41.347790 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2834e2d8-c479-4b11-abf1-7bcc5373e57b None None] PATH_INFO: `/` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:43:41.349356 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 653/1347] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:43:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:43:41.354573 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94d53e0-ca96-40f2-8da8-1007bbe3f786 None None] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:43:41.355003 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94d53e0-ca96-40f2-8da8-1007bbe3f786 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:43:41.355313 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94d53e0-ca96-40f2-8da8-1007bbe3f786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:43:41.374616 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.auth.core [None req-a94d53e0-ca96-40f2-8da8-1007bbe3f786 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:43:41.375808 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.fernet_utils [None req-a94d53e0-ca96-40f2-8da8-1007bbe3f786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:43:41.383821 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 695/1348] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:43:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:43:41.421806 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2834e2d8-c479-4b11-abf1-7bcc5373e57b None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:43:41.433470 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2834e2d8-c479-4b11-abf1-7bcc5373e57b None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:43:41.437046 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaa8637-f596-4794-804b-874805244345 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7cd031eea90473aaaa0f1375f04dc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42f35b96b9d4499abfb5e5f7cca49adc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:43:41.437684 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaa8637-f596-4794-804b-874805244345 service neutron] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:43:41.438009 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaa8637-f596-4794-804b-874805244345 service neutron] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:43:41.438294 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaa8637-f596-4794-804b-874805244345 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:43:41.447398 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adaa8637-f596-4794-804b-874805244345 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:43:41.448842 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adaa8637-f596-4794-804b-874805244345 service neutron] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:43:41.459081 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 654/1349] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 14:43:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:43:47.052725 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5db20d8-ff0a-445a-abd3-7a07788a6ba6 None None] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:43:47.052725 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5db20d8-ff0a-445a-abd3-7a07788a6ba6 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:43:47.052725 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5db20d8-ff0a-445a-abd3-7a07788a6ba6 None None] PATH_INFO: `/` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:43:47.052725 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 696/1350] 10.222.0.57 () {54 vars in 921 bytes} [Thu May 19 14:43:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:43:47.055148 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7d6c0c-ff2c-4f00-a438-c518cb36c6d4 None None] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:43:47.055858 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7d6c0c-ff2c-4f00-a438-c518cb36c6d4 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:43:47.056297 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7d6c0c-ff2c-4f00-a438-c518cb36c6d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:43:47.132332 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.auth.core [None req-6d7d6c0c-ff2c-4f00-a438-c518cb36c6d4 None None] MFA Rules not processed for user `b230516fafae4ac092c036f3b9aeb047`. Rule list: `[]` (Enabled: `True`). {{(pid=218624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:43:47.133951 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-6d7d6c0c-ff2c-4f00-a438-c518cb36c6d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:43:47.190381 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=100 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:43:47.233453 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 655/1351] 10.222.0.57 () {58 vars in 1018 bytes} [Thu May 19 14:43:47 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:43:51.369281 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=72 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:43:51.863195 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c61ee38-9bb0-4fce-9bb2-0b2404245ab8 None None] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:43:51.863381 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c61ee38-9bb0-4fce-9bb2-0b2404245ab8 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:43:51.863729 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c61ee38-9bb0-4fce-9bb2-0b2404245ab8 None None] PATH_INFO: `/` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:43:51.865382 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 697/1352] 10.222.0.57 () {54 vars in 921 bytes} [Thu May 19 14:43:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:43:51.870839 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72401d7d-fa39-4e7e-9e96-bd7ee8056619 None None] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:43:51.871432 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72401d7d-fa39-4e7e-9e96-bd7ee8056619 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:43:51.871657 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72401d7d-fa39-4e7e-9e96-bd7ee8056619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:43:51.892366 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.auth.core [None req-72401d7d-fa39-4e7e-9e96-bd7ee8056619 None None] MFA Rules not processed for user `b230516fafae4ac092c036f3b9aeb047`. Rule list: `[]` (Enabled: `True`). {{(pid=218624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:43:51.894208 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-72401d7d-fa39-4e7e-9e96-bd7ee8056619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:43:51.903367 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 656/1353] 10.222.0.57 () {58 vars in 1018 bytes} [Thu May 19 14:43:51 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:43:53.350105 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df05cd8-43b1-4a16-ad4d-b497b88e3980 None None] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:43:53.350658 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df05cd8-43b1-4a16-ad4d-b497b88e3980 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:43:53.350658 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df05cd8-43b1-4a16-ad4d-b497b88e3980 None None] PATH_INFO: `/` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:43:53.352059 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 698/1354] 10.222.0.57 () {54 vars in 921 bytes} [Thu May 19 14:43:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:43:53.357119 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e89b6c3-a17b-4bc7-8a53-f05601b90b9d None None] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:43:53.357558 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e89b6c3-a17b-4bc7-8a53-f05601b90b9d None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:43:53.357678 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e89b6c3-a17b-4bc7-8a53-f05601b90b9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:43:53.375705 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.auth.core [None req-6e89b6c3-a17b-4bc7-8a53-f05601b90b9d None None] MFA Rules not processed for user `b230516fafae4ac092c036f3b9aeb047`. Rule list: `[]` (Enabled: `True`). {{(pid=218624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:43:53.377445 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-6e89b6c3-a17b-4bc7-8a53-f05601b90b9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:43:53.386502 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 657/1355] 10.222.0.57 () {58 vars in 1018 bytes} [Thu May 19 14:43:53 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 14:43:57.525446 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99192442-7444-4704-a80e-764beaa1edfc None None] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:43:57.526412 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99192442-7444-4704-a80e-764beaa1edfc None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:43:57.526863 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99192442-7444-4704-a80e-764beaa1edfc None None] PATH_INFO: `/` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:43:57.528406 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 699/1356] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:43:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:43:57.534623 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4987d46-1049-4c31-920d-d2b1aa75f3f8 None None] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:43:57.535122 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4987d46-1049-4c31-920d-d2b1aa75f3f8 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:43:57.535217 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4987d46-1049-4c31-920d-d2b1aa75f3f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:43:57.556267 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.auth.core [None req-f4987d46-1049-4c31-920d-d2b1aa75f3f8 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:43:57.557555 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-f4987d46-1049-4c31-920d-d2b1aa75f3f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:43:57.566158 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 658/1357] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:43:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:43:57.628432 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03198a55-be80-4117-9c8e-62ee52116c2e None None] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:43:57.628722 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03198a55-be80-4117-9c8e-62ee52116c2e None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:43:57.629315 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03198a55-be80-4117-9c8e-62ee52116c2e None None] PATH_INFO: `/` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:43:57.630711 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 700/1358] 10.222.0.57 () {54 vars in 961 bytes} [Thu May 19 14:43:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:43:57.638693 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13ab4c9-d273-487c-88cd-82642704d1e3 None None] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:43:57.639071 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13ab4c9-d273-487c-88cd-82642704d1e3 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:43:57.639424 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13ab4c9-d273-487c-88cd-82642704d1e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:43:57.695561 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.auth.core [None req-c13ab4c9-d273-487c-88cd-82642704d1e3 None None] MFA Rules not processed for user `a9591727b2f54244baf90b11a8e2277f`. Rule list: `[]` (Enabled: `True`). {{(pid=218624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:43:57.696875 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-c13ab4c9-d273-487c-88cd-82642704d1e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:43:57.786685 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 659/1359] 10.222.0.57 () {58 vars in 1058 bytes} [Thu May 19 14:43:57 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:43:57.795534 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03198a55-be80-4117-9c8e-62ee52116c2e None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:43:57.811315 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03198a55-be80-4117-9c8e-62ee52116c2e None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:43:57.818129 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bb7037-3fb8-4f8f-80fa-8793f0ecc7cc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9591727b2f54244baf90b11a8e2277f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42f35b96b9d4499abfb5e5f7cca49adc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:43:57.819045 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bb7037-3fb8-4f8f-80fa-8793f0ecc7cc service nova] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:43:57.819591 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bb7037-3fb8-4f8f-80fa-8793f0ecc7cc service nova] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:43:57.820090 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bb7037-3fb8-4f8f-80fa-8793f0ecc7cc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:43:57.828995 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bb7037-3fb8-4f8f-80fa-8793f0ecc7cc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:43:57.831830 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bb7037-3fb8-4f8f-80fa-8793f0ecc7cc service nova] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:43:57.842346 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 701/1360] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 14:43:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:44:00.308096 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f395309-4391-4511-bf01-c2aae628bd6a None None] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:44:00.308652 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f395309-4391-4511-bf01-c2aae628bd6a None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:44:00.308652 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f395309-4391-4511-bf01-c2aae628bd6a None None] PATH_INFO: `/` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:44:00.310088 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 660/1361] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:44:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:44:00.315671 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89738aa-7b91-444d-8378-18a2366787b2 None None] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:44:00.316158 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89738aa-7b91-444d-8378-18a2366787b2 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:44:00.316282 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89738aa-7b91-444d-8378-18a2366787b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:44:00.334525 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.auth.core [None req-c89738aa-7b91-444d-8378-18a2366787b2 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:44:00.335741 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.fernet_utils [None req-c89738aa-7b91-444d-8378-18a2366787b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:44:00.344139 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 702/1362] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:44:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:44:00.388659 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f395309-4391-4511-bf01-c2aae628bd6a None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:44:00.403474 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f395309-4391-4511-bf01-c2aae628bd6a None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:44:00.410103 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bed767-705f-4c1a-97c0-43a8ceb2cb9e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9591727b2f54244baf90b11a8e2277f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42f35b96b9d4499abfb5e5f7cca49adc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:44:00.410820 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bed767-705f-4c1a-97c0-43a8ceb2cb9e service nova] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:44:00.411110 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bed767-705f-4c1a-97c0-43a8ceb2cb9e service nova] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:44:00.411393 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bed767-705f-4c1a-97c0-43a8ceb2cb9e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:44:00.420248 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bed767-705f-4c1a-97c0-43a8ceb2cb9e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:44:00.422368 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bed767-705f-4c1a-97c0-43a8ceb2cb9e service nova] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:44:00.432872 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 661/1363] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 14:44:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:44:03.107711 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9963d35b-f420-4b6c-b24e-dd64ac6f1032 None None] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:44:03.108254 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9963d35b-f420-4b6c-b24e-dd64ac6f1032 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:44:03.108254 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9963d35b-f420-4b6c-b24e-dd64ac6f1032 None None] PATH_INFO: `/` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:44:03.109552 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 703/1364] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:44:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:44:03.115131 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca06099f-a890-4fc1-b64e-eea23414920c None None] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:44:03.115601 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca06099f-a890-4fc1-b64e-eea23414920c None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:44:03.116041 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca06099f-a890-4fc1-b64e-eea23414920c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:44:03.135296 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.auth.core [None req-ca06099f-a890-4fc1-b64e-eea23414920c None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:44:03.136499 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-ca06099f-a890-4fc1-b64e-eea23414920c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:44:03.145468 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 662/1365] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:44:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:44:03.192018 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c5288f-d0fb-43cb-b045-6788ec46ae8f None None] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:44:03.192364 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c5288f-d0fb-43cb-b045-6788ec46ae8f None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:44:03.192584 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c5288f-d0fb-43cb-b045-6788ec46ae8f None None] PATH_INFO: `/` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:44:03.194168 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 704/1366] 10.222.0.57 () {54 vars in 961 bytes} [Thu May 19 14:44:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:44:03.199358 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86180b8-7dfb-4986-9dbc-6dc655502735 None None] REQUEST_METHOD: `POST` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:44:03.199656 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86180b8-7dfb-4986-9dbc-6dc655502735 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:44:03.199936 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86180b8-7dfb-4986-9dbc-6dc655502735 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:44:03.219304 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.auth.core [None req-a86180b8-7dfb-4986-9dbc-6dc655502735 None None] MFA Rules not processed for user `a9591727b2f54244baf90b11a8e2277f`. Rule list: `[]` (Enabled: `True`). {{(pid=218624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:44:03.220541 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.fernet_utils [None req-a86180b8-7dfb-4986-9dbc-6dc655502735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:44:03.230203 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 663/1367] 10.222.0.57 () {58 vars in 1058 bytes} [Thu May 19 14:44:03 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:44:03.238128 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c5288f-d0fb-43cb-b045-6788ec46ae8f None None] Authenticating user token {{(pid=218625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:44:03.253244 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c5288f-d0fb-43cb-b045-6788ec46ae8f None None] Validating token access rules against request {{(pid=218625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:44:03.258733 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d65eb7-d4e9-428e-a3f5-44a53fd1de47 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9591727b2f54244baf90b11a8e2277f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42f35b96b9d4499abfb5e5f7cca49adc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:44:03.259382 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d65eb7-d4e9-428e-a3f5-44a53fd1de47 service nova] REQUEST_METHOD: `GET` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:44:03.259667 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d65eb7-d4e9-428e-a3f5-44a53fd1de47 service nova] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:44:03.259943 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d65eb7-d4e9-428e-a3f5-44a53fd1de47 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:44:03.269098 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d65eb7-d4e9-428e-a3f5-44a53fd1de47 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:44:03.270724 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d65eb7-d4e9-428e-a3f5-44a53fd1de47 service nova] RBAC: Authorization granted {{(pid=218625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:44:03.280755 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 705/1368] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 14:44:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:44:04.097608 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc93f1e2-fb28-42a1-8c8e-edd5f670c91a None None] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:44:04.098225 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc93f1e2-fb28-42a1-8c8e-edd5f670c91a None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:44:04.098511 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc93f1e2-fb28-42a1-8c8e-edd5f670c91a None None] PATH_INFO: `/` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:44:04.100231 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 664/1369] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:44:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:44:04.106294 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0895d5-3e59-4b7b-ada5-31152f5474f1 None None] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:44:04.106673 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0895d5-3e59-4b7b-ada5-31152f5474f1 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:44:04.106862 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0895d5-3e59-4b7b-ada5-31152f5474f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:44:04.127356 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.auth.core [None req-1b0895d5-3e59-4b7b-ada5-31152f5474f1 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:44:04.128614 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.fernet_utils [None req-1b0895d5-3e59-4b7b-ada5-31152f5474f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:44:04.137997 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 706/1370] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:44:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:44:05.659795 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e45be1-e76b-4084-84d7-25a349e6e195 None None] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:44:05.660367 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e45be1-e76b-4084-84d7-25a349e6e195 None None] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:44:05.660367 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e45be1-e76b-4084-84d7-25a349e6e195 None None] PATH_INFO: `/` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:44:05.661742 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 665/1371] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:44:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:44:05.667223 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c63834-647b-4400-9a7a-e4927a6f0e40 None None] REQUEST_METHOD: `POST` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:44:05.667650 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c63834-647b-4400-9a7a-e4927a6f0e40 None None] SCRIPT_NAME: `/identity` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:44:05.667779 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c63834-647b-4400-9a7a-e4927a6f0e40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:44:05.686501 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.auth.core [None req-a2c63834-647b-4400-9a7a-e4927a6f0e40 None None] MFA Rules not processed for user `d8062aba2d374babbc9d68178a9546e3`. Rule list: `[]` (Enabled: `True`). {{(pid=218625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:44:05.687703 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG keystone.common.fernet_utils [None req-a2c63834-647b-4400-9a7a-e4927a6f0e40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:44:05.695920 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: [pid: 218625|app: 0|req: 707/1372] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:44:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:44:05.741209 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e45be1-e76b-4084-84d7-25a349e6e195 None None] Authenticating user token {{(pid=218624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:44:05.756353 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e45be1-e76b-4084-84d7-25a349e6e195 None None] Validating token access rules against request {{(pid=218624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:44:05.761881 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7f7b90-9c30-4449-a69d-e13cd087db87 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9591727b2f54244baf90b11a8e2277f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42f35b96b9d4499abfb5e5f7cca49adc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:44:05.762521 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7f7b90-9c30-4449-a69d-e13cd087db87 service nova] REQUEST_METHOD: `GET` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:44:05.762807 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7f7b90-9c30-4449-a69d-e13cd087db87 service nova] SCRIPT_NAME: `/identity` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:44:05.763083 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7f7b90-9c30-4449-a69d-e13cd087db87 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=218624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:44:05.771516 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7f7b90-9c30-4449-a69d-e13cd087db87 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:44:05.773094 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7f7b90-9c30-4449-a69d-e13cd087db87 service nova] RBAC: Authorization granted {{(pid=218624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:44:05.783280 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: [pid: 218624|app: 0|req: 666/1373] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 14:44:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:44:15.681308 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: DEBUG dbcounter [-] [218625] Writing DB stats keystone:SELECT=21 {{(pid=218625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:44:15.778829 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: DEBUG dbcounter [-] [218624] Writing DB stats keystone:SELECT=95 {{(pid=218624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:44:27.477347 dv-cinder-842541-1-iscsi systemd[1]: Stopping Devstack devstack@keystone.service... May 19 14:44:27.477922 dv-cinder-842541-1-iscsi devstack@keystone.service[218616]: Thu May 19 14:44:27 2022 - graceful shutdown triggered... May 19 14:44:27.478499 dv-cinder-842541-1-iscsi devstack@keystone.service[218624]: Gracefully killing worker 1 (pid: 218624)... May 19 14:44:27.478900 dv-cinder-842541-1-iscsi devstack@keystone.service[218625]: Gracefully killing worker 2 (pid: 218625)... May 19 14:44:28.479648 dv-cinder-842541-1-iscsi devstack@keystone.service[218616]: worker 1 buried after 1 seconds May 19 14:44:28.479648 dv-cinder-842541-1-iscsi devstack@keystone.service[218616]: worker 2 buried after 1 seconds May 19 14:44:28.479648 dv-cinder-842541-1-iscsi devstack@keystone.service[218616]: goodbye to uWSGI. May 19 14:44:28.481117 dv-cinder-842541-1-iscsi systemd[1]: devstack@keystone.service: Succeeded. May 19 14:44:28.481659 dv-cinder-842541-1-iscsi systemd[1]: Stopped Devstack devstack@keystone.service. May 19 14:50:22.535794 dv-cinder-842541-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... May 19 14:50:22.543273 dv-cinder-842541-1-iscsi devstack@keystone.service[281774]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini May 19 14:50:22.545271 dv-cinder-842541-1-iscsi devstack@keystone.service[281774]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu May 19 14:50:22 2022] *** May 19 14:50:22.545271 dv-cinder-842541-1-iscsi devstack@keystone.service[281774]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 May 19 14:50:22.545271 dv-cinder-842541-1-iscsi devstack@keystone.service[281774]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 May 19 14:50:22.545271 dv-cinder-842541-1-iscsi devstack@keystone.service[281774]: nodename: dv-cinder-842541-1-iscsi May 19 14:50:22.545271 dv-cinder-842541-1-iscsi devstack@keystone.service[281774]: machine: x86_64 May 19 14:50:22.545271 dv-cinder-842541-1-iscsi devstack@keystone.service[281774]: clock source: unix May 19 14:50:22.545271 dv-cinder-842541-1-iscsi devstack@keystone.service[281774]: pcre jit disabled May 19 14:50:22.545919 dv-cinder-842541-1-iscsi devstack@keystone.service[281774]: detected number of CPU cores: 4 May 19 14:50:22.545919 dv-cinder-842541-1-iscsi devstack@keystone.service[281774]: current working directory: / May 19 14:50:22.545919 dv-cinder-842541-1-iscsi devstack@keystone.service[281774]: detected binary path: /usr/bin/uwsgi-core May 19 14:50:22.545919 dv-cinder-842541-1-iscsi devstack@keystone.service[281774]: your processes number limit is 47680 May 19 14:50:22.545919 dv-cinder-842541-1-iscsi devstack@keystone.service[281774]: your memory page size is 4096 bytes May 19 14:50:22.545919 dv-cinder-842541-1-iscsi devstack@keystone.service[281774]: detected max file descriptor number: 2048 May 19 14:50:22.545919 dv-cinder-842541-1-iscsi devstack@keystone.service[281774]: lock engine: pthread robust mutexes May 19 14:50:22.545919 dv-cinder-842541-1-iscsi devstack@keystone.service[281774]: thunder lock: enabled May 19 14:50:22.545919 dv-cinder-842541-1-iscsi devstack@keystone.service[281774]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 May 19 14:50:22.545919 dv-cinder-842541-1-iscsi devstack@keystone.service[281774]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] May 19 14:50:22.597272 dv-cinder-842541-1-iscsi devstack@keystone.service[281774]: Python main interpreter initialized at 0x56071f8189c0 May 19 14:50:22.597272 dv-cinder-842541-1-iscsi devstack@keystone.service[281774]: python threads support enabled May 19 14:50:22.597272 dv-cinder-842541-1-iscsi devstack@keystone.service[281774]: your server socket listen backlog is limited to 100 connections May 19 14:50:22.597272 dv-cinder-842541-1-iscsi devstack@keystone.service[281774]: your mercy for graceful operations on workers is 90 seconds May 19 14:50:22.597690 dv-cinder-842541-1-iscsi devstack@keystone.service[281774]: mapped 403077 bytes (393 KB) for 2 cores May 19 14:50:22.597773 dv-cinder-842541-1-iscsi devstack@keystone.service[281774]: *** Operational MODE: preforking *** May 19 14:50:22.598013 dv-cinder-842541-1-iscsi devstack@keystone.service[281774]: *** uWSGI is running in multiple interpreter mode *** May 19 14:50:22.598013 dv-cinder-842541-1-iscsi devstack@keystone.service[281774]: spawned uWSGI master process (pid: 281774) May 19 14:50:22.598219 dv-cinder-842541-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. May 19 14:50:22.599302 dv-cinder-842541-1-iscsi devstack@keystone.service[281774]: spawned uWSGI worker 1 (pid: 281785, cores: 1) May 19 14:50:22.599302 dv-cinder-842541-1-iscsi devstack@keystone.service[281774]: spawned uWSGI worker 2 (pid: 281786, cores: 1) May 19 14:50:22.599302 dv-cinder-842541-1-iscsi devstack@keystone.service[281774]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... May 19 14:50:28.334145 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 14:50:28.334145 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: from cryptography.utils import int_from_bytes May 19 14:50:28.335082 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 14:50:28.335082 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: from cryptography.utils import int_from_bytes May 19 14:50:28.362693 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 14:50:28.362693 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: from cryptography.utils import int_from_bytes May 19 14:50:28.363278 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 14:50:28.363278 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: from cryptography.utils import int_from_bytes May 19 14:50:28.384983 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 19 14:50:28.385187 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 19 14:50:28.385374 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] command line args: [] {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 19 14:50:28.385651 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 19 14:50:28.385836 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] ================================================================================ {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 19 14:50:28.386126 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] admin_token = **** {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:50:28.386381 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] config_dir = [] {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:50:28.386574 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:50:28.386762 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] config_source = [] {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:50:28.386948 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] debug = True {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:50:28.387154 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:50:28.387650 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:50:28.387771 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] insecure_debug = False {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:50:28.387969 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:50:28.388225 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:50:28.388467 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] list_limit = None {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:50:28.388653 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] log_config_append = None {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:50:28.388837 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:50:28.389021 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] log_dir = None {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:50:28.389206 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] log_file = None {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:50:28.389450 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:50:28.389715 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:50:28.389995 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:50:28.390255 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:50:28.390441 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:50:28.390625 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:50:28.390880 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:50:28.391134 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(user_domain)s %(project_domain)s {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:50:28.391375 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:50:28.391618 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:50:28.391860 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:50:28.392101 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:50:28.392356 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:50:28.392609 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] notification_format = cadf {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:50:28.392894 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:50:28.393395 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.57/identity {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:50:28.393583 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] publish_errors = False {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:50:28.393824 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:50:28.394088 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:50:28.394271 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:50:28.394524 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:50:28.394766 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:50:28.395005 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] standard_threads = False {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:50:28.395244 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] strict_password_check = False {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:50:28.395493 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:50:28.395675 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] use_eventlog = False {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:50:28.395966 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] use_journal = False {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:50:28.396284 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] use_json = False {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:50:28.396609 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] use_stderr = False {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:50:28.396798 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] use_syslog = False {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:50:28.397043 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] watch_log_file = False {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:50:28.397310 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.397560 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.397831 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.398306 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.398436 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.398606 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.398864 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.399109 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.399368 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.399368 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.399880 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.400125 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.400625 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.400625 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.401110 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.401110 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.401333 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.401597 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.401883 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.402144 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.402389 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.402651 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.402922 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.403184 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.403442 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.403682 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.403923 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.404162 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.404401 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.404662 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.404920 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.405159 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.405399 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.405654 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.405939 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.406189 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.406448 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.406687 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.406931 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.407176 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.407415 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.407652 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.407892 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.408130 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.408375 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.408630 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.408870 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.409116 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.409354 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.409599 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.409891 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.410126 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.410367 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.410613 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.410859 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.411104 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.411351 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.412253 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.412507 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.412763 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.413010 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.413114 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 19 14:50:28.413207 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 19 14:50:28.413285 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.413402 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] command line args: [] {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 19 14:50:28.413585 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.413684 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 19 14:50:28.413783 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.413922 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] ================================================================================ {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 19 14:50:28.414019 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.414141 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] admin_token = **** {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:50:28.414250 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.414462 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] config_dir = [] {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:50:28.414611 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.414721 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:50:28.414816 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.414908 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] config_source = [] {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:50:28.415005 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] debug = True {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:50:28.415091 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.415183 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:50:28.415300 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.415403 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:50:28.415498 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.415590 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] insecure_debug = False {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:50:28.415682 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.415808 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:50:28.415910 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.416065 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:50:28.416172 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.416304 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] list_limit = None {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:50:28.416404 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.416496 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] log_config_append = None {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:50:28.416616 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] cache.enabled = True {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.416717 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:50:28.416824 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] log_dir = None {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:50:28.416924 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.417011 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] log_file = None {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:50:28.417117 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.417203 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:50:28.417338 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.417445 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:50:28.417587 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.417746 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:50:28.417835 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.417969 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:50:28.418071 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.418164 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:50:28.418302 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:50:28.418412 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.418550 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.418652 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:50:28.418800 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(user_domain)s %(project_domain)s {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:50:28.418899 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.419042 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:50:28.419140 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.419282 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:50:28.419385 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.419522 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:50:28.419616 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.419761 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:50:28.419860 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.420006 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.420113 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:50:28.420254 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] notification_format = cadf {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:50:28.420354 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.420499 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.420598 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:50:28.420740 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.420980 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.421093 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.57/identity {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:50:28.421208 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] publish_errors = False {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:50:28.421306 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.421448 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:50:28.421547 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.421685 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:50:28.421785 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.421901 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:50:28.421996 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.422146 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:50:28.422253 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.422390 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:50:28.422483 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.422632 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] standard_threads = False {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:50:28.422731 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.422872 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] strict_password_check = False {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:50:28.423032 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.423132 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:50:28.423291 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] use_eventlog = False {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:50:28.423389 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.423480 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] use_journal = False {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:50:28.423572 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.423669 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] use_json = False {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:50:28.423760 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] auth.external = None {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.423851 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] use_stderr = False {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:50:28.423976 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] use_syslog = False {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:50:28.424069 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] auth.mapped = None {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.424213 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] watch_log_file = False {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:50:28.424319 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.424488 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.424589 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.424743 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] auth.password = None {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.424859 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.424985 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] auth.token = None {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.425088 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.425235 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.425334 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.425480 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] catalog.caching = True {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.425580 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.425730 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.425947 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.426038 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.426128 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.426241 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.426342 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.426497 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.426604 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.426739 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.426840 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.426985 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] credential.caching = True {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.427705 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.427833 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.428032 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] credential.driver = sql {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.428133 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.428305 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.428399 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.428568 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.428678 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.428817 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.428944 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.429074 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.429182 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.429321 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.429451 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.429598 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.429719 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.429879 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.429997 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.430131 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.430236 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.430400 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.430500 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.430696 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.430797 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.430952 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.431052 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.431226 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.431327 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.431477 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.431577 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.431718 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.431830 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.431961 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.432061 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] federation.caching = True {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.432208 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.432301 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.432462 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] federation.driver = sql {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.432568 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.432718 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.432817 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.432957 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.433058 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.433197 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.433303 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.433448 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.433547 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.433702 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.433793 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.433970 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.434075 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.434227 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.434325 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.434469 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.434574 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.434721 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.434823 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.434962 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] identity.caching = True {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.435060 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.435185 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.435284 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.435433 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.435531 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.435652 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.435790 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.435893 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.435985 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.436121 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.436232 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] identity.driver = sql {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.436378 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.436491 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.436617 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.436716 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.436864 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.436968 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.437102 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.437203 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.437346 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.437451 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.437584 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.437688 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.437825 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.437951 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.438091 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.438191 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.438338 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.438449 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.438584 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.438684 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.438830 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.438924 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.439077 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.439177 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.439321 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.439415 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.439561 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.439676 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.439818 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.439916 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.440058 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.440158 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.440301 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.440393 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.440537 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.440646 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.440781 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.440876 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.441032 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.441143 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.441275 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.441373 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.441521 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.441615 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.441757 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.441914 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.442033 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.442126 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.442290 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.442399 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.442531 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.442685 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.442811 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.442906 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.443059 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.443164 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.443628 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.443753 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.443843 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.443945 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.444032 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] ldap.password = **** {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.444123 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] cache.enabled = True {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.444216 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.444313 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.444398 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.444496 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.444589 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.444674 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.444759 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.444852 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.444938 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.445035 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.445168 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.445263 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.445413 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.445513 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.445652 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.445750 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.445913 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.446010 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.446173 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.446292 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.446423 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.446523 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.446665 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.446758 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.446908 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.447014 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.447150 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.447250 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.447386 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] ldap.user = None {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.447499 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.447722 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.448076 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.448173 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.448263 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.448362 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.448454 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.448566 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.448658 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.448821 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.448918 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.449022 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.449114 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.449266 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.449380 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.449510 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.449609 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.449766 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.449912 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.450016 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.450108 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.450262 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.450394 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.450493 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.450671 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.450773 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.450925 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.451024 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.451169 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] auth.external = None {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.451262 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.451412 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] auth.mapped = None {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.451517 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.451694 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.451793 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.451931 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.452030 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.452175 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] auth.password = None {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.452279 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.452417 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] auth.token = None {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.452511 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.452656 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.452755 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.452900 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] catalog.caching = True {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.453005 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.453138 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.453244 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.453385 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.453491 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.453638 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.453742 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.453903 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.454009 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.454145 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.454238 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] policy.driver = sql {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.454391 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] credential.caching = True {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.454493 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.454641 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.454734 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] credential.driver = sql {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.454886 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.454985 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.455127 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] receipt.caching = True {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.455241 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.455370 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.455463 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.455618 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.455715 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.455862 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.455967 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.456097 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.456201 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.456340 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.456441 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.456587 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] resource.caching = True {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.456699 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.456844 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.456938 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.457098 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] resource.driver = sql {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.457190 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.457342 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.457445 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.457597 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.457697 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.457898 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.458008 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.458101 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] revoke.caching = True {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.458196 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.458468 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.458572 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] federation.caching = True {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.458719 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.458804 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.459046 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] role.cache_time = None {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.459156 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] federation.driver = sql {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.459245 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] role.caching = True {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.459335 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.459427 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] role.driver = sql {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.459512 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.459661 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] role.list_limit = None {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.459756 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.459917 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.460011 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.460174 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.460274 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.460417 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.460522 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.460677 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.460776 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.460919 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.461020 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.461170 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.461262 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.461410 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] identity.caching = True {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.461503 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.461664 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.461762 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.461928 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.462024 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.462174 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.462266 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.462418 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.462513 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.462673 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.462767 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] identity.driver = sql {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.462909 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.463000 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.463154 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.463335 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.463448 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.463581 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.463675 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.463839 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.463945 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.464096 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.464212 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.464337 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.464444 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.464588 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.464688 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.464827 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.464940 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.465081 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.465189 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.465339 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.465432 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.465583 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.465684 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.465824 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.465988 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.466077 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.466166 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.466311 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.466412 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.466566 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.466673 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.466822 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.466915 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.467073 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.467173 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.467314 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.467405 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.467559 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.467658 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.467798 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] token.cache_time = None {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.467892 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.468041 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] token.caching = True {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.468139 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.468277 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.468369 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.468530 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] token.provider = fernet {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.468622 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.468774 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.468868 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.469016 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.469123 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.469272 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.469372 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.469512 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.469617 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.469753 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.469873 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.470024 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.470124 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.470259 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] ldap.password = **** {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.470351 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] trust.driver = sql {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.470502 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.470596 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.470755 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.470843 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.470986 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.471096 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.471228 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.471326 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.471495 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.471584 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.471704 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.471802 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.471954 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.472048 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.472189 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.472281 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.472426 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.472526 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] database.connection = **** {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.472681 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.472784 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.472937 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.473031 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.473181 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.473280 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.473423 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.473517 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.473665 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.473765 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.474625 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.474729 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] ldap.user = None {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.474836 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.474924 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.474924 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.475090 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.475176 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.475272 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.475405 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.475514 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.475658 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.475758 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.475911 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.476014 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.476152 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.476256 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.476388 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.476488 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.476630 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.476744 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.476870 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.476964 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.477112 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.477204 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.477348 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.477454 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.477557 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=281785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 19 14:50:28.477643 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.477855 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.478114 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.478368 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.478619 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.478854 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.479106 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.479349 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.479591 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.479844 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.480086 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.480339 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.480626 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.480836 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.481098 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] policy.driver = sql {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.481336 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.481588 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.481831 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.482095 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] receipt.caching = True {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.482340 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.482593 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.482842 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.483080 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.483323 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.483570 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] resource.caching = True {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.483826 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.484081 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] resource.driver = sql {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.484323 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.484584 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.484841 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.485082 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] revoke.caching = True {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.485334 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.485578 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.485823 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] role.cache_time = None {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.486088 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] role.caching = True {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.486342 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] role.driver = sql {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.486580 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] role.list_limit = None {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.486835 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.487089 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.487343 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.487597 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.487850 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.488117 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.488370 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.488623 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.488865 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.489120 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.489375 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.490207 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.490466 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.490969 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.491216 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.491473 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.491729 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.491983 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.492238 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.492479 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.492737 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.492994 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.493238 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.493475 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.493712 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.493985 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.494249 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.494519 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.494777 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.495023 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.495268 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.495508 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] token.cache_time = None {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.495753 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] token.caching = True {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.495999 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.496255 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] token.provider = fernet {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.496500 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.496763 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.497017 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.497261 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.497511 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.497760 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.498040 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] trust.driver = sql {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.498284 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.498529 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.498841 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.499102 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.499358 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.499596 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.499847 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.500149 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.500417 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] database.connection = **** {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.500669 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.500921 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.501164 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.501411 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.501654 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.501920 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.502165 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.502408 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.502659 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.502902 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.503151 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.503393 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.503648 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.503890 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.504141 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.504383 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.504628 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.504870 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:50:28.505044 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=281786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 19 14:50:28.509318 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=281785) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:50:28.509552 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=281785) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:50:28.509779 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=281785) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:50:28.518690 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=281785) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:50:28.530893 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=281786) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:50:28.531130 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=281786) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:50:28.531362 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=281786) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:50:28.542307 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=281786) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:50:28.547450 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=281785) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:50:28.547687 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=281785) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:50:28.561565 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=281786) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:50:28.561800 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=281786) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:50:28.601185 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=281785) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:50:28.605862 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=281786) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:50:28.652860 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=281785) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:50:28.653132 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=281785) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:50:28.654325 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=281786) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:50:28.654598 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=281786) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:50:28.720395 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=281785) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:50:28.720620 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=281785) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:50:28.720845 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=281785) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:50:28.720969 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=281786) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:50:28.721065 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=281785) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:50:28.721160 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=281786) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:50:28.721265 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=281785) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:50:28.721360 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=281786) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:50:28.721484 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=281785) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:50:28.721591 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=281786) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:50:28.721757 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=281786) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:50:28.722020 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=281786) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:50:28.745319 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=281785) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:50:28.745553 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=281785) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:50:28.745677 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=281786) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:50:28.745771 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=281785) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:50:28.745886 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=281786) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:50:28.746020 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=281785) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:50:28.746117 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=281786) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:50:28.746234 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=281785) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:50:28.746324 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=281786) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:50:28.746449 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=281785) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:50:28.746545 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=281786) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:50:28.746670 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=281785) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:50:28.746781 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=281786) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:50:28.746948 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=281786) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:50:28.758037 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=281785) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:50:28.758379 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=281786) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:50:28.771660 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=281786) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:50:28.771933 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=281786) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:50:28.772058 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=281786) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:50:28.772287 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=281786) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:50:28.772517 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=281786) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:50:28.772777 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=281786) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:50:28.772999 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=281786) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:50:28.773104 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=281785) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:50:28.773315 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=281785) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:50:28.773546 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=281785) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:50:28.773774 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=281785) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:50:28.774034 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=281785) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:50:28.774258 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=281785) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:50:28.774479 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=281785) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:50:28.791026 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=281786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.791241 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=281786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 14:50:28.791340 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=281785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.791466 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=281785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 14:50:28.791575 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=281786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.791774 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=281786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 14:50:28.791876 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=281785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.792000 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=281785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 14:50:28.792102 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=281786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.792277 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=281785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.792372 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=281786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.792546 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=281785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.792640 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=281786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.792803 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=281786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.792905 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=281785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.793065 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=281785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.804547 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=281785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.804823 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=281785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.805093 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=281785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.805374 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=281785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.805661 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=281786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.805982 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=281786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.806258 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=281786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.806545 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=281786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.813573 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=281785) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:50:28.814748 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=281786) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:50:28.816919 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=281785) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:50:28.817203 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=281785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.817490 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=281785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.826678 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=281785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.826927 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=281786) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:50:28.827049 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=281785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.827049 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=281785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.827238 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=281786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.827341 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=281785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.827434 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=281786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.827533 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=281785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.827626 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=281786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.827719 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=281785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.827811 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=281786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.827903 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=281785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.827903 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=281785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.828085 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=281786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.828085 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=281786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.828085 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=281786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.828085 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=281786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.828085 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=281786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.828085 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=281786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.852688 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=281785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.854334 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=281785) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:50:28.854623 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=281785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.857424 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=281786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.859611 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=281786) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:50:28.859996 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=281786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.860110 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=281785) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:50:28.860301 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=281785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.860590 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=281785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.866870 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=281786) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:50:28.867153 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=281786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.867439 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=281786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.868727 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=281785) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:50:28.869011 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=281785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.873754 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=281785) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:50:28.874079 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=281785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.874368 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=281785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.874654 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=281785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.874927 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=281786) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:50:28.875033 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=281785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.875203 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=281785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.875304 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=281786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.875476 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=281785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.875750 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=281785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.880859 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=281786) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:50:28.881149 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=281786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.881437 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=281786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.881724 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=281786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.882105 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=281786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.882390 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=281786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.882665 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=281786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.882936 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=281786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.895264 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=281785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.895549 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=281785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.900188 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=281785) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:50:28.901820 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=281786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.902130 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=281786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.903859 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=281785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.904156 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=281785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.906699 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=281786) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:50:28.909165 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=281785) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:50:28.910359 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=281786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.910661 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=281786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.912944 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=281785) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:50:28.915698 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=281786) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:50:28.916827 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=281785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.917168 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=281785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.917496 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=281785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.917864 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=281785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.918217 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=281785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.918505 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=281785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.919404 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=281786) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:50:28.923327 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=281786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.923626 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=281786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.923914 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=281786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.924202 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=281786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.924485 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=281786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.924769 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=281786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.938657 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=281785) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:50:28.939083 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=281785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.939361 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=281785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.939641 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=281785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.947015 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=281786) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:50:28.947015 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=281786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.947015 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=281786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.948064 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=281786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.949233 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=281785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.951019 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=281785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.951311 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=281785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.954431 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=281785) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:50:28.954719 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=281785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.955014 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=281785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.955300 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=281785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.955581 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=281785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.955680 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=281786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.957343 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=281786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.957626 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=281786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.960826 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=281786) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:50:28.961107 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=281786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.961401 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=281786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.961709 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=281786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.962016 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=281786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.969262 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=281785) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:50:28.969548 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=281785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.969835 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=281785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.970142 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=281785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.970427 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=281785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.970704 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=281785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.970991 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=281785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.975566 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=281786) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:50:28.975852 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=281786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.976214 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=281786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.976513 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=281786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.976800 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=281786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.977079 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=281786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.977359 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=281786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.990681 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=281785) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:50:28.994024 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=281785) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:50:28.996265 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=281786) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:50:28.997368 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=281785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.999034 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=281785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:28.999574 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=281786) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:50:29.000703 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=281785) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:50:29.000995 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=281785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:29.001290 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=281785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:29.002955 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=281786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:29.004574 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=281786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:29.007086 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=281786) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:50:29.007374 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=281786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:29.007662 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=281786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:29.009399 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=281785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:29.011072 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=281785) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:50:29.014390 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=281785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:29.014685 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=281785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:29.014972 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=281785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:29.015205 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=281786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:29.015305 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=281785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:29.016852 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=281786) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:50:29.020154 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=281786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:29.020451 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=281786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:29.020741 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=281786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:29.021152 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=281786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:29.024791 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=281785) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:50:29.025081 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=281785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:29.025367 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=281785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:29.030637 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=281786) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:50:29.030934 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=281786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:29.031221 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=281786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:29.033399 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=281785) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:50:29.033690 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=281785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:29.034012 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=281785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:29.034298 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=281785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:29.034575 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=281785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:29.034870 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=281785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:29.035147 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=281785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:29.035416 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=281785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:29.035697 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=281785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:29.036160 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=281785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:29.036560 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=281785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:29.036886 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=281785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:29.037239 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=281785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:29.037545 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=281785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:29.039549 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=281786) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:50:29.039838 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=281786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:29.040122 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=281786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:29.040410 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=281786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:29.040691 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=281786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:29.040975 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=281786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:29.041259 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=281786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:29.041532 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=281786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:29.041902 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=281786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:29.042195 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=281786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:29.042480 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=281786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:29.042755 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=281786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:29.043025 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=281786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:29.043299 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=281786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:50:29.079204 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x56071f8189c0 pid: 281785 (default app) May 19 14:50:29.082760 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58882af-dc6c-49c9-a92e-103cba9b064a None None] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:29.083064 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58882af-dc6c-49c9-a92e-103cba9b064a None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:29.083476 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58882af-dc6c-49c9-a92e-103cba9b064a None None] PATH_INFO: `/v3` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:29.087784 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 1/1] 10.222.0.57 () {50 vars in 791 bytes} [Thu May 19 14:50:29 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:50:29.088137 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x56071f8189c0 pid: 281786 (default app) May 19 14:50:37.181666 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21fcf3a-5210-4d3e-959f-cff20705778c None None] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:37.182267 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21fcf3a-5210-4d3e-959f-cff20705778c None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:37.182385 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21fcf3a-5210-4d3e-959f-cff20705778c None None] PATH_INFO: `/` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:37.183698 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 2/2] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:50:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:50:37.191074 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e28c02-553c-4d34-b6f6-0611916c2310 None None] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:37.191492 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e28c02-553c-4d34-b6f6-0611916c2310 None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:37.191652 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e28c02-553c-4d34-b6f6-0611916c2310 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:37.265331 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: INFO dbcounter [None req-d6e28c02-553c-4d34-b6f6-0611916c2310 None None] Registered counter for database keystone May 19 14:50:37.292480 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG oslo_db.sqlalchemy.engines [None req-d6e28c02-553c-4d34-b6f6-0611916c2310 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=281786) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 19 14:50:37.299637 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writer thread running {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 19 14:50:37.480867 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG passlib.handlers.bcrypt [None req-d6e28c02-553c-4d34-b6f6-0611916c2310 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=281786) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 19 14:50:37.481193 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG passlib.handlers.bcrypt [None req-d6e28c02-553c-4d34-b6f6-0611916c2310 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=281786) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 19 14:50:37.536816 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.auth.core [None req-d6e28c02-553c-4d34-b6f6-0611916c2310 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:50:37.538640 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.fernet_utils [None req-d6e28c02-553c-4d34-b6f6-0611916c2310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:50:37.663055 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 1/3] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:50:37 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 475 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 19 14:50:37.686926 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21fcf3a-5210-4d3e-959f-cff20705778c None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:50:37.758305 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: INFO dbcounter [None req-a21fcf3a-5210-4d3e-959f-cff20705778c None None] Registered counter for database keystone May 19 14:50:37.778536 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG oslo_db.sqlalchemy.engines [None req-a21fcf3a-5210-4d3e-959f-cff20705778c None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=281785) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 19 14:50:37.784980 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writer thread running {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 19 14:50:37.909323 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21fcf3a-5210-4d3e-959f-cff20705778c None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:50:37.914042 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72928d03-f854-48d5-9371-78942c112c03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f00ebb3222e9428fb4f634c779ecb9a3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:50:37.914712 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72928d03-f854-48d5-9371-78942c112c03 admin admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:37.915010 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72928d03-f854-48d5-9371-78942c112c03 admin admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:37.915299 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72928d03-f854-48d5-9371-78942c112c03 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:37.923706 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72928d03-f854-48d5-9371-78942c112c03 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:50:37.976134 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: WARNING keystone.common.rbac_enforcer.enforcer [None req-72928d03-f854-48d5-9371-78942c112c03 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 19 14:50:37.976484 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72928d03-f854-48d5-9371-78942c112c03 admin admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:50:37.986349 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 3/4] 10.222.0.57 () {58 vars in 1324 bytes} [Thu May 19 14:50:37 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 19 14:50:37.993389 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e28c02-553c-4d34-b6f6-0611916c2310 None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:50:38.008884 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e28c02-553c-4d34-b6f6-0611916c2310 None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:50:38.013383 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7768f0d1-b462-4206-b351-87dd9538965f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f00ebb3222e9428fb4f634c779ecb9a3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:50:38.014560 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7768f0d1-b462-4206-b351-87dd9538965f admin admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:38.014853 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7768f0d1-b462-4206-b351-87dd9538965f admin admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:38.015166 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7768f0d1-b462-4206-b351-87dd9538965f admin admin] PATH_INFO: `/v3/projects/f00ebb3222e9428fb4f634c779ecb9a3` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:38.016834 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7768f0d1-b462-4206-b351-87dd9538965f admin admin] RBAC: Authorizing `identity:get_project(project_id=f00ebb3222e9428fb4f634c779ecb9a3)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:50:38.070507 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: WARNING keystone.common.rbac_enforcer.enforcer [None req-7768f0d1-b462-4206-b351-87dd9538965f admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 19 14:50:38.070857 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7768f0d1-b462-4206-b351-87dd9538965f admin admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:50:38.075888 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 2/5] 10.222.0.57 () {56 vars in 1207 bytes} [Thu May 19 14:50:37 2022] GET /identity/v3/projects/f00ebb3222e9428fb4f634c779ecb9a3 => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:50:40.306778 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dbf390-186f-4f18-940a-afd3649eaa5f None None] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:40.307356 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dbf390-186f-4f18-940a-afd3649eaa5f None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:40.307356 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dbf390-186f-4f18-940a-afd3649eaa5f None None] PATH_INFO: `/` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:40.311489 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 4/6] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:50:40 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:50:40.316986 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bcf36b-e6d1-47f2-8036-ec0e1de69d22 None None] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:40.318009 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bcf36b-e6d1-47f2-8036-ec0e1de69d22 None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:40.318313 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bcf36b-e6d1-47f2-8036-ec0e1de69d22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:40.337440 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.auth.core [None req-d9bcf36b-e6d1-47f2-8036-ec0e1de69d22 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:50:40.339042 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.fernet_utils [None req-d9bcf36b-e6d1-47f2-8036-ec0e1de69d22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:50:40.347147 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 3/7] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:50:40 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) May 19 14:50:40.370545 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dbf390-186f-4f18-940a-afd3649eaa5f None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:50:40.382889 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dbf390-186f-4f18-940a-afd3649eaa5f None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:50:40.387164 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df126992-486e-4b49-b77a-6a27a10c6eab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f00ebb3222e9428fb4f634c779ecb9a3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:50:40.387798 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df126992-486e-4b49-b77a-6a27a10c6eab admin admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:40.388092 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df126992-486e-4b49-b77a-6a27a10c6eab admin admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:40.388373 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df126992-486e-4b49-b77a-6a27a10c6eab admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:40.396240 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df126992-486e-4b49-b77a-6a27a10c6eab admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:50:40.398067 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df126992-486e-4b49-b77a-6a27a10c6eab admin admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:50:40.406896 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 5/8] 10.222.0.57 () {58 vars in 1324 bytes} [Thu May 19 14:50:40 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 14:50:40.413097 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bcf36b-e6d1-47f2-8036-ec0e1de69d22 None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:50:40.422300 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bcf36b-e6d1-47f2-8036-ec0e1de69d22 None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:50:40.427408 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04900641-e10b-4eee-836f-0cd95125414d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f00ebb3222e9428fb4f634c779ecb9a3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:50:40.428282 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04900641-e10b-4eee-836f-0cd95125414d admin admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:40.428576 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04900641-e10b-4eee-836f-0cd95125414d admin admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:40.428861 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04900641-e10b-4eee-836f-0cd95125414d admin admin] PATH_INFO: `/v3/users/8c487f3640744d568515b03847007a04` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:40.430328 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04900641-e10b-4eee-836f-0cd95125414d admin admin] RBAC: Authorizing `identity:get_user(user_id=8c487f3640744d568515b03847007a04)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:50:40.432166 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04900641-e10b-4eee-836f-0cd95125414d admin admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:50:40.433750 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 4/9] 10.222.0.57 () {56 vars in 1198 bytes} [Thu May 19 14:50:40 2022] GET /identity/v3/users/8c487f3640744d568515b03847007a04 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:50:42.723884 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e167780f-18a4-4440-bc7b-aa3cb74d0e01 None None] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:42.724561 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e167780f-18a4-4440-bc7b-aa3cb74d0e01 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:42.724561 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e167780f-18a4-4440-bc7b-aa3cb74d0e01 None None] PATH_INFO: `/` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:42.725821 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 6/10] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:50:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:50:42.731149 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b1ce40-3fbf-4f62-80d7-59dd368f62f1 None None] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:42.731573 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b1ce40-3fbf-4f62-80d7-59dd368f62f1 None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:42.731731 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b1ce40-3fbf-4f62-80d7-59dd368f62f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:42.749096 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.auth.core [None req-b4b1ce40-3fbf-4f62-80d7-59dd368f62f1 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:50:42.750328 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.fernet_utils [None req-b4b1ce40-3fbf-4f62-80d7-59dd368f62f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:50:42.768718 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c931d1-ac34-4621-8dfe-f568f9d79ec3 None None] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:42.769014 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c931d1-ac34-4621-8dfe-f568f9d79ec3 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:42.769301 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c931d1-ac34-4621-8dfe-f568f9d79ec3 None None] PATH_INFO: `/` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:42.771027 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 7/11] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:50:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:50:42.776158 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f267fefa-f759-47dd-aa8d-1d60917e46bd None None] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:42.776455 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f267fefa-f759-47dd-aa8d-1d60917e46bd None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:42.776743 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f267fefa-f759-47dd-aa8d-1d60917e46bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:42.818445 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG passlib.handlers.bcrypt [None req-f267fefa-f759-47dd-aa8d-1d60917e46bd None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=281785) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 19 14:50:42.818883 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG passlib.handlers.bcrypt [None req-f267fefa-f759-47dd-aa8d-1d60917e46bd None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=281785) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 19 14:50:42.833386 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 5/12] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:50:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:50:42.857015 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b1ce40-3fbf-4f62-80d7-59dd368f62f1 None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:50:42.866553 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.auth.core [None req-f267fefa-f759-47dd-aa8d-1d60917e46bd None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:50:42.868191 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-f267fefa-f759-47dd-aa8d-1d60917e46bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:50:42.880789 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:50:42.963209 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b1ce40-3fbf-4f62-80d7-59dd368f62f1 None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:50:42.970800 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 8/13] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:50:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:50:42.994811 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f267fefa-f759-47dd-aa8d-1d60917e46bd None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:50:42.997770 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6179cf-79af-42a2-8200-e562fabcfe83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:50:42.998607 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6179cf-79af-42a2-8200-e562fabcfe83 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:42.998900 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6179cf-79af-42a2-8200-e562fabcfe83 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:42.999183 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6179cf-79af-42a2-8200-e562fabcfe83 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:43.011791 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6179cf-79af-42a2-8200-e562fabcfe83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:50:43.023240 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-6b6179cf-79af-42a2-8200-e562fabcfe83 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:50:43.023240 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:50:43.023240 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:50:43.023240 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:50:43.023240 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:50:43.023240 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:50:43.023240 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:50:43.023240 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:50:43.023240 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:50:43.023240 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:50:43.023240 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:50:43.023240 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:50:43.023240 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:50:43.023240 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:50:43.023240 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:50:43.023240 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:50:43.024441 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:50:43.024441 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:50:43.024441 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:50:43.024441 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:50:43.024441 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:50:43.024441 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:50:43.024441 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:50:43.024441 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:50:43.024441 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:50:43.024441 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:50:43.024441 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:50:43.024441 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:50:43.024441 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:50:43.024441 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:50:43.024441 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:50:43.024441 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:50:43.025500 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:50:43.025500 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:50:43.025500 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:50:43.025500 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:50:43.025500 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:50:43.025500 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:50:43.025500 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:50:43.025500 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:50:43.025500 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:50:43.025500 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:50:43.025500 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:50:43.025500 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:50:43.025500 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 6/14] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:50:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:50:43.030224 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6179cf-79af-42a2-8200-e562fabcfe83 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:50:43.213923 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f267fefa-f759-47dd-aa8d-1d60917e46bd None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:50:43.216267 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:50:43.234357 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:50:43.252961 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc8778b-d0e6-425d-948a-6db0ccefa536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:50:43.253902 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc8778b-d0e6-425d-948a-6db0ccefa536 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:43.254188 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6179cf-79af-42a2-8200-e562fabcfe83 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:50:43.254311 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc8778b-d0e6-425d-948a-6db0ccefa536 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:43.254474 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc8778b-d0e6-425d-948a-6db0ccefa536 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:43.289782 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30d6cef-a05e-4e74-9750-746e6f00764b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:50:43.290489 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30d6cef-a05e-4e74-9750-746e6f00764b None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:43.290785 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30d6cef-a05e-4e74-9750-746e6f00764b None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:43.291071 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30d6cef-a05e-4e74-9750-746e6f00764b None admin] PATH_INFO: `/v3/roles` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:43.291843 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30d6cef-a05e-4e74-9750-746e6f00764b None admin] RBAC: Adding query filter params (name=admin) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:50:43.292160 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30d6cef-a05e-4e74-9750-746e6f00764b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:50:43.293573 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30d6cef-a05e-4e74-9750-746e6f00764b None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:50:43.306068 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 7/15] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:50:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:50:43.311637 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30d6cef-a05e-4e74-9750-746e6f00764b None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:50:43.324532 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc8778b-d0e6-425d-948a-6db0ccefa536 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:50:43.326106 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc8778b-d0e6-425d-948a-6db0ccefa536 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:50:43.411696 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30d6cef-a05e-4e74-9750-746e6f00764b None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:50:43.424805 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 9/16] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:50:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 14:50:43.431148 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc8778b-d0e6-425d-948a-6db0ccefa536 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:50:43.434438 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:50:43.447988 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8b34af-bccf-43e5-b1f3-c21b370881d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:50:43.448868 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8b34af-bccf-43e5-b1f3-c21b370881d9 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:43.449152 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8b34af-bccf-43e5-b1f3-c21b370881d9 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:43.449427 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8b34af-bccf-43e5-b1f3-c21b370881d9 None admin] PATH_INFO: `/v3/users/8c487f3640744d568515b03847007a04` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:43.450717 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8b34af-bccf-43e5-b1f3-c21b370881d9 None admin] RBAC: Authorizing `identity:get_user(user_id=8c487f3640744d568515b03847007a04)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:50:43.452177 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8b34af-bccf-43e5-b1f3-c21b370881d9 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:50:43.453880 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 8/17] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:50:43 2022] GET /identity/v3/users/8c487f3640744d568515b03847007a04 => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:50:43.459420 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8b34af-bccf-43e5-b1f3-c21b370881d9 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:50:43.474250 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:50:43.532248 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc8778b-d0e6-425d-948a-6db0ccefa536 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:50:43.559554 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8b34af-bccf-43e5-b1f3-c21b370881d9 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:50:43.567136 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc968937-f114-4d06-a126-7891d9895a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:50:43.568130 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc968937-f114-4d06-a126-7891d9895a39 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:43.568422 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc968937-f114-4d06-a126-7891d9895a39 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:43.568713 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc968937-f114-4d06-a126-7891d9895a39 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:43.589359 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc968937-f114-4d06-a126-7891d9895a39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:50:43.590883 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc968937-f114-4d06-a126-7891d9895a39 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:50:43.593074 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080986a0-8c07-4371-bef2-6551b176b513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:50:43.593888 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080986a0-8c07-4371-bef2-6551b176b513 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:43.594176 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080986a0-8c07-4371-bef2-6551b176b513 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:43.594453 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080986a0-8c07-4371-bef2-6551b176b513 None admin] PATH_INFO: `/v3/domains/default` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:43.595639 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080986a0-8c07-4371-bef2-6551b176b513 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:50:43.597090 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080986a0-8c07-4371-bef2-6551b176b513 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:50:43.599480 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 9/18] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:50:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:50:43.605872 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080986a0-8c07-4371-bef2-6551b176b513 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:50:43.616288 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-fc968937-f114-4d06-a126-7891d9895a39 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:50:43.616288 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:50:43.616288 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:50:43.616288 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:50:43.616288 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:50:43.616288 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:50:43.616288 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:50:43.616288 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:50:43.616288 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:50:43.616288 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:50:43.616288 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:50:43.616288 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:50:43.616288 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:50:43.616288 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:50:43.616288 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:50:43.617343 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:50:43.617343 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:50:43.617343 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:50:43.617343 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:50:43.617343 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:50:43.617343 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:50:43.617343 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:50:43.617343 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:50:43.617343 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:50:43.617343 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:50:43.617343 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:50:43.617343 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:50:43.617343 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:50:43.617343 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:50:43.617343 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:50:43.617343 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:50:43.617343 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:50:43.618361 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:50:43.618361 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:50:43.618361 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:50:43.618361 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:50:43.618361 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 10/19] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:50:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:50:43.623422 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc968937-f114-4d06-a126-7891d9895a39 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:50:43.682800 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:50:43.691090 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:50:43.711987 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080986a0-8c07-4371-bef2-6551b176b513 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:50:43.729014 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc968937-f114-4d06-a126-7891d9895a39 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:50:43.748783 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd5cfe2-17a1-42ab-9364-581c46865dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:50:43.749462 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd5cfe2-17a1-42ab-9364-581c46865dbd None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:43.749756 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd5cfe2-17a1-42ab-9364-581c46865dbd None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:43.750077 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd5cfe2-17a1-42ab-9364-581c46865dbd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:43.750930 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd5cfe2-17a1-42ab-9364-581c46865dbd None admin] RBAC: Adding query filter params (role.id=028379e01f9245e9b8fc56e8b9e8a6ad, scope.domain.id=default, user.id=8c487f3640744d568515b03847007a04) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:50:43.751255 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd5cfe2-17a1-42ab-9364-581c46865dbd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:50:43.752731 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd5cfe2-17a1-42ab-9364-581c46865dbd None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:50:43.760443 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 10/20] 10.222.0.57 () {56 vars in 1322 bytes} [Thu May 19 14:50:43 2022] GET /identity/v3/role_assignments?user.id=8c487f3640744d568515b03847007a04&scope.domain.id=default&role.id=028379e01f9245e9b8fc56e8b9e8a6ad => generated 233 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:50:43.764063 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47453a1b-c897-4d82-b5f6-5b3334fdd8d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:50:43.764747 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47453a1b-c897-4d82-b5f6-5b3334fdd8d2 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:43.765035 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47453a1b-c897-4d82-b5f6-5b3334fdd8d2 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:43.765314 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47453a1b-c897-4d82-b5f6-5b3334fdd8d2 None admin] PATH_INFO: `/v3/domains` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:43.766095 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47453a1b-c897-4d82-b5f6-5b3334fdd8d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:50:43.766412 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47453a1b-c897-4d82-b5f6-5b3334fdd8d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:50:43.767838 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47453a1b-c897-4d82-b5f6-5b3334fdd8d2 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:50:43.789449 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 11/21] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:50:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:50:46.286420 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a92d75-00a5-4d2b-989d-e8ae2cab7f5f None None] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:46.287561 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a92d75-00a5-4d2b-989d-e8ae2cab7f5f None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:46.288108 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a92d75-00a5-4d2b-989d-e8ae2cab7f5f None None] PATH_INFO: `/` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:46.289723 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 11/22] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:50:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:50:46.295464 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671c7c4b-2cec-48a8-9a56-45b6f67d177c None None] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:46.295895 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671c7c4b-2cec-48a8-9a56-45b6f67d177c None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:46.296037 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671c7c4b-2cec-48a8-9a56-45b6f67d177c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:46.320072 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.auth.core [None req-671c7c4b-2cec-48a8-9a56-45b6f67d177c None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:50:46.321651 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-671c7c4b-2cec-48a8-9a56-45b6f67d177c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:50:46.414059 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 12/23] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:50:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:50:46.450380 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a92d75-00a5-4d2b-989d-e8ae2cab7f5f None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:50:46.580345 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4be08fa-8e6b-4223-a811-13f7890e0910 None None] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:46.581004 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4be08fa-8e6b-4223-a811-13f7890e0910 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:46.581529 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4be08fa-8e6b-4223-a811-13f7890e0910 None None] PATH_INFO: `/` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:46.583905 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 13/24] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:50:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:50:46.601054 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da55745e-3ec3-4275-968a-5f5fed398dd2 None None] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:46.601684 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da55745e-3ec3-4275-968a-5f5fed398dd2 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:46.602214 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da55745e-3ec3-4275-968a-5f5fed398dd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:46.615226 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a92d75-00a5-4d2b-989d-e8ae2cab7f5f None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:50:46.634219 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.auth.core [None req-da55745e-3ec3-4275-968a-5f5fed398dd2 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:50:46.635782 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-da55745e-3ec3-4275-968a-5f5fed398dd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:50:46.694828 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c845a1-be0b-4639-bcfb-e6fbccf2c7e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:50:46.695663 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c845a1-be0b-4639-bcfb-e6fbccf2c7e9 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:46.695957 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c845a1-be0b-4639-bcfb-e6fbccf2c7e9 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:46.696251 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c845a1-be0b-4639-bcfb-e6fbccf2c7e9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:46.720345 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c845a1-be0b-4639-bcfb-e6fbccf2c7e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:50:46.735491 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-a4c845a1-be0b-4639-bcfb-e6fbccf2c7e9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:50:46.735491 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:50:46.735491 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:50:46.735491 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:50:46.735491 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:50:46.735491 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:50:46.735491 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:50:46.735491 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:50:46.735491 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:50:46.735491 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:50:46.735491 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:50:46.735491 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:50:46.735491 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:50:46.735491 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:50:46.735491 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:50:46.735491 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:50:46.736572 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:50:46.736572 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:50:46.736572 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:50:46.736572 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:50:46.736572 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:50:46.736572 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:50:46.736572 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:50:46.736572 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:50:46.736572 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:50:46.736572 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:50:46.736572 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:50:46.736572 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:50:46.736572 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:50:46.736572 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:50:46.736572 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:50:46.736572 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:50:46.737476 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:50:46.737476 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:50:46.737476 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:50:46.737476 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:50:46.737476 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:50:46.737476 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:50:46.737476 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:50:46.737476 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:50:46.737476 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:50:46.737476 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:50:46.737476 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:50:46.737476 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:50:46.741420 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 12/25] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:50:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:50:46.747635 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c845a1-be0b-4639-bcfb-e6fbccf2c7e9 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:50:46.755842 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:50:46.791974 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:50:46.877189 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 14/26] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:50:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:50:46.879504 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c038d3f-db20-4578-8be9-863ada0a4adb None None] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:46.879818 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c038d3f-db20-4578-8be9-863ada0a4adb None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:46.880107 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c038d3f-db20-4578-8be9-863ada0a4adb None None] PATH_INFO: `/` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:46.881752 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 15/27] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:50:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:50:46.892360 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1555dcd-3da2-48cd-9f45-6b5ce3d7e489 None None] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:46.897189 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1555dcd-3da2-48cd-9f45-6b5ce3d7e489 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:46.897189 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1555dcd-3da2-48cd-9f45-6b5ce3d7e489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:46.925634 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.auth.core [None req-a1555dcd-3da2-48cd-9f45-6b5ce3d7e489 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:50:46.927966 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-a1555dcd-3da2-48cd-9f45-6b5ce3d7e489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:50:46.964093 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c845a1-be0b-4639-bcfb-e6fbccf2c7e9 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:50:47.037227 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ae52e3-8872-479c-9b8b-f7664f0e4a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:50:47.038217 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ae52e3-8872-479c-9b8b-f7664f0e4a5e None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:47.038518 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ae52e3-8872-479c-9b8b-f7664f0e4a5e None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:47.038807 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ae52e3-8872-479c-9b8b-f7664f0e4a5e None admin] PATH_INFO: `/v3/roles` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:47.041389 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ae52e3-8872-479c-9b8b-f7664f0e4a5e None admin] RBAC: Adding query filter params (name=admin) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:50:47.041712 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ae52e3-8872-479c-9b8b-f7664f0e4a5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:50:47.043206 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ae52e3-8872-479c-9b8b-f7664f0e4a5e None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:50:47.066054 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 13/28] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:50:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:50:47.070743 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ae52e3-8872-479c-9b8b-f7664f0e4a5e None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:50:47.083277 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 16/29] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:50:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:50:47.090878 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1555dcd-3da2-48cd-9f45-6b5ce3d7e489 None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:50:47.160306 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:50:47.201064 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ae52e3-8872-479c-9b8b-f7664f0e4a5e None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:50:47.238961 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b741749-8d3f-4a98-8949-6ab18030fcd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:50:47.240059 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b741749-8d3f-4a98-8949-6ab18030fcd4 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:47.240627 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b741749-8d3f-4a98-8949-6ab18030fcd4 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:47.240983 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b741749-8d3f-4a98-8949-6ab18030fcd4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:47.253470 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b741749-8d3f-4a98-8949-6ab18030fcd4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:50:47.255636 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b741749-8d3f-4a98-8949-6ab18030fcd4 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:50:47.276613 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-5b741749-8d3f-4a98-8949-6ab18030fcd4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:50:47.276613 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:50:47.276613 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:50:47.276613 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:50:47.276613 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:50:47.276613 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:50:47.276613 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:50:47.276613 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:50:47.276613 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:50:47.276613 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:50:47.276613 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:50:47.276613 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:50:47.276613 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:50:47.276613 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:50:47.276613 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:50:47.277660 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:50:47.277660 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:50:47.277660 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:50:47.277660 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:50:47.277660 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:50:47.277660 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:50:47.277660 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:50:47.277660 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:50:47.277660 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:50:47.277660 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:50:47.277660 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:50:47.277660 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:50:47.277660 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:50:47.277660 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:50:47.277660 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:50:47.277660 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:50:47.277660 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:50:47.278685 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:50:47.278685 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:50:47.278685 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:50:47.278685 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:50:47.278685 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 14/30] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:50:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:50:47.284243 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b741749-8d3f-4a98-8949-6ab18030fcd4 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:50:47.293433 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1555dcd-3da2-48cd-9f45-6b5ce3d7e489 None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:50:47.305166 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:50:47.345037 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8d378f-51d7-4766-97bc-48df1acbc219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:50:47.346447 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8d378f-51d7-4766-97bc-48df1acbc219 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:47.347148 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8d378f-51d7-4766-97bc-48df1acbc219 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:47.348063 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8d378f-51d7-4766-97bc-48df1acbc219 None admin] PATH_INFO: `/v3/users/8c487f3640744d568515b03847007a04` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:47.351226 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8d378f-51d7-4766-97bc-48df1acbc219 None admin] RBAC: Authorizing `identity:get_user(user_id=8c487f3640744d568515b03847007a04)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:50:47.353576 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8d378f-51d7-4766-97bc-48df1acbc219 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:50:47.356250 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 17/31] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:50:47 2022] GET /identity/v3/users/8c487f3640744d568515b03847007a04 => generated 247 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:50:47.357752 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2442bf42-9ac7-4784-9284-196d382634db None None] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:47.358279 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2442bf42-9ac7-4784-9284-196d382634db None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:47.358908 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2442bf42-9ac7-4784-9284-196d382634db None None] PATH_INFO: `/` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:47.360953 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 18/32] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:50:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:50:47.365953 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2442bf42-9ac7-4784-9284-196d382634db None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:50:47.415577 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b741749-8d3f-4a98-8949-6ab18030fcd4 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:50:47.437798 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:50:47.458629 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e15d37-5c39-4704-86e6-32f92fdab08a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:50:47.459555 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e15d37-5c39-4704-86e6-32f92fdab08a None admin] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:47.460019 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e15d37-5c39-4704-86e6-32f92fdab08a None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:47.460283 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e15d37-5c39-4704-86e6-32f92fdab08a None admin] PATH_INFO: `/v3/roles` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:47.461174 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e15d37-5c39-4704-86e6-32f92fdab08a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:50:47.462722 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e15d37-5c39-4704-86e6-32f92fdab08a None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:50:47.498670 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2442bf42-9ac7-4784-9284-196d382634db None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:50:47.540232 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d992b307-3b71-4dc7-afbf-b367c134fbe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:50:47.541319 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d992b307-3b71-4dc7-afbf-b367c134fbe4 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:47.541914 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d992b307-3b71-4dc7-afbf-b367c134fbe4 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:47.542440 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d992b307-3b71-4dc7-afbf-b367c134fbe4 None admin] PATH_INFO: `/v3/domains` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:47.543486 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d992b307-3b71-4dc7-afbf-b367c134fbe4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:50:47.544013 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d992b307-3b71-4dc7-afbf-b367c134fbe4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:50:47.545727 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d992b307-3b71-4dc7-afbf-b367c134fbe4 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:50:47.564260 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 19/33] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:50:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:50:47.565593 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d992b307-3b71-4dc7-afbf-b367c134fbe4 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:50:47.605527 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:50:47.631691 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 15/34] 10.222.0.57 () {60 vars in 1131 bytes} [Thu May 19 14:50:47 2022] POST /identity/v3/roles => generated 225 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:50:47.634159 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fc264e-5837-46da-bf06-0881b80ad8a6 None None] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:47.634662 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fc264e-5837-46da-bf06-0881b80ad8a6 None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:47.635144 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fc264e-5837-46da-bf06-0881b80ad8a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:47.662215 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.auth.core [None req-53fc264e-5837-46da-bf06-0881b80ad8a6 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:50:47.664411 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.fernet_utils [None req-53fc264e-5837-46da-bf06-0881b80ad8a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:50:47.745017 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d992b307-3b71-4dc7-afbf-b367c134fbe4 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:50:47.794040 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 16/35] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:50:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:50:47.795355 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fc264e-5837-46da-bf06-0881b80ad8a6 None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:50:47.810396 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77a0700-ecc5-4986-b1af-9fa6bec061cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:50:47.811229 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77a0700-ecc5-4986-b1af-9fa6bec061cf None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:47.811528 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77a0700-ecc5-4986-b1af-9fa6bec061cf None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:47.811818 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77a0700-ecc5-4986-b1af-9fa6bec061cf None admin] PATH_INFO: `/v3/domains/default` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:47.818395 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77a0700-ecc5-4986-b1af-9fa6bec061cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:50:47.818395 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77a0700-ecc5-4986-b1af-9fa6bec061cf None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:50:47.819091 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 20/36] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:50:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:50:47.824622 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77a0700-ecc5-4986-b1af-9fa6bec061cf None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:50:47.946283 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fc264e-5837-46da-bf06-0881b80ad8a6 None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:50:47.961327 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:50:47.988312 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a434ec-3dbc-4ddd-852a-429ce63fd653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:50:47.989533 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a434ec-3dbc-4ddd-852a-429ce63fd653 None admin] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:47.990105 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a434ec-3dbc-4ddd-852a-429ce63fd653 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:47.990394 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a434ec-3dbc-4ddd-852a-429ce63fd653 None admin] PATH_INFO: `/v3/projects` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:47.991141 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a434ec-3dbc-4ddd-852a-429ce63fd653 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:50:47.992794 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a434ec-3dbc-4ddd-852a-429ce63fd653 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:50:47.993627 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77a0700-ecc5-4986-b1af-9fa6bec061cf None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:50:47.995460 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:50:48.034022 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 17/37] 10.222.0.57 () {60 vars in 1141 bytes} [Thu May 19 14:50:47 2022] POST /identity/v3/projects => generated 301 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:50:48.034990 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a434ec-3dbc-4ddd-852a-429ce63fd653 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:50:48.049761 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8260ef26-0e61-461c-80f1-e6ed78d31ee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:50:48.050885 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8260ef26-0e61-461c-80f1-e6ed78d31ee7 None admin] REQUEST_METHOD: `PUT` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:48.051234 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8260ef26-0e61-461c-80f1-e6ed78d31ee7 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:48.051734 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8260ef26-0e61-461c-80f1-e6ed78d31ee7 None admin] PATH_INFO: `/v3/domains/default/users/8c487f3640744d568515b03847007a04/roles/028379e01f9245e9b8fc56e8b9e8a6ad` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:48.055052 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8260ef26-0e61-461c-80f1-e6ed78d31ee7 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=8c487f3640744d568515b03847007a04, role_id=028379e01f9245e9b8fc56e8b9e8a6ad)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:50:48.056694 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8260ef26-0e61-461c-80f1-e6ed78d31ee7 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:50:48.165209 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a434ec-3dbc-4ddd-852a-429ce63fd653 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:50:48.204110 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b040779d-82ed-4d52-8a35-ac94c566f3eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:50:48.204720 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b040779d-82ed-4d52-8a35-ac94c566f3eb None admin] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:48.205012 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b040779d-82ed-4d52-8a35-ac94c566f3eb None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:48.205296 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b040779d-82ed-4d52-8a35-ac94c566f3eb None admin] PATH_INFO: `/v3/roles` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:48.206106 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b040779d-82ed-4d52-8a35-ac94c566f3eb None admin] RBAC: Authorizing `identity:create_role()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:50:48.207631 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b040779d-82ed-4d52-8a35-ac94c566f3eb None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:50:48.223067 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 18/38] 10.222.0.57 () {60 vars in 1131 bytes} [Thu May 19 14:50:48 2022] POST /identity/v3/roles => generated 219 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:50:48.224962 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f57d83-ad1b-4316-bb78-c11383fd43db None None] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:48.225324 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f57d83-ad1b-4316-bb78-c11383fd43db None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:48.225949 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f57d83-ad1b-4316-bb78-c11383fd43db None None] PATH_INFO: `/` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:48.232184 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 19/39] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:50:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:50:48.232416 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 21/40] 10.222.0.57 () {58 vars in 1361 bytes} [Thu May 19 14:50:47 2022] PUT /identity/v3/domains/default/users/8c487f3640744d568515b03847007a04/roles/028379e01f9245e9b8fc56e8b9e8a6ad => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 14:50:48.232522 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd52591-be36-42d8-8bfe-7c0dc22825ff None None] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:48.232522 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd52591-be36-42d8-8bfe-7c0dc22825ff None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:48.232522 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd52591-be36-42d8-8bfe-7c0dc22825ff None None] PATH_INFO: `/` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:48.236307 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 20/41] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:50:48 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:50:48.237876 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f3805e-4b85-45c2-a20e-dcde7ea3e0a2 None None] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:48.238181 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f3805e-4b85-45c2-a20e-dcde7ea3e0a2 None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:48.238475 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f3805e-4b85-45c2-a20e-dcde7ea3e0a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:48.240593 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4010804c-7681-46ea-984d-4dbc865066f9 None None] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:48.241347 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4010804c-7681-46ea-984d-4dbc865066f9 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:48.242116 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4010804c-7681-46ea-984d-4dbc865066f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:48.261209 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.auth.core [None req-26f3805e-4b85-45c2-a20e-dcde7ea3e0a2 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:50:48.262576 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.fernet_utils [None req-26f3805e-4b85-45c2-a20e-dcde7ea3e0a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:50:48.269403 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.auth.core [None req-4010804c-7681-46ea-984d-4dbc865066f9 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:50:48.271864 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-4010804c-7681-46ea-984d-4dbc865066f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:50:48.288148 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:50:48.370243 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 21/42] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:50:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:50:48.375850 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 22/43] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:50:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:50:48.395951 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f3805e-4b85-45c2-a20e-dcde7ea3e0a2 None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:50:48.402788 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4010804c-7681-46ea-984d-4dbc865066f9 None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:50:48.519790 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f3805e-4b85-45c2-a20e-dcde7ea3e0a2 None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:50:48.528555 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:50:48.546784 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4010804c-7681-46ea-984d-4dbc865066f9 None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:50:48.571494 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6ed1ab-2f55-407b-a45b-adff3a84d0ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:50:48.572191 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6ed1ab-2f55-407b-a45b-adff3a84d0ac None admin] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:48.573205 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6ed1ab-2f55-407b-a45b-adff3a84d0ac None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:48.573505 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6ed1ab-2f55-407b-a45b-adff3a84d0ac None admin] PATH_INFO: `/v3/roles` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:48.574314 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6ed1ab-2f55-407b-a45b-adff3a84d0ac None admin] RBAC: Authorizing `identity:create_role()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:50:48.575743 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6ed1ab-2f55-407b-a45b-adff3a84d0ac None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:50:48.589219 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3672c3d-fb09-4f8c-b717-fd9a98c5550a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:50:48.589720 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 22/44] 10.222.0.57 () {60 vars in 1131 bytes} [Thu May 19 14:50:48 2022] POST /identity/v3/roles => generated 223 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:50:48.590642 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3672c3d-fb09-4f8c-b717-fd9a98c5550a None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:48.591120 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3672c3d-fb09-4f8c-b717-fd9a98c5550a None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:48.591594 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3672c3d-fb09-4f8c-b717-fd9a98c5550a None admin] PATH_INFO: `/v3/domains/default` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:48.593099 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3672c3d-fb09-4f8c-b717-fd9a98c5550a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:50:48.595263 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3672c3d-fb09-4f8c-b717-fd9a98c5550a None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:50:48.600863 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 23/45] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:50:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:50:48.607423 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6ed1ab-2f55-407b-a45b-adff3a84d0ac None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:50:48.624379 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:50:48.744628 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6ed1ab-2f55-407b-a45b-adff3a84d0ac None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:50:48.787966 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b994f3-4712-4758-8659-274b4b189c1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:50:48.788645 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b994f3-4712-4758-8659-274b4b189c1f None admin] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:48.788873 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b994f3-4712-4758-8659-274b4b189c1f None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:48.789153 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b994f3-4712-4758-8659-274b4b189c1f None admin] PATH_INFO: `/v3/projects` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:48.789911 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b994f3-4712-4758-8659-274b4b189c1f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:50:48.791392 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b994f3-4712-4758-8659-274b4b189c1f None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:50:48.821575 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 23/46] 10.222.0.57 () {60 vars in 1141 bytes} [Thu May 19 14:50:48 2022] POST /identity/v3/projects => generated 312 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:50:50.564112 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb16410a-d349-497b-aa20-3cc7a5c67bb2 None None] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:50.564714 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb16410a-d349-497b-aa20-3cc7a5c67bb2 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:50.564714 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb16410a-d349-497b-aa20-3cc7a5c67bb2 None None] PATH_INFO: `/` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:50.566103 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 24/47] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:50:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:50:50.571669 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17a4198-cece-4c65-baae-af9a937c95b5 None None] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:50.572133 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17a4198-cece-4c65-baae-af9a937c95b5 None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:50.572245 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17a4198-cece-4c65-baae-af9a937c95b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:50.589504 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.auth.core [None req-f17a4198-cece-4c65-baae-af9a937c95b5 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:50:50.590740 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.fernet_utils [None req-f17a4198-cece-4c65-baae-af9a937c95b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:50:50.652704 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:50:50.664888 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 24/48] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:50:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:50:50.688314 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb16410a-d349-497b-aa20-3cc7a5c67bb2 None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:50:50.790964 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb16410a-d349-497b-aa20-3cc7a5c67bb2 None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:50:50.825070 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8064cd70-b88f-44ce-a436-fba7a8ce90e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:50:50.825799 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8064cd70-b88f-44ce-a436-fba7a8ce90e0 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:50.826124 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8064cd70-b88f-44ce-a436-fba7a8ce90e0 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:50.826410 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8064cd70-b88f-44ce-a436-fba7a8ce90e0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:50.838443 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8064cd70-b88f-44ce-a436-fba7a8ce90e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:50:50.847338 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-8064cd70-b88f-44ce-a436-fba7a8ce90e0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:50:50.847338 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:50:50.847338 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:50:50.847338 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:50:50.847338 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:50:50.847338 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:50:50.847338 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:50:50.847338 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:50:50.847338 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:50:50.847338 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:50:50.847338 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:50:50.847338 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:50:50.847338 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:50:50.847338 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:50:50.847338 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:50:50.847338 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:50:50.849040 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:50:50.849040 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:50:50.849040 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:50:50.849040 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:50:50.849040 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:50:50.849040 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:50:50.849040 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:50:50.849040 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:50:50.849040 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:50:50.849040 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:50:50.849040 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:50:50.849040 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:50:50.849040 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:50:50.849040 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:50:50.849040 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:50:50.849040 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:50:50.849996 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:50:50.849996 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:50:50.849996 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:50:50.849996 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:50:50.849996 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:50:50.849996 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:50:50.849996 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:50:50.849996 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:50:50.849996 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:50:50.849996 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:50:50.849996 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:50:50.849996 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:50:50.849996 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 25/49] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:50:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:50:50.855020 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17a4198-cece-4c65-baae-af9a937c95b5 None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:50:50.959308 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17a4198-cece-4c65-baae-af9a937c95b5 None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:50:50.993697 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d29837-44ff-4475-96f0-cc84800251ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:50:50.994477 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d29837-44ff-4475-96f0-cc84800251ef None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:50.994779 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d29837-44ff-4475-96f0-cc84800251ef None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:50.996068 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d29837-44ff-4475-96f0-cc84800251ef None admin] PATH_INFO: `/v3/roles` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:50.996826 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d29837-44ff-4475-96f0-cc84800251ef None admin] RBAC: Adding query filter params (name=admin) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:50:50.997142 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d29837-44ff-4475-96f0-cc84800251ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:50:50.998552 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d29837-44ff-4475-96f0-cc84800251ef None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:50:51.007117 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 25/50] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:50:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:50:51.013366 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8064cd70-b88f-44ce-a436-fba7a8ce90e0 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:50:51.041465 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:50:51.080295 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a2fe0c-c757-4713-8f79-28b1290b7c20 None None] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:51.080532 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a2fe0c-c757-4713-8f79-28b1290b7c20 None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:51.080824 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a2fe0c-c757-4713-8f79-28b1290b7c20 None None] PATH_INFO: `/` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:51.082259 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 26/51] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:50:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:50:51.087371 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9950a659-61b1-471d-97b4-0438290cb3bf None None] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:51.087673 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9950a659-61b1-471d-97b4-0438290cb3bf None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:51.087967 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9950a659-61b1-471d-97b4-0438290cb3bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:51.105317 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.auth.core [None req-9950a659-61b1-471d-97b4-0438290cb3bf None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:50:51.106567 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.fernet_utils [None req-9950a659-61b1-471d-97b4-0438290cb3bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:50:51.123128 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8064cd70-b88f-44ce-a436-fba7a8ce90e0 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:50:51.156998 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97e58a1-f002-43b4-b708-122c9612cefb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:50:51.157906 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97e58a1-f002-43b4-b708-122c9612cefb None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:51.158212 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97e58a1-f002-43b4-b708-122c9612cefb None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:51.158407 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:50:51.158512 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97e58a1-f002-43b4-b708-122c9612cefb None admin] PATH_INFO: `/v3/users/8c487f3640744d568515b03847007a04` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:51.159811 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97e58a1-f002-43b4-b708-122c9612cefb None admin] RBAC: Authorizing `identity:get_user(user_id=8c487f3640744d568515b03847007a04)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:50:51.162141 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97e58a1-f002-43b4-b708-122c9612cefb None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:50:51.163822 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 26/52] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:50:51 2022] GET /identity/v3/users/8c487f3640744d568515b03847007a04 => generated 247 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:50:51.169151 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97e58a1-f002-43b4-b708-122c9612cefb None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:50:51.177582 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 27/53] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:50:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:50:51.200714 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9950a659-61b1-471d-97b4-0438290cb3bf None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:50:51.263115 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:50:51.276578 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97e58a1-f002-43b4-b708-122c9612cefb None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:50:51.307058 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9950a659-61b1-471d-97b4-0438290cb3bf None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:50:51.310126 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764316ba-04ec-4765-a489-9e41ef571c01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:50:51.310927 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764316ba-04ec-4765-a489-9e41ef571c01 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:51.311223 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764316ba-04ec-4765-a489-9e41ef571c01 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:51.311510 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764316ba-04ec-4765-a489-9e41ef571c01 None admin] PATH_INFO: `/v3/domains/default` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:51.312713 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764316ba-04ec-4765-a489-9e41ef571c01 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:50:51.314244 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764316ba-04ec-4765-a489-9e41ef571c01 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:50:51.315891 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 27/54] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:50:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:50:51.321667 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764316ba-04ec-4765-a489-9e41ef571c01 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:50:51.342382 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17445fa-a6b0-4ef4-9c0f-39e7b646ee79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:50:51.343119 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17445fa-a6b0-4ef4-9c0f-39e7b646ee79 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:51.343411 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17445fa-a6b0-4ef4-9c0f-39e7b646ee79 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:51.343691 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17445fa-a6b0-4ef4-9c0f-39e7b646ee79 None admin] PATH_INFO: `/v3/domains/default` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:51.344896 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17445fa-a6b0-4ef4-9c0f-39e7b646ee79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:50:51.346405 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17445fa-a6b0-4ef4-9c0f-39e7b646ee79 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:50:51.348914 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 28/55] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:50:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:50:51.354685 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17445fa-a6b0-4ef4-9c0f-39e7b646ee79 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:50:51.408677 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:50:51.423683 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764316ba-04ec-4765-a489-9e41ef571c01 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:50:51.458499 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17445fa-a6b0-4ef4-9c0f-39e7b646ee79 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:50:51.459029 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89531566-8a94-48c3-b122-78c0cdbba724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:50:51.459718 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89531566-8a94-48c3-b122-78c0cdbba724 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:51.460157 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89531566-8a94-48c3-b122-78c0cdbba724 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:51.460308 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89531566-8a94-48c3-b122-78c0cdbba724 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:51.461123 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89531566-8a94-48c3-b122-78c0cdbba724 None admin] RBAC: Adding query filter params (role.id=028379e01f9245e9b8fc56e8b9e8a6ad, scope.domain.id=default, user.id=8c487f3640744d568515b03847007a04) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:50:51.461446 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89531566-8a94-48c3-b122-78c0cdbba724 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:50:51.462944 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89531566-8a94-48c3-b122-78c0cdbba724 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:50:51.471100 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 28/56] 10.222.0.57 () {56 vars in 1322 bytes} [Thu May 19 14:50:51 2022] GET /identity/v3/role_assignments?user.id=8c487f3640744d568515b03847007a04&scope.domain.id=default&role.id=028379e01f9245e9b8fc56e8b9e8a6ad => generated 530 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:50:51.499271 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d02a8f-737d-4e00-a34a-1996ce605c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:50:51.499978 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d02a8f-737d-4e00-a34a-1996ce605c2d None admin] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:51.500271 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d02a8f-737d-4e00-a34a-1996ce605c2d None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:51.500559 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d02a8f-737d-4e00-a34a-1996ce605c2d None admin] PATH_INFO: `/v3/projects` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:51.501303 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d02a8f-737d-4e00-a34a-1996ce605c2d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:50:51.502865 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d02a8f-737d-4e00-a34a-1996ce605c2d None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:50:51.532528 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 29/57] 10.222.0.57 () {60 vars in 1140 bytes} [Thu May 19 14:50:51 2022] POST /identity/v3/projects => generated 298 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 14:50:53.807682 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1055575-db5f-4391-8590-b578185e877e None None] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:53.808235 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1055575-db5f-4391-8590-b578185e877e None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:53.808336 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1055575-db5f-4391-8590-b578185e877e None None] PATH_INFO: `/` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:53.809656 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 29/58] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:50:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:50:53.815070 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dff742-c9d8-4197-bdcc-9bd282d8a93d None None] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:53.815463 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dff742-c9d8-4197-bdcc-9bd282d8a93d None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:53.815641 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dff742-c9d8-4197-bdcc-9bd282d8a93d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:53.833905 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.auth.core [None req-d5dff742-c9d8-4197-bdcc-9bd282d8a93d None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:50:53.835089 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.fernet_utils [None req-d5dff742-c9d8-4197-bdcc-9bd282d8a93d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:50:53.900537 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 30/59] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:50:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:50:53.925833 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1055575-db5f-4391-8590-b578185e877e None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:50:53.950225 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:50:54.029684 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1055575-db5f-4391-8590-b578185e877e None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:50:54.063412 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea0bff7-6be8-4f47-aff2-e7a19484e27d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:50:54.064210 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea0bff7-6be8-4f47-aff2-e7a19484e27d None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:54.064502 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea0bff7-6be8-4f47-aff2-e7a19484e27d None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:54.064783 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea0bff7-6be8-4f47-aff2-e7a19484e27d None admin] PATH_INFO: `/v3/domains/default` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:54.066056 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea0bff7-6be8-4f47-aff2-e7a19484e27d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:50:54.067557 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea0bff7-6be8-4f47-aff2-e7a19484e27d None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:50:54.069171 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 30/60] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:50:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:50:54.075020 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dff742-c9d8-4197-bdcc-9bd282d8a93d None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:50:54.108635 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:50:54.174992 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dff742-c9d8-4197-bdcc-9bd282d8a93d None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:50:54.209671 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0984d517-066c-4cd0-b515-2b9224f093a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:50:54.210313 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0984d517-066c-4cd0-b515-2b9224f093a2 None admin] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:54.210608 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0984d517-066c-4cd0-b515-2b9224f093a2 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:54.210894 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0984d517-066c-4cd0-b515-2b9224f093a2 None admin] PATH_INFO: `/v3/users` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:54.211625 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0984d517-066c-4cd0-b515-2b9224f093a2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:50:54.213097 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0984d517-066c-4cd0-b515-2b9224f093a2 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:50:54.239250 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 31/61] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 19 14:50:54 2022] POST /identity/v3/users => generated 275 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:50:57.623285 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51aa544-7e8e-4e92-85e9-2d76b57b01c2 None None] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:57.623847 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51aa544-7e8e-4e92-85e9-2d76b57b01c2 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:57.623847 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51aa544-7e8e-4e92-85e9-2d76b57b01c2 None None] PATH_INFO: `/` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:57.625478 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 31/62] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:50:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:50:57.632385 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cb8eac-7c5b-4b1e-baad-7aa22691beaf None None] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:57.633354 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cb8eac-7c5b-4b1e-baad-7aa22691beaf None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:57.633891 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cb8eac-7c5b-4b1e-baad-7aa22691beaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:57.661521 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.auth.core [None req-93cb8eac-7c5b-4b1e-baad-7aa22691beaf None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:50:57.665285 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.fernet_utils [None req-93cb8eac-7c5b-4b1e-baad-7aa22691beaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:50:57.756873 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 32/63] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:50:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:50:57.803325 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51aa544-7e8e-4e92-85e9-2d76b57b01c2 None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:50:57.853882 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b999aab-0204-4bf8-a2de-2c5c44018485 None None] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:57.854579 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b999aab-0204-4bf8-a2de-2c5c44018485 None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:57.855074 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b999aab-0204-4bf8-a2de-2c5c44018485 None None] PATH_INFO: `/` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:57.856699 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 33/64] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:50:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:50:57.862081 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd32c8e-7862-4297-899b-c01df766e037 None None] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:57.862665 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd32c8e-7862-4297-899b-c01df766e037 None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:57.863178 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd32c8e-7862-4297-899b-c01df766e037 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:57.895660 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.auth.core [None req-6fd32c8e-7862-4297-899b-c01df766e037 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:50:57.897623 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.fernet_utils [None req-6fd32c8e-7862-4297-899b-c01df766e037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:50:57.918644 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:50:57.926149 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:50:57.929283 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51aa544-7e8e-4e92-85e9-2d76b57b01c2 None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:50:57.988895 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b97b21-fcd9-401f-9288-fc88386a0b6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:50:57.994993 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b97b21-fcd9-401f-9288-fc88386a0b6c None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:57.995517 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b97b21-fcd9-401f-9288-fc88386a0b6c None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:57.996021 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b97b21-fcd9-401f-9288-fc88386a0b6c None admin] PATH_INFO: `/v3/roles/member` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:58.017599 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b97b21-fcd9-401f-9288-fc88386a0b6c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:50:58.035847 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-65b97b21-fcd9-401f-9288-fc88386a0b6c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:50:58.035847 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:50:58.035847 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:50:58.035847 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:50:58.035847 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:50:58.035847 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:50:58.035847 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:50:58.035847 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:50:58.035847 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:50:58.035847 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:50:58.035847 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:50:58.035847 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:50:58.035847 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:50:58.035847 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:50:58.035847 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:50:58.035847 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:50:58.036923 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:50:58.036923 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:50:58.036923 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:50:58.036923 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:50:58.036923 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:50:58.036923 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:50:58.036923 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:50:58.036923 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:50:58.036923 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:50:58.036923 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:50:58.036923 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:50:58.036923 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:50:58.036923 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:50:58.036923 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:50:58.036923 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:50:58.036923 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:50:58.037900 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:50:58.037900 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:50:58.037900 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:50:58.037900 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:50:58.037900 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:50:58.037900 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:50:58.037900 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:50:58.037900 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:50:58.037900 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:50:58.037900 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:50:58.037900 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:50:58.037900 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:50:58.043605 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 32/65] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:50:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:50:58.053250 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b97b21-fcd9-401f-9288-fc88386a0b6c None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:50:58.054975 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 34/66] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:50:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:50:58.106393 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd32c8e-7862-4297-899b-c01df766e037 None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:50:58.202669 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b97b21-fcd9-401f-9288-fc88386a0b6c None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:50:58.215608 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd32c8e-7862-4297-899b-c01df766e037 None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:50:58.242667 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f46179-8e41-478a-b435-a6f29d80a43d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:50:58.243474 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f46179-8e41-478a-b435-a6f29d80a43d None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:58.243766 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f46179-8e41-478a-b435-a6f29d80a43d None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:58.244238 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f46179-8e41-478a-b435-a6f29d80a43d None admin] PATH_INFO: `/v3/roles` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:58.245170 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f46179-8e41-478a-b435-a6f29d80a43d None admin] RBAC: Adding query filter params (name=member) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:50:58.245774 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f46179-8e41-478a-b435-a6f29d80a43d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:50:58.247481 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f46179-8e41-478a-b435-a6f29d80a43d None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:50:58.251216 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf25778-1f17-40fd-9276-6fc6d4734ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:50:58.252005 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf25778-1f17-40fd-9276-6fc6d4734ddf None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:58.252295 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf25778-1f17-40fd-9276-6fc6d4734ddf None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:58.252579 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf25778-1f17-40fd-9276-6fc6d4734ddf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:58.262555 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 33/67] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:50:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:50:58.266076 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf25778-1f17-40fd-9276-6fc6d4734ddf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:50:58.268312 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9346465f-646e-4718-8958-8124b0d63230 None None] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:58.268615 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9346465f-646e-4718-8958-8124b0d63230 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:58.269154 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9346465f-646e-4718-8958-8124b0d63230 None None] PATH_INFO: `/` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:58.270704 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 34/68] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:50:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:50:58.272073 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87794c1b-fda9-442e-9cd2-527d61e51eab None None] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:58.272356 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-4bf25778-1f17-40fd-9276-6fc6d4734ddf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:50:58.272356 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:50:58.272356 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:50:58.272356 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:50:58.272356 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:50:58.272356 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:50:58.272356 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:50:58.272356 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:50:58.272356 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:50:58.272356 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:50:58.272356 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:50:58.272356 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:50:58.272356 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:50:58.272356 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:50:58.272356 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:50:58.272356 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:50:58.273355 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:50:58.273355 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:50:58.273355 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:50:58.273355 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:50:58.273355 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:50:58.273355 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:50:58.273355 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:50:58.273355 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:50:58.273355 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:50:58.273355 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:50:58.273355 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:50:58.273355 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:50:58.273355 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:50:58.273355 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:50:58.273355 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:50:58.273355 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:50:58.274417 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:50:58.274417 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:50:58.274417 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:50:58.274417 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:50:58.274417 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:50:58.274417 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:50:58.274417 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:50:58.274417 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:50:58.274417 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:50:58.274417 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:50:58.274417 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:50:58.274417 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:50:58.275118 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87794c1b-fda9-442e-9cd2-527d61e51eab None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:58.275118 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87794c1b-fda9-442e-9cd2-527d61e51eab None None] PATH_INFO: `/` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:58.275268 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 35/69] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:50:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:50:58.275268 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf25778-1f17-40fd-9276-6fc6d4734ddf None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:50:58.278565 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 35/70] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:50:58 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:50:58.280007 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceaefbb-0e72-4c93-b808-335c5d261098 None None] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:58.280310 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceaefbb-0e72-4c93-b808-335c5d261098 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:58.280603 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceaefbb-0e72-4c93-b808-335c5d261098 None None] PATH_INFO: `/` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:58.281913 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 36/71] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:50:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:50:58.281913 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba84e2b0-da13-4047-a6aa-529e1aa0c975 None None] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:58.285998 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba84e2b0-da13-4047-a6aa-529e1aa0c975 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:58.286259 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba84e2b0-da13-4047-a6aa-529e1aa0c975 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:58.289734 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:50:58.304152 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.auth.core [None req-ba84e2b0-da13-4047-a6aa-529e1aa0c975 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:50:58.305363 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-ba84e2b0-da13-4047-a6aa-529e1aa0c975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:50:58.308696 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:50:58.374664 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 37/72] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:50:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:50:58.375525 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba84e2b0-da13-4047-a6aa-529e1aa0c975 None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:50:58.381402 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf25778-1f17-40fd-9276-6fc6d4734ddf None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:50:58.416987 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a966f970-b388-4c39-bc44-1d3ac3816ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:50:58.417903 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a966f970-b388-4c39-bc44-1d3ac3816ca6 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:58.418187 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a966f970-b388-4c39-bc44-1d3ac3816ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:58.418469 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a966f970-b388-4c39-bc44-1d3ac3816ca6 None admin] PATH_INFO: `/v3/users/2950d06ae84d46c081f68af26db145d4` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:58.431958 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a966f970-b388-4c39-bc44-1d3ac3816ca6 None admin] RBAC: Authorizing `identity:get_user(user_id=2950d06ae84d46c081f68af26db145d4)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:50:58.433470 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a966f970-b388-4c39-bc44-1d3ac3816ca6 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:50:58.435233 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 36/73] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:50:58 2022] GET /identity/v3/users/2950d06ae84d46c081f68af26db145d4 => generated 275 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:50:58.436527 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707616c5-b524-4aab-a080-5849b7557ea1 None None] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:58.436827 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707616c5-b524-4aab-a080-5849b7557ea1 None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:58.437116 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707616c5-b524-4aab-a080-5849b7557ea1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:58.454208 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.auth.core [None req-707616c5-b524-4aab-a080-5849b7557ea1 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:50:58.455371 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.fernet_utils [None req-707616c5-b524-4aab-a080-5849b7557ea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:50:58.477111 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba84e2b0-da13-4047-a6aa-529e1aa0c975 None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:50:58.511290 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f230f011-4406-4968-adee-ec32d54691dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:50:58.511968 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f230f011-4406-4968-adee-ec32d54691dc None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:58.512260 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f230f011-4406-4968-adee-ec32d54691dc None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:58.512542 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f230f011-4406-4968-adee-ec32d54691dc None admin] PATH_INFO: `/v3/roles` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:58.513300 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f230f011-4406-4968-adee-ec32d54691dc None admin] RBAC: Adding query filter params (name=admin) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:50:58.513615 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f230f011-4406-4968-adee-ec32d54691dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:50:58.515045 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f230f011-4406-4968-adee-ec32d54691dc None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:50:58.517038 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:50:58.523095 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 37/74] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:50:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:50:58.523845 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 38/75] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:50:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:50:58.524381 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd048fe5-d82c-435b-be5d-1140552de9b6 None None] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:58.524685 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd048fe5-d82c-435b-be5d-1140552de9b6 None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:58.524970 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd048fe5-d82c-435b-be5d-1140552de9b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:58.529128 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f230f011-4406-4968-adee-ec32d54691dc None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:50:58.531819 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:50:58.541762 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.auth.core [None req-dd048fe5-d82c-435b-be5d-1140552de9b6 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:50:58.542996 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.fernet_utils [None req-dd048fe5-d82c-435b-be5d-1140552de9b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:50:58.609567 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 38/76] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:50:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:50:58.610487 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd048fe5-d82c-435b-be5d-1140552de9b6 None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:50:58.645019 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f230f011-4406-4968-adee-ec32d54691dc None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:50:58.681920 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17d412d-0a4b-4dd9-a356-eadbb9114921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:50:58.682739 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17d412d-0a4b-4dd9-a356-eadbb9114921 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:58.683035 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17d412d-0a4b-4dd9-a356-eadbb9114921 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:58.683318 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17d412d-0a4b-4dd9-a356-eadbb9114921 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:58.693728 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17d412d-0a4b-4dd9-a356-eadbb9114921 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:50:58.702175 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-f17d412d-0a4b-4dd9-a356-eadbb9114921 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:50:58.702175 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:50:58.702175 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:50:58.702175 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:50:58.702175 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:50:58.702175 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:50:58.702175 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:50:58.702175 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:50:58.702175 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:50:58.702175 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:50:58.702175 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:50:58.702175 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:50:58.702175 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:50:58.702175 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:50:58.702175 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:50:58.702175 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:50:58.703371 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:50:58.703371 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:50:58.703371 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:50:58.703371 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:50:58.703371 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:50:58.703371 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:50:58.703371 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:50:58.703371 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:50:58.703371 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:50:58.703371 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:50:58.703371 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:50:58.703371 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:50:58.703371 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:50:58.703371 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:50:58.703371 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:50:58.703371 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:50:58.704381 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:50:58.704381 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:50:58.704381 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:50:58.704381 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:50:58.704381 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:50:58.704381 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:50:58.704381 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:50:58.704381 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:50:58.704381 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:50:58.704381 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:50:58.704381 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:50:58.704381 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:50:58.704381 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 39/77] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 19 14:50:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:50:58.704381 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17d412d-0a4b-4dd9-a356-eadbb9114921 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:50:58.724347 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd048fe5-d82c-435b-be5d-1140552de9b6 None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:50:58.752398 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:50:58.761169 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17023a66-333e-43f1-a6e4-4f192f3e4e84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:50:58.761829 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17023a66-333e-43f1-a6e4-4f192f3e4e84 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:58.762152 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17023a66-333e-43f1-a6e4-4f192f3e4e84 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:58.762437 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17023a66-333e-43f1-a6e4-4f192f3e4e84 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:58.769064 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:50:58.809591 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17d412d-0a4b-4dd9-a356-eadbb9114921 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:50:58.831920 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17023a66-333e-43f1-a6e4-4f192f3e4e84 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:50:58.834152 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17023a66-333e-43f1-a6e4-4f192f3e4e84 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:50:58.844542 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a75dbc5-b8e6-461d-b469-f45644a9835c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:50:58.845432 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a75dbc5-b8e6-461d-b469-f45644a9835c None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:58.845725 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a75dbc5-b8e6-461d-b469-f45644a9835c None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:58.846044 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a75dbc5-b8e6-461d-b469-f45644a9835c None admin] PATH_INFO: `/v3/users/8c487f3640744d568515b03847007a04` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:58.847272 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a75dbc5-b8e6-461d-b469-f45644a9835c None admin] RBAC: Authorizing `identity:get_user(user_id=8c487f3640744d568515b03847007a04)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:50:58.849477 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a75dbc5-b8e6-461d-b469-f45644a9835c None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:50:58.851173 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 40/78] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:50:58 2022] GET /identity/v3/users/8c487f3640744d568515b03847007a04 => generated 247 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:50:58.851956 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a75dbc5-b8e6-461d-b469-f45644a9835c None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:50:58.935480 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 39/79] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:50:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:50:58.936342 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17023a66-333e-43f1-a6e4-4f192f3e4e84 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:50:58.959005 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a75dbc5-b8e6-461d-b469-f45644a9835c None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:50:58.964308 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:50:58.987695 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:50:58.994352 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5272805-9c39-429b-a3f7-f6304e3cce7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:50:58.995167 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5272805-9c39-429b-a3f7-f6304e3cce7d None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:58.995466 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5272805-9c39-429b-a3f7-f6304e3cce7d None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:58.995754 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5272805-9c39-429b-a3f7-f6304e3cce7d None admin] PATH_INFO: `/v3/domains/default` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:58.996965 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5272805-9c39-429b-a3f7-f6304e3cce7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:50:58.998463 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5272805-9c39-429b-a3f7-f6304e3cce7d None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:50:59.000136 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 41/80] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:50:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:50:59.000908 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5272805-9c39-429b-a3f7-f6304e3cce7d None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:50:59.038457 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17023a66-333e-43f1-a6e4-4f192f3e4e84 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:50:59.080788 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542d0d41-1990-4e9f-ac42-41bef003a188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:50:59.081599 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542d0d41-1990-4e9f-ac42-41bef003a188 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:59.081915 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542d0d41-1990-4e9f-ac42-41bef003a188 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:59.082694 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542d0d41-1990-4e9f-ac42-41bef003a188 None admin] PATH_INFO: `/v3/roles/member` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:59.093444 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542d0d41-1990-4e9f-ac42-41bef003a188 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:50:59.101744 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-542d0d41-1990-4e9f-ac42-41bef003a188 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:50:59.101744 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:50:59.101744 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:50:59.101744 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:50:59.101744 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:50:59.101744 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:50:59.101744 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:50:59.101744 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:50:59.101744 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:50:59.101744 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:50:59.101744 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:50:59.101744 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:50:59.101744 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:50:59.101744 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:50:59.101744 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:50:59.101744 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:50:59.102951 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:50:59.102951 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:50:59.102951 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:50:59.102951 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:50:59.102951 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:50:59.102951 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:50:59.102951 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:50:59.102951 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:50:59.102951 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:50:59.102951 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:50:59.102951 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:50:59.102951 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:50:59.102951 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:50:59.102951 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:50:59.102951 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:50:59.102951 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:50:59.103878 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:50:59.103878 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:50:59.103878 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:50:59.103878 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:50:59.103878 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:50:59.103878 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:50:59.103878 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:50:59.103878 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:50:59.103878 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:50:59.103878 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:50:59.103878 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:50:59.103878 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:50:59.103878 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 40/81] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:50:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:50:59.106663 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542d0d41-1990-4e9f-ac42-41bef003a188 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:50:59.108568 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5272805-9c39-429b-a3f7-f6304e3cce7d None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:50:59.142284 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbb13e8-3365-4c96-b19a-947db5d85b75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:50:59.142955 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbb13e8-3365-4c96-b19a-947db5d85b75 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:59.143248 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbb13e8-3365-4c96-b19a-947db5d85b75 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:59.143539 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbb13e8-3365-4c96-b19a-947db5d85b75 None admin] PATH_INFO: `/v3/roles` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:59.144294 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbb13e8-3365-4c96-b19a-947db5d85b75 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:50:59.145266 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbb13e8-3365-4c96-b19a-947db5d85b75 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:50:59.146966 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbb13e8-3365-4c96-b19a-947db5d85b75 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:50:59.155236 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 42/82] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:50:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:50:59.156051 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbb13e8-3365-4c96-b19a-947db5d85b75 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:50:59.199653 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:50:59.213191 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542d0d41-1990-4e9f-ac42-41bef003a188 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:50:59.216821 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:50:59.246904 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5a11fa-5c4a-4c08-b82a-b21e8462a592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:50:59.247570 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5a11fa-5c4a-4c08-b82a-b21e8462a592 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:59.247865 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5a11fa-5c4a-4c08-b82a-b21e8462a592 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:59.248147 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5a11fa-5c4a-4c08-b82a-b21e8462a592 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:59.259214 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbb13e8-3365-4c96-b19a-947db5d85b75 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:50:59.293045 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2bb34a-a127-427b-9e35-1017046a1da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:50:59.293948 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2bb34a-a127-427b-9e35-1017046a1da3 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:59.294242 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2bb34a-a127-427b-9e35-1017046a1da3 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:59.294524 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2bb34a-a127-427b-9e35-1017046a1da3 None admin] PATH_INFO: `/v3/projects/aca8075ce986433eae657055d1366699` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:59.295728 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2bb34a-a127-427b-9e35-1017046a1da3 None admin] RBAC: Authorizing `identity:get_project(project_id=aca8075ce986433eae657055d1366699)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:50:59.297194 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2bb34a-a127-427b-9e35-1017046a1da3 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:50:59.298943 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 43/83] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:50:59 2022] GET /identity/v3/projects/aca8075ce986433eae657055d1366699 => generated 298 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:50:59.299776 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2bb34a-a127-427b-9e35-1017046a1da3 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:50:59.320444 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5a11fa-5c4a-4c08-b82a-b21e8462a592 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:50:59.321983 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5a11fa-5c4a-4c08-b82a-b21e8462a592 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:50:59.399974 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2bb34a-a127-427b-9e35-1017046a1da3 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:50:59.407913 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:50:59.421039 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 41/84] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:50:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:50:59.425950 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5a11fa-5c4a-4c08-b82a-b21e8462a592 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:50:59.430212 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:50:59.439291 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080c633c-21fe-4142-bb45-2e0d19364f3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:50:59.439978 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080c633c-21fe-4142-bb45-2e0d19364f3e None admin] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:59.440274 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080c633c-21fe-4142-bb45-2e0d19364f3e None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:59.440564 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080c633c-21fe-4142-bb45-2e0d19364f3e None admin] PATH_INFO: `/v3/users` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:59.441291 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080c633c-21fe-4142-bb45-2e0d19364f3e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:50:59.443324 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080c633c-21fe-4142-bb45-2e0d19364f3e None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:50:59.470459 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 44/85] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 19 14:50:59 2022] POST /identity/v3/users => generated 289 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:50:59.471284 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080c633c-21fe-4142-bb45-2e0d19364f3e None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:50:59.535267 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5a11fa-5c4a-4c08-b82a-b21e8462a592 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:50:59.586073 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3036f75-ec5f-4d64-ac03-db5632b6cf0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:50:59.586780 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3036f75-ec5f-4d64-ac03-db5632b6cf0b None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:59.587075 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3036f75-ec5f-4d64-ac03-db5632b6cf0b None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:59.587364 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3036f75-ec5f-4d64-ac03-db5632b6cf0b None admin] PATH_INFO: `/v3/roles` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:59.588145 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3036f75-ec5f-4d64-ac03-db5632b6cf0b None admin] RBAC: Adding query filter params (name=member) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:50:59.588470 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3036f75-ec5f-4d64-ac03-db5632b6cf0b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:50:59.589963 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3036f75-ec5f-4d64-ac03-db5632b6cf0b None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:50:59.600761 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 42/86] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:50:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:50:59.601669 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3036f75-ec5f-4d64-ac03-db5632b6cf0b None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:50:59.617649 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080c633c-21fe-4142-bb45-2e0d19364f3e None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:50:59.664395 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9999c9ba-3caf-4adf-829d-aff7a075ff86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:50:59.664951 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:50:59.665050 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9999c9ba-3caf-4adf-829d-aff7a075ff86 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:59.665050 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9999c9ba-3caf-4adf-829d-aff7a075ff86 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:59.665050 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9999c9ba-3caf-4adf-829d-aff7a075ff86 None admin] PATH_INFO: `/v3/users/2950d06ae84d46c081f68af26db145d4` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:59.667015 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9999c9ba-3caf-4adf-829d-aff7a075ff86 None admin] RBAC: Authorizing `identity:get_user(user_id=2950d06ae84d46c081f68af26db145d4)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:50:59.667851 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9999c9ba-3caf-4adf-829d-aff7a075ff86 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:50:59.669983 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 45/87] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:50:59 2022] GET /identity/v3/users/2950d06ae84d46c081f68af26db145d4 => generated 275 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:50:59.670457 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9999c9ba-3caf-4adf-829d-aff7a075ff86 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:50:59.731241 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3036f75-ec5f-4d64-ac03-db5632b6cf0b None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:50:59.736727 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:50:59.772513 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7662f66a-bb9d-40f2-893e-8c1947a4f7ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:50:59.773197 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7662f66a-bb9d-40f2-893e-8c1947a4f7ad None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:59.773492 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7662f66a-bb9d-40f2-893e-8c1947a4f7ad None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:59.773779 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7662f66a-bb9d-40f2-893e-8c1947a4f7ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:59.775348 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7662f66a-bb9d-40f2-893e-8c1947a4f7ad None admin] RBAC: Adding query filter params (role.id=aed9aa9ef7f7448bb3b40c9ad082d545, scope.project.id=aca8075ce986433eae657055d1366699, user.id=2950d06ae84d46c081f68af26db145d4) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:50:59.775348 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7662f66a-bb9d-40f2-893e-8c1947a4f7ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:50:59.776546 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7662f66a-bb9d-40f2-893e-8c1947a4f7ad None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:50:59.790022 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 43/88] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:50:59 2022] GET /identity/v3/role_assignments?user.id=2950d06ae84d46c081f68af26db145d4&scope.project.id=aca8075ce986433eae657055d1366699&role.id=aed9aa9ef7f7448bb3b40c9ad082d545 => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:50:59.790022 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7662f66a-bb9d-40f2-893e-8c1947a4f7ad None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:50:59.824245 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9999c9ba-3caf-4adf-829d-aff7a075ff86 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:50:59.869130 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee55fd67-882d-4698-a295-a99639e67998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:50:59.870168 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee55fd67-882d-4698-a295-a99639e67998 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:59.870469 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee55fd67-882d-4698-a295-a99639e67998 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:59.871026 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee55fd67-882d-4698-a295-a99639e67998 None admin] PATH_INFO: `/v3/projects/aca8075ce986433eae657055d1366699` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:59.872298 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee55fd67-882d-4698-a295-a99639e67998 None admin] RBAC: Authorizing `identity:get_project(project_id=aca8075ce986433eae657055d1366699)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:50:59.873783 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee55fd67-882d-4698-a295-a99639e67998 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:50:59.876924 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 46/89] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:50:59 2022] GET /identity/v3/projects/aca8075ce986433eae657055d1366699 => generated 298 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:50:59.877764 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee55fd67-882d-4698-a295-a99639e67998 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:50:59.931486 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7662f66a-bb9d-40f2-893e-8c1947a4f7ad None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:50:59.962238 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:00.003538 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504e5bd4-7cf6-4418-85bb-1470723b20d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:00.004904 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504e5bd4-7cf6-4418-85bb-1470723b20d0 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:00.005193 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504e5bd4-7cf6-4418-85bb-1470723b20d0 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:00.005476 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504e5bd4-7cf6-4418-85bb-1470723b20d0 None admin] PATH_INFO: `/v3/users/2950d06ae84d46c081f68af26db145d4` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:00.007156 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504e5bd4-7cf6-4418-85bb-1470723b20d0 None admin] RBAC: Authorizing `identity:get_user(user_id=2950d06ae84d46c081f68af26db145d4)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:00.008684 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504e5bd4-7cf6-4418-85bb-1470723b20d0 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:00.010471 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 44/90] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:50:59 2022] GET /identity/v3/users/2950d06ae84d46c081f68af26db145d4 => generated 275 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:00.014036 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504e5bd4-7cf6-4418-85bb-1470723b20d0 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:00.075216 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee55fd67-882d-4698-a295-a99639e67998 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:00.114267 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:00.141691 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a526255b-9932-4e30-bc68-97180e512772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:00.142389 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a526255b-9932-4e30-bc68-97180e512772 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:00.142735 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a526255b-9932-4e30-bc68-97180e512772 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:00.143024 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a526255b-9932-4e30-bc68-97180e512772 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:00.200032 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504e5bd4-7cf6-4418-85bb-1470723b20d0 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:00.261360 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101ace30-8c45-4cee-9eac-38092ba86a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:00.261978 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101ace30-8c45-4cee-9eac-38092ba86a49 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:00.262270 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101ace30-8c45-4cee-9eac-38092ba86a49 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:00.262552 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101ace30-8c45-4cee-9eac-38092ba86a49 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:00.263403 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101ace30-8c45-4cee-9eac-38092ba86a49 None admin] RBAC: Adding query filter params (role.id=028379e01f9245e9b8fc56e8b9e8a6ad, scope.project.id=aca8075ce986433eae657055d1366699, user.id=8c487f3640744d568515b03847007a04) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:00.263724 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101ace30-8c45-4cee-9eac-38092ba86a49 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:00.265191 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101ace30-8c45-4cee-9eac-38092ba86a49 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:00.279631 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a526255b-9932-4e30-bc68-97180e512772 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:00.281121 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a526255b-9932-4e30-bc68-97180e512772 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:00.289996 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 45/91] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:51:00 2022] GET /identity/v3/role_assignments?user.id=8c487f3640744d568515b03847007a04&scope.project.id=aca8075ce986433eae657055d1366699&role.id=028379e01f9245e9b8fc56e8b9e8a6ad => generated 259 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:00.291059 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101ace30-8c45-4cee-9eac-38092ba86a49 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:00.378144 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:00.519244 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 47/92] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:50:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 641 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:51:00.524010 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101ace30-8c45-4cee-9eac-38092ba86a49 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:00.526052 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a526255b-9932-4e30-bc68-97180e512772 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:00.553083 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:00.630782 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fcc93b-8912-4364-8a9d-7aee258aa9ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:00.631639 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fcc93b-8912-4364-8a9d-7aee258aa9ab None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:00.631931 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fcc93b-8912-4364-8a9d-7aee258aa9ab None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:00.632213 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fcc93b-8912-4364-8a9d-7aee258aa9ab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:00.724019 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fcc93b-8912-4364-8a9d-7aee258aa9ab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:00.725518 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fcc93b-8912-4364-8a9d-7aee258aa9ab None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:00.729975 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a526255b-9932-4e30-bc68-97180e512772 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:00.780210 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954d40f4-d801-41df-9e8b-6ceab05e0bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:00.781475 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954d40f4-d801-41df-9e8b-6ceab05e0bdd None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:00.782034 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954d40f4-d801-41df-9e8b-6ceab05e0bdd None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:00.782563 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954d40f4-d801-41df-9e8b-6ceab05e0bdd None admin] PATH_INFO: `/v3/projects/aca8075ce986433eae657055d1366699` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:00.786584 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954d40f4-d801-41df-9e8b-6ceab05e0bdd None admin] RBAC: Authorizing `identity:get_project(project_id=aca8075ce986433eae657055d1366699)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:00.788333 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954d40f4-d801-41df-9e8b-6ceab05e0bdd None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:00.790380 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 48/93] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:51:00 2022] GET /identity/v3/projects/aca8075ce986433eae657055d1366699 => generated 298 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:51:00.796297 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954d40f4-d801-41df-9e8b-6ceab05e0bdd None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:00.811039 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:00.918212 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 46/94] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:51:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 628 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:51:00.923440 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fcc93b-8912-4364-8a9d-7aee258aa9ab None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:00.943086 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:00.950743 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954d40f4-d801-41df-9e8b-6ceab05e0bdd None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:01.032951 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dab0272-d05b-4aa3-9571-dd64bf70aefc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:01.034043 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dab0272-d05b-4aa3-9571-dd64bf70aefc None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:01.034560 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dab0272-d05b-4aa3-9571-dd64bf70aefc None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:01.035077 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dab0272-d05b-4aa3-9571-dd64bf70aefc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:01.036305 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dab0272-d05b-4aa3-9571-dd64bf70aefc None admin] RBAC: Adding query filter params (role.id=3b9c36a010504ba7bc63a8ccf0a9bca0, scope.project.id=aca8075ce986433eae657055d1366699, user.id=2950d06ae84d46c081f68af26db145d4) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:01.036873 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dab0272-d05b-4aa3-9571-dd64bf70aefc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:01.038605 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dab0272-d05b-4aa3-9571-dd64bf70aefc None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:01.053016 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 49/95] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:51:00 2022] GET /identity/v3/role_assignments?user.id=2950d06ae84d46c081f68af26db145d4&scope.project.id=aca8075ce986433eae657055d1366699&role.id=3b9c36a010504ba7bc63a8ccf0a9bca0 => generated 259 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:51:01.073523 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fcc93b-8912-4364-8a9d-7aee258aa9ab None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:01.114004 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e351dd62-6b3e-4e05-ae44-62ce171be689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:01.115711 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e351dd62-6b3e-4e05-ae44-62ce171be689 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:01.116006 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e351dd62-6b3e-4e05-ae44-62ce171be689 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:01.116293 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e351dd62-6b3e-4e05-ae44-62ce171be689 None admin] PATH_INFO: `/v3/projects/ab05694ea13841e182a0be2091bf3654` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:01.117554 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e351dd62-6b3e-4e05-ae44-62ce171be689 None admin] RBAC: Authorizing `identity:get_project(project_id=ab05694ea13841e182a0be2091bf3654)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:01.119104 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e351dd62-6b3e-4e05-ae44-62ce171be689 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:01.120920 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 47/96] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:51:00 2022] GET /identity/v3/projects/ab05694ea13841e182a0be2091bf3654 => generated 312 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:51:01.127071 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dab0272-d05b-4aa3-9571-dd64bf70aefc None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:01.260710 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dab0272-d05b-4aa3-9571-dd64bf70aefc None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:01.298998 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78c35ef-4f08-45bf-90eb-68080b276f40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:01.300074 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78c35ef-4f08-45bf-90eb-68080b276f40 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:01.300594 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78c35ef-4f08-45bf-90eb-68080b276f40 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:01.301936 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78c35ef-4f08-45bf-90eb-68080b276f40 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:01.303060 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78c35ef-4f08-45bf-90eb-68080b276f40 None admin] RBAC: Adding query filter params (role.id=aed9aa9ef7f7448bb3b40c9ad082d545, scope.project.id=ab05694ea13841e182a0be2091bf3654, user.id=2950d06ae84d46c081f68af26db145d4) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:01.303652 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78c35ef-4f08-45bf-90eb-68080b276f40 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:01.305375 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78c35ef-4f08-45bf-90eb-68080b276f40 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:01.312084 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 50/97] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:51:01 2022] GET /identity/v3/role_assignments?user.id=2950d06ae84d46c081f68af26db145d4&scope.project.id=ab05694ea13841e182a0be2091bf3654&role.id=aed9aa9ef7f7448bb3b40c9ad082d545 => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:02.432862 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b77436-018e-451f-8022-1c7107d154ce None None] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:02.433457 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b77436-018e-451f-8022-1c7107d154ce None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:02.433457 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b77436-018e-451f-8022-1c7107d154ce None None] PATH_INFO: `/` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:02.434903 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 48/98] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:51:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:51:02.440832 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb4261b-13e5-4cd9-ab20-bb36cf20b9c7 None None] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:02.441312 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb4261b-13e5-4cd9-ab20-bb36cf20b9c7 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:02.442106 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb4261b-13e5-4cd9-ab20-bb36cf20b9c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:02.475046 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.auth.core [None req-1eb4261b-13e5-4cd9-ab20-bb36cf20b9c7 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:51:02.476312 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-1eb4261b-13e5-4cd9-ab20-bb36cf20b9c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:51:02.488893 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:02.516152 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91672d31-e5cd-49ff-9caa-8ae8a3d152eb None None] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:02.516499 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91672d31-e5cd-49ff-9caa-8ae8a3d152eb None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:02.516732 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91672d31-e5cd-49ff-9caa-8ae8a3d152eb None None] PATH_INFO: `/` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:02.518310 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 49/99] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:51:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:51:02.525129 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a61568f-a8da-4a81-9d2f-366c98635aef None None] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:02.525761 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a61568f-a8da-4a81-9d2f-366c98635aef None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:02.526515 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a61568f-a8da-4a81-9d2f-366c98635aef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:02.560004 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.auth.core [None req-5a61568f-a8da-4a81-9d2f-366c98635aef None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:51:02.561243 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.fernet_utils [None req-5a61568f-a8da-4a81-9d2f-366c98635aef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:51:02.590384 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:02.608754 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 51/100] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:51:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:51:02.702849 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 50/101] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:51:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:51:02.704389 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb4261b-13e5-4cd9-ab20-bb36cf20b9c7 None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:02.752097 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a61568f-a8da-4a81-9d2f-366c98635aef None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:02.877997 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb4261b-13e5-4cd9-ab20-bb36cf20b9c7 None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:02.970289 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb51b5c-e6a9-42d0-83cc-4a907ba6b429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:02.971094 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb51b5c-e6a9-42d0-83cc-4a907ba6b429 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:02.971397 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb51b5c-e6a9-42d0-83cc-4a907ba6b429 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:02.971685 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb51b5c-e6a9-42d0-83cc-4a907ba6b429 None admin] PATH_INFO: `/v3/domains/default` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:02.974198 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb51b5c-e6a9-42d0-83cc-4a907ba6b429 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:02.975692 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb51b5c-e6a9-42d0-83cc-4a907ba6b429 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:02.977394 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 52/102] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:51:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:02.984920 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb51b5c-e6a9-42d0-83cc-4a907ba6b429 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:03.010369 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a61568f-a8da-4a81-9d2f-366c98635aef None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:03.027381 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:03.072576 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4c205b-0463-453d-8584-303252771405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:03.073831 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4c205b-0463-453d-8584-303252771405 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:03.076823 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4c205b-0463-453d-8584-303252771405 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:03.080399 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4c205b-0463-453d-8584-303252771405 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:03.110473 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4c205b-0463-453d-8584-303252771405 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:51:03.118497 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-0d4c205b-0463-453d-8584-303252771405 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:51:03.118497 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:51:03.118497 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:51:03.118497 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:51:03.118497 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:51:03.118497 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:51:03.118497 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:51:03.118497 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:51:03.118497 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:51:03.118497 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:51:03.118497 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:51:03.118497 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:51:03.118497 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:51:03.118497 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:51:03.118497 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:51:03.118497 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:51:03.119632 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:51:03.119632 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:51:03.119632 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:51:03.119632 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:51:03.119632 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:51:03.119632 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:51:03.119632 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:51:03.119632 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:51:03.119632 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:51:03.119632 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:51:03.119632 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:51:03.119632 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:51:03.119632 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:51:03.119632 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:51:03.119632 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:51:03.119632 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:51:03.120653 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:51:03.120653 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:51:03.120653 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:51:03.120653 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:51:03.120653 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:51:03.120653 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:51:03.120653 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:51:03.120653 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:51:03.120653 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:51:03.120653 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:51:03.120653 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:51:03.120653 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:51:03.120653 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 51/103] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:51:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 368 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:51:03.121504 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe6ed26-4164-49e4-b5ca-5e95e401d0b4 None None] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:03.122196 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe6ed26-4164-49e4-b5ca-5e95e401d0b4 None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:03.122196 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe6ed26-4164-49e4-b5ca-5e95e401d0b4 None None] PATH_INFO: `/` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:03.123553 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 52/104] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:51:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:51:03.130395 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c223fb-f4a8-4cd2-9dd3-6769a2c34967 None None] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:03.130707 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c223fb-f4a8-4cd2-9dd3-6769a2c34967 None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:03.130972 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c223fb-f4a8-4cd2-9dd3-6769a2c34967 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:03.151791 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:03.175337 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.auth.core [None req-24c223fb-f4a8-4cd2-9dd3-6769a2c34967 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:51:03.176708 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.fernet_utils [None req-24c223fb-f4a8-4cd2-9dd3-6769a2c34967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:51:03.199567 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb51b5c-e6a9-42d0-83cc-4a907ba6b429 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:03.258795 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbff4a6-fc4b-42b2-b808-8da633b9e438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:03.259426 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbff4a6-fc4b-42b2-b808-8da633b9e438 None admin] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:03.259720 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbff4a6-fc4b-42b2-b808-8da633b9e438 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:03.260001 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbff4a6-fc4b-42b2-b808-8da633b9e438 None admin] PATH_INFO: `/v3/projects` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:03.260834 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbff4a6-fc4b-42b2-b808-8da633b9e438 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:03.262515 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbff4a6-fc4b-42b2-b808-8da633b9e438 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:03.306724 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 53/105] 10.222.0.57 () {60 vars in 1141 bytes} [Thu May 19 14:51:02 2022] POST /identity/v3/projects => generated 302 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:51:03.307576 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbff4a6-fc4b-42b2-b808-8da633b9e438 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:03.332443 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 53/106] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:51:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:51:03.398469 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c223fb-f4a8-4cd2-9dd3-6769a2c34967 None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:03.444471 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:03.515024 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbff4a6-fc4b-42b2-b808-8da633b9e438 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:03.584135 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b538472-d076-4540-8c15-bcfdf6574ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:03.584852 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b538472-d076-4540-8c15-bcfdf6574ed7 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:03.585152 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b538472-d076-4540-8c15-bcfdf6574ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:03.585443 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b538472-d076-4540-8c15-bcfdf6574ed7 None admin] PATH_INFO: `/v3/roles` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:03.586266 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b538472-d076-4540-8c15-bcfdf6574ed7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:03.586591 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b538472-d076-4540-8c15-bcfdf6574ed7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:03.588063 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b538472-d076-4540-8c15-bcfdf6574ed7 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:03.593483 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c223fb-f4a8-4cd2-9dd3-6769a2c34967 None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:03.608454 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 54/107] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:51:03 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:03.614193 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b538472-d076-4540-8c15-bcfdf6574ed7 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:03.669752 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c9e203-895d-40ad-b99f-dbf61050bd57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:03.669752 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c9e203-895d-40ad-b99f-dbf61050bd57 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:03.669752 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c9e203-895d-40ad-b99f-dbf61050bd57 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:03.669752 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c9e203-895d-40ad-b99f-dbf61050bd57 None admin] PATH_INFO: `/v3/roles/member` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:03.673483 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:03.697501 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c9e203-895d-40ad-b99f-dbf61050bd57 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:51:03.710289 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-b8c9e203-895d-40ad-b99f-dbf61050bd57 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:51:03.710289 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:51:03.710289 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:51:03.710289 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:51:03.710289 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:51:03.710289 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:51:03.710289 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:51:03.710289 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:51:03.710289 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:51:03.710289 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:51:03.710289 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:51:03.710289 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:51:03.710289 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:51:03.710289 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:51:03.710289 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:51:03.710289 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:51:03.711409 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:51:03.711409 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:51:03.711409 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:51:03.711409 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:51:03.711409 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:51:03.711409 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:51:03.711409 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:51:03.711409 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:51:03.711409 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:51:03.711409 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:51:03.711409 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:51:03.711409 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:51:03.711409 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:51:03.711409 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:51:03.711409 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:51:03.711409 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:51:03.712341 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:51:03.712341 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:51:03.712341 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:51:03.712341 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:51:03.712341 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:51:03.712341 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:51:03.712341 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:51:03.712341 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:51:03.712341 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:51:03.712341 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:51:03.712341 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:51:03.712341 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:51:03.712341 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 54/108] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:51:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:51:03.725421 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c9e203-895d-40ad-b99f-dbf61050bd57 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:03.810950 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b538472-d076-4540-8c15-bcfdf6574ed7 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:03.855228 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:03.874967 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba07812-7c08-4936-b8f6-c5744b07facd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:03.875871 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba07812-7c08-4936-b8f6-c5744b07facd None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:03.876167 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba07812-7c08-4936-b8f6-c5744b07facd None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:03.876456 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba07812-7c08-4936-b8f6-c5744b07facd None admin] PATH_INFO: `/v3/users/bce930239cd6428ca707fea57a0d4abb` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:03.905921 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba07812-7c08-4936-b8f6-c5744b07facd None admin] RBAC: Authorizing `identity:get_user(user_id=bce930239cd6428ca707fea57a0d4abb)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:03.908688 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba07812-7c08-4936-b8f6-c5744b07facd None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:03.912350 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 55/109] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:51:03 2022] GET /identity/v3/users/bce930239cd6428ca707fea57a0d4abb => generated 289 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:03.917639 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba07812-7c08-4936-b8f6-c5744b07facd None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:03.923697 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c9e203-895d-40ad-b99f-dbf61050bd57 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:03.964185 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc2d477-e7b3-4d6c-a580-6a4cdc6f90ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:03.964883 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc2d477-e7b3-4d6c-a580-6a4cdc6f90ea None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:03.965179 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc2d477-e7b3-4d6c-a580-6a4cdc6f90ea None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:03.965530 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc2d477-e7b3-4d6c-a580-6a4cdc6f90ea None admin] PATH_INFO: `/v3/roles` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:03.967232 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc2d477-e7b3-4d6c-a580-6a4cdc6f90ea None admin] RBAC: Adding query filter params (name=member) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:03.967585 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc2d477-e7b3-4d6c-a580-6a4cdc6f90ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:03.969026 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc2d477-e7b3-4d6c-a580-6a4cdc6f90ea None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:03.980025 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 55/110] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:51:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:03.985606 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc2d477-e7b3-4d6c-a580-6a4cdc6f90ea None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:04.058483 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:04.062502 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba07812-7c08-4936-b8f6-c5744b07facd None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:04.140777 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc2d477-e7b3-4d6c-a580-6a4cdc6f90ea None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:04.170589 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca111a19-bf7f-4635-85af-815049eeedfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:04.171364 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca111a19-bf7f-4635-85af-815049eeedfa None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:04.171669 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca111a19-bf7f-4635-85af-815049eeedfa None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:04.171961 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca111a19-bf7f-4635-85af-815049eeedfa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:04.178411 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a7c94a-c32b-4040-9477-171d28a4e04f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:04.179313 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a7c94a-c32b-4040-9477-171d28a4e04f None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:04.179608 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a7c94a-c32b-4040-9477-171d28a4e04f None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:04.179899 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a7c94a-c32b-4040-9477-171d28a4e04f None admin] PATH_INFO: `/v3/users/2950d06ae84d46c081f68af26db145d4` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:04.181158 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a7c94a-c32b-4040-9477-171d28a4e04f None admin] RBAC: Authorizing `identity:get_user(user_id=2950d06ae84d46c081f68af26db145d4)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:04.182690 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a7c94a-c32b-4040-9477-171d28a4e04f None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:04.184513 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 56/111] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:51:03 2022] GET /identity/v3/users/2950d06ae84d46c081f68af26db145d4 => generated 275 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:51:04.185889 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7835a4e6-0924-4bfc-a370-f563f957a23e None None] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:04.186201 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7835a4e6-0924-4bfc-a370-f563f957a23e None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:04.186497 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7835a4e6-0924-4bfc-a370-f563f957a23e None None] PATH_INFO: `/` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:04.187757 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 57/112] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:51:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:51:04.193945 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7835a4e6-0924-4bfc-a370-f563f957a23e None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:04.294705 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:04.314950 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7835a4e6-0924-4bfc-a370-f563f957a23e None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:04.330965 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca111a19-bf7f-4635-85af-815049eeedfa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:04.333385 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca111a19-bf7f-4635-85af-815049eeedfa None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:04.342210 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:04.357885 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c571d7-ff73-479c-9bdc-a2ae6617e5c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:04.359218 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c571d7-ff73-479c-9bdc-a2ae6617e5c4 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:04.359665 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c571d7-ff73-479c-9bdc-a2ae6617e5c4 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:04.360102 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c571d7-ff73-479c-9bdc-a2ae6617e5c4 None admin] PATH_INFO: `/v3/projects/aca8075ce986433eae657055d1366699` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:04.361639 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c571d7-ff73-479c-9bdc-a2ae6617e5c4 None admin] RBAC: Authorizing `identity:get_project(project_id=aca8075ce986433eae657055d1366699)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:04.363765 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c571d7-ff73-479c-9bdc-a2ae6617e5c4 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:04.365736 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 58/113] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:51:04 2022] GET /identity/v3/projects/aca8075ce986433eae657055d1366699 => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:51:04.371661 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97a4601-63af-40bc-a6ea-2254c3f89f81 None None] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:04.372211 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97a4601-63af-40bc-a6ea-2254c3f89f81 None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:04.372706 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97a4601-63af-40bc-a6ea-2254c3f89f81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:04.406011 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.auth.core [None req-e97a4601-63af-40bc-a6ea-2254c3f89f81 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:51:04.413789 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.fernet_utils [None req-e97a4601-63af-40bc-a6ea-2254c3f89f81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:51:04.502726 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 56/114] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:51:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 585 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:51:04.503270 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 59/115] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:51:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:51:04.508704 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca111a19-bf7f-4635-85af-815049eeedfa None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:04.510154 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f76e47-6c9a-4b6b-884e-4c98546b442d None None] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:04.510457 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f76e47-6c9a-4b6b-884e-4c98546b442d None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:04.510744 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f76e47-6c9a-4b6b-884e-4c98546b442d None None] PATH_INFO: `/` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:04.512260 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 60/116] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:51:04 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:51:04.512848 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f76e47-6c9a-4b6b-884e-4c98546b442d None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:04.636566 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:04.655163 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca111a19-bf7f-4635-85af-815049eeedfa None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:04.655760 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f76e47-6c9a-4b6b-884e-4c98546b442d None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:04.690126 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:04.700991 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3b22cb-9337-419c-9de5-530526c68e20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:04.702427 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3b22cb-9337-419c-9de5-530526c68e20 None admin] REQUEST_METHOD: `PUT` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:04.702838 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3b22cb-9337-419c-9de5-530526c68e20 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:04.703136 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3b22cb-9337-419c-9de5-530526c68e20 None admin] PATH_INFO: `/v3/projects/aca8075ce986433eae657055d1366699/users/2950d06ae84d46c081f68af26db145d4/roles/aed9aa9ef7f7448bb3b40c9ad082d545` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:04.705392 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3b22cb-9337-419c-9de5-530526c68e20 None admin] RBAC: Authorizing `identity:create_grant(project_id=aca8075ce986433eae657055d1366699, user_id=2950d06ae84d46c081f68af26db145d4, role_id=aed9aa9ef7f7448bb3b40c9ad082d545)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:04.707133 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b64cfd-3181-43fc-ac48-27d9cf665784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:04.707396 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3b22cb-9337-419c-9de5-530526c68e20 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:04.708667 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b64cfd-3181-43fc-ac48-27d9cf665784 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:04.709135 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b64cfd-3181-43fc-ac48-27d9cf665784 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:04.709733 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b64cfd-3181-43fc-ac48-27d9cf665784 None admin] PATH_INFO: `/v3/projects/aca8075ce986433eae657055d1366699` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:04.711416 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b64cfd-3181-43fc-ac48-27d9cf665784 None admin] RBAC: Authorizing `identity:get_project(project_id=aca8075ce986433eae657055d1366699)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:04.713590 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b64cfd-3181-43fc-ac48-27d9cf665784 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:04.721553 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 61/117] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:51:04 2022] GET /identity/v3/projects/aca8075ce986433eae657055d1366699 => generated 298 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:04.723276 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9203a8a2-ee32-455f-bbb1-ccbe0868f960 None None] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:04.723602 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9203a8a2-ee32-455f-bbb1-ccbe0868f960 None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:04.723743 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 57/118] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 14:51:04 2022] PUT /identity/v3/projects/aca8075ce986433eae657055d1366699/users/2950d06ae84d46c081f68af26db145d4/roles/aed9aa9ef7f7448bb3b40c9ad082d545 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:51:04.723860 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9203a8a2-ee32-455f-bbb1-ccbe0868f960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:04.724657 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3b22cb-9337-419c-9de5-530526c68e20 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:04.744558 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.auth.core [None req-9203a8a2-ee32-455f-bbb1-ccbe0868f960 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:51:04.745986 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.fernet_utils [None req-9203a8a2-ee32-455f-bbb1-ccbe0868f960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:51:04.842250 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 62/119] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:51:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:51:04.843380 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9203a8a2-ee32-455f-bbb1-ccbe0868f960 None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:04.861100 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3b22cb-9337-419c-9de5-530526c68e20 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:04.917610 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8bbad3-8eda-43b3-98aa-9e0822daa2b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:04.918395 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8bbad3-8eda-43b3-98aa-9e0822daa2b4 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:04.918696 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8bbad3-8eda-43b3-98aa-9e0822daa2b4 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:04.918988 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8bbad3-8eda-43b3-98aa-9e0822daa2b4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:04.933726 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8bbad3-8eda-43b3-98aa-9e0822daa2b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:51:04.943037 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-4e8bbad3-8eda-43b3-98aa-9e0822daa2b4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:51:04.943037 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:51:04.943037 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:51:04.943037 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:51:04.943037 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:51:04.943037 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:51:04.943037 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:51:04.943037 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:51:04.943037 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:51:04.943037 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:51:04.943037 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:51:04.943037 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:51:04.943037 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:51:04.943037 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:51:04.943037 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:51:04.943037 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:51:04.944185 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:51:04.944185 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:51:04.944185 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:51:04.944185 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:51:04.944185 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:51:04.944185 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:51:04.944185 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:51:04.944185 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:51:04.944185 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:51:04.944185 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:51:04.944185 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:51:04.944185 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:51:04.944185 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:51:04.944185 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:51:04.944185 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:51:04.944185 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:51:04.945100 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:51:04.945100 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:51:04.945100 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:51:04.945100 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:51:04.945100 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:51:04.945100 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:51:04.945100 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:51:04.945100 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:51:04.945100 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:51:04.945100 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:51:04.945100 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:51:04.945100 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:51:04.945100 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 58/120] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:51:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:51:04.945912 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e3553f-6279-401e-968d-3bcd60936760 None None] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:04.946607 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e3553f-6279-401e-968d-3bcd60936760 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:04.946607 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e3553f-6279-401e-968d-3bcd60936760 None None] PATH_INFO: `/` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:04.948015 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 59/121] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:51:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:51:04.948609 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e3553f-6279-401e-968d-3bcd60936760 None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:04.959694 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:04.995958 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9203a8a2-ee32-455f-bbb1-ccbe0868f960 None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:05.018618 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:05.038778 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d8051c-6477-41e1-914d-7538e4f79d1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:05.039508 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d8051c-6477-41e1-914d-7538e4f79d1f None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:05.039808 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d8051c-6477-41e1-914d-7538e4f79d1f None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:05.040098 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d8051c-6477-41e1-914d-7538e4f79d1f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:05.041221 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d8051c-6477-41e1-914d-7538e4f79d1f None admin] RBAC: Adding query filter params (role.id=e56389e17ad24e20bf3bba1ca7684062, scope.project.id=aca8075ce986433eae657055d1366699, user.id=bce930239cd6428ca707fea57a0d4abb) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:05.041548 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d8051c-6477-41e1-914d-7538e4f79d1f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:05.043838 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d8051c-6477-41e1-914d-7538e4f79d1f None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:05.052877 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 63/122] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:51:04 2022] GET /identity/v3/role_assignments?user.id=bce930239cd6428ca707fea57a0d4abb&scope.project.id=aca8075ce986433eae657055d1366699&role.id=e56389e17ad24e20bf3bba1ca7684062 => generated 259 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:05.054187 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900fdecd-fbb1-49c5-8fcc-f3fb5066d362 None None] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:05.054487 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900fdecd-fbb1-49c5-8fcc-f3fb5066d362 None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:05.054829 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900fdecd-fbb1-49c5-8fcc-f3fb5066d362 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:05.078545 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.auth.core [None req-900fdecd-fbb1-49c5-8fcc-f3fb5066d362 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:51:05.080064 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.fernet_utils [None req-900fdecd-fbb1-49c5-8fcc-f3fb5066d362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:51:05.094611 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e3553f-6279-401e-968d-3bcd60936760 None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:05.139030 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1800a428-eaba-4fd9-a5bc-359ab0c1792d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:05.140249 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1800a428-eaba-4fd9-a5bc-359ab0c1792d None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:05.140822 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1800a428-eaba-4fd9-a5bc-359ab0c1792d None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:05.141335 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1800a428-eaba-4fd9-a5bc-359ab0c1792d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:05.155074 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1800a428-eaba-4fd9-a5bc-359ab0c1792d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:51:05.165178 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-1800a428-eaba-4fd9-a5bc-359ab0c1792d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:51:05.165178 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:51:05.165178 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:51:05.165178 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:51:05.165178 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:51:05.165178 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:51:05.165178 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:51:05.165178 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:51:05.165178 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:51:05.165178 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:51:05.165178 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:51:05.165178 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:51:05.165178 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:51:05.165178 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:51:05.165178 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:51:05.165178 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:51:05.166307 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:51:05.166307 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:51:05.166307 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:51:05.166307 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:51:05.166307 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:51:05.166307 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:51:05.166307 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:51:05.166307 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:51:05.166307 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:51:05.166307 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:51:05.166307 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:51:05.166307 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:51:05.166307 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:51:05.166307 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:51:05.166307 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:51:05.166307 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:51:05.167247 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:51:05.167247 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:51:05.167247 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:51:05.167247 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:51:05.167247 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:51:05.167247 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:51:05.167247 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:51:05.167247 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:51:05.167247 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:51:05.167247 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:51:05.167247 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:51:05.167247 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:51:05.169992 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 60/123] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 19 14:51:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:51:05.171165 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1800a428-eaba-4fd9-a5bc-359ab0c1792d None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:05.183487 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 64/124] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:51:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:51:05.184537 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900fdecd-fbb1-49c5-8fcc-f3fb5066d362 None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:05.323618 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900fdecd-fbb1-49c5-8fcc-f3fb5066d362 None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:05.341220 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:05.354243 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:05.369475 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebb2773-6a83-43e4-99fc-4ee8a7437e1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:05.370221 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebb2773-6a83-43e4-99fc-4ee8a7437e1c None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:05.370796 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebb2773-6a83-43e4-99fc-4ee8a7437e1c None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:05.371099 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebb2773-6a83-43e4-99fc-4ee8a7437e1c None admin] PATH_INFO: `/v3/roles` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:05.371877 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebb2773-6a83-43e4-99fc-4ee8a7437e1c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:05.372396 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebb2773-6a83-43e4-99fc-4ee8a7437e1c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:05.374768 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebb2773-6a83-43e4-99fc-4ee8a7437e1c None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:05.385080 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 65/125] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:51:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:05.386287 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebb2773-6a83-43e4-99fc-4ee8a7437e1c None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:05.414295 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1800a428-eaba-4fd9-a5bc-359ab0c1792d None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:05.515485 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebb2773-6a83-43e4-99fc-4ee8a7437e1c None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:05.526616 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e34cda0-de10-4061-9c66-eb84e4ccab36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:05.527321 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e34cda0-de10-4061-9c66-eb84e4ccab36 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:05.527623 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e34cda0-de10-4061-9c66-eb84e4ccab36 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:05.527913 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e34cda0-de10-4061-9c66-eb84e4ccab36 None admin] PATH_INFO: `/v3/roles` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:05.528698 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e34cda0-de10-4061-9c66-eb84e4ccab36 None admin] RBAC: Adding query filter params (name=admin) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:05.529016 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e34cda0-de10-4061-9c66-eb84e4ccab36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:05.530471 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e34cda0-de10-4061-9c66-eb84e4ccab36 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:05.553168 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 61/126] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:51:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 382 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:05.554218 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e34cda0-de10-4061-9c66-eb84e4ccab36 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:05.572132 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acc462b-ddf7-4e89-924b-96617a786e77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:05.573522 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acc462b-ddf7-4e89-924b-96617a786e77 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:05.574025 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acc462b-ddf7-4e89-924b-96617a786e77 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:05.574316 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acc462b-ddf7-4e89-924b-96617a786e77 None admin] PATH_INFO: `/v3/roles/member` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:05.600549 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acc462b-ddf7-4e89-924b-96617a786e77 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:51:05.614469 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-7acc462b-ddf7-4e89-924b-96617a786e77 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:51:05.614469 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:51:05.614469 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:51:05.614469 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:51:05.614469 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:51:05.614469 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:51:05.614469 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:51:05.614469 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:51:05.614469 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:51:05.614469 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:51:05.614469 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:51:05.614469 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:51:05.614469 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:51:05.614469 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:51:05.614469 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:51:05.614469 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:51:05.615704 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:51:05.615704 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:51:05.615704 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:51:05.615704 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:51:05.615704 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:51:05.615704 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:51:05.615704 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:51:05.615704 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:51:05.615704 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:51:05.615704 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:51:05.615704 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:51:05.615704 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:51:05.615704 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:51:05.615704 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:51:05.615704 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:51:05.615704 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:51:05.617784 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:51:05.617784 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:51:05.617784 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:51:05.617784 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:51:05.617784 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:51:05.617784 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:51:05.617784 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:51:05.617784 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:51:05.617784 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:51:05.617784 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:51:05.617784 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:51:05.617784 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:51:05.617784 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 66/127] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:51:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:51:05.617784 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acc462b-ddf7-4e89-924b-96617a786e77 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:05.670226 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:05.762559 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acc462b-ddf7-4e89-924b-96617a786e77 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:05.770398 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e34cda0-de10-4061-9c66-eb84e4ccab36 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:05.798595 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:05.809281 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03cecd8-d66e-4478-9b8d-34d605c4f6c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:05.810526 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03cecd8-d66e-4478-9b8d-34d605c4f6c5 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:05.811043 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03cecd8-d66e-4478-9b8d-34d605c4f6c5 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:05.811551 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03cecd8-d66e-4478-9b8d-34d605c4f6c5 None admin] PATH_INFO: `/v3/users/2950d06ae84d46c081f68af26db145d4` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:05.815039 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03cecd8-d66e-4478-9b8d-34d605c4f6c5 None admin] RBAC: Authorizing `identity:get_user(user_id=2950d06ae84d46c081f68af26db145d4)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:05.818048 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03cecd8-d66e-4478-9b8d-34d605c4f6c5 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:05.819992 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 62/128] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:51:05 2022] GET /identity/v3/users/2950d06ae84d46c081f68af26db145d4 => generated 275 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:05.821043 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03cecd8-d66e-4478-9b8d-34d605c4f6c5 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:05.822698 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945d81ad-89c5-49d5-a751-888bfdf46e0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:05.823603 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945d81ad-89c5-49d5-a751-888bfdf46e0d None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:05.823896 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945d81ad-89c5-49d5-a751-888bfdf46e0d None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:05.824186 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945d81ad-89c5-49d5-a751-888bfdf46e0d None admin] PATH_INFO: `/v3/users/8c487f3640744d568515b03847007a04` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:05.829671 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945d81ad-89c5-49d5-a751-888bfdf46e0d None admin] RBAC: Authorizing `identity:get_user(user_id=8c487f3640744d568515b03847007a04)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:05.831262 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945d81ad-89c5-49d5-a751-888bfdf46e0d None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:05.833383 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 67/129] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:51:05 2022] GET /identity/v3/users/8c487f3640744d568515b03847007a04 => generated 247 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:05.838659 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945d81ad-89c5-49d5-a751-888bfdf46e0d None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:05.937437 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03cecd8-d66e-4478-9b8d-34d605c4f6c5 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:05.981368 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbbc202-1590-4741-bc46-bd451adbd4e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:05.982553 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbbc202-1590-4741-bc46-bd451adbd4e4 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:05.983099 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbbc202-1590-4741-bc46-bd451adbd4e4 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:05.983604 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbbc202-1590-4741-bc46-bd451adbd4e4 None admin] PATH_INFO: `/v3/roles` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:05.984728 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbbc202-1590-4741-bc46-bd451adbd4e4 None admin] RBAC: Adding query filter params (name=member) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:05.985284 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbbc202-1590-4741-bc46-bd451adbd4e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:05.987043 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbbc202-1590-4741-bc46-bd451adbd4e4 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:05.996918 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 63/130] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:51:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:05.998260 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbbc202-1590-4741-bc46-bd451adbd4e4 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:06.048179 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945d81ad-89c5-49d5-a751-888bfdf46e0d None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:06.051687 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:06.071835 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:06.113108 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4685afc1-24fa-43e6-8e72-b1e13884d6d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:06.114865 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4685afc1-24fa-43e6-8e72-b1e13884d6d5 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:06.115162 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4685afc1-24fa-43e6-8e72-b1e13884d6d5 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:06.115452 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4685afc1-24fa-43e6-8e72-b1e13884d6d5 None admin] PATH_INFO: `/v3/projects/aca8075ce986433eae657055d1366699` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:06.118324 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbbc202-1590-4741-bc46-bd451adbd4e4 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:06.120996 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4685afc1-24fa-43e6-8e72-b1e13884d6d5 None admin] RBAC: Authorizing `identity:get_project(project_id=aca8075ce986433eae657055d1366699)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:06.122672 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4685afc1-24fa-43e6-8e72-b1e13884d6d5 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:06.124668 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 68/131] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:51:05 2022] GET /identity/v3/projects/aca8075ce986433eae657055d1366699 => generated 298 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:06.125385 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4685afc1-24fa-43e6-8e72-b1e13884d6d5 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:06.158147 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580cfb99-8c53-4846-bb58-d1fe0f993873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:06.159562 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580cfb99-8c53-4846-bb58-d1fe0f993873 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:06.160749 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580cfb99-8c53-4846-bb58-d1fe0f993873 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:06.161259 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580cfb99-8c53-4846-bb58-d1fe0f993873 None admin] PATH_INFO: `/v3/projects/aca8075ce986433eae657055d1366699` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:06.163141 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580cfb99-8c53-4846-bb58-d1fe0f993873 None admin] RBAC: Authorizing `identity:get_project(project_id=aca8075ce986433eae657055d1366699)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:06.165018 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580cfb99-8c53-4846-bb58-d1fe0f993873 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:06.167154 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 64/132] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:51:05 2022] GET /identity/v3/projects/aca8075ce986433eae657055d1366699 => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:06.168188 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580cfb99-8c53-4846-bb58-d1fe0f993873 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:06.275057 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4685afc1-24fa-43e6-8e72-b1e13884d6d5 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:06.292457 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580cfb99-8c53-4846-bb58-d1fe0f993873 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:06.337607 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d53487-69fc-4be1-b666-0ae56a6b4d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:06.338454 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d53487-69fc-4be1-b666-0ae56a6b4d7b None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:06.338749 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d53487-69fc-4be1-b666-0ae56a6b4d7b None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:06.338880 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:06.339046 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d53487-69fc-4be1-b666-0ae56a6b4d7b None admin] PATH_INFO: `/v3/users/2950d06ae84d46c081f68af26db145d4` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:06.341300 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d53487-69fc-4be1-b666-0ae56a6b4d7b None admin] RBAC: Authorizing `identity:get_user(user_id=2950d06ae84d46c081f68af26db145d4)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:06.343098 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d53487-69fc-4be1-b666-0ae56a6b4d7b None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:06.344771 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 69/133] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:51:06 2022] GET /identity/v3/users/2950d06ae84d46c081f68af26db145d4 => generated 275 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:06.347990 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d53487-69fc-4be1-b666-0ae56a6b4d7b None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:06.361064 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80d840e-8c10-4977-a6a1-b445e63a7782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:06.362022 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80d840e-8c10-4977-a6a1-b445e63a7782 None admin] REQUEST_METHOD: `PUT` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:06.362320 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80d840e-8c10-4977-a6a1-b445e63a7782 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:06.362608 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80d840e-8c10-4977-a6a1-b445e63a7782 None admin] PATH_INFO: `/v3/projects/aca8075ce986433eae657055d1366699/users/2950d06ae84d46c081f68af26db145d4/roles/3b9c36a010504ba7bc63a8ccf0a9bca0` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:06.367183 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80d840e-8c10-4977-a6a1-b445e63a7782 None admin] RBAC: Authorizing `identity:create_grant(project_id=aca8075ce986433eae657055d1366699, user_id=2950d06ae84d46c081f68af26db145d4, role_id=3b9c36a010504ba7bc63a8ccf0a9bca0)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:06.368681 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80d840e-8c10-4977-a6a1-b445e63a7782 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:06.389134 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 65/134] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 14:51:06 2022] PUT /identity/v3/projects/aca8075ce986433eae657055d1366699/users/2950d06ae84d46c081f68af26db145d4/roles/3b9c36a010504ba7bc63a8ccf0a9bca0 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:51:06.390016 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80d840e-8c10-4977-a6a1-b445e63a7782 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:06.451347 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:06.559219 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d53487-69fc-4be1-b666-0ae56a6b4d7b None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:06.605679 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1a82a6-a461-47c4-aa25-23fb793defb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:06.607446 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1a82a6-a461-47c4-aa25-23fb793defb1 None admin] REQUEST_METHOD: `PUT` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:06.607958 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1a82a6-a461-47c4-aa25-23fb793defb1 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:06.608452 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1a82a6-a461-47c4-aa25-23fb793defb1 None admin] PATH_INFO: `/v3/projects/aca8075ce986433eae657055d1366699/users/8c487f3640744d568515b03847007a04/roles/028379e01f9245e9b8fc56e8b9e8a6ad` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:06.611708 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1a82a6-a461-47c4-aa25-23fb793defb1 None admin] RBAC: Authorizing `identity:create_grant(project_id=aca8075ce986433eae657055d1366699, user_id=8c487f3640744d568515b03847007a04, role_id=028379e01f9245e9b8fc56e8b9e8a6ad)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:06.613516 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1a82a6-a461-47c4-aa25-23fb793defb1 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:06.634741 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 70/135] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 14:51:06 2022] PUT /identity/v3/projects/aca8075ce986433eae657055d1366699/users/8c487f3640744d568515b03847007a04/roles/028379e01f9245e9b8fc56e8b9e8a6ad => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:51:06.637141 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80d840e-8c10-4977-a6a1-b445e63a7782 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:06.699666 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b051ea99-978d-4936-bd0b-34669f1d0265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:06.700575 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b051ea99-978d-4936-bd0b-34669f1d0265 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:06.700874 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b051ea99-978d-4936-bd0b-34669f1d0265 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:06.701165 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b051ea99-978d-4936-bd0b-34669f1d0265 None admin] PATH_INFO: `/v3/projects/ab05694ea13841e182a0be2091bf3654` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:06.702427 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b051ea99-978d-4936-bd0b-34669f1d0265 None admin] RBAC: Authorizing `identity:get_project(project_id=ab05694ea13841e182a0be2091bf3654)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:06.703916 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b051ea99-978d-4936-bd0b-34669f1d0265 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:06.705915 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 66/136] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:51:06 2022] GET /identity/v3/projects/ab05694ea13841e182a0be2091bf3654 => generated 312 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:06.712105 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1a82a6-a461-47c4-aa25-23fb793defb1 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:06.866458 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:06.870455 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1a82a6-a461-47c4-aa25-23fb793defb1 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:06.898432 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf1b826-d69b-4168-a264-369d5cdc06e6 None None] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:06.898736 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf1b826-d69b-4168-a264-369d5cdc06e6 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:06.899040 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf1b826-d69b-4168-a264-369d5cdc06e6 None None] PATH_INFO: `/` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:06.900453 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 67/137] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:51:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:51:06.905707 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f198f6-6bac-405b-9a64-cb7393f89f75 None None] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:06.906045 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f198f6-6bac-405b-9a64-cb7393f89f75 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:06.906346 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f198f6-6bac-405b-9a64-cb7393f89f75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:06.931556 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c358e8b6-bf8a-4027-a894-76ad749b558f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:06.932485 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c358e8b6-bf8a-4027-a894-76ad749b558f None admin] REQUEST_METHOD: `PUT` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:06.932781 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c358e8b6-bf8a-4027-a894-76ad749b558f None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:06.933063 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c358e8b6-bf8a-4027-a894-76ad749b558f None admin] PATH_INFO: `/v3/projects/ab05694ea13841e182a0be2091bf3654/users/2950d06ae84d46c081f68af26db145d4/roles/aed9aa9ef7f7448bb3b40c9ad082d545` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:06.934737 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.auth.core [None req-06f198f6-6bac-405b-9a64-cb7393f89f75 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:51:06.935984 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-06f198f6-6bac-405b-9a64-cb7393f89f75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:51:06.939335 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c358e8b6-bf8a-4027-a894-76ad749b558f None admin] RBAC: Authorizing `identity:create_grant(project_id=ab05694ea13841e182a0be2091bf3654, user_id=2950d06ae84d46c081f68af26db145d4, role_id=aed9aa9ef7f7448bb3b40c9ad082d545)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:06.940852 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c358e8b6-bf8a-4027-a894-76ad749b558f None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:06.966894 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 71/138] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 14:51:06 2022] PUT /identity/v3/projects/ab05694ea13841e182a0be2091bf3654/users/2950d06ae84d46c081f68af26db145d4/roles/aed9aa9ef7f7448bb3b40c9ad082d545 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:51:07.016440 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:07.038878 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 68/139] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:51:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:51:07.065963 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c358e8b6-bf8a-4027-a894-76ad749b558f None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:07.199609 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c358e8b6-bf8a-4027-a894-76ad749b558f None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:07.241752 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354a64a8-6dbd-4097-9742-cf858715cd67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:07.242659 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354a64a8-6dbd-4097-9742-cf858715cd67 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:07.242958 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354a64a8-6dbd-4097-9742-cf858715cd67 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:07.243241 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354a64a8-6dbd-4097-9742-cf858715cd67 None admin] PATH_INFO: `/v3/domains/default` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:07.244463 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354a64a8-6dbd-4097-9742-cf858715cd67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:07.246042 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354a64a8-6dbd-4097-9742-cf858715cd67 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:07.247803 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 72/140] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:51:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:07.253962 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f198f6-6bac-405b-9a64-cb7393f89f75 None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:07.388667 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f198f6-6bac-405b-9a64-cb7393f89f75 None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:07.430294 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba52d1f-7ef2-4370-a926-f3daa4ba6485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:07.431372 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba52d1f-7ef2-4370-a926-f3daa4ba6485 None admin] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:07.431945 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba52d1f-7ef2-4370-a926-f3daa4ba6485 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:07.432489 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba52d1f-7ef2-4370-a926-f3daa4ba6485 None admin] PATH_INFO: `/v3/users` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:07.433478 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba52d1f-7ef2-4370-a926-f3daa4ba6485 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:07.435178 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba52d1f-7ef2-4370-a926-f3daa4ba6485 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:07.466808 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 69/141] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 19 14:51:07 2022] POST /identity/v3/users => generated 283 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:51:07.573427 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa336a6-492e-4c54-ac68-71341021d283 None None] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:07.573783 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa336a6-492e-4c54-ac68-71341021d283 None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:07.574034 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa336a6-492e-4c54-ac68-71341021d283 None None] PATH_INFO: `/` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:07.575439 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 73/142] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:51:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:51:07.581496 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f8e9c8-2128-448f-96f1-8d37c1bac9e6 None None] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:07.582314 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f8e9c8-2128-448f-96f1-8d37c1bac9e6 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:07.582996 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f8e9c8-2128-448f-96f1-8d37c1bac9e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:07.606858 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.auth.core [None req-77f8e9c8-2128-448f-96f1-8d37c1bac9e6 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:51:07.608489 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-77f8e9c8-2128-448f-96f1-8d37c1bac9e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:51:07.671851 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:07.716403 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 70/143] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:51:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:51:07.755690 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa336a6-492e-4c54-ac68-71341021d283 None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:07.801181 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:07.892026 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa336a6-492e-4c54-ac68-71341021d283 None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:07.929604 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9bdab2-4955-4171-9ba2-6d897588aee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:07.930500 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9bdab2-4955-4171-9ba2-6d897588aee8 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:07.930798 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9bdab2-4955-4171-9ba2-6d897588aee8 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:07.931084 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9bdab2-4955-4171-9ba2-6d897588aee8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:07.942817 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9bdab2-4955-4171-9ba2-6d897588aee8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:51:07.954979 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-5d9bdab2-4955-4171-9ba2-6d897588aee8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:51:07.954979 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:51:07.954979 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:51:07.954979 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:51:07.954979 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:51:07.954979 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:51:07.954979 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:51:07.954979 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:51:07.954979 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:51:07.954979 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:51:07.954979 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:51:07.954979 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:51:07.954979 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:51:07.954979 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:51:07.954979 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:51:07.954979 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:51:07.956043 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:51:07.956043 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:51:07.956043 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:51:07.956043 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:51:07.956043 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:51:07.956043 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:51:07.956043 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:51:07.956043 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:51:07.956043 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:51:07.956043 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:51:07.956043 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:51:07.956043 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:51:07.956043 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:51:07.956043 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:51:07.956043 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:51:07.956043 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:51:07.957527 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:51:07.957527 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:51:07.957527 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:51:07.957527 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:51:07.957527 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:51:07.957527 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:51:07.957527 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:51:07.957527 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:51:07.957527 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:51:07.957527 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:51:07.957527 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:51:07.957527 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:51:07.957527 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 74/144] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:51:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:51:07.962784 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f8e9c8-2128-448f-96f1-8d37c1bac9e6 None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:08.088420 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f8e9c8-2128-448f-96f1-8d37c1bac9e6 None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:08.134970 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdd7160-7c76-43a1-99a4-5adbfbe43f05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:08.135685 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdd7160-7c76-43a1-99a4-5adbfbe43f05 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:08.135982 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdd7160-7c76-43a1-99a4-5adbfbe43f05 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:08.136272 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdd7160-7c76-43a1-99a4-5adbfbe43f05 None admin] PATH_INFO: `/v3/roles` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:08.137063 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdd7160-7c76-43a1-99a4-5adbfbe43f05 None admin] RBAC: Adding query filter params (name=member) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:08.137384 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdd7160-7c76-43a1-99a4-5adbfbe43f05 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:08.138886 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdd7160-7c76-43a1-99a4-5adbfbe43f05 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:08.149643 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 71/145] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:51:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:08.155772 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9bdab2-4955-4171-9ba2-6d897588aee8 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:08.255233 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:08.267717 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9bdab2-4955-4171-9ba2-6d897588aee8 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:08.309605 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45da038b-ad11-4edf-ba8e-9172f5873136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:08.310536 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45da038b-ad11-4edf-ba8e-9172f5873136 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:08.310825 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45da038b-ad11-4edf-ba8e-9172f5873136 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:08.311109 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45da038b-ad11-4edf-ba8e-9172f5873136 None admin] PATH_INFO: `/v3/users/2950d06ae84d46c081f68af26db145d4` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:08.312367 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45da038b-ad11-4edf-ba8e-9172f5873136 None admin] RBAC: Authorizing `identity:get_user(user_id=2950d06ae84d46c081f68af26db145d4)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:08.313853 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45da038b-ad11-4edf-ba8e-9172f5873136 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:08.315518 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 75/146] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:51:08 2022] GET /identity/v3/users/2950d06ae84d46c081f68af26db145d4 => generated 275 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:08.321265 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdd7160-7c76-43a1-99a4-5adbfbe43f05 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:08.327797 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf0d20b-b89b-49b2-ae0b-5beb0b0fac2a None None] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:08.328097 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf0d20b-b89b-49b2-ae0b-5beb0b0fac2a None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:08.328377 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf0d20b-b89b-49b2-ae0b-5beb0b0fac2a None None] PATH_INFO: `/` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:08.329763 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 76/147] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:51:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:51:08.336298 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5850c8e5-33d7-4b8a-96bd-f4170dbca0bc None None] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:08.336608 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5850c8e5-33d7-4b8a-96bd-f4170dbca0bc None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:08.336887 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5850c8e5-33d7-4b8a-96bd-f4170dbca0bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:08.368506 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.auth.core [None req-5850c8e5-33d7-4b8a-96bd-f4170dbca0bc None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:51:08.369662 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.fernet_utils [None req-5850c8e5-33d7-4b8a-96bd-f4170dbca0bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:51:08.388114 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:08.473263 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 77/148] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:51:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:51:08.502059 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdd7160-7c76-43a1-99a4-5adbfbe43f05 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:08.546427 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94e4bdc-ebec-4642-92e8-14a076c5cdc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:08.547096 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94e4bdc-ebec-4642-92e8-14a076c5cdc6 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:08.547391 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94e4bdc-ebec-4642-92e8-14a076c5cdc6 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:08.547726 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94e4bdc-ebec-4642-92e8-14a076c5cdc6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:08.590678 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5850c8e5-33d7-4b8a-96bd-f4170dbca0bc None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:08.626406 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94e4bdc-ebec-4642-92e8-14a076c5cdc6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:08.627907 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94e4bdc-ebec-4642-92e8-14a076c5cdc6 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:08.662251 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:08.760570 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:08.762562 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 72/149] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:51:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:51:08.768311 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94e4bdc-ebec-4642-92e8-14a076c5cdc6 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:08.789087 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5850c8e5-33d7-4b8a-96bd-f4170dbca0bc None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:08.863385 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485bbece-156d-4936-b810-26ceb5e8a693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:08.864337 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485bbece-156d-4936-b810-26ceb5e8a693 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:08.864834 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485bbece-156d-4936-b810-26ceb5e8a693 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:08.865126 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485bbece-156d-4936-b810-26ceb5e8a693 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:08.886012 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94e4bdc-ebec-4642-92e8-14a076c5cdc6 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:08.888302 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485bbece-156d-4936-b810-26ceb5e8a693 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:51:08.896909 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-485bbece-156d-4936-b810-26ceb5e8a693 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:51:08.896909 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:51:08.896909 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:51:08.896909 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:51:08.896909 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:51:08.896909 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:51:08.896909 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:51:08.896909 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:51:08.896909 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:51:08.896909 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:51:08.896909 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:51:08.896909 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:51:08.896909 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:51:08.896909 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:51:08.896909 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:51:08.896909 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:51:08.897986 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:51:08.897986 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:51:08.897986 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:51:08.897986 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:51:08.897986 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:51:08.897986 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:51:08.897986 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:51:08.897986 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:51:08.897986 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:51:08.897986 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:51:08.897986 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:51:08.897986 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:51:08.897986 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:51:08.897986 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:51:08.897986 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:51:08.897986 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:51:08.898944 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:51:08.898944 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:51:08.898944 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:51:08.898944 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:51:08.898944 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:51:08.898944 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:51:08.898944 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:51:08.898944 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:51:08.898944 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:51:08.898944 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:51:08.898944 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:51:08.898944 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:51:08.902470 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 78/150] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:51:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:51:08.908389 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485bbece-156d-4936-b810-26ceb5e8a693 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:08.934840 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:08.954450 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1985dabc-0144-4c89-8a98-023ae154489d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:08.955405 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1985dabc-0144-4c89-8a98-023ae154489d None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:08.955706 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1985dabc-0144-4c89-8a98-023ae154489d None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:08.955990 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1985dabc-0144-4c89-8a98-023ae154489d None admin] PATH_INFO: `/v3/projects/aca8075ce986433eae657055d1366699` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:08.957388 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1985dabc-0144-4c89-8a98-023ae154489d None admin] RBAC: Authorizing `identity:get_project(project_id=aca8075ce986433eae657055d1366699)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:08.958882 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1985dabc-0144-4c89-8a98-023ae154489d None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:08.960720 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 73/151] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:51:08 2022] GET /identity/v3/projects/aca8075ce986433eae657055d1366699 => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:51:08.966713 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1985dabc-0144-4c89-8a98-023ae154489d None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:09.132429 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1985dabc-0144-4c89-8a98-023ae154489d None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:09.134081 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485bbece-156d-4936-b810-26ceb5e8a693 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:09.191240 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71924dbd-85f2-41f8-abb3-99a682a7c1d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:09.192417 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71924dbd-85f2-41f8-abb3-99a682a7c1d7 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:09.192714 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71924dbd-85f2-41f8-abb3-99a682a7c1d7 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:09.193044 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71924dbd-85f2-41f8-abb3-99a682a7c1d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:09.193924 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71924dbd-85f2-41f8-abb3-99a682a7c1d7 None admin] RBAC: Adding query filter params (role.id=aed9aa9ef7f7448bb3b40c9ad082d545, scope.project.id=aca8075ce986433eae657055d1366699, user.id=2950d06ae84d46c081f68af26db145d4) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:09.194248 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71924dbd-85f2-41f8-abb3-99a682a7c1d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:09.195702 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71924dbd-85f2-41f8-abb3-99a682a7c1d7 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:09.199461 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102b2d2c-22cd-4060-b78e-7f78738748be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:09.200191 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102b2d2c-22cd-4060-b78e-7f78738748be None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:09.200562 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102b2d2c-22cd-4060-b78e-7f78738748be None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:09.200853 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102b2d2c-22cd-4060-b78e-7f78738748be None admin] PATH_INFO: `/v3/roles` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:09.201630 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102b2d2c-22cd-4060-b78e-7f78738748be None admin] RBAC: Adding query filter params (name=reader) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:09.201973 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102b2d2c-22cd-4060-b78e-7f78738748be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:09.203368 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102b2d2c-22cd-4060-b78e-7f78738748be None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:09.204399 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 74/152] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:51:08 2022] GET /identity/v3/role_assignments?user.id=2950d06ae84d46c081f68af26db145d4&scope.project.id=aca8075ce986433eae657055d1366699&role.id=aed9aa9ef7f7448bb3b40c9ad082d545 => generated 608 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:51:09.221926 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 79/153] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:51:08 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:09.227604 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71924dbd-85f2-41f8-abb3-99a682a7c1d7 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:09.316868 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:09.363641 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71924dbd-85f2-41f8-abb3-99a682a7c1d7 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:09.416204 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51de32d-2ede-43cc-9738-28528c61ed53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:09.417110 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51de32d-2ede-43cc-9738-28528c61ed53 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:09.417403 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51de32d-2ede-43cc-9738-28528c61ed53 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:09.417687 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51de32d-2ede-43cc-9738-28528c61ed53 None admin] PATH_INFO: `/v3/users/bce930239cd6428ca707fea57a0d4abb` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:09.418967 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51de32d-2ede-43cc-9738-28528c61ed53 None admin] RBAC: Authorizing `identity:get_user(user_id=bce930239cd6428ca707fea57a0d4abb)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:09.420437 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51de32d-2ede-43cc-9738-28528c61ed53 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:09.422165 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 75/154] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:51:09 2022] GET /identity/v3/users/bce930239cd6428ca707fea57a0d4abb => generated 289 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:09.428305 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102b2d2c-22cd-4060-b78e-7f78738748be None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:09.449598 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:09.558352 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102b2d2c-22cd-4060-b78e-7f78738748be None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:09.603947 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6837fa4f-920a-476b-a946-d5d011736964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:09.604833 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6837fa4f-920a-476b-a946-d5d011736964 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:09.605124 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6837fa4f-920a-476b-a946-d5d011736964 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:09.605406 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6837fa4f-920a-476b-a946-d5d011736964 None admin] PATH_INFO: `/v3/projects/aca8075ce986433eae657055d1366699` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:09.606674 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6837fa4f-920a-476b-a946-d5d011736964 None admin] RBAC: Authorizing `identity:get_project(project_id=aca8075ce986433eae657055d1366699)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:09.608180 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6837fa4f-920a-476b-a946-d5d011736964 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:09.609951 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 80/155] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:51:09 2022] GET /identity/v3/projects/aca8075ce986433eae657055d1366699 => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:09.616174 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51de32d-2ede-43cc-9738-28528c61ed53 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:09.772201 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51de32d-2ede-43cc-9738-28528c61ed53 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:09.811997 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:09.817811 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f739126-6fe1-43b3-a230-1f67fe66b0ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:09.819160 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f739126-6fe1-43b3-a230-1f67fe66b0ac None admin] REQUEST_METHOD: `PUT` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:09.819649 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f739126-6fe1-43b3-a230-1f67fe66b0ac None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:09.820137 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f739126-6fe1-43b3-a230-1f67fe66b0ac None admin] PATH_INFO: `/v3/projects/aca8075ce986433eae657055d1366699/users/bce930239cd6428ca707fea57a0d4abb/roles/e56389e17ad24e20bf3bba1ca7684062` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:09.822516 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f739126-6fe1-43b3-a230-1f67fe66b0ac None admin] RBAC: Authorizing `identity:create_grant(project_id=aca8075ce986433eae657055d1366699, user_id=bce930239cd6428ca707fea57a0d4abb, role_id=e56389e17ad24e20bf3bba1ca7684062)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:09.823991 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f739126-6fe1-43b3-a230-1f67fe66b0ac None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:09.847305 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 76/156] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 14:51:09 2022] PUT /identity/v3/projects/aca8075ce986433eae657055d1366699/users/bce930239cd6428ca707fea57a0d4abb/roles/e56389e17ad24e20bf3bba1ca7684062 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:51:10.209626 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b308b367-86cf-4259-b555-96529dab4601 None None] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:10.210225 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b308b367-86cf-4259-b555-96529dab4601 None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:10.210225 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b308b367-86cf-4259-b555-96529dab4601 None None] PATH_INFO: `/` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:10.212041 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 81/157] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:51:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:51:10.217622 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d1d0c2-8c2b-4e7a-b5c7-825c21ad2e56 None None] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:10.217964 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d1d0c2-8c2b-4e7a-b5c7-825c21ad2e56 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:10.218240 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d1d0c2-8c2b-4e7a-b5c7-825c21ad2e56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:10.239030 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.auth.core [None req-34d1d0c2-8c2b-4e7a-b5c7-825c21ad2e56 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:51:10.240294 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-34d1d0c2-8c2b-4e7a-b5c7-825c21ad2e56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:51:10.323432 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 77/158] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:51:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:51:10.372027 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b308b367-86cf-4259-b555-96529dab4601 None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:10.417479 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b25337-3cd5-41c7-a223-b0fdc72abf75 None None] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:10.417807 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b25337-3cd5-41c7-a223-b0fdc72abf75 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:10.418110 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b25337-3cd5-41c7-a223-b0fdc72abf75 None None] PATH_INFO: `/` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:10.419693 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 78/159] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:51:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:51:10.424813 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e9e875-59bb-4527-bd5c-4e2903604575 None None] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:10.425112 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e9e875-59bb-4527-bd5c-4e2903604575 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:10.425407 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e9e875-59bb-4527-bd5c-4e2903604575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:10.459041 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.auth.core [None req-e2e9e875-59bb-4527-bd5c-4e2903604575 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:51:10.460342 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-e2e9e875-59bb-4527-bd5c-4e2903604575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:51:10.467680 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:10.510218 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b308b367-86cf-4259-b555-96529dab4601 None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:10.577520 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce27f66-aef6-44f8-8fb9-60cf7365282a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:10.578632 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce27f66-aef6-44f8-8fb9-60cf7365282a None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:10.578921 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce27f66-aef6-44f8-8fb9-60cf7365282a None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:10.579210 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce27f66-aef6-44f8-8fb9-60cf7365282a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:10.609114 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 79/160] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:51:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:51:10.613305 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce27f66-aef6-44f8-8fb9-60cf7365282a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:51:10.621509 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-7ce27f66-aef6-44f8-8fb9-60cf7365282a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:51:10.621509 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:51:10.621509 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:51:10.621509 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:51:10.621509 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:51:10.621509 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:51:10.621509 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:51:10.621509 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:51:10.621509 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:51:10.621509 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:51:10.621509 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:51:10.621509 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:51:10.621509 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:51:10.621509 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:51:10.621509 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:51:10.621509 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:51:10.622643 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:51:10.622643 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:51:10.622643 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:51:10.622643 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:51:10.622643 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:51:10.622643 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:51:10.622643 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:51:10.622643 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:51:10.622643 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:51:10.622643 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:51:10.622643 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:51:10.622643 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:51:10.622643 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:51:10.622643 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:51:10.622643 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:51:10.622643 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:51:10.623552 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:51:10.623552 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:51:10.623552 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:51:10.623552 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:51:10.623552 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:51:10.623552 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:51:10.623552 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:51:10.623552 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:51:10.623552 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:51:10.623552 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:51:10.623552 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:51:10.623552 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:51:10.627141 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 82/161] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 19 14:51:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:51:10.633451 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e9e875-59bb-4527-bd5c-4e2903604575 None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:10.633748 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce27f66-aef6-44f8-8fb9-60cf7365282a None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:10.745765 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:10.759298 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce27f66-aef6-44f8-8fb9-60cf7365282a None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:10.803481 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb415e0-4567-4a3e-9013-023713b3c953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:10.804664 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb415e0-4567-4a3e-9013-023713b3c953 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:10.804963 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb415e0-4567-4a3e-9013-023713b3c953 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:10.805252 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb415e0-4567-4a3e-9013-023713b3c953 None admin] PATH_INFO: `/v3/roles` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:10.806049 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb415e0-4567-4a3e-9013-023713b3c953 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:10.806371 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb415e0-4567-4a3e-9013-023713b3c953 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:10.808094 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb415e0-4567-4a3e-9013-023713b3c953 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:10.811144 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:10.820140 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 83/162] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:51:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:51:10.826375 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb415e0-4567-4a3e-9013-023713b3c953 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:10.842371 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e9e875-59bb-4527-bd5c-4e2903604575 None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:10.915758 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ac2d99-f89a-407c-a65c-60ea4cb5cb01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:10.916583 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ac2d99-f89a-407c-a65c-60ea4cb5cb01 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:10.918059 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ac2d99-f89a-407c-a65c-60ea4cb5cb01 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:10.918351 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ac2d99-f89a-407c-a65c-60ea4cb5cb01 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:10.939159 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ac2d99-f89a-407c-a65c-60ea4cb5cb01 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:51:10.948773 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-53ac2d99-f89a-407c-a65c-60ea4cb5cb01 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:51:10.948773 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:51:10.948773 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:51:10.948773 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:51:10.948773 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:51:10.948773 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:51:10.948773 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:51:10.948773 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:51:10.948773 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:51:10.948773 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:51:10.948773 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:51:10.948773 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:51:10.948773 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:51:10.948773 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:51:10.948773 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:51:10.948773 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:51:10.949811 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:51:10.949811 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:51:10.949811 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:51:10.949811 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:51:10.949811 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:51:10.949811 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:51:10.949811 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:51:10.949811 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:51:10.949811 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:51:10.949811 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:51:10.949811 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:51:10.949811 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:51:10.949811 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:51:10.949811 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:51:10.949811 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:51:10.949811 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:51:10.950819 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:51:10.950819 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:51:10.950819 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:51:10.950819 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:51:10.950819 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:51:10.950819 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:51:10.950819 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:51:10.950819 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:51:10.950819 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:51:10.950819 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:51:10.950819 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:51:10.950819 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:51:10.952529 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 80/163] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:51:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:51:10.959474 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ac2d99-f89a-407c-a65c-60ea4cb5cb01 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:11.034982 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb415e0-4567-4a3e-9013-023713b3c953 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:11.078639 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b436f3a-be2a-448f-8132-29fae196248c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:11.079739 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b436f3a-be2a-448f-8132-29fae196248c None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:11.079739 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b436f3a-be2a-448f-8132-29fae196248c None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:11.080286 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b436f3a-be2a-448f-8132-29fae196248c None admin] PATH_INFO: `/v3/users/2950d06ae84d46c081f68af26db145d4` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:11.082479 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b436f3a-be2a-448f-8132-29fae196248c None admin] RBAC: Authorizing `identity:get_user(user_id=2950d06ae84d46c081f68af26db145d4)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:11.084582 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b436f3a-be2a-448f-8132-29fae196248c None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:11.086582 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 84/164] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:51:10 2022] GET /identity/v3/users/2950d06ae84d46c081f68af26db145d4 => generated 275 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:11.095725 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b436f3a-be2a-448f-8132-29fae196248c None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:11.177011 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ac2d99-f89a-407c-a65c-60ea4cb5cb01 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:11.186989 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:11.212235 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:11.254468 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b436f3a-be2a-448f-8132-29fae196248c None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:11.259721 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58641f63-63ac-49da-9687-c7aef10763ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:11.260470 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58641f63-63ac-49da-9687-c7aef10763ce None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:11.260771 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58641f63-63ac-49da-9687-c7aef10763ce None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:11.261057 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58641f63-63ac-49da-9687-c7aef10763ce None admin] PATH_INFO: `/v3/roles` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:11.261828 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58641f63-63ac-49da-9687-c7aef10763ce None admin] RBAC: Adding query filter params (name=admin) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:11.262166 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58641f63-63ac-49da-9687-c7aef10763ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:11.263568 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58641f63-63ac-49da-9687-c7aef10763ce None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:11.279377 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 81/165] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:51:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:51:11.284555 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58641f63-63ac-49da-9687-c7aef10763ce None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:11.324665 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f6d047-b3f9-4e65-b67b-530f1ac5240c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:11.325769 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f6d047-b3f9-4e65-b67b-530f1ac5240c None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:11.326294 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f6d047-b3f9-4e65-b67b-530f1ac5240c None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:11.327240 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f6d047-b3f9-4e65-b67b-530f1ac5240c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:11.462651 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f6d047-b3f9-4e65-b67b-530f1ac5240c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:11.464649 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f6d047-b3f9-4e65-b67b-530f1ac5240c None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:11.530087 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58641f63-63ac-49da-9687-c7aef10763ce None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:11.583403 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:11.618620 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fa0d66-5c46-4ba5-8d19-dc865fb13538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:11.619533 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fa0d66-5c46-4ba5-8d19-dc865fb13538 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:11.619827 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fa0d66-5c46-4ba5-8d19-dc865fb13538 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:11.620112 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fa0d66-5c46-4ba5-8d19-dc865fb13538 None admin] PATH_INFO: `/v3/users/8c487f3640744d568515b03847007a04` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:11.621411 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fa0d66-5c46-4ba5-8d19-dc865fb13538 None admin] RBAC: Authorizing `identity:get_user(user_id=8c487f3640744d568515b03847007a04)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:11.622916 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fa0d66-5c46-4ba5-8d19-dc865fb13538 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:11.624627 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 82/166] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:51:11 2022] GET /identity/v3/users/8c487f3640744d568515b03847007a04 => generated 247 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:51:11.625821 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a3a1de-bd75-4641-bc8c-e45df569bad5 None None] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:11.626146 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a3a1de-bd75-4641-bc8c-e45df569bad5 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:11.626433 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a3a1de-bd75-4641-bc8c-e45df569bad5 None None] PATH_INFO: `/` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:11.627667 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 83/167] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:51:11 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:51:11.641177 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdb43a5-9ade-42eb-a619-01f4195f9789 None None] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:11.641482 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdb43a5-9ade-42eb-a619-01f4195f9789 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:11.641777 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdb43a5-9ade-42eb-a619-01f4195f9789 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:11.653874 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 85/168] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:51:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 559 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 14:51:11.654852 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f6d047-b3f9-4e65-b67b-530f1ac5240c None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:11.682301 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.auth.core [None req-dbdb43a5-9ade-42eb-a619-01f4195f9789 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:51:11.683526 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-dbdb43a5-9ade-42eb-a619-01f4195f9789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:51:11.756754 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:11.812482 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f6d047-b3f9-4e65-b67b-530f1ac5240c None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:11.832508 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 84/169] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:51:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:51:11.833498 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdb43a5-9ade-42eb-a619-01f4195f9789 None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:11.870999 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f936e9-9be0-4488-83fd-43b3c9ac0335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:11.871375 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f936e9-9be0-4488-83fd-43b3c9ac0335 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:11.871659 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f936e9-9be0-4488-83fd-43b3c9ac0335 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:11.871944 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f936e9-9be0-4488-83fd-43b3c9ac0335 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:11.934458 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:12.002272 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f936e9-9be0-4488-83fd-43b3c9ac0335 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:12.004238 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f936e9-9be0-4488-83fd-43b3c9ac0335 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:12.045338 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdb43a5-9ade-42eb-a619-01f4195f9789 None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:12.117219 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfebcb9-01d4-49a2-8256-fc5287cfba28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:12.121865 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfebcb9-01d4-49a2-8256-fc5287cfba28 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:12.121865 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfebcb9-01d4-49a2-8256-fc5287cfba28 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:12.121865 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfebcb9-01d4-49a2-8256-fc5287cfba28 None admin] PATH_INFO: `/v3/projects/aca8075ce986433eae657055d1366699` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:12.123001 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfebcb9-01d4-49a2-8256-fc5287cfba28 None admin] RBAC: Authorizing `identity:get_project(project_id=aca8075ce986433eae657055d1366699)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:12.124608 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfebcb9-01d4-49a2-8256-fc5287cfba28 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:12.127870 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 85/170] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:51:11 2022] GET /identity/v3/projects/aca8075ce986433eae657055d1366699 => generated 298 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:12.127870 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfebcb9-01d4-49a2-8256-fc5287cfba28 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:12.186305 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 86/171] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:51:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 532 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:51:12.187200 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f936e9-9be0-4488-83fd-43b3c9ac0335 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:12.207230 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:12.284251 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:12.303470 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfebcb9-01d4-49a2-8256-fc5287cfba28 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:12.343721 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b694ddf-b118-4e56-8353-2f5f4f9df106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:12.344542 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b694ddf-b118-4e56-8353-2f5f4f9df106 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:12.344834 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b694ddf-b118-4e56-8353-2f5f4f9df106 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:12.345117 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b694ddf-b118-4e56-8353-2f5f4f9df106 None admin] PATH_INFO: `/v3/roles/member` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:12.358153 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f936e9-9be0-4488-83fd-43b3c9ac0335 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:12.374135 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b694ddf-b118-4e56-8353-2f5f4f9df106 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:51:12.382385 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-5b694ddf-b118-4e56-8353-2f5f4f9df106 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:51:12.382385 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:51:12.382385 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:51:12.382385 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:51:12.382385 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:51:12.382385 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:51:12.382385 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:51:12.382385 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:51:12.382385 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:51:12.382385 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:51:12.382385 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:51:12.382385 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:51:12.382385 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:51:12.382385 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:51:12.382385 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:51:12.382385 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:51:12.383393 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:51:12.383393 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:51:12.383393 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:51:12.383393 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:51:12.383393 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:51:12.383393 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:51:12.383393 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:51:12.383393 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:51:12.383393 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:51:12.383393 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:51:12.383393 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:51:12.383393 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:51:12.383393 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:51:12.383393 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:51:12.383393 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:51:12.383393 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:51:12.384326 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:51:12.384326 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:51:12.384326 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:51:12.384326 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:51:12.384326 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:51:12.384326 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:51:12.384326 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:51:12.384326 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:51:12.384326 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:51:12.384326 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:51:12.384326 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:51:12.384326 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:51:12.384326 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 86/172] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:51:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:51:12.385087 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b694ddf-b118-4e56-8353-2f5f4f9df106 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:12.443448 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5db8905-f5f9-4110-a9ea-f5cace68ea36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:12.444143 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5db8905-f5f9-4110-a9ea-f5cace68ea36 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:12.444433 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5db8905-f5f9-4110-a9ea-f5cace68ea36 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:12.444713 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5db8905-f5f9-4110-a9ea-f5cace68ea36 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:12.445586 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5db8905-f5f9-4110-a9ea-f5cace68ea36 None admin] RBAC: Adding query filter params (role.id=3b9c36a010504ba7bc63a8ccf0a9bca0, scope.project.id=aca8075ce986433eae657055d1366699, user.id=2950d06ae84d46c081f68af26db145d4) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:12.445934 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5db8905-f5f9-4110-a9ea-f5cace68ea36 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:12.447714 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5db8905-f5f9-4110-a9ea-f5cace68ea36 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:12.463680 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 87/173] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:51:12 2022] GET /identity/v3/role_assignments?user.id=2950d06ae84d46c081f68af26db145d4&scope.project.id=aca8075ce986433eae657055d1366699&role.id=3b9c36a010504ba7bc63a8ccf0a9bca0 => generated 608 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:12.464564 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5db8905-f5f9-4110-a9ea-f5cace68ea36 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:12.622307 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b694ddf-b118-4e56-8353-2f5f4f9df106 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:12.626293 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:12.658834 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db0085f-7d4e-49e4-91e2-596d2f53552f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:12.660308 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db0085f-7d4e-49e4-91e2-596d2f53552f None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:12.660868 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db0085f-7d4e-49e4-91e2-596d2f53552f None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:12.661255 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db0085f-7d4e-49e4-91e2-596d2f53552f None admin] PATH_INFO: `/v3/projects/aca8075ce986433eae657055d1366699` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:12.662871 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db0085f-7d4e-49e4-91e2-596d2f53552f None admin] RBAC: Authorizing `identity:get_project(project_id=aca8075ce986433eae657055d1366699)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:12.664465 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db0085f-7d4e-49e4-91e2-596d2f53552f None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:12.666857 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 87/174] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:51:12 2022] GET /identity/v3/projects/aca8075ce986433eae657055d1366699 => generated 298 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:12.675186 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db0085f-7d4e-49e4-91e2-596d2f53552f None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:12.677403 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5db8905-f5f9-4110-a9ea-f5cace68ea36 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:12.766437 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:12.794522 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c81088-a025-41ba-8973-1f9e94adee10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:12.795517 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c81088-a025-41ba-8973-1f9e94adee10 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:12.795960 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c81088-a025-41ba-8973-1f9e94adee10 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:12.796403 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c81088-a025-41ba-8973-1f9e94adee10 None admin] PATH_INFO: `/v3/roles` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:12.797340 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c81088-a025-41ba-8973-1f9e94adee10 None admin] RBAC: Adding query filter params (name=member) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:12.797822 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c81088-a025-41ba-8973-1f9e94adee10 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:12.799518 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c81088-a025-41ba-8973-1f9e94adee10 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:12.824626 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 88/175] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:51:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:12.829505 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f476475-9def-4d76-bc3e-c6d0f1234d9b None None] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:12.830153 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f476475-9def-4d76-bc3e-c6d0f1234d9b None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:12.830671 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f476475-9def-4d76-bc3e-c6d0f1234d9b None None] PATH_INFO: `/` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:12.833429 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 89/176] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:51:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:51:12.834631 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f476475-9def-4d76-bc3e-c6d0f1234d9b None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:12.856272 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db0085f-7d4e-49e4-91e2-596d2f53552f None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:12.902933 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb095404-660c-4f97-b5bc-599aafef22ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:12.903643 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb095404-660c-4f97-b5bc-599aafef22ed None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:12.903935 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb095404-660c-4f97-b5bc-599aafef22ed None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:12.904214 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb095404-660c-4f97-b5bc-599aafef22ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:12.905198 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb095404-660c-4f97-b5bc-599aafef22ed None admin] RBAC: Adding query filter params (role.id=028379e01f9245e9b8fc56e8b9e8a6ad, scope.project.id=aca8075ce986433eae657055d1366699, user.id=8c487f3640744d568515b03847007a04) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:12.905532 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb095404-660c-4f97-b5bc-599aafef22ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:12.907212 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb095404-660c-4f97-b5bc-599aafef22ed None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:12.915817 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 88/177] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:51:12 2022] GET /identity/v3/role_assignments?user.id=8c487f3640744d568515b03847007a04&scope.project.id=aca8075ce986433eae657055d1366699&role.id=028379e01f9245e9b8fc56e8b9e8a6ad => generated 608 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:12.917196 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051c093c-dd42-4387-b33a-a0a844118064 None None] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:12.917489 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051c093c-dd42-4387-b33a-a0a844118064 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:12.917773 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051c093c-dd42-4387-b33a-a0a844118064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:12.937859 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.auth.core [None req-051c093c-dd42-4387-b33a-a0a844118064 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:51:12.939374 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-051c093c-dd42-4387-b33a-a0a844118064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:51:12.973248 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:13.024580 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f476475-9def-4d76-bc3e-c6d0f1234d9b None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:13.025476 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 89/178] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:51:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:51:13.054224 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051c093c-dd42-4387-b33a-a0a844118064 None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:13.074125 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5870de5e-4fe8-46c1-9520-e86ef284666b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:13.075037 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5870de5e-4fe8-46c1-9520-e86ef284666b None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:13.075328 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5870de5e-4fe8-46c1-9520-e86ef284666b None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:13.075610 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5870de5e-4fe8-46c1-9520-e86ef284666b None admin] PATH_INFO: `/v3/users/2950d06ae84d46c081f68af26db145d4` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:13.076951 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5870de5e-4fe8-46c1-9520-e86ef284666b None admin] RBAC: Authorizing `identity:get_user(user_id=2950d06ae84d46c081f68af26db145d4)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:13.078616 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5870de5e-4fe8-46c1-9520-e86ef284666b None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:13.080660 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 90/179] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:51:12 2022] GET /identity/v3/users/2950d06ae84d46c081f68af26db145d4 => generated 275 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:13.086977 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5870de5e-4fe8-46c1-9520-e86ef284666b None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:13.179303 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:13.193993 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051c093c-dd42-4387-b33a-a0a844118064 None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:13.246702 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5870de5e-4fe8-46c1-9520-e86ef284666b None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:13.270539 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9901704b-e88c-40ae-bf85-70aa2e9e0127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:13.271982 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9901704b-e88c-40ae-bf85-70aa2e9e0127 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:13.272498 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9901704b-e88c-40ae-bf85-70aa2e9e0127 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:13.272989 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9901704b-e88c-40ae-bf85-70aa2e9e0127 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:13.295013 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c359e75-c885-43f4-8b9b-4163f29ba19f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:13.296345 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c359e75-c885-43f4-8b9b-4163f29ba19f None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:13.297099 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c359e75-c885-43f4-8b9b-4163f29ba19f None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:13.297529 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9901704b-e88c-40ae-bf85-70aa2e9e0127 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:51:13.297721 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c359e75-c885-43f4-8b9b-4163f29ba19f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:13.306739 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-9901704b-e88c-40ae-bf85-70aa2e9e0127 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:51:13.306739 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:51:13.306739 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:51:13.306739 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:51:13.306739 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:51:13.306739 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:51:13.306739 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:51:13.306739 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:51:13.306739 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:51:13.306739 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:51:13.306739 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:51:13.306739 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:51:13.306739 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:51:13.306739 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:51:13.306739 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:51:13.306739 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:51:13.308039 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:51:13.308039 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:51:13.308039 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:51:13.308039 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:51:13.308039 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:51:13.308039 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:51:13.308039 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:51:13.308039 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:51:13.308039 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:51:13.308039 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:51:13.308039 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:51:13.308039 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:51:13.308039 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:51:13.308039 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:51:13.308039 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:51:13.308039 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:51:13.309040 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:51:13.309040 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:51:13.309040 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:51:13.309040 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:51:13.309040 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:51:13.309040 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:51:13.309040 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:51:13.309040 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:51:13.309040 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:51:13.309040 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:51:13.309040 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:51:13.309040 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:51:13.309040 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 90/180] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:51:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:51:13.309806 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0dc0a2-0fd3-4743-b178-31f4367dd864 None None] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:13.309806 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0dc0a2-0fd3-4743-b178-31f4367dd864 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:13.309976 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0dc0a2-0fd3-4743-b178-31f4367dd864 None None] PATH_INFO: `/` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:13.314262 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 91/181] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:51:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:51:13.314262 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3f20b7-2d2a-4f3e-a97e-4ff9bdf94211 None None] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:13.314262 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3f20b7-2d2a-4f3e-a97e-4ff9bdf94211 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:13.314262 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3f20b7-2d2a-4f3e-a97e-4ff9bdf94211 None None] PATH_INFO: `/` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:13.314868 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 92/182] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:51:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:51:13.321041 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cf1b0f-9f4e-4b3a-8802-eae7cc29fea6 None None] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:13.321376 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cf1b0f-9f4e-4b3a-8802-eae7cc29fea6 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:13.321618 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cf1b0f-9f4e-4b3a-8802-eae7cc29fea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:13.340293 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.auth.core [None req-51cf1b0f-9f4e-4b3a-8802-eae7cc29fea6 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:51:13.341516 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-51cf1b0f-9f4e-4b3a-8802-eae7cc29fea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:51:13.364953 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:13.409425 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c359e75-c885-43f4-8b9b-4163f29ba19f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:13.411603 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c359e75-c885-43f4-8b9b-4163f29ba19f None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:13.439178 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 93/183] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:51:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:51:13.439774 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cf1b0f-9f4e-4b3a-8802-eae7cc29fea6 None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:13.489075 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:13.538506 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 91/184] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:51:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 14:51:13.544019 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b61f24-8cea-4ce1-bb48-5350e0bdf0b9 None None] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:13.544312 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b61f24-8cea-4ce1-bb48-5350e0bdf0b9 None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:13.544595 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b61f24-8cea-4ce1-bb48-5350e0bdf0b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:13.558516 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cf1b0f-9f4e-4b3a-8802-eae7cc29fea6 None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:13.562711 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.auth.core [None req-c8b61f24-8cea-4ce1-bb48-5350e0bdf0b9 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:51:13.563974 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.fernet_utils [None req-c8b61f24-8cea-4ce1-bb48-5350e0bdf0b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:51:13.599425 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96856ec-02ea-4ec7-8528-e5f2f736f776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:13.600111 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96856ec-02ea-4ec7-8528-e5f2f736f776 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:13.600406 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96856ec-02ea-4ec7-8528-e5f2f736f776 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:13.600692 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96856ec-02ea-4ec7-8528-e5f2f736f776 None admin] PATH_INFO: `/v3/roles` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:13.601460 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96856ec-02ea-4ec7-8528-e5f2f736f776 None admin] RBAC: Adding query filter params (name=admin) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:13.601777 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96856ec-02ea-4ec7-8528-e5f2f736f776 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:13.603207 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96856ec-02ea-4ec7-8528-e5f2f736f776 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:13.617109 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 94/185] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:51:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:13.617109 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96856ec-02ea-4ec7-8528-e5f2f736f776 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:13.633320 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:13.645370 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 92/186] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:51:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:51:13.646269 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b61f24-8cea-4ce1-bb48-5350e0bdf0b9 None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:13.740829 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96856ec-02ea-4ec7-8528-e5f2f736f776 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:13.754240 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:13.764892 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b61f24-8cea-4ce1-bb48-5350e0bdf0b9 None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:13.778574 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9acecf5-37fb-4c81-836c-2c7f4af1f38b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:13.779382 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9acecf5-37fb-4c81-836c-2c7f4af1f38b None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:13.779672 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9acecf5-37fb-4c81-836c-2c7f4af1f38b None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:13.779951 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9acecf5-37fb-4c81-836c-2c7f4af1f38b None admin] PATH_INFO: `/v3/domains/default` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:13.781122 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9acecf5-37fb-4c81-836c-2c7f4af1f38b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:13.782596 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9acecf5-37fb-4c81-836c-2c7f4af1f38b None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:13.784220 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 95/187] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:51:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:13.785556 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9acecf5-37fb-4c81-836c-2c7f4af1f38b None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:13.804030 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1a53ae-7059-4750-90bc-ac3e58729b46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:13.804910 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1a53ae-7059-4750-90bc-ac3e58729b46 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:13.805197 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1a53ae-7059-4750-90bc-ac3e58729b46 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:13.805477 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1a53ae-7059-4750-90bc-ac3e58729b46 None admin] PATH_INFO: `/v3/projects/ab05694ea13841e182a0be2091bf3654` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:13.806708 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1a53ae-7059-4750-90bc-ac3e58729b46 None admin] RBAC: Authorizing `identity:get_project(project_id=ab05694ea13841e182a0be2091bf3654)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:13.808169 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1a53ae-7059-4750-90bc-ac3e58729b46 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:13.809902 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 93/188] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:51:13 2022] GET /identity/v3/projects/ab05694ea13841e182a0be2091bf3654 => generated 312 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:13.814586 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1a53ae-7059-4750-90bc-ac3e58729b46 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:13.872468 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:13.892023 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9acecf5-37fb-4c81-836c-2c7f4af1f38b None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:13.913340 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1a53ae-7059-4750-90bc-ac3e58729b46 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:13.928184 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62137fdf-772d-487b-bacc-b76e8517b63c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:13.929239 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62137fdf-772d-487b-bacc-b76e8517b63c None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:13.929533 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62137fdf-772d-487b-bacc-b76e8517b63c None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:13.929818 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62137fdf-772d-487b-bacc-b76e8517b63c None admin] PATH_INFO: `/v3/users/61be9336f6054259b36e1e8ded1ccc79` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:13.943524 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62137fdf-772d-487b-bacc-b76e8517b63c None admin] RBAC: Authorizing `identity:get_user(user_id=61be9336f6054259b36e1e8ded1ccc79)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:13.944982 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62137fdf-772d-487b-bacc-b76e8517b63c None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:13.946964 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 96/189] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:51:13 2022] GET /identity/v3/users/61be9336f6054259b36e1e8ded1ccc79 => generated 283 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:13.947221 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7573bc24-0c70-48ca-a738-dd295d4a38af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:13.948040 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7573bc24-0c70-48ca-a738-dd295d4a38af None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:13.948323 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659b5ebf-e4c5-4e07-b597-e8d10c8358dd None None] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:13.948427 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7573bc24-0c70-48ca-a738-dd295d4a38af None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:13.948597 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7573bc24-0c70-48ca-a738-dd295d4a38af None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:13.948703 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659b5ebf-e4c5-4e07-b597-e8d10c8358dd None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:13.948878 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659b5ebf-e4c5-4e07-b597-e8d10c8358dd None None] PATH_INFO: `/` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:13.950343 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 97/190] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:51:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:51:13.950944 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659b5ebf-e4c5-4e07-b597-e8d10c8358dd None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:13.963861 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7573bc24-0c70-48ca-a738-dd295d4a38af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:51:13.972115 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-7573bc24-0c70-48ca-a738-dd295d4a38af None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:51:13.972115 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:51:13.972115 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:51:13.972115 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:51:13.972115 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:51:13.972115 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:51:13.972115 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:51:13.972115 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:51:13.972115 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:51:13.972115 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:51:13.972115 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:51:13.972115 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:51:13.972115 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:51:13.972115 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:51:13.972115 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:51:13.972115 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:51:13.973686 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:51:13.973686 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:51:13.973686 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:51:13.973686 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:51:13.973686 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:51:13.973686 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:51:13.973686 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:51:13.973686 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:51:13.973686 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:51:13.973686 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:51:13.973686 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:51:13.973686 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:51:13.973686 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:51:13.973686 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:51:13.973686 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:51:13.973686 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:51:13.974661 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:51:13.974661 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:51:13.974661 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:51:13.974661 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:51:13.974661 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:51:13.974661 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:51:13.974661 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:51:13.974661 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:51:13.974661 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:51:13.974661 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:51:13.974661 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:51:13.974661 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:51:13.976755 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 94/191] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 19 14:51:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:51:13.976755 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7573bc24-0c70-48ca-a738-dd295d4a38af None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:13.995573 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:14.058099 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659b5ebf-e4c5-4e07-b597-e8d10c8358dd None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:14.085140 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7573bc24-0c70-48ca-a738-dd295d4a38af None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:14.094039 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420cf874-41fa-4ddc-9e9f-0bda6fb880c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:14.094710 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420cf874-41fa-4ddc-9e9f-0bda6fb880c0 None admin] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:14.094999 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420cf874-41fa-4ddc-9e9f-0bda6fb880c0 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:14.095284 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420cf874-41fa-4ddc-9e9f-0bda6fb880c0 None admin] PATH_INFO: `/v3/users` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:14.095992 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420cf874-41fa-4ddc-9e9f-0bda6fb880c0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:14.098193 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420cf874-41fa-4ddc-9e9f-0bda6fb880c0 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:14.108181 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:14.121097 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 98/192] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 19 14:51:13 2022] POST /identity/v3/users => generated 297 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:51:14.121934 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420cf874-41fa-4ddc-9e9f-0bda6fb880c0 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:14.129325 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9cfa02-827c-4f4a-a069-113704a7b6b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:14.130198 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9cfa02-827c-4f4a-a069-113704a7b6b9 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:14.130489 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9cfa02-827c-4f4a-a069-113704a7b6b9 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:14.130774 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9cfa02-827c-4f4a-a069-113704a7b6b9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:14.131935 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9cfa02-827c-4f4a-a069-113704a7b6b9 None admin] RBAC: Adding query filter params (role.id=aed9aa9ef7f7448bb3b40c9ad082d545, scope.project.id=ab05694ea13841e182a0be2091bf3654, user.id=2950d06ae84d46c081f68af26db145d4) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:14.132258 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9cfa02-827c-4f4a-a069-113704a7b6b9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:14.133814 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9cfa02-827c-4f4a-a069-113704a7b6b9 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:14.140860 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 95/193] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:51:13 2022] GET /identity/v3/role_assignments?user.id=2950d06ae84d46c081f68af26db145d4&scope.project.id=ab05694ea13841e182a0be2091bf3654&role.id=aed9aa9ef7f7448bb3b40c9ad082d545 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:14.142252 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673e0b66-127a-4fd3-b576-816933ede1ce None None] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:14.142551 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673e0b66-127a-4fd3-b576-816933ede1ce None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:14.142837 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673e0b66-127a-4fd3-b576-816933ede1ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:14.163396 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.auth.core [None req-673e0b66-127a-4fd3-b576-816933ede1ce None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:51:14.164810 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.fernet_utils [None req-673e0b66-127a-4fd3-b576-816933ede1ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:51:14.265430 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 96/194] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:51:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:51:14.267641 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673e0b66-127a-4fd3-b576-816933ede1ce None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:14.285935 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420cf874-41fa-4ddc-9e9f-0bda6fb880c0 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:14.291378 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:14.334321 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976f17d6-871e-46e0-9a64-f844a3bd5dc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:14.335043 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976f17d6-871e-46e0-9a64-f844a3bd5dc4 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:14.335390 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976f17d6-871e-46e0-9a64-f844a3bd5dc4 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:14.335682 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976f17d6-871e-46e0-9a64-f844a3bd5dc4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:14.415938 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673e0b66-127a-4fd3-b576-816933ede1ce None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:14.420315 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:14.425781 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976f17d6-871e-46e0-9a64-f844a3bd5dc4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:14.428221 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976f17d6-871e-46e0-9a64-f844a3bd5dc4 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:14.461101 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da90e4c-025d-493c-8a7b-73145dd76329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:14.462201 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da90e4c-025d-493c-8a7b-73145dd76329 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:14.462708 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da90e4c-025d-493c-8a7b-73145dd76329 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:14.463203 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da90e4c-025d-493c-8a7b-73145dd76329 None admin] PATH_INFO: `/v3/roles` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:14.464188 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da90e4c-025d-493c-8a7b-73145dd76329 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:14.464751 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da90e4c-025d-493c-8a7b-73145dd76329 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:14.466936 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da90e4c-025d-493c-8a7b-73145dd76329 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:14.477923 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 97/195] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:51:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:14.478543 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da90e4c-025d-493c-8a7b-73145dd76329 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:14.551317 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 99/196] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:51:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:51:14.552127 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976f17d6-871e-46e0-9a64-f844a3bd5dc4 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:14.572128 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:14.606430 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da90e4c-025d-493c-8a7b-73145dd76329 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:14.665119 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:14.673918 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976f17d6-871e-46e0-9a64-f844a3bd5dc4 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:14.679300 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180e45ff-863a-4531-8f59-d0ec1660f81b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:14.680384 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180e45ff-863a-4531-8f59-d0ec1660f81b None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:14.680884 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180e45ff-863a-4531-8f59-d0ec1660f81b None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:14.681345 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180e45ff-863a-4531-8f59-d0ec1660f81b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:14.697057 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180e45ff-863a-4531-8f59-d0ec1660f81b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:51:14.710175 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-180e45ff-863a-4531-8f59-d0ec1660f81b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:51:14.710175 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:51:14.710175 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:51:14.710175 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:51:14.710175 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:51:14.710175 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:51:14.710175 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:51:14.710175 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:51:14.710175 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:51:14.710175 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:51:14.710175 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:51:14.710175 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:51:14.710175 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:51:14.710175 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:51:14.710175 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:51:14.710175 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:51:14.711420 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:51:14.711420 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:51:14.711420 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:51:14.711420 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:51:14.711420 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:51:14.711420 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:51:14.711420 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:51:14.711420 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:51:14.711420 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:51:14.711420 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:51:14.711420 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:51:14.711420 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:51:14.711420 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:51:14.711420 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:51:14.711420 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:51:14.711420 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:51:14.712617 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:51:14.712617 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:51:14.712617 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:51:14.712617 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:51:14.712617 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:51:14.712617 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:51:14.712617 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:51:14.712617 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:51:14.712617 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:51:14.712617 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:51:14.712617 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:51:14.712617 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:51:14.715593 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 98/197] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:51:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:51:14.716783 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180e45ff-863a-4531-8f59-d0ec1660f81b None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:14.717281 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491cd3b8-5f9e-413a-a998-869deeaefdc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:14.718273 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491cd3b8-5f9e-413a-a998-869deeaefdc6 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:14.718635 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491cd3b8-5f9e-413a-a998-869deeaefdc6 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:14.718986 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491cd3b8-5f9e-413a-a998-869deeaefdc6 None admin] PATH_INFO: `/v3/users/61be9336f6054259b36e1e8ded1ccc79` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:14.720607 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491cd3b8-5f9e-413a-a998-869deeaefdc6 None admin] RBAC: Authorizing `identity:get_user(user_id=61be9336f6054259b36e1e8ded1ccc79)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:14.723158 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491cd3b8-5f9e-413a-a998-869deeaefdc6 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:14.724295 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 100/198] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:51:14 2022] GET /identity/v3/users/61be9336f6054259b36e1e8ded1ccc79 => generated 283 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:14.725131 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491cd3b8-5f9e-413a-a998-869deeaefdc6 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:14.840941 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180e45ff-863a-4531-8f59-d0ec1660f81b None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:14.850554 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491cd3b8-5f9e-413a-a998-869deeaefdc6 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:14.885787 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee0223a-4b55-46d4-99b3-926c9c8684ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:14.886937 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee0223a-4b55-46d4-99b3-926c9c8684ba None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:14.887324 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee0223a-4b55-46d4-99b3-926c9c8684ba None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:14.887745 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee0223a-4b55-46d4-99b3-926c9c8684ba None admin] PATH_INFO: `/v3/projects/caf48e96564645f3b3e590d230116a2d` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:14.889253 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee0223a-4b55-46d4-99b3-926c9c8684ba None admin] RBAC: Authorizing `identity:get_project(project_id=caf48e96564645f3b3e590d230116a2d)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:14.890829 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee0223a-4b55-46d4-99b3-926c9c8684ba None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:14.892647 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 99/199] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:51:14 2022] GET /identity/v3/projects/caf48e96564645f3b3e590d230116a2d => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:14.893509 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee0223a-4b55-46d4-99b3-926c9c8684ba None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:14.896810 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:14.898601 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e742ceb4-56d4-4380-92da-e516e4c34101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:14.900195 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e742ceb4-56d4-4380-92da-e516e4c34101 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:14.900507 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e742ceb4-56d4-4380-92da-e516e4c34101 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:14.900796 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e742ceb4-56d4-4380-92da-e516e4c34101 None admin] PATH_INFO: `/v3/roles` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:14.901579 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e742ceb4-56d4-4380-92da-e516e4c34101 None admin] RBAC: Adding query filter params (name=reader) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:14.902264 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e742ceb4-56d4-4380-92da-e516e4c34101 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:14.903693 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e742ceb4-56d4-4380-92da-e516e4c34101 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:14.916112 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 101/200] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:51:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:14.917197 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e742ceb4-56d4-4380-92da-e516e4c34101 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:14.958423 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:15.019378 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee0223a-4b55-46d4-99b3-926c9c8684ba None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:15.057577 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3daf9f6-d826-4a90-85a4-8b550fa7a4a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:15.058636 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e742ceb4-56d4-4380-92da-e516e4c34101 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:15.059179 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3daf9f6-d826-4a90-85a4-8b550fa7a4a3 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:15.059663 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3daf9f6-d826-4a90-85a4-8b550fa7a4a3 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:15.060235 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3daf9f6-d826-4a90-85a4-8b550fa7a4a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:15.104643 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372c2c81-6507-40d7-b5eb-2cdc123dcd8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:15.105233 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372c2c81-6507-40d7-b5eb-2cdc123dcd8e None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:15.105524 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372c2c81-6507-40d7-b5eb-2cdc123dcd8e None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:15.105809 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372c2c81-6507-40d7-b5eb-2cdc123dcd8e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:15.107284 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372c2c81-6507-40d7-b5eb-2cdc123dcd8e None admin] RBAC: Adding query filter params (role.id=028379e01f9245e9b8fc56e8b9e8a6ad, scope.project.id=caf48e96564645f3b3e590d230116a2d, user.id=61be9336f6054259b36e1e8ded1ccc79) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:15.107609 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372c2c81-6507-40d7-b5eb-2cdc123dcd8e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:15.109677 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372c2c81-6507-40d7-b5eb-2cdc123dcd8e None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:15.118050 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 102/201] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:51:14 2022] GET /identity/v3/role_assignments?user.id=61be9336f6054259b36e1e8ded1ccc79&scope.project.id=caf48e96564645f3b3e590d230116a2d&role.id=028379e01f9245e9b8fc56e8b9e8a6ad => generated 259 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:15.122344 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372c2c81-6507-40d7-b5eb-2cdc123dcd8e None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:15.165814 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3daf9f6-d826-4a90-85a4-8b550fa7a4a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:15.168124 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3daf9f6-d826-4a90-85a4-8b550fa7a4a3 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:15.173478 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:15.327880 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:15.342650 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372c2c81-6507-40d7-b5eb-2cdc123dcd8e None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:15.371701 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 100/202] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:51:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 478 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:51:15.379005 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3daf9f6-d826-4a90-85a4-8b550fa7a4a3 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:15.406726 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbee76a-a15b-43a5-b84e-3105dcd64d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:15.407613 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbee76a-a15b-43a5-b84e-3105dcd64d61 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:15.407913 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbee76a-a15b-43a5-b84e-3105dcd64d61 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:15.408200 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbee76a-a15b-43a5-b84e-3105dcd64d61 None admin] PATH_INFO: `/v3/users/bce930239cd6428ca707fea57a0d4abb` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:15.409721 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbee76a-a15b-43a5-b84e-3105dcd64d61 None admin] RBAC: Authorizing `identity:get_user(user_id=bce930239cd6428ca707fea57a0d4abb)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:15.411376 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbee76a-a15b-43a5-b84e-3105dcd64d61 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:15.413392 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 103/203] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:51:15 2022] GET /identity/v3/users/bce930239cd6428ca707fea57a0d4abb => generated 289 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:15.419002 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbee76a-a15b-43a5-b84e-3105dcd64d61 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:15.498218 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3daf9f6-d826-4a90-85a4-8b550fa7a4a3 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:15.501099 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:15.541292 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318e0897-bee8-4860-a4c7-d1a9edd89556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:15.542745 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318e0897-bee8-4860-a4c7-d1a9edd89556 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:15.543261 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318e0897-bee8-4860-a4c7-d1a9edd89556 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:15.543750 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318e0897-bee8-4860-a4c7-d1a9edd89556 None admin] PATH_INFO: `/v3/projects/caf48e96564645f3b3e590d230116a2d` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:15.545176 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318e0897-bee8-4860-a4c7-d1a9edd89556 None admin] RBAC: Authorizing `identity:get_project(project_id=caf48e96564645f3b3e590d230116a2d)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:15.546891 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318e0897-bee8-4860-a4c7-d1a9edd89556 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:15.549051 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 101/204] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:51:15 2022] GET /identity/v3/projects/caf48e96564645f3b3e590d230116a2d => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:51:15.556177 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318e0897-bee8-4860-a4c7-d1a9edd89556 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:15.668991 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbee76a-a15b-43a5-b84e-3105dcd64d61 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:15.673661 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318e0897-bee8-4860-a4c7-d1a9edd89556 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:15.716123 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3fad9b-c58d-4246-9117-5828d2b9e39f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:15.716939 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3fad9b-c58d-4246-9117-5828d2b9e39f None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:15.717332 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3fad9b-c58d-4246-9117-5828d2b9e39f None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:15.717716 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3fad9b-c58d-4246-9117-5828d2b9e39f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:15.719039 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3fad9b-c58d-4246-9117-5828d2b9e39f None admin] RBAC: Adding query filter params (role.id=3b9c36a010504ba7bc63a8ccf0a9bca0, scope.project.id=caf48e96564645f3b3e590d230116a2d, user.id=61be9336f6054259b36e1e8ded1ccc79) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:15.719367 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3fad9b-c58d-4246-9117-5828d2b9e39f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:15.720839 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3fad9b-c58d-4246-9117-5828d2b9e39f None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:15.728549 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 102/205] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:51:15 2022] GET /identity/v3/role_assignments?user.id=61be9336f6054259b36e1e8ded1ccc79&scope.project.id=caf48e96564645f3b3e590d230116a2d&role.id=3b9c36a010504ba7bc63a8ccf0a9bca0 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:51:15.768832 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5109d6-6dca-4c1e-bfbd-d902ec7d1449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:15.769920 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5109d6-6dca-4c1e-bfbd-d902ec7d1449 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:15.770228 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5109d6-6dca-4c1e-bfbd-d902ec7d1449 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:15.770519 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5109d6-6dca-4c1e-bfbd-d902ec7d1449 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:15.817085 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:15.882117 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5109d6-6dca-4c1e-bfbd-d902ec7d1449 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:15.883654 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5109d6-6dca-4c1e-bfbd-d902ec7d1449 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:16.023147 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 104/206] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:51:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 604 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:51:16.028971 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3fad9b-c58d-4246-9117-5828d2b9e39f None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:16.067116 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:16.140609 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3fad9b-c58d-4246-9117-5828d2b9e39f None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:16.178003 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c46e54-29ee-4a99-8839-8267b0b9256d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:16.178961 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c46e54-29ee-4a99-8839-8267b0b9256d None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:16.179245 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c46e54-29ee-4a99-8839-8267b0b9256d None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:16.179525 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c46e54-29ee-4a99-8839-8267b0b9256d None admin] PATH_INFO: `/v3/projects/aca8075ce986433eae657055d1366699` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:16.180759 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c46e54-29ee-4a99-8839-8267b0b9256d None admin] RBAC: Authorizing `identity:get_project(project_id=aca8075ce986433eae657055d1366699)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:16.182259 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c46e54-29ee-4a99-8839-8267b0b9256d None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:16.184159 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 103/207] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:51:16 2022] GET /identity/v3/projects/aca8075ce986433eae657055d1366699 => generated 298 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:51:16.191041 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5109d6-6dca-4c1e-bfbd-d902ec7d1449 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:16.254983 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:16.314634 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5109d6-6dca-4c1e-bfbd-d902ec7d1449 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:16.357506 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe415b8-4420-425d-96de-9dd8a043a4c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:16.358219 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe415b8-4420-425d-96de-9dd8a043a4c1 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:16.358511 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe415b8-4420-425d-96de-9dd8a043a4c1 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:16.358795 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe415b8-4420-425d-96de-9dd8a043a4c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:16.359653 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe415b8-4420-425d-96de-9dd8a043a4c1 None admin] RBAC: Adding query filter params (role.id=e56389e17ad24e20bf3bba1ca7684062, scope.project.id=aca8075ce986433eae657055d1366699, user.id=bce930239cd6428ca707fea57a0d4abb) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:16.359979 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe415b8-4420-425d-96de-9dd8a043a4c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:16.361455 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe415b8-4420-425d-96de-9dd8a043a4c1 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:16.368356 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 105/208] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:51:16 2022] GET /identity/v3/role_assignments?user.id=bce930239cd6428ca707fea57a0d4abb&scope.project.id=aca8075ce986433eae657055d1366699&role.id=e56389e17ad24e20bf3bba1ca7684062 => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:16.986307 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd305892-c9ca-4e84-bae2-aa00e99eac9a None None] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:16.986978 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd305892-c9ca-4e84-bae2-aa00e99eac9a None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:16.986978 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd305892-c9ca-4e84-bae2-aa00e99eac9a None None] PATH_INFO: `/` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:16.989148 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 104/209] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:51:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:51:16.997057 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52565c9a-0772-4ac8-8eb2-d9afdc735bbb None None] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:16.997430 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52565c9a-0772-4ac8-8eb2-d9afdc735bbb None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:16.997769 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52565c9a-0772-4ac8-8eb2-d9afdc735bbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:17.022423 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.auth.core [None req-52565c9a-0772-4ac8-8eb2-d9afdc735bbb None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:51:17.023646 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-52565c9a-0772-4ac8-8eb2-d9afdc735bbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:51:17.073494 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf2f7da-ec31-4dfb-bbbd-3e1e9064ee18 None None] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:17.074286 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf2f7da-ec31-4dfb-bbbd-3e1e9064ee18 None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:17.074812 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf2f7da-ec31-4dfb-bbbd-3e1e9064ee18 None None] PATH_INFO: `/` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:17.077937 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 105/210] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:51:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:51:17.083858 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f4f97a-2710-403b-b4af-e7d2d5d1998e None None] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:17.084482 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f4f97a-2710-403b-b4af-e7d2d5d1998e None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:17.084994 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f4f97a-2710-403b-b4af-e7d2d5d1998e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:17.105763 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.auth.core [None req-95f4f97a-2710-403b-b4af-e7d2d5d1998e None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:51:17.107335 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.fernet_utils [None req-95f4f97a-2710-403b-b4af-e7d2d5d1998e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:51:17.110903 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 106/211] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:51:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:51:17.140176 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52565c9a-0772-4ac8-8eb2-d9afdc735bbb None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:17.183379 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 106/212] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:51:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:51:17.200109 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:17.208941 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f4f97a-2710-403b-b4af-e7d2d5d1998e None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:17.243524 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:17.259679 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52565c9a-0772-4ac8-8eb2-d9afdc735bbb None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:17.298248 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edf4255-00ab-454c-9ab3-b5342d9a9271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:17.299526 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edf4255-00ab-454c-9ab3-b5342d9a9271 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:17.300058 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edf4255-00ab-454c-9ab3-b5342d9a9271 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:17.300616 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edf4255-00ab-454c-9ab3-b5342d9a9271 None admin] PATH_INFO: `/v3/roles/member` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:17.312601 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edf4255-00ab-454c-9ab3-b5342d9a9271 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:51:17.322506 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-6edf4255-00ab-454c-9ab3-b5342d9a9271 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:51:17.322506 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:51:17.322506 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:51:17.322506 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:51:17.322506 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:51:17.322506 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:51:17.322506 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:51:17.322506 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:51:17.322506 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:51:17.322506 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:51:17.322506 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:51:17.322506 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:51:17.322506 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:51:17.322506 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:51:17.322506 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:51:17.322506 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:51:17.323544 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:51:17.323544 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:51:17.323544 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:51:17.323544 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:51:17.323544 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:51:17.323544 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:51:17.323544 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:51:17.323544 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:51:17.323544 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:51:17.323544 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:51:17.323544 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:51:17.323544 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:51:17.323544 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:51:17.323544 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:51:17.323544 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:51:17.323544 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:51:17.324558 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:51:17.324558 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:51:17.324558 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:51:17.324558 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:51:17.324558 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:51:17.324558 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:51:17.324558 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:51:17.324558 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:51:17.324558 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:51:17.324558 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:51:17.324558 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:51:17.324558 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:51:17.325253 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f4f97a-2710-403b-b4af-e7d2d5d1998e None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:17.327337 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 107/213] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:51:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:51:17.335616 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edf4255-00ab-454c-9ab3-b5342d9a9271 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:17.371236 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ca1410-e91b-47ee-8876-50dbf3bc61a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:17.373099 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ca1410-e91b-47ee-8876-50dbf3bc61a7 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:17.373918 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ca1410-e91b-47ee-8876-50dbf3bc61a7 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:17.374414 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ca1410-e91b-47ee-8876-50dbf3bc61a7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:17.376712 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ca1410-e91b-47ee-8876-50dbf3bc61a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:17.379284 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ca1410-e91b-47ee-8876-50dbf3bc61a7 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:17.382474 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 107/214] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:51:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:51:17.387620 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ca1410-e91b-47ee-8876-50dbf3bc61a7 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:17.459693 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edf4255-00ab-454c-9ab3-b5342d9a9271 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:17.474732 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:17.498411 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5cef74-85ab-4e39-a65c-f7b3e6c99ed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:17.499432 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5cef74-85ab-4e39-a65c-f7b3e6c99ed9 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:17.499947 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5cef74-85ab-4e39-a65c-f7b3e6c99ed9 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:17.500451 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5cef74-85ab-4e39-a65c-f7b3e6c99ed9 None admin] PATH_INFO: `/v3/roles` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:17.501516 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5cef74-85ab-4e39-a65c-f7b3e6c99ed9 None admin] RBAC: Adding query filter params (name=member) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:17.502181 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5cef74-85ab-4e39-a65c-f7b3e6c99ed9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:17.503986 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5cef74-85ab-4e39-a65c-f7b3e6c99ed9 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:17.513683 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 108/215] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:51:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:51:17.521939 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5cef74-85ab-4e39-a65c-f7b3e6c99ed9 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:17.585433 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:17.591227 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ca1410-e91b-47ee-8876-50dbf3bc61a7 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:17.634989 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedb40c3-3e2a-43ad-a82f-c71aec088ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:17.636459 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedb40c3-3e2a-43ad-a82f-c71aec088ba5 None admin] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:17.636988 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedb40c3-3e2a-43ad-a82f-c71aec088ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:17.637503 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedb40c3-3e2a-43ad-a82f-c71aec088ba5 None admin] PATH_INFO: `/v3/users` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:17.638702 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedb40c3-3e2a-43ad-a82f-c71aec088ba5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:17.640963 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedb40c3-3e2a-43ad-a82f-c71aec088ba5 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:17.673547 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5cef74-85ab-4e39-a65c-f7b3e6c99ed9 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:17.683038 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 108/216] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 19 14:51:17 2022] POST /identity/v3/users => generated 297 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:51:17.720455 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec17c3f-b86d-4c07-963b-18e18e394a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:17.721433 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec17c3f-b86d-4c07-963b-18e18e394a09 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:17.721727 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec17c3f-b86d-4c07-963b-18e18e394a09 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:17.722078 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec17c3f-b86d-4c07-963b-18e18e394a09 None admin] PATH_INFO: `/v3/users/c33895867866448a957a662b630081bf` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:17.739307 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec17c3f-b86d-4c07-963b-18e18e394a09 None admin] RBAC: Authorizing `identity:get_user(user_id=c33895867866448a957a662b630081bf)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:17.740819 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec17c3f-b86d-4c07-963b-18e18e394a09 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:17.743062 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 109/217] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:51:17 2022] GET /identity/v3/users/c33895867866448a957a662b630081bf => generated 297 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:17.751549 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedb40c3-3e2a-43ad-a82f-c71aec088ba5 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:17.856833 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bee1d7-88fe-4417-bd51-64310e8945d0 None None] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:17.857201 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bee1d7-88fe-4417-bd51-64310e8945d0 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:17.857436 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bee1d7-88fe-4417-bd51-64310e8945d0 None None] PATH_INFO: `/` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:17.859033 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 110/218] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:51:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:51:17.864870 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d02b61-4110-483c-92d4-d7bae5425ae0 None None] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:17.865065 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d02b61-4110-483c-92d4-d7bae5425ae0 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:17.865360 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d02b61-4110-483c-92d4-d7bae5425ae0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:17.883363 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedb40c3-3e2a-43ad-a82f-c71aec088ba5 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:17.883593 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.auth.core [None req-41d02b61-4110-483c-92d4-d7bae5425ae0 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:51:17.884749 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-41d02b61-4110-483c-92d4-d7bae5425ae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:51:17.915258 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:17.926295 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c538e9eb-a87b-47c3-962c-bdcab4728b9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:17.927394 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c538e9eb-a87b-47c3-962c-bdcab4728b9a None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:17.927929 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c538e9eb-a87b-47c3-962c-bdcab4728b9a None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:17.928443 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c538e9eb-a87b-47c3-962c-bdcab4728b9a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:17.969251 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:17.971591 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 111/219] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:51:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:51:17.997332 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d02b61-4110-483c-92d4-d7bae5425ae0 None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:18.024789 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c538e9eb-a87b-47c3-962c-bdcab4728b9a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:18.026916 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c538e9eb-a87b-47c3-962c-bdcab4728b9a None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:18.142140 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d02b61-4110-483c-92d4-d7bae5425ae0 None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:18.170897 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 109/220] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:51:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:51:18.179007 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c538e9eb-a87b-47c3-962c-bdcab4728b9a None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:18.193682 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf51d8b-961e-425f-8144-3419b3f1653d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:18.194695 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf51d8b-961e-425f-8144-3419b3f1653d None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:18.195083 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf51d8b-961e-425f-8144-3419b3f1653d None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:18.195567 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf51d8b-961e-425f-8144-3419b3f1653d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:18.208429 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf51d8b-961e-425f-8144-3419b3f1653d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:51:18.217589 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-ccf51d8b-961e-425f-8144-3419b3f1653d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:51:18.217589 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:51:18.217589 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:51:18.217589 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:51:18.217589 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:51:18.217589 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:51:18.217589 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:51:18.217589 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:51:18.217589 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:51:18.217589 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:51:18.217589 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:51:18.217589 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:51:18.217589 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:51:18.217589 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:51:18.217589 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:51:18.217589 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:51:18.219600 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:51:18.219600 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:51:18.219600 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:51:18.219600 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:51:18.219600 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:51:18.219600 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:51:18.219600 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:51:18.219600 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:51:18.219600 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:51:18.219600 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:51:18.219600 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:51:18.219600 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:51:18.219600 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:51:18.219600 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:51:18.219600 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:51:18.219600 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:51:18.220731 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:51:18.220731 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:51:18.220731 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:51:18.220731 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:51:18.220731 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:51:18.220731 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:51:18.220731 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:51:18.220731 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:51:18.220731 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:51:18.220731 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:51:18.220731 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:51:18.220731 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:51:18.222128 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 112/221] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:51:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:51:18.228293 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf51d8b-961e-425f-8144-3419b3f1653d None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:18.250640 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:18.262312 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:18.303388 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c538e9eb-a87b-47c3-962c-bdcab4728b9a None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:18.339784 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16469f8f-68f2-4df2-b447-9da7ccc6c243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:18.340659 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16469f8f-68f2-4df2-b447-9da7ccc6c243 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:18.340948 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16469f8f-68f2-4df2-b447-9da7ccc6c243 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:18.341230 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16469f8f-68f2-4df2-b447-9da7ccc6c243 None admin] PATH_INFO: `/v3/projects/caf48e96564645f3b3e590d230116a2d` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:18.342477 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16469f8f-68f2-4df2-b447-9da7ccc6c243 None admin] RBAC: Authorizing `identity:get_project(project_id=caf48e96564645f3b3e590d230116a2d)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:18.343937 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16469f8f-68f2-4df2-b447-9da7ccc6c243 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:18.345992 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 110/222] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:51:18 2022] GET /identity/v3/projects/caf48e96564645f3b3e590d230116a2d => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:18.351910 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16469f8f-68f2-4df2-b447-9da7ccc6c243 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:18.352847 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf51d8b-961e-425f-8144-3419b3f1653d None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:18.397298 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bcc0cd-e057-4e0e-8bb3-76be90b8a14f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:18.398030 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bcc0cd-e057-4e0e-8bb3-76be90b8a14f None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:18.398327 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bcc0cd-e057-4e0e-8bb3-76be90b8a14f None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:18.398613 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bcc0cd-e057-4e0e-8bb3-76be90b8a14f None admin] PATH_INFO: `/v3/roles` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:18.399383 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bcc0cd-e057-4e0e-8bb3-76be90b8a14f None admin] RBAC: Adding query filter params (name=admin) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:18.399700 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bcc0cd-e057-4e0e-8bb3-76be90b8a14f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:18.405907 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bcc0cd-e057-4e0e-8bb3-76be90b8a14f None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:18.420682 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 113/223] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:51:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:51:18.427074 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bcc0cd-e057-4e0e-8bb3-76be90b8a14f None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:18.495041 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16469f8f-68f2-4df2-b447-9da7ccc6c243 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:18.522729 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:18.536393 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375ba84b-9391-4b95-a0b7-e4a86d78e014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:18.537348 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375ba84b-9391-4b95-a0b7-e4a86d78e014 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:18.537634 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375ba84b-9391-4b95-a0b7-e4a86d78e014 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:18.537966 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375ba84b-9391-4b95-a0b7-e4a86d78e014 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:18.538824 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375ba84b-9391-4b95-a0b7-e4a86d78e014 None admin] RBAC: Adding query filter params (role.id=aed9aa9ef7f7448bb3b40c9ad082d545, scope.project.id=caf48e96564645f3b3e590d230116a2d, user.id=c33895867866448a957a662b630081bf) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:18.539143 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375ba84b-9391-4b95-a0b7-e4a86d78e014 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:18.540642 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375ba84b-9391-4b95-a0b7-e4a86d78e014 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:18.547032 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 111/224] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:51:18 2022] GET /identity/v3/role_assignments?user.id=c33895867866448a957a662b630081bf&scope.project.id=caf48e96564645f3b3e590d230116a2d&role.id=aed9aa9ef7f7448bb3b40c9ad082d545 => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:18.548276 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a5de10-4ff3-4969-b715-c84e5bfc996c None None] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:18.548584 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a5de10-4ff3-4969-b715-c84e5bfc996c None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:18.548863 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a5de10-4ff3-4969-b715-c84e5bfc996c None None] PATH_INFO: `/` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:18.553965 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 112/225] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:51:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:51:18.554569 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:18.560887 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486611eb-f9b6-4e4a-a229-3822907cb637 None None] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:18.561225 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486611eb-f9b6-4e4a-a229-3822907cb637 None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:18.561451 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486611eb-f9b6-4e4a-a229-3822907cb637 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:18.595054 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.auth.core [None req-486611eb-f9b6-4e4a-a229-3822907cb637 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:51:18.596318 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.fernet_utils [None req-486611eb-f9b6-4e4a-a229-3822907cb637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:51:18.601657 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bcc0cd-e057-4e0e-8bb3-76be90b8a14f None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:18.679380 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea5f378-3720-45cf-ba15-1810f38a3ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:18.680350 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea5f378-3720-45cf-ba15-1810f38a3ce4 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:18.680702 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea5f378-3720-45cf-ba15-1810f38a3ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:18.680992 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea5f378-3720-45cf-ba15-1810f38a3ce4 None admin] PATH_INFO: `/v3/users/61be9336f6054259b36e1e8ded1ccc79` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:18.685605 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea5f378-3720-45cf-ba15-1810f38a3ce4 None admin] RBAC: Authorizing `identity:get_user(user_id=61be9336f6054259b36e1e8ded1ccc79)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:18.687172 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea5f378-3720-45cf-ba15-1810f38a3ce4 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:18.693879 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 114/226] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:51:18 2022] GET /identity/v3/users/61be9336f6054259b36e1e8ded1ccc79 => generated 283 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:51:18.695338 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea5f378-3720-45cf-ba15-1810f38a3ce4 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:18.754977 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 113/227] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:51:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:51:18.790360 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486611eb-f9b6-4e4a-a229-3822907cb637 None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:18.844639 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea5f378-3720-45cf-ba15-1810f38a3ce4 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:18.900688 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f0c1fd-0b08-40f4-835e-e75047934fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:18.901591 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f0c1fd-0b08-40f4-835e-e75047934fe5 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:18.901908 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f0c1fd-0b08-40f4-835e-e75047934fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:18.902201 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f0c1fd-0b08-40f4-835e-e75047934fe5 None admin] PATH_INFO: `/v3/projects/caf48e96564645f3b3e590d230116a2d` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:18.906305 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f0c1fd-0b08-40f4-835e-e75047934fe5 None admin] RBAC: Authorizing `identity:get_project(project_id=caf48e96564645f3b3e590d230116a2d)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:18.908520 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f0c1fd-0b08-40f4-835e-e75047934fe5 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:18.911612 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 115/228] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:51:18 2022] GET /identity/v3/projects/caf48e96564645f3b3e590d230116a2d => generated 302 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:51:18.917365 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f0c1fd-0b08-40f4-835e-e75047934fe5 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:18.935023 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486611eb-f9b6-4e4a-a229-3822907cb637 None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:18.958177 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:18.974439 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77ac1ea-8c08-41c8-8542-474b1e58c280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:18.975277 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77ac1ea-8c08-41c8-8542-474b1e58c280 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:18.975568 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77ac1ea-8c08-41c8-8542-474b1e58c280 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:18.975851 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77ac1ea-8c08-41c8-8542-474b1e58c280 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:18.977506 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:19.002948 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77ac1ea-8c08-41c8-8542-474b1e58c280 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:51:19.018062 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-d77ac1ea-8c08-41c8-8542-474b1e58c280 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:51:19.018062 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:51:19.018062 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:51:19.018062 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:51:19.018062 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:51:19.018062 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:51:19.018062 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:51:19.018062 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:51:19.018062 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:51:19.018062 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:51:19.018062 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:51:19.018062 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:51:19.018062 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:51:19.018062 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:51:19.018062 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:51:19.018062 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:51:19.019840 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:51:19.019840 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:51:19.019840 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:51:19.019840 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:51:19.019840 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:51:19.019840 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:51:19.019840 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:51:19.019840 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:51:19.019840 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:51:19.019840 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:51:19.019840 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:51:19.019840 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:51:19.019840 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:51:19.019840 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:51:19.019840 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:51:19.019840 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:51:19.020805 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:51:19.020805 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:51:19.020805 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:51:19.020805 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:51:19.020805 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:51:19.020805 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:51:19.020805 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:51:19.020805 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:51:19.020805 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:51:19.020805 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:51:19.020805 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:51:19.020805 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:51:19.024971 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 114/229] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 19 14:51:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:51:19.026990 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77ac1ea-8c08-41c8-8542-474b1e58c280 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:19.141349 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77ac1ea-8c08-41c8-8542-474b1e58c280 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:19.177705 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030c3e5a-0119-4e12-91bd-89b748bc9165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:19.178472 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030c3e5a-0119-4e12-91bd-89b748bc9165 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:19.178758 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030c3e5a-0119-4e12-91bd-89b748bc9165 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:19.179093 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030c3e5a-0119-4e12-91bd-89b748bc9165 None admin] PATH_INFO: `/v3/roles` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:19.179853 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030c3e5a-0119-4e12-91bd-89b748bc9165 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:19.180164 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030c3e5a-0119-4e12-91bd-89b748bc9165 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:19.181763 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030c3e5a-0119-4e12-91bd-89b748bc9165 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:19.196325 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 115/230] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:51:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:51:19.197887 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030c3e5a-0119-4e12-91bd-89b748bc9165 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:19.240375 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f0c1fd-0b08-40f4-835e-e75047934fe5 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:19.254504 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:19.326293 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030c3e5a-0119-4e12-91bd-89b748bc9165 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:19.343134 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8b8fc2-7c13-4929-ab91-a3ba11173b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:19.344066 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8b8fc2-7c13-4929-ab91-a3ba11173b13 None admin] REQUEST_METHOD: `PUT` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:19.344358 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8b8fc2-7c13-4929-ab91-a3ba11173b13 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:19.344650 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8b8fc2-7c13-4929-ab91-a3ba11173b13 None admin] PATH_INFO: `/v3/projects/caf48e96564645f3b3e590d230116a2d/users/61be9336f6054259b36e1e8ded1ccc79/roles/028379e01f9245e9b8fc56e8b9e8a6ad` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:19.346734 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8b8fc2-7c13-4929-ab91-a3ba11173b13 None admin] RBAC: Authorizing `identity:create_grant(project_id=caf48e96564645f3b3e590d230116a2d, user_id=61be9336f6054259b36e1e8ded1ccc79, role_id=028379e01f9245e9b8fc56e8b9e8a6ad)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:19.349038 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8b8fc2-7c13-4929-ab91-a3ba11173b13 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:19.368646 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736ab4b3-2b21-43be-a179-5123c215f54b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:19.369824 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736ab4b3-2b21-43be-a179-5123c215f54b None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:19.370341 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736ab4b3-2b21-43be-a179-5123c215f54b None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:19.370856 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736ab4b3-2b21-43be-a179-5123c215f54b None admin] PATH_INFO: `/v3/users/61be9336f6054259b36e1e8ded1ccc79` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:19.372770 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736ab4b3-2b21-43be-a179-5123c215f54b None admin] RBAC: Authorizing `identity:get_user(user_id=61be9336f6054259b36e1e8ded1ccc79)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:19.374618 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736ab4b3-2b21-43be-a179-5123c215f54b None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:19.381996 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 116/231] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:51:19 2022] GET /identity/v3/users/61be9336f6054259b36e1e8ded1ccc79 => generated 283 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:19.383168 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 116/232] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 14:51:18 2022] PUT /identity/v3/projects/caf48e96564645f3b3e590d230116a2d/users/61be9336f6054259b36e1e8ded1ccc79/roles/028379e01f9245e9b8fc56e8b9e8a6ad => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:51:19.383764 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736ab4b3-2b21-43be-a179-5123c215f54b None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:19.498412 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736ab4b3-2b21-43be-a179-5123c215f54b None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:19.510482 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:19.540396 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d381254-0684-4b20-af09-8c1b39e0c1db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:19.541285 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d381254-0684-4b20-af09-8c1b39e0c1db None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:19.541572 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d381254-0684-4b20-af09-8c1b39e0c1db None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:19.541871 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d381254-0684-4b20-af09-8c1b39e0c1db None admin] PATH_INFO: `/v3/projects/caf48e96564645f3b3e590d230116a2d` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:19.543153 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d381254-0684-4b20-af09-8c1b39e0c1db None admin] RBAC: Authorizing `identity:get_project(project_id=caf48e96564645f3b3e590d230116a2d)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:19.544760 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d381254-0684-4b20-af09-8c1b39e0c1db None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:19.551624 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 117/233] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:51:19 2022] GET /identity/v3/projects/caf48e96564645f3b3e590d230116a2d => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:51:19.554961 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8b8fc2-7c13-4929-ab91-a3ba11173b13 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:19.687154 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:19.711552 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8b8fc2-7c13-4929-ab91-a3ba11173b13 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:19.749385 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82eaf315-e0a7-4dc8-9c90-1c9391f9a0fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:19.750243 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eaf315-e0a7-4dc8-9c90-1c9391f9a0fc None admin] REQUEST_METHOD: `PUT` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:19.750531 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eaf315-e0a7-4dc8-9c90-1c9391f9a0fc None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:19.750808 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eaf315-e0a7-4dc8-9c90-1c9391f9a0fc None admin] PATH_INFO: `/v3/projects/caf48e96564645f3b3e590d230116a2d/users/61be9336f6054259b36e1e8ded1ccc79/roles/3b9c36a010504ba7bc63a8ccf0a9bca0` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:19.753061 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82eaf315-e0a7-4dc8-9c90-1c9391f9a0fc None admin] RBAC: Authorizing `identity:create_grant(project_id=caf48e96564645f3b3e590d230116a2d, user_id=61be9336f6054259b36e1e8ded1ccc79, role_id=3b9c36a010504ba7bc63a8ccf0a9bca0)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:19.754933 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82eaf315-e0a7-4dc8-9c90-1c9391f9a0fc None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:19.773731 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 117/234] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 14:51:19 2022] PUT /identity/v3/projects/caf48e96564645f3b3e590d230116a2d/users/61be9336f6054259b36e1e8ded1ccc79/roles/3b9c36a010504ba7bc63a8ccf0a9bca0 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:51:20.583819 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a46c0ab-4e82-4b54-bdee-1df76f37c80b None None] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:20.584704 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a46c0ab-4e82-4b54-bdee-1df76f37c80b None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:20.585180 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a46c0ab-4e82-4b54-bdee-1df76f37c80b None None] PATH_INFO: `/` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:20.587066 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 118/235] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:51:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:51:20.592870 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47458e0-7128-43e2-9147-b79239c687ef None None] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:20.593738 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47458e0-7128-43e2-9147-b79239c687ef None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:20.594296 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47458e0-7128-43e2-9147-b79239c687ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:20.611963 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.auth.core [None req-c47458e0-7128-43e2-9147-b79239c687ef None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:51:20.614182 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-c47458e0-7128-43e2-9147-b79239c687ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:51:20.705226 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 118/236] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:51:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:51:20.729544 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a46c0ab-4e82-4b54-bdee-1df76f37c80b None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:20.732065 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a375094-4835-4d94-8cc5-6532fa5ac7f5 None None] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:20.732482 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a375094-4835-4d94-8cc5-6532fa5ac7f5 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:20.732776 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a375094-4835-4d94-8cc5-6532fa5ac7f5 None None] PATH_INFO: `/` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:20.734618 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 119/237] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:51:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:51:20.746429 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6713e6e-66e7-4e4c-b6ee-67ca44ef8499 None None] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:20.746725 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6713e6e-66e7-4e4c-b6ee-67ca44ef8499 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:20.747010 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6713e6e-66e7-4e4c-b6ee-67ca44ef8499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:20.775823 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.auth.core [None req-d6713e6e-66e7-4e4c-b6ee-67ca44ef8499 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:51:20.777012 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-d6713e6e-66e7-4e4c-b6ee-67ca44ef8499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:51:20.881024 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:20.888986 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 120/238] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:51:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:51:20.918513 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a46c0ab-4e82-4b54-bdee-1df76f37c80b None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:20.937376 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6713e6e-66e7-4e4c-b6ee-67ca44ef8499 None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:20.981322 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e16079d-a8a4-4576-b67b-07930a3a2f70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:20.982172 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e16079d-a8a4-4576-b67b-07930a3a2f70 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:20.982498 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e16079d-a8a4-4576-b67b-07930a3a2f70 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:20.982783 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e16079d-a8a4-4576-b67b-07930a3a2f70 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:21.002608 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e16079d-a8a4-4576-b67b-07930a3a2f70 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:51:21.019648 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-5e16079d-a8a4-4576-b67b-07930a3a2f70 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:51:21.019648 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:51:21.019648 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:51:21.019648 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:51:21.019648 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:51:21.019648 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:51:21.019648 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:51:21.019648 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:51:21.019648 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:51:21.019648 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:51:21.019648 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:51:21.019648 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:51:21.019648 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:51:21.019648 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:51:21.019648 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:51:21.019648 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:51:21.022210 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:51:21.022210 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:51:21.022210 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:51:21.022210 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:51:21.022210 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:51:21.022210 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:51:21.022210 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:51:21.022210 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:51:21.022210 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:51:21.022210 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:51:21.022210 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:51:21.022210 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:51:21.022210 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:51:21.022210 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:51:21.022210 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:51:21.022210 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:51:21.024280 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:51:21.024280 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:51:21.024280 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:51:21.024280 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:51:21.024280 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:51:21.024280 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:51:21.024280 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:51:21.024280 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:51:21.024280 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:51:21.024280 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:51:21.024280 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:51:21.024280 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:51:21.024280 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 119/239] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:51:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:51:21.030776 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e16079d-a8a4-4576-b67b-07930a3a2f70 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:21.098234 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:21.120658 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6713e6e-66e7-4e4c-b6ee-67ca44ef8499 None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:21.185974 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0b405e-65de-458b-87cd-b88e5519a386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:21.186710 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0b405e-65de-458b-87cd-b88e5519a386 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:21.187000 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0b405e-65de-458b-87cd-b88e5519a386 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:21.187285 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0b405e-65de-458b-87cd-b88e5519a386 None admin] PATH_INFO: `/v3/domains/default` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:21.191360 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0b405e-65de-458b-87cd-b88e5519a386 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:21.192836 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0b405e-65de-458b-87cd-b88e5519a386 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:21.194540 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 121/240] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:51:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:51:21.200072 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0b405e-65de-458b-87cd-b88e5519a386 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:21.216786 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e16079d-a8a4-4576-b67b-07930a3a2f70 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:21.291443 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da8d7cd-b3eb-481b-90e8-fd0c63bb5c8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:21.292198 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da8d7cd-b3eb-481b-90e8-fd0c63bb5c8b None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:21.292496 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da8d7cd-b3eb-481b-90e8-fd0c63bb5c8b None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:21.292783 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da8d7cd-b3eb-481b-90e8-fd0c63bb5c8b None admin] PATH_INFO: `/v3/roles` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:21.293559 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da8d7cd-b3eb-481b-90e8-fd0c63bb5c8b None admin] RBAC: Adding query filter params (name=reader) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:21.293895 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da8d7cd-b3eb-481b-90e8-fd0c63bb5c8b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:21.295332 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da8d7cd-b3eb-481b-90e8-fd0c63bb5c8b None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:21.313817 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 120/241] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:51:21 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:21.322853 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da8d7cd-b3eb-481b-90e8-fd0c63bb5c8b None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:21.342365 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:21.366001 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0b405e-65de-458b-87cd-b88e5519a386 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:21.406239 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989480bc-7a2b-45f1-abf0-7a5abc354162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:21.406928 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989480bc-7a2b-45f1-abf0-7a5abc354162 None admin] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:21.407226 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989480bc-7a2b-45f1-abf0-7a5abc354162 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:21.407514 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989480bc-7a2b-45f1-abf0-7a5abc354162 None admin] PATH_INFO: `/v3/users` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:21.408254 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989480bc-7a2b-45f1-abf0-7a5abc354162 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:21.410342 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989480bc-7a2b-45f1-abf0-7a5abc354162 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:21.438010 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 122/242] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 19 14:51:21 2022] POST /identity/v3/users => generated 293 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:51:21.468169 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598b5992-d84e-48b6-a6c4-a922f64c8d46 None None] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:21.468830 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598b5992-d84e-48b6-a6c4-a922f64c8d46 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:21.469357 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598b5992-d84e-48b6-a6c4-a922f64c8d46 None None] PATH_INFO: `/` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:21.471499 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 123/243] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:51:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:51:21.481685 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9ba9df-6ede-4bb2-a1bf-db3bb4759f94 None None] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:21.482386 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9ba9df-6ede-4bb2-a1bf-db3bb4759f94 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:21.482911 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9ba9df-6ede-4bb2-a1bf-db3bb4759f94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:21.494659 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:21.500148 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da8d7cd-b3eb-481b-90e8-fd0c63bb5c8b None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:21.527039 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.auth.core [None req-8d9ba9df-6ede-4bb2-a1bf-db3bb4759f94 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:51:21.528438 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-8d9ba9df-6ede-4bb2-a1bf-db3bb4759f94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:51:21.547291 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2216e4d6-bbc8-4b41-8496-ff9d603db2a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:21.549032 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2216e4d6-bbc8-4b41-8496-ff9d603db2a7 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:21.549490 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2216e4d6-bbc8-4b41-8496-ff9d603db2a7 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:21.550067 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2216e4d6-bbc8-4b41-8496-ff9d603db2a7 None admin] PATH_INFO: `/v3/users/a5e2805af9a0450daaecf8b4e12404e3` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:21.568495 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2216e4d6-bbc8-4b41-8496-ff9d603db2a7 None admin] RBAC: Authorizing `identity:get_user(user_id=a5e2805af9a0450daaecf8b4e12404e3)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:21.570488 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2216e4d6-bbc8-4b41-8496-ff9d603db2a7 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:21.573480 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 121/244] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:51:21 2022] GET /identity/v3/users/a5e2805af9a0450daaecf8b4e12404e3 => generated 297 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:21.579942 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2216e4d6-bbc8-4b41-8496-ff9d603db2a7 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:21.710218 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2216e4d6-bbc8-4b41-8496-ff9d603db2a7 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:21.756967 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269ee8b5-5ae4-441f-8186-fa6b705e6ce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:21.758031 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269ee8b5-5ae4-441f-8186-fa6b705e6ce6 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:21.758531 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269ee8b5-5ae4-441f-8186-fa6b705e6ce6 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:21.758975 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269ee8b5-5ae4-441f-8186-fa6b705e6ce6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:21.771019 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 124/245] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:51:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:51:21.789420 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:21.798781 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9ba9df-6ede-4bb2-a1bf-db3bb4759f94 None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:21.839310 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269ee8b5-5ae4-441f-8186-fa6b705e6ce6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:21.840805 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269ee8b5-5ae4-441f-8186-fa6b705e6ce6 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:21.868037 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:21.919439 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9ba9df-6ede-4bb2-a1bf-db3bb4759f94 None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:21.949893 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 122/246] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:51:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:51:21.954921 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269ee8b5-5ae4-441f-8186-fa6b705e6ce6 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:21.968151 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c579ae8-a658-4f55-9bed-2c90e99298ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:21.969115 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c579ae8-a658-4f55-9bed-2c90e99298ac None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:21.969490 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c579ae8-a658-4f55-9bed-2c90e99298ac None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:21.969803 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c579ae8-a658-4f55-9bed-2c90e99298ac None admin] PATH_INFO: `/v3/roles/member` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:21.982881 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c579ae8-a658-4f55-9bed-2c90e99298ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:51:21.992812 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-2c579ae8-a658-4f55-9bed-2c90e99298ac None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:51:21.992812 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:51:21.992812 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:51:21.992812 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:51:21.992812 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:51:21.992812 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:51:21.992812 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:51:21.992812 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:51:21.992812 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:51:21.992812 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:51:21.992812 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:51:21.992812 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:51:21.992812 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:51:21.992812 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:51:21.992812 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:51:21.992812 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:51:21.994129 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:51:21.994129 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:51:21.994129 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:51:21.994129 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:51:21.994129 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:51:21.994129 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:51:21.994129 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:51:21.994129 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:51:21.994129 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:51:21.994129 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:51:21.994129 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:51:21.994129 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:51:21.994129 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:51:21.994129 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:51:21.994129 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:51:21.994129 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:51:21.995262 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:51:21.995262 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:51:21.995262 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:51:21.995262 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:51:21.995262 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:51:21.995262 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:51:21.995262 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:51:21.995262 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:51:21.995262 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:51:21.995262 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:51:21.995262 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:51:21.995262 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:51:21.997720 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 125/247] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:51:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:51:22.004495 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c579ae8-a658-4f55-9bed-2c90e99298ac None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:22.022647 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:22.077326 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269ee8b5-5ae4-441f-8186-fa6b705e6ce6 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:22.117699 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057015c6-edf9-4b61-9212-df271e7e1569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:22.118634 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057015c6-edf9-4b61-9212-df271e7e1569 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:22.118929 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057015c6-edf9-4b61-9212-df271e7e1569 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:22.119218 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057015c6-edf9-4b61-9212-df271e7e1569 None admin] PATH_INFO: `/v3/projects/caf48e96564645f3b3e590d230116a2d` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:22.120505 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057015c6-edf9-4b61-9212-df271e7e1569 None admin] RBAC: Authorizing `identity:get_project(project_id=caf48e96564645f3b3e590d230116a2d)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:22.122057 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057015c6-edf9-4b61-9212-df271e7e1569 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:22.124192 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 123/248] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:51:21 2022] GET /identity/v3/projects/caf48e96564645f3b3e590d230116a2d => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:51:22.132439 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057015c6-edf9-4b61-9212-df271e7e1569 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:22.147583 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c579ae8-a658-4f55-9bed-2c90e99298ac None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:22.174947 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:22.205749 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798b7d4c-7d20-4c0d-bd77-6ed08f79b662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:22.207046 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798b7d4c-7d20-4c0d-bd77-6ed08f79b662 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:22.207349 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798b7d4c-7d20-4c0d-bd77-6ed08f79b662 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:22.207802 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798b7d4c-7d20-4c0d-bd77-6ed08f79b662 None admin] PATH_INFO: `/v3/roles` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:22.208599 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798b7d4c-7d20-4c0d-bd77-6ed08f79b662 None admin] RBAC: Adding query filter params (name=member) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:22.208918 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798b7d4c-7d20-4c0d-bd77-6ed08f79b662 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:22.211261 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798b7d4c-7d20-4c0d-bd77-6ed08f79b662 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:22.221890 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 126/249] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:51:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:22.228924 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798b7d4c-7d20-4c0d-bd77-6ed08f79b662 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:22.293983 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057015c6-edf9-4b61-9212-df271e7e1569 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:22.320674 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:22.343787 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5578fe12-8d4c-40a9-a296-a15ae1425940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:22.344952 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5578fe12-8d4c-40a9-a296-a15ae1425940 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:22.345247 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5578fe12-8d4c-40a9-a296-a15ae1425940 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:22.345821 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5578fe12-8d4c-40a9-a296-a15ae1425940 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:22.347615 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5578fe12-8d4c-40a9-a296-a15ae1425940 None admin] RBAC: Adding query filter params (role.id=e56389e17ad24e20bf3bba1ca7684062, scope.project.id=caf48e96564645f3b3e590d230116a2d, user.id=a5e2805af9a0450daaecf8b4e12404e3) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:22.348068 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5578fe12-8d4c-40a9-a296-a15ae1425940 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:22.350533 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5578fe12-8d4c-40a9-a296-a15ae1425940 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:22.359417 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 124/250] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:51:22 2022] GET /identity/v3/role_assignments?user.id=a5e2805af9a0450daaecf8b4e12404e3&scope.project.id=caf48e96564645f3b3e590d230116a2d&role.id=e56389e17ad24e20bf3bba1ca7684062 => generated 259 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:22.361478 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a56da6-68e5-4753-bf70-43d25a095426 None None] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:22.361773 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a56da6-68e5-4753-bf70-43d25a095426 None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:22.362111 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a56da6-68e5-4753-bf70-43d25a095426 None None] PATH_INFO: `/` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:22.364542 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 125/251] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:51:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:51:22.368335 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798b7d4c-7d20-4c0d-bd77-6ed08f79b662 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:22.370328 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9ea76b-f826-47ff-8f12-ecc1dd81e21c None None] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:22.370731 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9ea76b-f826-47ff-8f12-ecc1dd81e21c None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:22.371021 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9ea76b-f826-47ff-8f12-ecc1dd81e21c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:22.390674 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.auth.core [None req-0e9ea76b-f826-47ff-8f12-ecc1dd81e21c None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:51:22.391922 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.fernet_utils [None req-0e9ea76b-f826-47ff-8f12-ecc1dd81e21c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:51:22.409068 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18308ff7-b08c-433b-83ed-7f9e8b9f0eec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:22.410556 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18308ff7-b08c-433b-83ed-7f9e8b9f0eec None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:22.410898 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18308ff7-b08c-433b-83ed-7f9e8b9f0eec None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:22.411178 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18308ff7-b08c-433b-83ed-7f9e8b9f0eec None admin] PATH_INFO: `/v3/users/c33895867866448a957a662b630081bf` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:22.412615 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18308ff7-b08c-433b-83ed-7f9e8b9f0eec None admin] RBAC: Authorizing `identity:get_user(user_id=c33895867866448a957a662b630081bf)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:22.414222 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18308ff7-b08c-433b-83ed-7f9e8b9f0eec None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:22.419718 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 127/252] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:51:22 2022] GET /identity/v3/users/c33895867866448a957a662b630081bf => generated 297 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:22.422160 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18308ff7-b08c-433b-83ed-7f9e8b9f0eec None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:22.475223 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:22.475846 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 126/253] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:51:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:51:22.503719 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9ea76b-f826-47ff-8f12-ecc1dd81e21c None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:22.542877 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18308ff7-b08c-433b-83ed-7f9e8b9f0eec None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:22.582265 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01dfb6c-db96-4e96-b856-423c0ac48a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:22.583254 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01dfb6c-db96-4e96-b856-423c0ac48a6e None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:22.584281 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01dfb6c-db96-4e96-b856-423c0ac48a6e None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:22.584579 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01dfb6c-db96-4e96-b856-423c0ac48a6e None admin] PATH_INFO: `/v3/projects/caf48e96564645f3b3e590d230116a2d` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:22.586343 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01dfb6c-db96-4e96-b856-423c0ac48a6e None admin] RBAC: Authorizing `identity:get_project(project_id=caf48e96564645f3b3e590d230116a2d)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:22.588889 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01dfb6c-db96-4e96-b856-423c0ac48a6e None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:22.593949 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 128/254] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:51:22 2022] GET /identity/v3/projects/caf48e96564645f3b3e590d230116a2d => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:51:22.599286 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01dfb6c-db96-4e96-b856-423c0ac48a6e None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:22.652848 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9ea76b-f826-47ff-8f12-ecc1dd81e21c None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:22.689832 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:22.726420 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c336047-7820-4d4e-8b88-db2853389155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:22.727183 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c336047-7820-4d4e-8b88-db2853389155 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:22.727451 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c336047-7820-4d4e-8b88-db2853389155 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:22.727735 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c336047-7820-4d4e-8b88-db2853389155 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:22.752186 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c336047-7820-4d4e-8b88-db2853389155 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:51:22.761237 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-8c336047-7820-4d4e-8b88-db2853389155 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:51:22.761237 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:51:22.761237 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:51:22.761237 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:51:22.761237 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:51:22.761237 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:51:22.761237 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:51:22.761237 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:51:22.761237 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:51:22.761237 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:51:22.761237 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:51:22.761237 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:51:22.761237 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:51:22.761237 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:51:22.761237 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:51:22.761237 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:51:22.762393 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:51:22.762393 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:51:22.762393 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:51:22.762393 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:51:22.762393 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:51:22.762393 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:51:22.762393 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:51:22.762393 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:51:22.762393 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:51:22.762393 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:51:22.762393 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:51:22.762393 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:51:22.762393 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:51:22.762393 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:51:22.762393 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:51:22.762393 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:51:22.763316 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:51:22.763316 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:51:22.763316 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:51:22.763316 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:51:22.763316 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:51:22.763316 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:51:22.763316 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:51:22.763316 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:51:22.763316 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:51:22.763316 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:51:22.763316 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:51:22.763316 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:51:22.763316 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 127/255] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:51:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:51:22.765624 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a619770-e3eb-4e4e-af45-cca00ef2be56 None None] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:22.765624 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a619770-e3eb-4e4e-af45-cca00ef2be56 None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:22.765624 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a619770-e3eb-4e4e-af45-cca00ef2be56 None None] PATH_INFO: `/` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:22.772468 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 128/256] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:51:22 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:51:22.772980 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a619770-e3eb-4e4e-af45-cca00ef2be56 None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:22.791301 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01dfb6c-db96-4e96-b856-423c0ac48a6e None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:22.807942 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:22.837606 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1d3dc8-fad7-4f28-8efb-97def1b166d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:22.838691 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1d3dc8-fad7-4f28-8efb-97def1b166d9 None admin] REQUEST_METHOD: `PUT` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:22.838987 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1d3dc8-fad7-4f28-8efb-97def1b166d9 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:22.839438 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1d3dc8-fad7-4f28-8efb-97def1b166d9 None admin] PATH_INFO: `/v3/projects/caf48e96564645f3b3e590d230116a2d/users/c33895867866448a957a662b630081bf/roles/aed9aa9ef7f7448bb3b40c9ad082d545` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:22.841796 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1d3dc8-fad7-4f28-8efb-97def1b166d9 None admin] RBAC: Authorizing `identity:create_grant(project_id=caf48e96564645f3b3e590d230116a2d, user_id=c33895867866448a957a662b630081bf, role_id=aed9aa9ef7f7448bb3b40c9ad082d545)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:22.843779 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1d3dc8-fad7-4f28-8efb-97def1b166d9 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:22.860486 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 129/257] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 14:51:22 2022] PUT /identity/v3/projects/caf48e96564645f3b3e590d230116a2d/users/c33895867866448a957a662b630081bf/roles/aed9aa9ef7f7448bb3b40c9ad082d545 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:51:22.861783 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48002b22-3f65-4a23-81f7-4efd6392c0bb None None] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:22.862110 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48002b22-3f65-4a23-81f7-4efd6392c0bb None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:22.862406 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48002b22-3f65-4a23-81f7-4efd6392c0bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:22.881087 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.auth.core [None req-48002b22-3f65-4a23-81f7-4efd6392c0bb None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:51:22.882403 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-48002b22-3f65-4a23-81f7-4efd6392c0bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:51:22.935605 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a619770-e3eb-4e4e-af45-cca00ef2be56 None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:22.983123 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e15ca6-3b56-49b5-bc5c-ba9a9d4868fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:22.983717 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e15ca6-3b56-49b5-bc5c-ba9a9d4868fe None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:22.984003 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e15ca6-3b56-49b5-bc5c-ba9a9d4868fe None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:22.984279 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e15ca6-3b56-49b5-bc5c-ba9a9d4868fe None admin] PATH_INFO: `/v3/roles` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:22.985037 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e15ca6-3b56-49b5-bc5c-ba9a9d4868fe None admin] RBAC: Adding query filter params (name=admin) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:22.985346 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e15ca6-3b56-49b5-bc5c-ba9a9d4868fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:22.986793 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e15ca6-3b56-49b5-bc5c-ba9a9d4868fe None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:22.997434 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 129/258] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:51:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:23.008874 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e15ca6-3b56-49b5-bc5c-ba9a9d4868fe None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:23.009758 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 130/259] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:51:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:51:23.056478 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48002b22-3f65-4a23-81f7-4efd6392c0bb None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:23.078730 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:23.172228 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e15ca6-3b56-49b5-bc5c-ba9a9d4868fe None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:23.209910 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e727d5-b73b-485c-ad35-35b5da65b2fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:23.210946 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e727d5-b73b-485c-ad35-35b5da65b2fd None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:23.211239 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e727d5-b73b-485c-ad35-35b5da65b2fd None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:23.211518 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e727d5-b73b-485c-ad35-35b5da65b2fd None admin] PATH_INFO: `/v3/users/61be9336f6054259b36e1e8ded1ccc79` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:23.212782 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e727d5-b73b-485c-ad35-35b5da65b2fd None admin] RBAC: Authorizing `identity:get_user(user_id=61be9336f6054259b36e1e8ded1ccc79)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:23.214300 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e727d5-b73b-485c-ad35-35b5da65b2fd None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:23.216448 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 130/260] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:51:23 2022] GET /identity/v3/users/61be9336f6054259b36e1e8ded1ccc79 => generated 283 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:51:23.223139 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e727d5-b73b-485c-ad35-35b5da65b2fd None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:23.255041 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48002b22-3f65-4a23-81f7-4efd6392c0bb None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:23.263997 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:23.313250 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4928ce-b614-48e3-8539-3410e72f5d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:23.314386 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4928ce-b614-48e3-8539-3410e72f5d15 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:23.314792 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4928ce-b614-48e3-8539-3410e72f5d15 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:23.315186 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4928ce-b614-48e3-8539-3410e72f5d15 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:23.331908 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4928ce-b614-48e3-8539-3410e72f5d15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:51:23.343825 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-5c4928ce-b614-48e3-8539-3410e72f5d15 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:51:23.343825 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:51:23.343825 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:51:23.343825 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:51:23.343825 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:51:23.343825 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:51:23.343825 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:51:23.343825 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:51:23.343825 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:51:23.343825 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:51:23.343825 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:51:23.343825 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:51:23.343825 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:51:23.343825 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:51:23.343825 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:51:23.343825 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:51:23.345100 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:51:23.345100 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:51:23.345100 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:51:23.345100 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:51:23.345100 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:51:23.345100 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:51:23.345100 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:51:23.345100 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:51:23.345100 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:51:23.345100 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:51:23.345100 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:51:23.345100 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:51:23.345100 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:51:23.345100 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:51:23.345100 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:51:23.345100 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:51:23.346274 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:51:23.346274 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:51:23.346274 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:51:23.346274 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:51:23.346274 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:51:23.346274 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:51:23.346274 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:51:23.346274 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:51:23.346274 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:51:23.346274 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:51:23.346274 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:51:23.346274 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:51:23.347999 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 131/261] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 19 14:51:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:51:23.349925 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e727d5-b73b-485c-ad35-35b5da65b2fd None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:23.353746 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4928ce-b614-48e3-8539-3410e72f5d15 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:23.362332 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:23.401092 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69daa92-ccb7-4553-9b9b-b6f9d5b51d9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:23.405074 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69daa92-ccb7-4553-9b9b-b6f9d5b51d9f None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:23.405074 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69daa92-ccb7-4553-9b9b-b6f9d5b51d9f None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:23.405074 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69daa92-ccb7-4553-9b9b-b6f9d5b51d9f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:23.510176 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69daa92-ccb7-4553-9b9b-b6f9d5b51d9f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:23.511585 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69daa92-ccb7-4553-9b9b-b6f9d5b51d9f None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:23.548935 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4928ce-b614-48e3-8539-3410e72f5d15 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:23.638554 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b71da6c-f90a-464e-a039-f5892a8a9918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:23.639299 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b71da6c-f90a-464e-a039-f5892a8a9918 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:23.639601 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b71da6c-f90a-464e-a039-f5892a8a9918 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:23.639889 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b71da6c-f90a-464e-a039-f5892a8a9918 None admin] PATH_INFO: `/v3/roles` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:23.640663 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b71da6c-f90a-464e-a039-f5892a8a9918 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:23.640982 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b71da6c-f90a-464e-a039-f5892a8a9918 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:23.642419 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b71da6c-f90a-464e-a039-f5892a8a9918 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:23.670284 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 132/262] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:51:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:51:23.677573 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b71da6c-f90a-464e-a039-f5892a8a9918 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:23.723260 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 131/263] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:51:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 501 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:51:23.728822 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69daa92-ccb7-4553-9b9b-b6f9d5b51d9f None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:23.737124 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:23.748335 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:23.875054 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69daa92-ccb7-4553-9b9b-b6f9d5b51d9f None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:23.914417 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bfa840-e037-4bd1-a78b-aa14937324d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:23.915795 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bfa840-e037-4bd1-a78b-aa14937324d2 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:23.916318 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bfa840-e037-4bd1-a78b-aa14937324d2 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:23.916813 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bfa840-e037-4bd1-a78b-aa14937324d2 None admin] PATH_INFO: `/v3/projects/caf48e96564645f3b3e590d230116a2d` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:23.918275 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bfa840-e037-4bd1-a78b-aa14937324d2 None admin] RBAC: Authorizing `identity:get_project(project_id=caf48e96564645f3b3e590d230116a2d)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:23.919956 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bfa840-e037-4bd1-a78b-aa14937324d2 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:23.920395 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b71da6c-f90a-464e-a039-f5892a8a9918 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:23.923016 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 132/264] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:51:23 2022] GET /identity/v3/projects/caf48e96564645f3b3e590d230116a2d => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:51:23.930042 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bfa840-e037-4bd1-a78b-aa14937324d2 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:23.999532 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ce5d00-f137-449d-a91c-67a6ebc6922d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:24.000440 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ce5d00-f137-449d-a91c-67a6ebc6922d None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:24.000738 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ce5d00-f137-449d-a91c-67a6ebc6922d None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:24.001028 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ce5d00-f137-449d-a91c-67a6ebc6922d None admin] PATH_INFO: `/v3/users/61be9336f6054259b36e1e8ded1ccc79` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:24.002295 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ce5d00-f137-449d-a91c-67a6ebc6922d None admin] RBAC: Authorizing `identity:get_user(user_id=61be9336f6054259b36e1e8ded1ccc79)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:24.004356 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ce5d00-f137-449d-a91c-67a6ebc6922d None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:24.014204 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 133/265] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:51:23 2022] GET /identity/v3/users/61be9336f6054259b36e1e8ded1ccc79 => generated 283 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:51:24.015134 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ce5d00-f137-449d-a91c-67a6ebc6922d None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:24.019851 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:24.049095 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bfa840-e037-4bd1-a78b-aa14937324d2 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:24.089518 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcdcefd-9ba3-4647-9f36-ebeb9c98bebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:24.090705 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcdcefd-9ba3-4647-9f36-ebeb9c98bebb None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:24.091314 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcdcefd-9ba3-4647-9f36-ebeb9c98bebb None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:24.091854 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcdcefd-9ba3-4647-9f36-ebeb9c98bebb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:24.092920 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcdcefd-9ba3-4647-9f36-ebeb9c98bebb None admin] RBAC: Adding query filter params (role.id=028379e01f9245e9b8fc56e8b9e8a6ad, scope.project.id=caf48e96564645f3b3e590d230116a2d, user.id=61be9336f6054259b36e1e8ded1ccc79) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:24.093482 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcdcefd-9ba3-4647-9f36-ebeb9c98bebb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:24.095313 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcdcefd-9ba3-4647-9f36-ebeb9c98bebb None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:24.103303 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 133/266] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:51:23 2022] GET /identity/v3/role_assignments?user.id=61be9336f6054259b36e1e8ded1ccc79&scope.project.id=caf48e96564645f3b3e590d230116a2d&role.id=028379e01f9245e9b8fc56e8b9e8a6ad => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:51:24.206262 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:24.216542 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ce5d00-f137-449d-a91c-67a6ebc6922d None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:24.263526 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9c14b0-c54b-475a-ba34-a19cd2e61383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:24.264201 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9c14b0-c54b-475a-ba34-a19cd2e61383 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:24.264515 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9c14b0-c54b-475a-ba34-a19cd2e61383 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:24.264772 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9c14b0-c54b-475a-ba34-a19cd2e61383 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:24.347361 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9c14b0-c54b-475a-ba34-a19cd2e61383 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:24.348890 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9c14b0-c54b-475a-ba34-a19cd2e61383 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:24.476678 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:24.488001 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 134/267] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:51:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:51:24.493688 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcdcefd-9ba3-4647-9f36-ebeb9c98bebb None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:24.634668 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcdcefd-9ba3-4647-9f36-ebeb9c98bebb None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:24.677750 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7e73e3-0df0-445b-b902-4910ab790b9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:24.678699 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7e73e3-0df0-445b-b902-4910ab790b9a None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:24.678991 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7e73e3-0df0-445b-b902-4910ab790b9a None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:24.679278 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7e73e3-0df0-445b-b902-4910ab790b9a None admin] PATH_INFO: `/v3/projects/caf48e96564645f3b3e590d230116a2d` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:24.680506 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7e73e3-0df0-445b-b902-4910ab790b9a None admin] RBAC: Authorizing `identity:get_project(project_id=caf48e96564645f3b3e590d230116a2d)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:24.682005 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7e73e3-0df0-445b-b902-4910ab790b9a None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:24.684060 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 134/268] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:51:24 2022] GET /identity/v3/projects/caf48e96564645f3b3e590d230116a2d => generated 302 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:24.690455 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9c14b0-c54b-475a-ba34-a19cd2e61383 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:24.821382 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9c14b0-c54b-475a-ba34-a19cd2e61383 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:24.875218 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed652500-9f9c-434a-b3b8-bbe2a66ee51e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:24.875893 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed652500-9f9c-434a-b3b8-bbe2a66ee51e None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:24.876192 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed652500-9f9c-434a-b3b8-bbe2a66ee51e None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:24.876476 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed652500-9f9c-434a-b3b8-bbe2a66ee51e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:24.877328 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed652500-9f9c-434a-b3b8-bbe2a66ee51e None admin] RBAC: Adding query filter params (role.id=3b9c36a010504ba7bc63a8ccf0a9bca0, scope.project.id=caf48e96564645f3b3e590d230116a2d, user.id=61be9336f6054259b36e1e8ded1ccc79) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:24.877647 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed652500-9f9c-434a-b3b8-bbe2a66ee51e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:24.880337 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed652500-9f9c-434a-b3b8-bbe2a66ee51e None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:24.890897 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 135/269] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:51:24 2022] GET /identity/v3/role_assignments?user.id=61be9336f6054259b36e1e8ded1ccc79&scope.project.id=caf48e96564645f3b3e590d230116a2d&role.id=3b9c36a010504ba7bc63a8ccf0a9bca0 => generated 608 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:25.381774 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10d67fa-5260-4b29-8fae-400cf02b7d0a None None] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:25.382329 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10d67fa-5260-4b29-8fae-400cf02b7d0a None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:25.382412 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10d67fa-5260-4b29-8fae-400cf02b7d0a None None] PATH_INFO: `/` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:25.384025 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 135/270] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:51:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:51:25.389595 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1da96a1-7c14-4aca-9e86-b65ea8615a9b None None] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:25.390079 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1da96a1-7c14-4aca-9e86-b65ea8615a9b None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:25.390264 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1da96a1-7c14-4aca-9e86-b65ea8615a9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:25.407771 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.auth.core [None req-b1da96a1-7c14-4aca-9e86-b65ea8615a9b None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:51:25.409001 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-b1da96a1-7c14-4aca-9e86-b65ea8615a9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:51:25.466595 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:25.485746 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 136/271] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:51:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:51:25.494682 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec32eb0-1a5f-43c5-9c24-12748d4e600e None None] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:25.494989 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec32eb0-1a5f-43c5-9c24-12748d4e600e None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:25.495279 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec32eb0-1a5f-43c5-9c24-12748d4e600e None None] PATH_INFO: `/` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:25.496686 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 136/272] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:51:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:51:25.502228 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae1c886-dfd7-4c9d-b304-07e0393f2497 None None] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:25.502528 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae1c886-dfd7-4c9d-b304-07e0393f2497 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:25.502820 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae1c886-dfd7-4c9d-b304-07e0393f2497 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:25.513965 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec32eb0-1a5f-43c5-9c24-12748d4e600e None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:25.522082 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.auth.core [None req-fae1c886-dfd7-4c9d-b304-07e0393f2497 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:51:25.523292 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-fae1c886-dfd7-4c9d-b304-07e0393f2497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:51:25.528027 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:25.607720 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 137/273] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:51:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:51:25.634897 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae1c886-dfd7-4c9d-b304-07e0393f2497 None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:25.664929 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec32eb0-1a5f-43c5-9c24-12748d4e600e None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:25.705121 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ede737-6b7d-414b-b787-25ec276da7f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:25.706329 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ede737-6b7d-414b-b787-25ec276da7f7 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:25.706546 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ede737-6b7d-414b-b787-25ec276da7f7 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:25.707001 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ede737-6b7d-414b-b787-25ec276da7f7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:25.708410 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ede737-6b7d-414b-b787-25ec276da7f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:25.710363 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ede737-6b7d-414b-b787-25ec276da7f7 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:25.712710 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 137/274] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:51:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:25.718231 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ede737-6b7d-414b-b787-25ec276da7f7 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:25.746824 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae1c886-dfd7-4c9d-b304-07e0393f2497 None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:25.765309 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:25.786565 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858a7a2e-e854-4e1b-933f-2dd30f99ab96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:25.787691 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858a7a2e-e854-4e1b-933f-2dd30f99ab96 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:25.788154 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858a7a2e-e854-4e1b-933f-2dd30f99ab96 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:25.788694 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858a7a2e-e854-4e1b-933f-2dd30f99ab96 None admin] PATH_INFO: `/v3/roles/member` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:25.801297 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858a7a2e-e854-4e1b-933f-2dd30f99ab96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:51:25.809086 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:25.810909 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-858a7a2e-e854-4e1b-933f-2dd30f99ab96 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:51:25.810909 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:51:25.810909 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:51:25.810909 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:51:25.810909 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:51:25.810909 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:51:25.810909 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:51:25.810909 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:51:25.810909 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:51:25.810909 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:51:25.810909 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:51:25.810909 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:51:25.810909 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:51:25.810909 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:51:25.810909 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:51:25.810909 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:51:25.817513 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:51:25.817513 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:51:25.817513 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:51:25.817513 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:51:25.817513 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:51:25.817513 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:51:25.817513 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:51:25.817513 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:51:25.817513 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:51:25.817513 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:51:25.817513 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:51:25.817513 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:51:25.817513 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:51:25.817513 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:51:25.817513 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:51:25.817513 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:51:25.818566 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:51:25.818566 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:51:25.818566 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:51:25.818566 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:51:25.818566 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:51:25.818566 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:51:25.818566 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:51:25.818566 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:51:25.818566 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:51:25.818566 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:51:25.818566 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:51:25.818566 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:51:25.818566 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 138/275] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:51:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:51:25.822135 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858a7a2e-e854-4e1b-933f-2dd30f99ab96 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:25.836308 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ede737-6b7d-414b-b787-25ec276da7f7 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:25.873346 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f385496-6c50-4aa9-b310-28cc96b82952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:25.874057 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f385496-6c50-4aa9-b310-28cc96b82952 None admin] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:25.874350 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f385496-6c50-4aa9-b310-28cc96b82952 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:25.874634 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f385496-6c50-4aa9-b310-28cc96b82952 None admin] PATH_INFO: `/v3/users` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:25.875435 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f385496-6c50-4aa9-b310-28cc96b82952 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:25.876925 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f385496-6c50-4aa9-b310-28cc96b82952 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:25.900422 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 138/276] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 19 14:51:25 2022] POST /identity/v3/users => generated 293 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 14:51:25.901680 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8b287c-50c6-47f7-b578-d1b28c4a38cd None None] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:25.902274 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8b287c-50c6-47f7-b578-d1b28c4a38cd None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:25.902274 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8b287c-50c6-47f7-b578-d1b28c4a38cd None None] PATH_INFO: `/` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:25.903584 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 139/277] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:51:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:51:25.910056 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f5af4d-dfd8-46f7-a197-cb60f6377aaa None None] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:25.911640 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f5af4d-dfd8-46f7-a197-cb60f6377aaa None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:25.912080 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f5af4d-dfd8-46f7-a197-cb60f6377aaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:25.933516 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.auth.core [None req-55f5af4d-dfd8-46f7-a197-cb60f6377aaa None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:51:25.935203 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.fernet_utils [None req-55f5af4d-dfd8-46f7-a197-cb60f6377aaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:51:25.957162 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858a7a2e-e854-4e1b-933f-2dd30f99ab96 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:26.007372 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cf9949-cce4-414c-803c-ae0c25bb8f7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:26.009047 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cf9949-cce4-414c-803c-ae0c25bb8f7d None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:26.009637 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cf9949-cce4-414c-803c-ae0c25bb8f7d None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:26.010176 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cf9949-cce4-414c-803c-ae0c25bb8f7d None admin] PATH_INFO: `/v3/roles` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:26.011502 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cf9949-cce4-414c-803c-ae0c25bb8f7d None admin] RBAC: Adding query filter params (name=member) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:26.012570 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cf9949-cce4-414c-803c-ae0c25bb8f7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:26.014538 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cf9949-cce4-414c-803c-ae0c25bb8f7d None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:26.027662 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 140/278] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:51:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:51:26.028085 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 139/279] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:51:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:51:26.037191 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f5af4d-dfd8-46f7-a197-cb60f6377aaa None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:26.052680 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cf9949-cce4-414c-803c-ae0c25bb8f7d None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:26.114693 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:26.120696 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:26.168962 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f5af4d-dfd8-46f7-a197-cb60f6377aaa None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:26.204004 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cf9949-cce4-414c-803c-ae0c25bb8f7d None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:26.210533 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dde956d-a764-4427-9980-47e6c32331ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:26.211668 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dde956d-a764-4427-9980-47e6c32331ab None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:26.211796 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dde956d-a764-4427-9980-47e6c32331ab None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:26.211974 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dde956d-a764-4427-9980-47e6c32331ab None admin] PATH_INFO: `/v3/users/cdd1247fb66645729ea4e48a0560c265` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:26.226960 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dde956d-a764-4427-9980-47e6c32331ab None admin] RBAC: Authorizing `identity:get_user(user_id=cdd1247fb66645729ea4e48a0560c265)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:26.229823 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dde956d-a764-4427-9980-47e6c32331ab None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:26.230290 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 141/280] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:51:26 2022] GET /identity/v3/users/cdd1247fb66645729ea4e48a0560c265 => generated 293 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:26.231642 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f9f71b-303b-4c05-983a-f6c96dca128e None None] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:26.231938 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f9f71b-303b-4c05-983a-f6c96dca128e None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:26.232224 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f9f71b-303b-4c05-983a-f6c96dca128e None None] PATH_INFO: `/` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:26.233490 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 142/281] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:51:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:51:26.236471 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f9f71b-303b-4c05-983a-f6c96dca128e None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:26.250190 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64fa8f8-58f1-43a8-818f-8a647cb03581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:26.251004 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64fa8f8-58f1-43a8-818f-8a647cb03581 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:26.251302 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64fa8f8-58f1-43a8-818f-8a647cb03581 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:26.251566 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64fa8f8-58f1-43a8-818f-8a647cb03581 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:26.268496 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64fa8f8-58f1-43a8-818f-8a647cb03581 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:51:26.277575 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-b64fa8f8-58f1-43a8-818f-8a647cb03581 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:51:26.277575 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:51:26.277575 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:51:26.277575 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:51:26.277575 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:51:26.277575 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:51:26.277575 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:51:26.277575 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:51:26.277575 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:51:26.277575 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:51:26.277575 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:51:26.277575 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:51:26.277575 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:51:26.277575 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:51:26.277575 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:51:26.277575 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:51:26.278689 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:51:26.278689 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:51:26.278689 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:51:26.278689 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:51:26.278689 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:51:26.278689 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:51:26.278689 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:51:26.278689 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:51:26.278689 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:51:26.278689 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:51:26.278689 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:51:26.278689 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:51:26.278689 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:51:26.278689 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:51:26.278689 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:51:26.278689 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:51:26.279608 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:51:26.279608 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:51:26.279608 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:51:26.279608 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:51:26.279608 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:51:26.279608 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:51:26.279608 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:51:26.279608 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:51:26.279608 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:51:26.279608 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:51:26.279608 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:51:26.279608 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:51:26.279608 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 140/282] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:51:26 2022] GET /identity/v3/roles/reader => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:51:26.280394 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a1e182-7260-4473-ac06-a501778c969b None None] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:26.281219 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a1e182-7260-4473-ac06-a501778c969b None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:26.281219 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a1e182-7260-4473-ac06-a501778c969b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:26.300406 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.auth.core [None req-83a1e182-7260-4473-ac06-a501778c969b None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:51:26.301639 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-83a1e182-7260-4473-ac06-a501778c969b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:51:26.378959 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f9f71b-303b-4c05-983a-f6c96dca128e None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:26.390660 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 141/283] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:51:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:51:26.392234 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a1e182-7260-4473-ac06-a501778c969b None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:26.406729 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:26.426394 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:26.428169 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845339c5-61b6-4201-bb97-2a9643d54b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:26.429118 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845339c5-61b6-4201-bb97-2a9643d54b94 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:26.429414 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845339c5-61b6-4201-bb97-2a9643d54b94 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:26.429695 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845339c5-61b6-4201-bb97-2a9643d54b94 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:26.430566 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845339c5-61b6-4201-bb97-2a9643d54b94 None admin] RBAC: Adding query filter params (role.id=aed9aa9ef7f7448bb3b40c9ad082d545, user.id=cdd1247fb66645729ea4e48a0560c265, scope.system=all) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:26.430884 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845339c5-61b6-4201-bb97-2a9643d54b94 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:26.432559 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845339c5-61b6-4201-bb97-2a9643d54b94 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:26.441149 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 143/284] 10.222.0.57 () {56 vars in 1308 bytes} [Thu May 19 14:51:26 2022] GET /identity/v3/role_assignments?user.id=cdd1247fb66645729ea4e48a0560c265&scope.system=all&role.id=aed9aa9ef7f7448bb3b40c9ad082d545 => generated 226 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:51:26.442356 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845339c5-61b6-4201-bb97-2a9643d54b94 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:26.517992 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a1e182-7260-4473-ac06-a501778c969b None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:26.559097 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7af2e8-4796-457e-bb6b-aee09cf216f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:26.560190 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7af2e8-4796-457e-bb6b-aee09cf216f0 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:26.560791 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7af2e8-4796-457e-bb6b-aee09cf216f0 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:26.561291 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7af2e8-4796-457e-bb6b-aee09cf216f0 None admin] PATH_INFO: `/v3/roles` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:26.562399 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7af2e8-4796-457e-bb6b-aee09cf216f0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:26.563002 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7af2e8-4796-457e-bb6b-aee09cf216f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:26.564677 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7af2e8-4796-457e-bb6b-aee09cf216f0 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:26.575077 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 142/285] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:51:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:26.581741 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7af2e8-4796-457e-bb6b-aee09cf216f0 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:26.583329 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845339c5-61b6-4201-bb97-2a9643d54b94 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:26.630622 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0890136-659a-45f1-b205-7084769bf532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:26.631466 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0890136-659a-45f1-b205-7084769bf532 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:26.631754 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0890136-659a-45f1-b205-7084769bf532 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:26.632035 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0890136-659a-45f1-b205-7084769bf532 None admin] PATH_INFO: `/v3/roles/member` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:26.646905 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0890136-659a-45f1-b205-7084769bf532 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:51:26.657640 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-a0890136-659a-45f1-b205-7084769bf532 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:51:26.657640 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:51:26.657640 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:51:26.657640 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:51:26.657640 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:51:26.657640 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:51:26.657640 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:51:26.657640 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:51:26.657640 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:51:26.657640 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:51:26.657640 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:51:26.657640 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:51:26.657640 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:51:26.657640 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:51:26.657640 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:51:26.657640 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:51:26.658851 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:51:26.658851 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:51:26.658851 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:51:26.658851 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:51:26.658851 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:51:26.658851 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:51:26.658851 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:51:26.658851 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:51:26.658851 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:51:26.658851 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:51:26.658851 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:51:26.658851 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:51:26.658851 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:51:26.658851 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:51:26.658851 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:51:26.658851 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:51:26.660467 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:51:26.660467 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:51:26.660467 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:51:26.660467 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:51:26.660467 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:51:26.660467 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:51:26.660467 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:51:26.660467 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:51:26.660467 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:51:26.660467 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:51:26.660467 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:51:26.660467 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:51:26.664959 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 144/286] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:51:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:51:26.675037 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0890136-659a-45f1-b205-7084769bf532 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:26.709010 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:26.723539 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7af2e8-4796-457e-bb6b-aee09cf216f0 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:26.753362 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:26.777267 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d07440-a23d-4528-b077-f6dcca6159fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:26.778585 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d07440-a23d-4528-b077-f6dcca6159fc None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:26.778966 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d07440-a23d-4528-b077-f6dcca6159fc None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:26.779179 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d07440-a23d-4528-b077-f6dcca6159fc None admin] PATH_INFO: `/v3/users/a5e2805af9a0450daaecf8b4e12404e3` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:26.780839 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d07440-a23d-4528-b077-f6dcca6159fc None admin] RBAC: Authorizing `identity:get_user(user_id=a5e2805af9a0450daaecf8b4e12404e3)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:26.782357 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d07440-a23d-4528-b077-f6dcca6159fc None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:26.785209 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 143/287] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:51:26 2022] GET /identity/v3/users/a5e2805af9a0450daaecf8b4e12404e3 => generated 297 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:51:26.791121 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d07440-a23d-4528-b077-f6dcca6159fc None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:26.844815 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0890136-659a-45f1-b205-7084769bf532 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:26.920191 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d07440-a23d-4528-b077-f6dcca6159fc None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:26.941082 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be1a5aa-83c5-4921-977e-473158309e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:26.941768 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be1a5aa-83c5-4921-977e-473158309e7d None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:26.942083 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be1a5aa-83c5-4921-977e-473158309e7d None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:26.942371 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be1a5aa-83c5-4921-977e-473158309e7d None admin] PATH_INFO: `/v3/roles` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:26.943199 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be1a5aa-83c5-4921-977e-473158309e7d None admin] RBAC: Adding query filter params (name=member) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:26.943511 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be1a5aa-83c5-4921-977e-473158309e7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:26.945822 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be1a5aa-83c5-4921-977e-473158309e7d None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:26.964288 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 145/288] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:51:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:26.969251 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be1a5aa-83c5-4921-977e-473158309e7d None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:26.980745 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66da9bb-82b8-4a09-81ee-9bbcab73e33f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:26.981630 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66da9bb-82b8-4a09-81ee-9bbcab73e33f None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:26.981952 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66da9bb-82b8-4a09-81ee-9bbcab73e33f None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:26.982292 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66da9bb-82b8-4a09-81ee-9bbcab73e33f None admin] PATH_INFO: `/v3/projects/caf48e96564645f3b3e590d230116a2d` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:26.983599 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66da9bb-82b8-4a09-81ee-9bbcab73e33f None admin] RBAC: Authorizing `identity:get_project(project_id=caf48e96564645f3b3e590d230116a2d)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:26.985057 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66da9bb-82b8-4a09-81ee-9bbcab73e33f None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:26.992334 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 144/289] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:51:26 2022] GET /identity/v3/projects/caf48e96564645f3b3e590d230116a2d => generated 302 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:51:26.994192 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66da9bb-82b8-4a09-81ee-9bbcab73e33f None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:27.051941 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:27.112222 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66da9bb-82b8-4a09-81ee-9bbcab73e33f None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:27.150825 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d68faa7-ae49-4ba6-98a3-951b78688214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:27.151865 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d68faa7-ae49-4ba6-98a3-951b78688214 None admin] REQUEST_METHOD: `PUT` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:27.152160 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d68faa7-ae49-4ba6-98a3-951b78688214 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:27.152443 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d68faa7-ae49-4ba6-98a3-951b78688214 None admin] PATH_INFO: `/v3/projects/caf48e96564645f3b3e590d230116a2d/users/a5e2805af9a0450daaecf8b4e12404e3/roles/e56389e17ad24e20bf3bba1ca7684062` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:27.154734 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d68faa7-ae49-4ba6-98a3-951b78688214 None admin] RBAC: Authorizing `identity:create_grant(project_id=caf48e96564645f3b3e590d230116a2d, user_id=a5e2805af9a0450daaecf8b4e12404e3, role_id=e56389e17ad24e20bf3bba1ca7684062)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:27.156548 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d68faa7-ae49-4ba6-98a3-951b78688214 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:27.181682 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 145/290] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 14:51:26 2022] PUT /identity/v3/projects/caf48e96564645f3b3e590d230116a2d/users/a5e2805af9a0450daaecf8b4e12404e3/roles/e56389e17ad24e20bf3bba1ca7684062 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:51:27.222756 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be1a5aa-83c5-4921-977e-473158309e7d None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:27.231652 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:27.269265 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4099674a-7d1a-4353-b045-2995f307c74f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:27.270681 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4099674a-7d1a-4353-b045-2995f307c74f None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:27.271190 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4099674a-7d1a-4353-b045-2995f307c74f None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:27.271862 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4099674a-7d1a-4353-b045-2995f307c74f None admin] PATH_INFO: `/v3/users/c33895867866448a957a662b630081bf` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:27.273615 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4099674a-7d1a-4353-b045-2995f307c74f None admin] RBAC: Authorizing `identity:get_user(user_id=c33895867866448a957a662b630081bf)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:27.275388 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4099674a-7d1a-4353-b045-2995f307c74f None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:27.277752 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 146/291] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:51:26 2022] GET /identity/v3/users/c33895867866448a957a662b630081bf => generated 297 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:27.285965 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d68faa7-ae49-4ba6-98a3-951b78688214 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:27.410952 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:27.436217 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d68faa7-ae49-4ba6-98a3-951b78688214 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:27.477732 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24486b8a-e735-4e6a-b26c-98bde9b178cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:27.478686 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24486b8a-e735-4e6a-b26c-98bde9b178cd None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:27.479157 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24486b8a-e735-4e6a-b26c-98bde9b178cd None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:27.479600 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24486b8a-e735-4e6a-b26c-98bde9b178cd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:27.564032 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24486b8a-e735-4e6a-b26c-98bde9b178cd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:27.566471 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24486b8a-e735-4e6a-b26c-98bde9b178cd None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:27.687806 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:27.704340 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 146/292] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:51:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:51:27.710965 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4099674a-7d1a-4353-b045-2995f307c74f None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:27.860975 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4099674a-7d1a-4353-b045-2995f307c74f None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:27.896701 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b124fa-e6af-419a-937f-d9b208cdefb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:27.897580 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b124fa-e6af-419a-937f-d9b208cdefb9 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:27.897893 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b124fa-e6af-419a-937f-d9b208cdefb9 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:27.898183 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b124fa-e6af-419a-937f-d9b208cdefb9 None admin] PATH_INFO: `/v3/projects/caf48e96564645f3b3e590d230116a2d` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:27.899384 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b124fa-e6af-419a-937f-d9b208cdefb9 None admin] RBAC: Authorizing `identity:get_project(project_id=caf48e96564645f3b3e590d230116a2d)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:27.900893 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b124fa-e6af-419a-937f-d9b208cdefb9 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:27.903002 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 147/293] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:51:27 2022] GET /identity/v3/projects/caf48e96564645f3b3e590d230116a2d => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:27.908383 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24486b8a-e735-4e6a-b26c-98bde9b178cd None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:28.025055 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24486b8a-e735-4e6a-b26c-98bde9b178cd None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:28.061211 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d0db17-4261-49fd-b0da-896464a5e78d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:28.061967 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d0db17-4261-49fd-b0da-896464a5e78d None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:28.062266 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d0db17-4261-49fd-b0da-896464a5e78d None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:28.062556 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d0db17-4261-49fd-b0da-896464a5e78d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:28.063473 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d0db17-4261-49fd-b0da-896464a5e78d None admin] RBAC: Adding query filter params (role.id=aed9aa9ef7f7448bb3b40c9ad082d545, scope.project.id=caf48e96564645f3b3e590d230116a2d, user.id=c33895867866448a957a662b630081bf) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:28.063791 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d0db17-4261-49fd-b0da-896464a5e78d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:28.065374 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d0db17-4261-49fd-b0da-896464a5e78d None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:28.072705 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 147/294] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:51:27 2022] GET /identity/v3/role_assignments?user.id=c33895867866448a957a662b630081bf&scope.project.id=caf48e96564645f3b3e590d230116a2d&role.id=aed9aa9ef7f7448bb3b40c9ad082d545 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:28.767475 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df8127c-8911-44ea-8930-a5547d8f99ee None None] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:28.768972 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df8127c-8911-44ea-8930-a5547d8f99ee None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:28.769510 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df8127c-8911-44ea-8930-a5547d8f99ee None None] PATH_INFO: `/` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:28.771685 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 148/295] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:51:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:51:28.776928 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018187ae-2a75-4bd6-b4b8-384b7881cd87 None None] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:28.777811 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018187ae-2a75-4bd6-b4b8-384b7881cd87 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:28.778343 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018187ae-2a75-4bd6-b4b8-384b7881cd87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:28.798646 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.auth.core [None req-018187ae-2a75-4bd6-b4b8-384b7881cd87 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:51:28.800952 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-018187ae-2a75-4bd6-b4b8-384b7881cd87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:51:28.852673 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4177a761-39b7-48b3-b6a6-113ab8064c5a None None] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:28.853332 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4177a761-39b7-48b3-b6a6-113ab8064c5a None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:28.853836 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4177a761-39b7-48b3-b6a6-113ab8064c5a None None] PATH_INFO: `/` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:28.856990 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 149/296] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:51:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:51:28.861276 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e980d376-10aa-4b74-93db-7f71ed84291f None None] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:28.861969 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:28.863436 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e980d376-10aa-4b74-93db-7f71ed84291f None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:28.863436 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e980d376-10aa-4b74-93db-7f71ed84291f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:28.883221 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 148/297] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:51:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:51:28.887374 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.auth.core [None req-e980d376-10aa-4b74-93db-7f71ed84291f None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:51:28.888594 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.fernet_utils [None req-e980d376-10aa-4b74-93db-7f71ed84291f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:51:28.905971 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018187ae-2a75-4bd6-b4b8-384b7881cd87 None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:28.935303 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:28.973430 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 150/298] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:51:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:51:28.997614 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e980d376-10aa-4b74-93db-7f71ed84291f None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:29.038156 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018187ae-2a75-4bd6-b4b8-384b7881cd87 None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:29.083686 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d41d21-79cc-4952-ba39-bc48a8a8e740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:29.084902 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d41d21-79cc-4952-ba39-bc48a8a8e740 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:29.085433 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d41d21-79cc-4952-ba39-bc48a8a8e740 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:29.085971 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d41d21-79cc-4952-ba39-bc48a8a8e740 None admin] PATH_INFO: `/v3/domains/default` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:29.087391 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d41d21-79cc-4952-ba39-bc48a8a8e740 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:29.089278 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d41d21-79cc-4952-ba39-bc48a8a8e740 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:29.091537 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 149/299] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:51:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:51:29.101537 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d41d21-79cc-4952-ba39-bc48a8a8e740 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:29.116983 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e980d376-10aa-4b74-93db-7f71ed84291f None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:29.150091 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:29.158341 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df59b161-c99b-43b9-ba0e-c86049fa565a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:29.159723 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df59b161-c99b-43b9-ba0e-c86049fa565a None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:29.160547 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df59b161-c99b-43b9-ba0e-c86049fa565a None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:29.160836 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df59b161-c99b-43b9-ba0e-c86049fa565a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:29.177795 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df59b161-c99b-43b9-ba0e-c86049fa565a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:51:29.189171 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-df59b161-c99b-43b9-ba0e-c86049fa565a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:51:29.189171 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:51:29.189171 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:51:29.189171 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:51:29.189171 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:51:29.189171 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:51:29.189171 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:51:29.189171 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:51:29.189171 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:51:29.189171 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:51:29.189171 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:51:29.189171 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:51:29.189171 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:51:29.189171 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:51:29.189171 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:51:29.189171 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:51:29.190252 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:51:29.190252 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:51:29.190252 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:51:29.190252 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:51:29.190252 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:51:29.190252 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:51:29.190252 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:51:29.190252 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:51:29.190252 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:51:29.190252 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:51:29.190252 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:51:29.190252 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:51:29.190252 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:51:29.190252 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:51:29.190252 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:51:29.190252 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:51:29.192157 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:51:29.192157 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:51:29.192157 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:51:29.192157 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:51:29.192157 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:51:29.192157 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:51:29.192157 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:51:29.192157 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:51:29.192157 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:51:29.192157 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:51:29.192157 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:51:29.192157 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:51:29.192157 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 151/300] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:51:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:51:29.202922 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df59b161-c99b-43b9-ba0e-c86049fa565a None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:29.215569 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d41d21-79cc-4952-ba39-bc48a8a8e740 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:29.245049 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:29.255783 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede96799-febd-4ad6-9a7f-75bd57c9db90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:29.256778 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede96799-febd-4ad6-9a7f-75bd57c9db90 None admin] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:29.257278 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede96799-febd-4ad6-9a7f-75bd57c9db90 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:29.257729 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede96799-febd-4ad6-9a7f-75bd57c9db90 None admin] PATH_INFO: `/v3/groups` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:29.258662 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede96799-febd-4ad6-9a7f-75bd57c9db90 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:29.260337 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede96799-febd-4ad6-9a7f-75bd57c9db90 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:29.280185 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 150/301] 10.222.0.57 () {60 vars in 1134 bytes} [Thu May 19 14:51:29 2022] POST /identity/v3/groups => generated 229 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 14:51:29.346929 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df59b161-c99b-43b9-ba0e-c86049fa565a None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:29.372694 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c7c967-7aa0-4eef-8f26-76febf75dc94 None None] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:29.373176 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c7c967-7aa0-4eef-8f26-76febf75dc94 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:29.373570 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c7c967-7aa0-4eef-8f26-76febf75dc94 None None] PATH_INFO: `/` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:29.375477 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 151/302] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:51:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:51:29.382117 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8ce697-8ca2-4c46-9231-196c7381ec44 None None] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:29.382547 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8ce697-8ca2-4c46-9231-196c7381ec44 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:29.382937 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8ce697-8ca2-4c46-9231-196c7381ec44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:29.392501 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6573ab4c-ae1f-4f82-a0ee-7e16ec045199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:29.393643 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6573ab4c-ae1f-4f82-a0ee-7e16ec045199 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:29.394281 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6573ab4c-ae1f-4f82-a0ee-7e16ec045199 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:29.394754 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6573ab4c-ae1f-4f82-a0ee-7e16ec045199 None admin] PATH_INFO: `/v3/roles` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:29.395716 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6573ab4c-ae1f-4f82-a0ee-7e16ec045199 None admin] RBAC: Adding query filter params (name=reader) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:29.396194 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6573ab4c-ae1f-4f82-a0ee-7e16ec045199 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:29.398769 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6573ab4c-ae1f-4f82-a0ee-7e16ec045199 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:29.403275 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.auth.core [None req-7b8ce697-8ca2-4c46-9231-196c7381ec44 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:51:29.404648 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-7b8ce697-8ca2-4c46-9231-196c7381ec44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:51:29.416895 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 152/303] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:51:29 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:29.418764 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6573ab4c-ae1f-4f82-a0ee-7e16ec045199 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:29.500146 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 152/304] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:51:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:51:29.523428 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8ce697-8ca2-4c46-9231-196c7381ec44 None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:29.555547 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:29.563376 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6573ab4c-ae1f-4f82-a0ee-7e16ec045199 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:29.584842 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:29.615339 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bb0aff-5fd3-47c8-b677-37ab344e3eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:29.616602 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bb0aff-5fd3-47c8-b677-37ab344e3eb8 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:29.616968 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bb0aff-5fd3-47c8-b677-37ab344e3eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:29.617329 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bb0aff-5fd3-47c8-b677-37ab344e3eb8 None admin] PATH_INFO: `/v3/users/6074527c5b9e4554a5f1182ebf0536be` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:29.637305 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bb0aff-5fd3-47c8-b677-37ab344e3eb8 None admin] RBAC: Authorizing `identity:get_user(user_id=6074527c5b9e4554a5f1182ebf0536be)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:29.639211 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bb0aff-5fd3-47c8-b677-37ab344e3eb8 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:29.646405 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 153/305] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:51:29 2022] GET /identity/v3/users/6074527c5b9e4554a5f1182ebf0536be => generated 293 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:51:29.649223 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bb0aff-5fd3-47c8-b677-37ab344e3eb8 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:29.671064 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8ce697-8ca2-4c46-9231-196c7381ec44 None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:29.728118 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d80a958-fffa-4d20-9f29-31a1212390d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:29.729213 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d80a958-fffa-4d20-9f29-31a1212390d4 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:29.729510 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d80a958-fffa-4d20-9f29-31a1212390d4 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:29.729797 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d80a958-fffa-4d20-9f29-31a1212390d4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:29.745598 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d80a958-fffa-4d20-9f29-31a1212390d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:51:29.756786 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-6d80a958-fffa-4d20-9f29-31a1212390d4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:51:29.756786 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:51:29.756786 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:51:29.756786 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:51:29.756786 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:51:29.756786 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:51:29.756786 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:51:29.756786 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:51:29.756786 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:51:29.756786 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:51:29.756786 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:51:29.756786 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:51:29.756786 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:51:29.756786 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:51:29.756786 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:51:29.756786 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:51:29.758112 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:51:29.758112 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:51:29.758112 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:51:29.758112 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:51:29.758112 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:51:29.758112 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:51:29.758112 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:51:29.758112 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:51:29.758112 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:51:29.758112 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:51:29.758112 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:51:29.758112 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:51:29.758112 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:51:29.758112 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:51:29.758112 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:51:29.758112 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:51:29.759131 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:51:29.759131 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:51:29.759131 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:51:29.759131 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:51:29.759131 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:51:29.759131 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:51:29.759131 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:51:29.759131 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:51:29.759131 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:51:29.759131 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:51:29.759131 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:51:29.759131 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:51:29.759131 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 153/306] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:51:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:51:29.765510 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d80a958-fffa-4d20-9f29-31a1212390d4 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:29.787553 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bb0aff-5fd3-47c8-b677-37ab344e3eb8 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:29.835154 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b7329c-5234-4841-a86e-6ba9b45d23b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:29.836242 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b7329c-5234-4841-a86e-6ba9b45d23b1 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:29.836560 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b7329c-5234-4841-a86e-6ba9b45d23b1 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:29.836845 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b7329c-5234-4841-a86e-6ba9b45d23b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:29.838043 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b7329c-5234-4841-a86e-6ba9b45d23b1 None admin] RBAC: Adding query filter params (role.id=e56389e17ad24e20bf3bba1ca7684062, user.id=6074527c5b9e4554a5f1182ebf0536be, scope.system=all) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:29.838370 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b7329c-5234-4841-a86e-6ba9b45d23b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:29.840110 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b7329c-5234-4841-a86e-6ba9b45d23b1 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:29.853873 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 154/307] 10.222.0.57 () {56 vars in 1308 bytes} [Thu May 19 14:51:29 2022] GET /identity/v3/role_assignments?user.id=6074527c5b9e4554a5f1182ebf0536be&scope.system=all&role.id=e56389e17ad24e20bf3bba1ca7684062 => generated 226 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:29.880118 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2855d342-7f42-40e2-876e-af3d37f8b3b4 None None] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:29.880478 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2855d342-7f42-40e2-876e-af3d37f8b3b4 None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:29.882044 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2855d342-7f42-40e2-876e-af3d37f8b3b4 None None] PATH_INFO: `/` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:29.882604 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 155/308] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:51:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:51:29.887712 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:29.889482 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6c0f5f-2eb9-4a87-b02d-da659de16cad None None] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:29.889765 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6c0f5f-2eb9-4a87-b02d-da659de16cad None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:29.890085 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6c0f5f-2eb9-4a87-b02d-da659de16cad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:29.892140 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d80a958-fffa-4d20-9f29-31a1212390d4 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:29.909023 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.auth.core [None req-1a6c0f5f-2eb9-4a87-b02d-da659de16cad None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:51:29.911836 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.fernet_utils [None req-1a6c0f5f-2eb9-4a87-b02d-da659de16cad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:51:29.938928 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:29.941069 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bc13bd-6d5b-4e82-a714-85eeb35a360f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:29.942263 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bc13bd-6d5b-4e82-a714-85eeb35a360f None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:29.942720 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bc13bd-6d5b-4e82-a714-85eeb35a360f None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:29.943330 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bc13bd-6d5b-4e82-a714-85eeb35a360f None admin] PATH_INFO: `/v3/roles` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:29.944742 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bc13bd-6d5b-4e82-a714-85eeb35a360f None admin] RBAC: Adding query filter params (name=member) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:29.945522 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bc13bd-6d5b-4e82-a714-85eeb35a360f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:29.947926 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bc13bd-6d5b-4e82-a714-85eeb35a360f None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:29.959454 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 154/309] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:51:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:29.965595 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bc13bd-6d5b-4e82-a714-85eeb35a360f None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:30.017595 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 156/310] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:51:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:51:30.040321 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6c0f5f-2eb9-4a87-b02d-da659de16cad None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:30.101822 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bc13bd-6d5b-4e82-a714-85eeb35a360f None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:30.147969 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dea9912-daba-4b72-a2c6-da7a2e395630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:30.148983 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dea9912-daba-4b72-a2c6-da7a2e395630 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:30.149281 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dea9912-daba-4b72-a2c6-da7a2e395630 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:30.149571 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dea9912-daba-4b72-a2c6-da7a2e395630 None admin] PATH_INFO: `/v3/users/cdd1247fb66645729ea4e48a0560c265` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:30.150217 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6c0f5f-2eb9-4a87-b02d-da659de16cad None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:30.150852 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dea9912-daba-4b72-a2c6-da7a2e395630 None admin] RBAC: Authorizing `identity:get_user(user_id=cdd1247fb66645729ea4e48a0560c265)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:30.153526 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dea9912-daba-4b72-a2c6-da7a2e395630 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:30.155653 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 155/311] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:51:29 2022] GET /identity/v3/users/cdd1247fb66645729ea4e48a0560c265 => generated 293 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:51:30.162350 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dea9912-daba-4b72-a2c6-da7a2e395630 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:30.194754 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e925a763-6024-483b-8410-044bf1fca603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:30.195693 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e925a763-6024-483b-8410-044bf1fca603 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:30.196096 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e925a763-6024-483b-8410-044bf1fca603 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:30.196704 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e925a763-6024-483b-8410-044bf1fca603 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:30.203838 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:30.211593 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e925a763-6024-483b-8410-044bf1fca603 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:51:30.220124 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-e925a763-6024-483b-8410-044bf1fca603 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:51:30.220124 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:51:30.220124 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:51:30.220124 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:51:30.220124 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:51:30.220124 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:51:30.220124 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:51:30.220124 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:51:30.220124 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:51:30.220124 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:51:30.220124 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:51:30.220124 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:51:30.220124 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:51:30.220124 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:51:30.220124 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:51:30.220124 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:51:30.221223 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:51:30.221223 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:51:30.221223 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:51:30.221223 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:51:30.221223 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:51:30.221223 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:51:30.221223 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:51:30.221223 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:51:30.221223 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:51:30.221223 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:51:30.221223 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:51:30.221223 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:51:30.221223 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:51:30.221223 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:51:30.221223 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:51:30.221223 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:51:30.222507 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:51:30.222507 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:51:30.222507 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:51:30.222507 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:51:30.222507 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:51:30.222507 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:51:30.222507 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:51:30.222507 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:51:30.222507 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:51:30.222507 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:51:30.222507 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:51:30.222507 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:51:30.222507 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 157/312] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:51:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:51:30.227306 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e925a763-6024-483b-8410-044bf1fca603 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:30.239926 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:30.299406 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dea9912-daba-4b72-a2c6-da7a2e395630 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:30.339021 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daebe6c-57f3-4159-afd2-f863177b8e31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:30.339934 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daebe6c-57f3-4159-afd2-f863177b8e31 None admin] REQUEST_METHOD: `PUT` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:30.340240 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daebe6c-57f3-4159-afd2-f863177b8e31 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:30.340532 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daebe6c-57f3-4159-afd2-f863177b8e31 None admin] PATH_INFO: `/v3/system/users/cdd1247fb66645729ea4e48a0560c265/roles/aed9aa9ef7f7448bb3b40c9ad082d545` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:30.342096 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daebe6c-57f3-4159-afd2-f863177b8e31 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=cdd1247fb66645729ea4e48a0560c265, role_id=aed9aa9ef7f7448bb3b40c9ad082d545)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:30.343504 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daebe6c-57f3-4159-afd2-f863177b8e31 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:30.357889 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 156/313] 10.222.0.57 () {58 vars in 1334 bytes} [Thu May 19 14:51:30 2022] PUT /identity/v3/system/users/cdd1247fb66645729ea4e48a0560c265/roles/aed9aa9ef7f7448bb3b40c9ad082d545 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:51:30.390774 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e925a763-6024-483b-8410-044bf1fca603 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:30.429135 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98c3e9b-70a0-422c-95e4-7422227dcb74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:30.429831 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98c3e9b-70a0-422c-95e4-7422227dcb74 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:30.430181 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98c3e9b-70a0-422c-95e4-7422227dcb74 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:30.430430 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98c3e9b-70a0-422c-95e4-7422227dcb74 None admin] PATH_INFO: `/v3/roles` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:30.431258 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98c3e9b-70a0-422c-95e4-7422227dcb74 None admin] RBAC: Adding query filter params (name=reader) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:30.431568 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98c3e9b-70a0-422c-95e4-7422227dcb74 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:30.433002 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98c3e9b-70a0-422c-95e4-7422227dcb74 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:30.443405 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 158/314] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:51:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:51:30.449527 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daebe6c-57f3-4159-afd2-f863177b8e31 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:30.572724 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:30.584877 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daebe6c-57f3-4159-afd2-f863177b8e31 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:30.622286 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9fabf0-98bf-43ae-8e22-5c37dd6ad49c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:30.623179 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9fabf0-98bf-43ae-8e22-5c37dd6ad49c None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:30.623471 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9fabf0-98bf-43ae-8e22-5c37dd6ad49c None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:30.623756 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9fabf0-98bf-43ae-8e22-5c37dd6ad49c None admin] PATH_INFO: `/v3/users/a5e2805af9a0450daaecf8b4e12404e3` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:30.624983 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9fabf0-98bf-43ae-8e22-5c37dd6ad49c None admin] RBAC: Authorizing `identity:get_user(user_id=a5e2805af9a0450daaecf8b4e12404e3)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:30.626517 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9fabf0-98bf-43ae-8e22-5c37dd6ad49c None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:30.629268 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 157/315] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:51:30 2022] GET /identity/v3/users/a5e2805af9a0450daaecf8b4e12404e3 => generated 297 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:30.634843 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98c3e9b-70a0-422c-95e4-7422227dcb74 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:30.712853 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:30.752440 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98c3e9b-70a0-422c-95e4-7422227dcb74 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:30.790258 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3bc7ae-8cc3-42a2-977e-97b68aa3b832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:30.791742 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3bc7ae-8cc3-42a2-977e-97b68aa3b832 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:30.792265 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3bc7ae-8cc3-42a2-977e-97b68aa3b832 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:30.792777 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3bc7ae-8cc3-42a2-977e-97b68aa3b832 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:30.865918 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3bc7ae-8cc3-42a2-977e-97b68aa3b832 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:30.867770 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3bc7ae-8cc3-42a2-977e-97b68aa3b832 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:30.940754 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:30.983440 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 159/316] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:51:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:51:30.990073 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9fabf0-98bf-43ae-8e22-5c37dd6ad49c None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:31.126613 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9fabf0-98bf-43ae-8e22-5c37dd6ad49c None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:31.166199 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f6f29b-45d8-4eb4-9254-d887bd42430e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:31.168763 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f6f29b-45d8-4eb4-9254-d887bd42430e None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:31.168763 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f6f29b-45d8-4eb4-9254-d887bd42430e None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:31.168763 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f6f29b-45d8-4eb4-9254-d887bd42430e None admin] PATH_INFO: `/v3/projects/caf48e96564645f3b3e590d230116a2d` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:31.170647 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f6f29b-45d8-4eb4-9254-d887bd42430e None admin] RBAC: Authorizing `identity:get_project(project_id=caf48e96564645f3b3e590d230116a2d)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:31.172169 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f6f29b-45d8-4eb4-9254-d887bd42430e None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:31.173736 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 158/317] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:51:30 2022] GET /identity/v3/projects/caf48e96564645f3b3e590d230116a2d => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:31.183330 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3bc7ae-8cc3-42a2-977e-97b68aa3b832 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:31.327075 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3bc7ae-8cc3-42a2-977e-97b68aa3b832 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:31.369066 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fa4619-bd68-4363-9a37-837c53d46e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:31.370164 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fa4619-bd68-4363-9a37-837c53d46e5d None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:31.370678 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fa4619-bd68-4363-9a37-837c53d46e5d None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:31.372711 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fa4619-bd68-4363-9a37-837c53d46e5d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:31.374748 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fa4619-bd68-4363-9a37-837c53d46e5d None admin] RBAC: Adding query filter params (role.id=e56389e17ad24e20bf3bba1ca7684062, scope.project.id=caf48e96564645f3b3e590d230116a2d, user.id=a5e2805af9a0450daaecf8b4e12404e3) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:31.375302 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fa4619-bd68-4363-9a37-837c53d46e5d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:31.378573 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fa4619-bd68-4363-9a37-837c53d46e5d None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:31.386307 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 160/318] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:51:31 2022] GET /identity/v3/role_assignments?user.id=a5e2805af9a0450daaecf8b4e12404e3&scope.project.id=caf48e96564645f3b3e590d230116a2d&role.id=e56389e17ad24e20bf3bba1ca7684062 => generated 608 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:31.717949 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb38711e-70e7-4cff-a389-cd92268a18c9 None None] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:31.718303 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb38711e-70e7-4cff-a389-cd92268a18c9 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:31.718527 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb38711e-70e7-4cff-a389-cd92268a18c9 None None] PATH_INFO: `/` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:31.720687 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 159/319] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:51:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:51:31.727049 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04243f6-e0de-4495-b27a-2bda0675814d None None] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:31.727349 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04243f6-e0de-4495-b27a-2bda0675814d None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:31.727682 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04243f6-e0de-4495-b27a-2bda0675814d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:31.745395 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.auth.core [None req-e04243f6-e0de-4495-b27a-2bda0675814d None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:51:31.747335 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.fernet_utils [None req-e04243f6-e0de-4495-b27a-2bda0675814d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:51:31.777483 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:31.832082 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 161/320] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:51:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:51:31.856262 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb38711e-70e7-4cff-a389-cd92268a18c9 None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:31.889734 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:31.970555 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb38711e-70e7-4cff-a389-cd92268a18c9 None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:32.004978 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6f58f9-0272-4093-b3a4-715a7777167a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:32.006009 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6f58f9-0272-4093-b3a4-715a7777167a None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:32.006009 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6f58f9-0272-4093-b3a4-715a7777167a None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:32.006364 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6f58f9-0272-4093-b3a4-715a7777167a None admin] PATH_INFO: `/v3/domains/default` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:32.007522 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6f58f9-0272-4093-b3a4-715a7777167a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:32.009032 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6f58f9-0272-4093-b3a4-715a7777167a None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:32.010924 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 160/321] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:51:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:51:32.016357 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04243f6-e0de-4495-b27a-2bda0675814d None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:32.125463 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04243f6-e0de-4495-b27a-2bda0675814d None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:32.164912 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d153d00d-b493-43d2-b62b-72dc7e5879aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:32.165595 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d153d00d-b493-43d2-b62b-72dc7e5879aa None admin] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:32.165912 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d153d00d-b493-43d2-b62b-72dc7e5879aa None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:32.166246 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d153d00d-b493-43d2-b62b-72dc7e5879aa None admin] PATH_INFO: `/v3/groups` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:32.166966 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d153d00d-b493-43d2-b62b-72dc7e5879aa None admin] RBAC: Authorizing `identity:create_group()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:32.168586 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d153d00d-b493-43d2-b62b-72dc7e5879aa None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:32.184207 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 162/322] 10.222.0.57 () {60 vars in 1134 bytes} [Thu May 19 14:51:32 2022] POST /identity/v3/groups => generated 226 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:51:32.305281 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ae9878-4562-4995-9f62-8641625c960e None None] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:32.305493 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ae9878-4562-4995-9f62-8641625c960e None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:32.305774 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ae9878-4562-4995-9f62-8641625c960e None None] PATH_INFO: `/` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:32.307417 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 161/323] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:51:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:51:32.313101 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c32887-de73-4644-8cc8-54cbf939d6ba None None] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:32.313425 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c32887-de73-4644-8cc8-54cbf939d6ba None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:32.313679 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c32887-de73-4644-8cc8-54cbf939d6ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:32.333488 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.auth.core [None req-76c32887-de73-4644-8cc8-54cbf939d6ba None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:51:32.334638 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.fernet_utils [None req-76c32887-de73-4644-8cc8-54cbf939d6ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:51:32.354176 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:32.411656 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 163/324] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:51:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:51:32.434207 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ae9878-4562-4995-9f62-8641625c960e None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:32.545108 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:32.553961 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ae9878-4562-4995-9f62-8641625c960e None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:32.595382 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964414d2-1ab5-4906-8330-f4d9ba8656d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:32.596147 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964414d2-1ab5-4906-8330-f4d9ba8656d3 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:32.596483 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964414d2-1ab5-4906-8330-f4d9ba8656d3 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:32.596989 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964414d2-1ab5-4906-8330-f4d9ba8656d3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:32.610427 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964414d2-1ab5-4906-8330-f4d9ba8656d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:51:32.619437 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-964414d2-1ab5-4906-8330-f4d9ba8656d3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:51:32.619437 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:51:32.619437 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:51:32.619437 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:51:32.619437 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:51:32.619437 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:51:32.619437 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:51:32.619437 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:51:32.619437 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:51:32.619437 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:51:32.619437 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:51:32.619437 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:51:32.619437 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:51:32.619437 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:51:32.619437 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:51:32.619437 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:51:32.620624 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:51:32.620624 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:51:32.620624 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:51:32.620624 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:51:32.620624 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:51:32.620624 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:51:32.620624 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:51:32.620624 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:51:32.620624 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:51:32.620624 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:51:32.620624 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:51:32.620624 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:51:32.620624 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:51:32.620624 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:51:32.620624 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:51:32.620624 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:51:32.621637 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:51:32.621637 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:51:32.621637 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:51:32.621637 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:51:32.621637 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:51:32.621637 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:51:32.621637 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:51:32.621637 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:51:32.621637 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:51:32.621637 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:51:32.621637 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:51:32.621637 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:51:32.621637 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 162/325] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:51:32 2022] GET /identity/v3/roles/reader => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:51:32.626839 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c32887-de73-4644-8cc8-54cbf939d6ba None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:32.748100 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c32887-de73-4644-8cc8-54cbf939d6ba None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:32.784575 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a02d35f-4ddc-40fd-b0c5-6efb85f0fadb None None] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:32.784876 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a02d35f-4ddc-40fd-b0c5-6efb85f0fadb None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:32.785244 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a02d35f-4ddc-40fd-b0c5-6efb85f0fadb None None] PATH_INFO: `/` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:32.788855 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 163/326] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:51:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:51:32.790707 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059963c2-75c6-45ca-92f2-7d3bd34b3508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:32.791395 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059963c2-75c6-45ca-92f2-7d3bd34b3508 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:32.792004 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059963c2-75c6-45ca-92f2-7d3bd34b3508 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:32.792292 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059963c2-75c6-45ca-92f2-7d3bd34b3508 None admin] PATH_INFO: `/v3/roles` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:32.793066 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059963c2-75c6-45ca-92f2-7d3bd34b3508 None admin] RBAC: Adding query filter params (name=reader) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:32.793191 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7217f22-5098-4f6e-b9dc-def90d0e42da None None] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:32.793385 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059963c2-75c6-45ca-92f2-7d3bd34b3508 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:32.793500 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7217f22-5098-4f6e-b9dc-def90d0e42da None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:32.793728 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7217f22-5098-4f6e-b9dc-def90d0e42da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:32.794850 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059963c2-75c6-45ca-92f2-7d3bd34b3508 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:32.806044 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 164/327] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:51:32 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:32.811751 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059963c2-75c6-45ca-92f2-7d3bd34b3508 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:32.812871 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.auth.core [None req-f7217f22-5098-4f6e-b9dc-def90d0e42da None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:51:32.814252 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-f7217f22-5098-4f6e-b9dc-def90d0e42da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:51:32.832987 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:32.917947 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 164/328] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:51:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:51:32.941445 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7217f22-5098-4f6e-b9dc-def90d0e42da None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:32.943454 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059963c2-75c6-45ca-92f2-7d3bd34b3508 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:32.980538 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9c5406-c1a0-456a-adcd-a9b3f8fd4c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:32.981458 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9c5406-c1a0-456a-adcd-a9b3f8fd4c41 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:32.981749 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9c5406-c1a0-456a-adcd-a9b3f8fd4c41 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:32.982070 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9c5406-c1a0-456a-adcd-a9b3f8fd4c41 None admin] PATH_INFO: `/v3/users/6074527c5b9e4554a5f1182ebf0536be` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:32.983376 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9c5406-c1a0-456a-adcd-a9b3f8fd4c41 None admin] RBAC: Authorizing `identity:get_user(user_id=6074527c5b9e4554a5f1182ebf0536be)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:32.984963 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9c5406-c1a0-456a-adcd-a9b3f8fd4c41 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:32.987200 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 165/329] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:51:32 2022] GET /identity/v3/users/6074527c5b9e4554a5f1182ebf0536be => generated 293 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:51:32.994403 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9c5406-c1a0-456a-adcd-a9b3f8fd4c41 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:33.035255 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:33.052395 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7217f22-5098-4f6e-b9dc-def90d0e42da None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:33.091963 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:33.093410 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99310d58-237e-4e26-ac94-7b21f5e5491a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:33.094353 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99310d58-237e-4e26-ac94-7b21f5e5491a None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:33.095892 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99310d58-237e-4e26-ac94-7b21f5e5491a None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:33.095892 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99310d58-237e-4e26-ac94-7b21f5e5491a None admin] PATH_INFO: `/v3/roles/member` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:33.108792 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99310d58-237e-4e26-ac94-7b21f5e5491a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:51:33.114023 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9c5406-c1a0-456a-adcd-a9b3f8fd4c41 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:33.122541 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-99310d58-237e-4e26-ac94-7b21f5e5491a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:51:33.122541 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:51:33.122541 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:51:33.122541 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:51:33.122541 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:51:33.122541 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:51:33.122541 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:51:33.122541 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:51:33.122541 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:51:33.122541 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:51:33.122541 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:51:33.122541 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:51:33.122541 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:51:33.122541 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:51:33.122541 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:51:33.122541 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:51:33.123622 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:51:33.123622 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:51:33.123622 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:51:33.123622 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:51:33.123622 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:51:33.123622 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:51:33.123622 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:51:33.123622 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:51:33.123622 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:51:33.123622 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:51:33.123622 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:51:33.123622 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:51:33.123622 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:51:33.123622 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:51:33.123622 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:51:33.123622 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:51:33.124621 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:51:33.124621 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:51:33.124621 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:51:33.124621 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:51:33.124621 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:51:33.124621 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:51:33.124621 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:51:33.124621 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:51:33.124621 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:51:33.124621 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:51:33.124621 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:51:33.124621 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:51:33.127349 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 165/330] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:51:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:51:33.134493 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99310d58-237e-4e26-ac94-7b21f5e5491a None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:33.153028 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e6c06e-4642-4af1-9c57-4139f1687ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:33.153709 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e6c06e-4642-4af1-9c57-4139f1687ae5 None admin] REQUEST_METHOD: `PUT` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:33.154037 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e6c06e-4642-4af1-9c57-4139f1687ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:33.154323 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e6c06e-4642-4af1-9c57-4139f1687ae5 None admin] PATH_INFO: `/v3/system/users/6074527c5b9e4554a5f1182ebf0536be/roles/e56389e17ad24e20bf3bba1ca7684062` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:33.155862 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e6c06e-4642-4af1-9c57-4139f1687ae5 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=6074527c5b9e4554a5f1182ebf0536be, role_id=e56389e17ad24e20bf3bba1ca7684062)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:33.157290 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e6c06e-4642-4af1-9c57-4139f1687ae5 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:33.170282 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 166/331] 10.222.0.57 () {58 vars in 1334 bytes} [Thu May 19 14:51:32 2022] PUT /identity/v3/system/users/6074527c5b9e4554a5f1182ebf0536be/roles/e56389e17ad24e20bf3bba1ca7684062 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 14:51:33.327635 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99310d58-237e-4e26-ac94-7b21f5e5491a None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:33.381510 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4486ae11-edb0-4176-94f0-d515e3b9733d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:33.382243 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4486ae11-edb0-4176-94f0-d515e3b9733d None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:33.382536 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4486ae11-edb0-4176-94f0-d515e3b9733d None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:33.382824 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4486ae11-edb0-4176-94f0-d515e3b9733d None admin] PATH_INFO: `/v3/roles` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:33.383612 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4486ae11-edb0-4176-94f0-d515e3b9733d None admin] RBAC: Adding query filter params (name=member) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:33.383935 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4486ae11-edb0-4176-94f0-d515e3b9733d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:33.385378 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4486ae11-edb0-4176-94f0-d515e3b9733d None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:33.399168 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 166/332] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:51:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:51:33.405972 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e6c06e-4642-4af1-9c57-4139f1687ae5 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:33.555725 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e6c06e-4642-4af1-9c57-4139f1687ae5 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:33.613303 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6517a280-3c98-4123-a8f5-5031ec7692a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:33.614219 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6517a280-3c98-4123-a8f5-5031ec7692a7 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:33.614513 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6517a280-3c98-4123-a8f5-5031ec7692a7 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:33.614798 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6517a280-3c98-4123-a8f5-5031ec7692a7 None admin] PATH_INFO: `/v3/users/cdd1247fb66645729ea4e48a0560c265` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:33.618596 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6517a280-3c98-4123-a8f5-5031ec7692a7 None admin] RBAC: Authorizing `identity:get_user(user_id=cdd1247fb66645729ea4e48a0560c265)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:33.620217 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6517a280-3c98-4123-a8f5-5031ec7692a7 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:33.621933 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 167/333] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:51:33 2022] GET /identity/v3/users/cdd1247fb66645729ea4e48a0560c265 => generated 293 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:33.629093 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4486ae11-edb0-4176-94f0-d515e3b9733d None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:33.647199 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:33.782242 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4486ae11-edb0-4176-94f0-d515e3b9733d None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:33.820736 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8f99a9-2257-41b7-9bf8-010a3346dc9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:33.821591 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8f99a9-2257-41b7-9bf8-010a3346dc9c None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:33.821907 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8f99a9-2257-41b7-9bf8-010a3346dc9c None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:33.822195 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8f99a9-2257-41b7-9bf8-010a3346dc9c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:33.823051 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8f99a9-2257-41b7-9bf8-010a3346dc9c None admin] RBAC: Adding query filter params (role.id=aed9aa9ef7f7448bb3b40c9ad082d545, user.id=cdd1247fb66645729ea4e48a0560c265, scope.system=all) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:33.823371 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8f99a9-2257-41b7-9bf8-010a3346dc9c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:33.824823 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8f99a9-2257-41b7-9bf8-010a3346dc9c None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:33.842186 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 167/334] 10.222.0.57 () {56 vars in 1308 bytes} [Thu May 19 14:51:33 2022] GET /identity/v3/role_assignments?user.id=cdd1247fb66645729ea4e48a0560c265&scope.system=all&role.id=aed9aa9ef7f7448bb3b40c9ad082d545 => generated 510 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:35.758522 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a04010-6fdc-4f78-baeb-b3682d9ec2e1 None None] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:35.759075 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a04010-6fdc-4f78-baeb-b3682d9ec2e1 None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:35.759075 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a04010-6fdc-4f78-baeb-b3682d9ec2e1 None None] PATH_INFO: `/` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:35.760772 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 168/335] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:51:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:51:35.766491 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f78890-9cc0-4a88-a305-5307ba90cc31 None None] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:35.766970 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f78890-9cc0-4a88-a305-5307ba90cc31 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:35.767066 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f78890-9cc0-4a88-a305-5307ba90cc31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:35.787763 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9907a87c-a241-48c0-9d9c-1c6d3cc0c91b None None] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:35.788438 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9907a87c-a241-48c0-9d9c-1c6d3cc0c91b None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:35.788945 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9907a87c-a241-48c0-9d9c-1c6d3cc0c91b None None] PATH_INFO: `/` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:35.789064 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.auth.core [None req-98f78890-9cc0-4a88-a305-5307ba90cc31 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:51:35.790207 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-98f78890-9cc0-4a88-a305-5307ba90cc31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:51:35.791222 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 169/336] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:51:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:51:35.796180 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a534728f-2b5e-4b3f-98c2-f88b1b45f5b8 None None] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:35.796755 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a534728f-2b5e-4b3f-98c2-f88b1b45f5b8 None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:35.797259 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a534728f-2b5e-4b3f-98c2-f88b1b45f5b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:35.827043 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.auth.core [None req-a534728f-2b5e-4b3f-98c2-f88b1b45f5b8 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:51:35.828653 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.fernet_utils [None req-a534728f-2b5e-4b3f-98c2-f88b1b45f5b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:51:35.844524 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:35.879732 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:35.894768 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 168/337] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:51:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:51:35.934918 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15b5e64-b7c8-4e0c-a839-3f1fdd225c11 None None] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:35.935220 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15b5e64-b7c8-4e0c-a839-3f1fdd225c11 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:35.935425 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15b5e64-b7c8-4e0c-a839-3f1fdd225c11 None None] PATH_INFO: `/` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:35.937096 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 169/338] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:51:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:51:35.942898 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15b5e64-b7c8-4e0c-a839-3f1fdd225c11 None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:35.951511 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 170/339] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:51:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:51:35.953153 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5652e4-6124-4d46-847e-54a307736307 None None] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:35.953674 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5652e4-6124-4d46-847e-54a307736307 None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:35.954183 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5652e4-6124-4d46-847e-54a307736307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:35.976253 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.auth.core [None req-6b5652e4-6124-4d46-847e-54a307736307 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:51:35.977835 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.fernet_utils [None req-6b5652e4-6124-4d46-847e-54a307736307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:51:36.098880 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 171/340] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:51:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:51:36.100148 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5652e4-6124-4d46-847e-54a307736307 None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:36.119071 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15b5e64-b7c8-4e0c-a839-3f1fdd225c11 None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:36.178702 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb34068-f7b6-421f-9344-f9d40d7e49a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:36.179545 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb34068-f7b6-421f-9344-f9d40d7e49a9 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:36.179830 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb34068-f7b6-421f-9344-f9d40d7e49a9 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:36.180119 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb34068-f7b6-421f-9344-f9d40d7e49a9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:36.199955 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb34068-f7b6-421f-9344-f9d40d7e49a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:51:36.204974 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:36.212530 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-bdb34068-f7b6-421f-9344-f9d40d7e49a9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:51:36.212530 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:51:36.212530 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:51:36.212530 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:51:36.212530 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:51:36.212530 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:51:36.212530 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:51:36.212530 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:51:36.212530 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:51:36.212530 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:51:36.212530 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:51:36.212530 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:51:36.212530 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:51:36.212530 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:51:36.212530 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:51:36.212530 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:51:36.213631 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:51:36.213631 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:51:36.213631 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:51:36.213631 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:51:36.213631 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:51:36.213631 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:51:36.213631 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:51:36.213631 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:51:36.213631 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:51:36.213631 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:51:36.213631 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:51:36.213631 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:51:36.213631 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:51:36.213631 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:51:36.213631 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:51:36.213631 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:51:36.214586 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:51:36.214586 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:51:36.214586 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:51:36.214586 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:51:36.214586 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:51:36.214586 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:51:36.214586 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:51:36.214586 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:51:36.214586 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:51:36.214586 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:51:36.214586 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:51:36.214586 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:51:36.216179 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 170/341] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:51:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:51:36.218424 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb34068-f7b6-421f-9344-f9d40d7e49a9 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:36.257405 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5652e4-6124-4d46-847e-54a307736307 None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:36.336318 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4552009-724b-4a5a-94c4-55aeb0cce02b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:36.337436 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4552009-724b-4a5a-94c4-55aeb0cce02b None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:36.337944 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4552009-724b-4a5a-94c4-55aeb0cce02b None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:36.340208 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4552009-724b-4a5a-94c4-55aeb0cce02b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:36.345740 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:36.362322 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4552009-724b-4a5a-94c4-55aeb0cce02b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:51:36.374499 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-e4552009-724b-4a5a-94c4-55aeb0cce02b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:51:36.374499 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:51:36.374499 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:51:36.374499 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:51:36.374499 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:51:36.374499 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:51:36.374499 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:51:36.374499 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:51:36.374499 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:51:36.374499 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:51:36.374499 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:51:36.374499 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:51:36.374499 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:51:36.374499 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:51:36.374499 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:51:36.374499 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:51:36.375556 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:51:36.375556 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:51:36.375556 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:51:36.375556 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:51:36.375556 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:51:36.375556 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:51:36.375556 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:51:36.375556 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:51:36.375556 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:51:36.375556 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:51:36.375556 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:51:36.375556 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:51:36.375556 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:51:36.375556 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:51:36.375556 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:51:36.375556 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:51:36.376472 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:51:36.376472 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:51:36.376472 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:51:36.376472 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:51:36.376472 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:51:36.376472 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:51:36.376472 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:51:36.376472 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:51:36.376472 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:51:36.376472 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:51:36.376472 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:51:36.376472 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:51:36.376472 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 172/342] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 19 14:51:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:51:36.377260 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4552009-724b-4a5a-94c4-55aeb0cce02b None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:36.456584 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb34068-f7b6-421f-9344-f9d40d7e49a9 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:36.513915 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4552009-724b-4a5a-94c4-55aeb0cce02b None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:36.548551 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8465c3d3-b3e8-4aba-9c68-07990ca2d15f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:36.549371 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8465c3d3-b3e8-4aba-9c68-07990ca2d15f None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:36.550240 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8465c3d3-b3e8-4aba-9c68-07990ca2d15f None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:36.550565 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8465c3d3-b3e8-4aba-9c68-07990ca2d15f None admin] PATH_INFO: `/v3/roles/member` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:36.555715 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff1d42d-86a8-400d-9da2-5008a8d1bc91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:36.556411 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff1d42d-86a8-400d-9da2-5008a8d1bc91 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:36.556700 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff1d42d-86a8-400d-9da2-5008a8d1bc91 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:36.556982 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff1d42d-86a8-400d-9da2-5008a8d1bc91 None admin] PATH_INFO: `/v3/roles` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:36.557758 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff1d42d-86a8-400d-9da2-5008a8d1bc91 None admin] RBAC: Adding query filter params (name=member) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:36.558100 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff1d42d-86a8-400d-9da2-5008a8d1bc91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:36.559560 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff1d42d-86a8-400d-9da2-5008a8d1bc91 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:36.563848 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:36.577512 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 173/343] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:51:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:36.581355 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff1d42d-86a8-400d-9da2-5008a8d1bc91 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:36.582682 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8465c3d3-b3e8-4aba-9c68-07990ca2d15f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:51:36.590902 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-8465c3d3-b3e8-4aba-9c68-07990ca2d15f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:51:36.590902 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:51:36.590902 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:51:36.590902 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:51:36.590902 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:51:36.590902 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:51:36.590902 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:51:36.590902 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:51:36.590902 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:51:36.590902 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:51:36.590902 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:51:36.590902 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:51:36.590902 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:51:36.590902 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:51:36.590902 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:51:36.590902 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:51:36.592026 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:51:36.592026 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:51:36.592026 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:51:36.592026 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:51:36.592026 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:51:36.592026 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:51:36.592026 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:51:36.592026 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:51:36.592026 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:51:36.592026 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:51:36.592026 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:51:36.592026 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:51:36.592026 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:51:36.592026 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:51:36.592026 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:51:36.592026 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:51:36.592937 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:51:36.592937 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:51:36.592937 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:51:36.592937 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:51:36.592937 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:51:36.592937 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:51:36.592937 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:51:36.592937 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:51:36.592937 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:51:36.592937 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:51:36.592937 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:51:36.592937 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:51:36.592937 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 171/344] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:51:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 375 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:51:36.603010 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8465c3d3-b3e8-4aba-9c68-07990ca2d15f None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:36.742090 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff1d42d-86a8-400d-9da2-5008a8d1bc91 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:36.783641 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8465c3d3-b3e8-4aba-9c68-07990ca2d15f None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:36.787897 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a618268-fd82-4e50-b9f4-1d28f3b7d615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:36.789405 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a618268-fd82-4e50-b9f4-1d28f3b7d615 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:36.789972 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a618268-fd82-4e50-b9f4-1d28f3b7d615 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:36.790471 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a618268-fd82-4e50-b9f4-1d28f3b7d615 None admin] PATH_INFO: `/v3/roles` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:36.791461 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a618268-fd82-4e50-b9f4-1d28f3b7d615 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:36.792102 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a618268-fd82-4e50-b9f4-1d28f3b7d615 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:36.793768 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a618268-fd82-4e50-b9f4-1d28f3b7d615 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:36.804284 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 174/345] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:51:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:36.805653 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a618268-fd82-4e50-b9f4-1d28f3b7d615 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:36.808200 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:36.831466 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7155a06e-7767-4cc6-9110-b8bc5358aeb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:36.832280 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7155a06e-7767-4cc6-9110-b8bc5358aeb1 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:36.832577 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7155a06e-7767-4cc6-9110-b8bc5358aeb1 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:36.832874 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7155a06e-7767-4cc6-9110-b8bc5358aeb1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:36.876024 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:36.915740 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7155a06e-7767-4cc6-9110-b8bc5358aeb1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:36.917889 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7155a06e-7767-4cc6-9110-b8bc5358aeb1 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:36.925268 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a618268-fd82-4e50-b9f4-1d28f3b7d615 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:36.962855 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52a0403-22c1-4baa-b16d-4d405bdc7eda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:36.963815 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52a0403-22c1-4baa-b16d-4d405bdc7eda None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:36.964363 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52a0403-22c1-4baa-b16d-4d405bdc7eda None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:36.964825 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52a0403-22c1-4baa-b16d-4d405bdc7eda None admin] PATH_INFO: `/v3/roles` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:36.965753 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52a0403-22c1-4baa-b16d-4d405bdc7eda None admin] RBAC: Adding query filter params (name=member) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:36.966269 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52a0403-22c1-4baa-b16d-4d405bdc7eda None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:36.967837 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52a0403-22c1-4baa-b16d-4d405bdc7eda None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:36.977388 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 175/346] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:51:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:36.979002 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5813517a-30eb-4163-81d3-2d2271211640 None None] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:36.979493 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5813517a-30eb-4163-81d3-2d2271211640 None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:36.979946 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5813517a-30eb-4163-81d3-2d2271211640 None None] PATH_INFO: `/` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:36.981644 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 176/347] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:51:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:51:36.982570 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5813517a-30eb-4163-81d3-2d2271211640 None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:37.032185 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 172/348] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:51:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:51:37.032987 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7155a06e-7767-4cc6-9110-b8bc5358aeb1 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:37.054497 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:37.118511 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5813517a-30eb-4163-81d3-2d2271211640 None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:37.149453 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:37.163060 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7155a06e-7767-4cc6-9110-b8bc5358aeb1 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:37.172092 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c93b2a3-925b-4f9f-b852-cffa20002e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:37.172770 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c93b2a3-925b-4f9f-b852-cffa20002e70 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:37.173063 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c93b2a3-925b-4f9f-b852-cffa20002e70 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:37.173349 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c93b2a3-925b-4f9f-b852-cffa20002e70 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:37.209644 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11194fef-e184-438d-b95e-154c3ef6c1cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:37.210503 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11194fef-e184-438d-b95e-154c3ef6c1cc None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:37.210798 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11194fef-e184-438d-b95e-154c3ef6c1cc None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:37.211082 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11194fef-e184-438d-b95e-154c3ef6c1cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:37.277288 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c93b2a3-925b-4f9f-b852-cffa20002e70 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:37.278853 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c93b2a3-925b-4f9f-b852-cffa20002e70 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:37.319491 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11194fef-e184-438d-b95e-154c3ef6c1cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:37.320913 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11194fef-e184-438d-b95e-154c3ef6c1cc None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:37.344331 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:37.411384 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 177/349] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:51:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:51:37.412848 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ea7569-9e88-4c13-b2ec-1cfb0682b267 None None] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:37.413219 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ea7569-9e88-4c13-b2ec-1cfb0682b267 None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:37.414404 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ea7569-9e88-4c13-b2ec-1cfb0682b267 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:37.436590 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.auth.core [None req-26ea7569-9e88-4c13-b2ec-1cfb0682b267 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:51:37.437834 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.fernet_utils [None req-26ea7569-9e88-4c13-b2ec-1cfb0682b267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:51:37.455006 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:37.456980 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 173/350] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:51:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:51:37.458332 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e943cec7-d00b-4d34-8096-bc711045eb20 None None] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:37.458636 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e943cec7-d00b-4d34-8096-bc711045eb20 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:37.458998 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e943cec7-d00b-4d34-8096-bc711045eb20 None None] PATH_INFO: `/` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:37.460397 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 174/351] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:51:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:51:37.464013 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e943cec7-d00b-4d34-8096-bc711045eb20 None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:37.517043 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 178/352] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:51:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:51:37.517364 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ea7569-9e88-4c13-b2ec-1cfb0682b267 None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:37.599331 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e943cec7-d00b-4d34-8096-bc711045eb20 None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:37.618962 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:37.638420 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ea7569-9e88-4c13-b2ec-1cfb0682b267 None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:37.646016 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9d8b12-92de-431e-8cbb-234e6777e69b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:37.647376 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9d8b12-92de-431e-8cbb-234e6777e69b None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:37.647674 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9d8b12-92de-431e-8cbb-234e6777e69b None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:37.647961 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9d8b12-92de-431e-8cbb-234e6777e69b None admin] PATH_INFO: `/v3/projects/caf48e96564645f3b3e590d230116a2d` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:37.649196 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9d8b12-92de-431e-8cbb-234e6777e69b None admin] RBAC: Authorizing `identity:get_project(project_id=caf48e96564645f3b3e590d230116a2d)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:37.650749 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9d8b12-92de-431e-8cbb-234e6777e69b None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:37.657299 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 175/353] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:51:37 2022] GET /identity/v3/projects/caf48e96564645f3b3e590d230116a2d => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:37.657299 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d5189f-f52a-48f8-a06d-46bee1188dc5 None None] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:37.657299 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d5189f-f52a-48f8-a06d-46bee1188dc5 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:37.657299 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d5189f-f52a-48f8-a06d-46bee1188dc5 None None] PATH_INFO: `/` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:37.657299 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 176/354] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:51:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:51:37.657299 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d5189f-f52a-48f8-a06d-46bee1188dc5 None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:37.682317 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ff31e3-2df7-4a08-8bbf-46068cf15f1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:37.683202 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ff31e3-2df7-4a08-8bbf-46068cf15f1d None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:37.684953 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ff31e3-2df7-4a08-8bbf-46068cf15f1d None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:37.684953 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ff31e3-2df7-4a08-8bbf-46068cf15f1d None admin] PATH_INFO: `/v3/projects/aca8075ce986433eae657055d1366699` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:37.685603 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ff31e3-2df7-4a08-8bbf-46068cf15f1d None admin] RBAC: Authorizing `identity:get_project(project_id=aca8075ce986433eae657055d1366699)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:37.687139 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ff31e3-2df7-4a08-8bbf-46068cf15f1d None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:37.689170 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 179/355] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:51:37 2022] GET /identity/v3/projects/aca8075ce986433eae657055d1366699 => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:37.690650 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85297e88-f6ae-4718-96c6-543171747c49 None None] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:37.691184 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85297e88-f6ae-4718-96c6-543171747c49 None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:37.691454 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85297e88-f6ae-4718-96c6-543171747c49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:37.711170 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.auth.core [None req-85297e88-f6ae-4718-96c6-543171747c49 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:51:37.712379 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.fernet_utils [None req-85297e88-f6ae-4718-96c6-543171747c49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:51:37.729635 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:37.794158 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 180/356] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:51:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:51:37.795771 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85297e88-f6ae-4718-96c6-543171747c49 None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:37.802581 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d5189f-f52a-48f8-a06d-46bee1188dc5 None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:37.846949 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d318dd-f0f5-43d5-b22c-77339f64c896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:37.847837 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d318dd-f0f5-43d5-b22c-77339f64c896 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:37.848153 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d318dd-f0f5-43d5-b22c-77339f64c896 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:37.848936 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d318dd-f0f5-43d5-b22c-77339f64c896 None admin] PATH_INFO: `/v3/projects/aca8075ce986433eae657055d1366699` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:37.849753 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d318dd-f0f5-43d5-b22c-77339f64c896 None admin] RBAC: Authorizing `identity:get_project(project_id=aca8075ce986433eae657055d1366699)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:37.851350 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d318dd-f0f5-43d5-b22c-77339f64c896 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:37.852857 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 177/357] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:51:37 2022] GET /identity/v3/projects/aca8075ce986433eae657055d1366699 => generated 298 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:37.856227 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d318dd-f0f5-43d5-b22c-77339f64c896 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:37.916193 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85297e88-f6ae-4718-96c6-543171747c49 None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:37.923285 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:37.958661 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658446d0-478c-4e50-b217-d17b1b3c78a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:37.959763 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658446d0-478c-4e50-b217-d17b1b3c78a8 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:37.960051 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658446d0-478c-4e50-b217-d17b1b3c78a8 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:37.960332 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658446d0-478c-4e50-b217-d17b1b3c78a8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:37.973891 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658446d0-478c-4e50-b217-d17b1b3c78a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:51:37.982053 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-658446d0-478c-4e50-b217-d17b1b3c78a8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:51:37.982053 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:51:37.982053 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:51:37.982053 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:51:37.982053 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:51:37.982053 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:51:37.982053 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:51:37.982053 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:51:37.982053 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:51:37.982053 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:51:37.982053 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:51:37.982053 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:51:37.982053 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:51:37.982053 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:51:37.982053 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:51:37.982053 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:51:37.983071 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:51:37.983071 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:51:37.983071 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:51:37.983071 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:51:37.983071 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:51:37.983071 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:51:37.983071 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:51:37.983071 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:51:37.983071 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:51:37.983071 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:51:37.983071 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:51:37.983071 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:51:37.983071 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:51:37.983071 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:51:37.983071 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:51:37.983071 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:51:37.984292 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:51:37.984292 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:51:37.984292 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:51:37.984292 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:51:37.984292 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:51:37.984292 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:51:37.984292 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:51:37.984292 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:51:37.984292 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:51:37.984292 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:51:37.984292 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:51:37.984292 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:51:37.985312 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d318dd-f0f5-43d5-b22c-77339f64c896 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:37.985421 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 181/358] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 19 14:51:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:51:37.987656 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6981921c-bb36-450f-988a-0d6ac71224ea None None] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:37.987656 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6981921c-bb36-450f-988a-0d6ac71224ea None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:37.987656 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6981921c-bb36-450f-988a-0d6ac71224ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:38.007262 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.auth.core [None req-6981921c-bb36-450f-988a-0d6ac71224ea None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:51:38.008457 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.fernet_utils [None req-6981921c-bb36-450f-988a-0d6ac71224ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:51:38.012043 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:38.029248 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17ab856-5215-4c04-8fc3-f307edb44083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:38.030157 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17ab856-5215-4c04-8fc3-f307edb44083 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:38.030456 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17ab856-5215-4c04-8fc3-f307edb44083 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:38.030742 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17ab856-5215-4c04-8fc3-f307edb44083 None admin] PATH_INFO: `/v3/groups/ee1c47ac2fd14de2abac0ec34d47b7d9` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:38.038288 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17ab856-5215-4c04-8fc3-f307edb44083 None admin] RBAC: Authorizing `identity:get_group(group_id=ee1c47ac2fd14de2abac0ec34d47b7d9)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:38.039813 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17ab856-5215-4c04-8fc3-f307edb44083 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:38.041628 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 178/359] 10.222.0.57 () {56 vars in 1180 bytes} [Thu May 19 14:51:37 2022] GET /identity/v3/groups/ee1c47ac2fd14de2abac0ec34d47b7d9 => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:38.047015 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17ab856-5215-4c04-8fc3-f307edb44083 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:38.090931 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 182/360] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:51:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:51:38.092572 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6981921c-bb36-450f-988a-0d6ac71224ea None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:38.170213 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17ab856-5215-4c04-8fc3-f307edb44083 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:38.191737 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:38.211868 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f8ef1f-5070-4683-ba54-74b93d23b5d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:38.212737 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f8ef1f-5070-4683-ba54-74b93d23b5d1 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:38.213026 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f8ef1f-5070-4683-ba54-74b93d23b5d1 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:38.213298 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f8ef1f-5070-4683-ba54-74b93d23b5d1 None admin] PATH_INFO: `/v3/groups/ee1c47ac2fd14de2abac0ec34d47b7d9` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:38.214559 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f8ef1f-5070-4683-ba54-74b93d23b5d1 None admin] RBAC: Authorizing `identity:get_group(group_id=ee1c47ac2fd14de2abac0ec34d47b7d9)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:38.215992 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f8ef1f-5070-4683-ba54-74b93d23b5d1 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:38.221885 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 179/361] 10.222.0.57 () {56 vars in 1180 bytes} [Thu May 19 14:51:38 2022] GET /identity/v3/groups/ee1c47ac2fd14de2abac0ec34d47b7d9 => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:38.221885 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f8ef1f-5070-4683-ba54-74b93d23b5d1 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:38.222156 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6981921c-bb36-450f-988a-0d6ac71224ea None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:38.270587 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd48a14-dc65-4127-86fe-08d5d14c27ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:38.271428 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd48a14-dc65-4127-86fe-08d5d14c27ce None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:38.271781 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd48a14-dc65-4127-86fe-08d5d14c27ce None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:38.272064 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd48a14-dc65-4127-86fe-08d5d14c27ce None admin] PATH_INFO: `/v3/roles/reader` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:38.280373 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:38.286181 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd48a14-dc65-4127-86fe-08d5d14c27ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:51:38.294244 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-1fd48a14-dc65-4127-86fe-08d5d14c27ce None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:51:38.294244 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:51:38.294244 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:51:38.294244 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:51:38.294244 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:51:38.294244 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:51:38.294244 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:51:38.294244 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:51:38.294244 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:51:38.294244 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:51:38.294244 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:51:38.294244 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:51:38.294244 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:51:38.294244 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:51:38.294244 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:51:38.294244 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:51:38.295278 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:51:38.295278 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:51:38.295278 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:51:38.295278 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:51:38.295278 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:51:38.295278 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:51:38.295278 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:51:38.295278 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:51:38.295278 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:51:38.295278 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:51:38.295278 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:51:38.295278 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:51:38.295278 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:51:38.295278 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:51:38.295278 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:51:38.295278 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:51:38.296197 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:51:38.296197 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:51:38.296197 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:51:38.296197 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:51:38.296197 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:51:38.296197 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:51:38.296197 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:51:38.296197 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:51:38.296197 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:51:38.296197 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:51:38.296197 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:51:38.296197 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:51:38.298638 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 183/362] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:51:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:51:38.299841 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd48a14-dc65-4127-86fe-08d5d14c27ce None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:38.345032 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f8ef1f-5070-4683-ba54-74b93d23b5d1 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:38.390975 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0ce619-4270-4c3a-b86f-114be386eb5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:38.392293 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0ce619-4270-4c3a-b86f-114be386eb5f None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:38.392592 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0ce619-4270-4c3a-b86f-114be386eb5f None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:38.392884 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0ce619-4270-4c3a-b86f-114be386eb5f None admin] PATH_INFO: `/v3/groups/ee1c47ac2fd14de2abac0ec34d47b7d9` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:38.394571 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0ce619-4270-4c3a-b86f-114be386eb5f None admin] RBAC: Authorizing `identity:get_group(group_id=ee1c47ac2fd14de2abac0ec34d47b7d9)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:38.396023 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0ce619-4270-4c3a-b86f-114be386eb5f None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:38.397715 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 180/363] 10.222.0.57 () {56 vars in 1180 bytes} [Thu May 19 14:51:38 2022] GET /identity/v3/groups/ee1c47ac2fd14de2abac0ec34d47b7d9 => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:38.398528 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0ce619-4270-4c3a-b86f-114be386eb5f None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:38.426362 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd48a14-dc65-4127-86fe-08d5d14c27ce None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:38.466859 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:38.473224 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b3958e-c3a3-4f14-8742-bb63003de32c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:38.473997 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b3958e-c3a3-4f14-8742-bb63003de32c None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:38.474524 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b3958e-c3a3-4f14-8742-bb63003de32c None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:38.474816 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b3958e-c3a3-4f14-8742-bb63003de32c None admin] PATH_INFO: `/v3/roles` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:38.475937 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b3958e-c3a3-4f14-8742-bb63003de32c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:38.476539 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b3958e-c3a3-4f14-8742-bb63003de32c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:38.478486 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b3958e-c3a3-4f14-8742-bb63003de32c None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:38.487673 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 184/364] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:51:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:38.488522 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b3958e-c3a3-4f14-8742-bb63003de32c None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:38.521984 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0ce619-4270-4c3a-b86f-114be386eb5f None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:38.549034 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:38.557808 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8507d946-c651-4631-baa6-fe172b2f0614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:38.558503 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8507d946-c651-4631-baa6-fe172b2f0614 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:38.558984 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8507d946-c651-4631-baa6-fe172b2f0614 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:38.559273 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8507d946-c651-4631-baa6-fe172b2f0614 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:38.560115 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8507d946-c651-4631-baa6-fe172b2f0614 None admin] RBAC: Adding query filter params (group.id=ee1c47ac2fd14de2abac0ec34d47b7d9, role.id=aed9aa9ef7f7448bb3b40c9ad082d545, scope.project.id=caf48e96564645f3b3e590d230116a2d) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:38.560435 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8507d946-c651-4631-baa6-fe172b2f0614 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:38.562276 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8507d946-c651-4631-baa6-fe172b2f0614 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:38.575512 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 181/365] 10.222.0.57 () {56 vars in 1376 bytes} [Thu May 19 14:51:38 2022] GET /identity/v3/role_assignments?group.id=ee1c47ac2fd14de2abac0ec34d47b7d9&scope.project.id=caf48e96564645f3b3e590d230116a2d&role.id=aed9aa9ef7f7448bb3b40c9ad082d545 => generated 260 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:38.575512 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8507d946-c651-4631-baa6-fe172b2f0614 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:38.605669 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b3958e-c3a3-4f14-8742-bb63003de32c None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:38.646219 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a8f877-13a1-4711-b10a-11edadbc8465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:38.647041 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a8f877-13a1-4711-b10a-11edadbc8465 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:38.647334 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a8f877-13a1-4711-b10a-11edadbc8465 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:38.647617 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a8f877-13a1-4711-b10a-11edadbc8465 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:38.662862 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a8f877-13a1-4711-b10a-11edadbc8465 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:51:38.670816 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-27a8f877-13a1-4711-b10a-11edadbc8465 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:51:38.670816 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:51:38.670816 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:51:38.670816 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:51:38.670816 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:51:38.670816 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:51:38.670816 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:51:38.670816 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:51:38.670816 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:51:38.670816 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:51:38.670816 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:51:38.670816 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:51:38.670816 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:51:38.670816 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:51:38.670816 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:51:38.670816 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:51:38.671920 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:51:38.671920 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:51:38.671920 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:51:38.671920 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:51:38.671920 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:51:38.671920 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:51:38.671920 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:51:38.671920 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:51:38.671920 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:51:38.671920 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:51:38.671920 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:51:38.671920 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:51:38.671920 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:51:38.671920 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:51:38.671920 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:51:38.671920 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:51:38.672860 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:51:38.672860 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:51:38.672860 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:51:38.672860 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:51:38.672860 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:51:38.672860 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:51:38.672860 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:51:38.672860 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:51:38.672860 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:51:38.672860 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:51:38.672860 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:51:38.672860 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:51:38.672860 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 185/366] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:51:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:51:38.675439 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a8f877-13a1-4711-b10a-11edadbc8465 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:38.704044 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8507d946-c651-4631-baa6-fe172b2f0614 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:38.723227 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:38.746450 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10bca20-4315-481c-9708-321c4645e05f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:38.747008 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10bca20-4315-481c-9708-321c4645e05f None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:38.747298 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10bca20-4315-481c-9708-321c4645e05f None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:38.747589 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10bca20-4315-481c-9708-321c4645e05f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:38.748435 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10bca20-4315-481c-9708-321c4645e05f None admin] RBAC: Adding query filter params (group.id=ee1c47ac2fd14de2abac0ec34d47b7d9, role.id=3b9c36a010504ba7bc63a8ccf0a9bca0, scope.project.id=aca8075ce986433eae657055d1366699) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:38.748755 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10bca20-4315-481c-9708-321c4645e05f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:38.750277 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10bca20-4315-481c-9708-321c4645e05f None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:38.757696 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 182/367] 10.222.0.57 () {56 vars in 1376 bytes} [Thu May 19 14:51:38 2022] GET /identity/v3/role_assignments?group.id=ee1c47ac2fd14de2abac0ec34d47b7d9&scope.project.id=aca8075ce986433eae657055d1366699&role.id=3b9c36a010504ba7bc63a8ccf0a9bca0 => generated 260 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:38.758182 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10bca20-4315-481c-9708-321c4645e05f None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:38.808308 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a8f877-13a1-4711-b10a-11edadbc8465 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:38.839001 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:38.857474 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533bab49-e17f-46d4-91b7-def441ab8f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:38.859309 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533bab49-e17f-46d4-91b7-def441ab8f39 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:38.859878 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533bab49-e17f-46d4-91b7-def441ab8f39 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:38.860995 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533bab49-e17f-46d4-91b7-def441ab8f39 None admin] PATH_INFO: `/v3/roles` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:38.862799 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533bab49-e17f-46d4-91b7-def441ab8f39 None admin] RBAC: Adding query filter params (name=reader) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:38.863938 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533bab49-e17f-46d4-91b7-def441ab8f39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:38.866290 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533bab49-e17f-46d4-91b7-def441ab8f39 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:38.880056 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 186/368] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:51:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:38.882178 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533bab49-e17f-46d4-91b7-def441ab8f39 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:38.889190 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10bca20-4315-481c-9708-321c4645e05f None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:38.933697 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bb7eba-e32c-4505-8b7a-99ead0523808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:38.934472 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bb7eba-e32c-4505-8b7a-99ead0523808 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:38.934763 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bb7eba-e32c-4505-8b7a-99ead0523808 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:38.935101 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bb7eba-e32c-4505-8b7a-99ead0523808 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:38.935957 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bb7eba-e32c-4505-8b7a-99ead0523808 None admin] RBAC: Adding query filter params (group.id=ee1c47ac2fd14de2abac0ec34d47b7d9, role.id=aed9aa9ef7f7448bb3b40c9ad082d545, scope.project.id=aca8075ce986433eae657055d1366699) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:38.936279 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bb7eba-e32c-4505-8b7a-99ead0523808 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:38.937813 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bb7eba-e32c-4505-8b7a-99ead0523808 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:38.946831 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 183/369] 10.222.0.57 () {56 vars in 1376 bytes} [Thu May 19 14:51:38 2022] GET /identity/v3/role_assignments?group.id=ee1c47ac2fd14de2abac0ec34d47b7d9&scope.project.id=aca8075ce986433eae657055d1366699&role.id=aed9aa9ef7f7448bb3b40c9ad082d545 => generated 260 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:38.947733 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bb7eba-e32c-4505-8b7a-99ead0523808 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:39.009335 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:39.074682 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bb7eba-e32c-4505-8b7a-99ead0523808 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:39.103890 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533bab49-e17f-46d4-91b7-def441ab8f39 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:39.114191 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bae499-f49e-405e-9948-f7906bfa56ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:39.115287 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bae499-f49e-405e-9948-f7906bfa56ad None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:39.115818 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bae499-f49e-405e-9948-f7906bfa56ad None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:39.116340 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bae499-f49e-405e-9948-f7906bfa56ad None admin] PATH_INFO: `/v3/roles` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:39.117338 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bae499-f49e-405e-9948-f7906bfa56ad None admin] RBAC: Adding query filter params (name=admin) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:39.117896 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bae499-f49e-405e-9948-f7906bfa56ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:39.119619 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bae499-f49e-405e-9948-f7906bfa56ad None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:39.130279 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 184/370] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:51:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:39.131615 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bae499-f49e-405e-9948-f7906bfa56ad None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:39.239574 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7912d895-dec6-48b4-98ae-33d41e3b5fd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:39.240252 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7912d895-dec6-48b4-98ae-33d41e3b5fd3 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:39.240546 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7912d895-dec6-48b4-98ae-33d41e3b5fd3 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:39.240833 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7912d895-dec6-48b4-98ae-33d41e3b5fd3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:39.263509 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bae499-f49e-405e-9948-f7906bfa56ad None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:39.274338 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:39.305390 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:39.319050 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1a5c28-b30f-4435-a048-afcaa41f0e9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:39.319974 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1a5c28-b30f-4435-a048-afcaa41f0e9d None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:39.321367 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1a5c28-b30f-4435-a048-afcaa41f0e9d None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:39.321367 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1a5c28-b30f-4435-a048-afcaa41f0e9d None admin] PATH_INFO: `/v3/users/6074527c5b9e4554a5f1182ebf0536be` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:39.322131 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1a5c28-b30f-4435-a048-afcaa41f0e9d None admin] RBAC: Authorizing `identity:get_user(user_id=6074527c5b9e4554a5f1182ebf0536be)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:39.323587 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1a5c28-b30f-4435-a048-afcaa41f0e9d None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:39.325250 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 185/371] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:51:39 2022] GET /identity/v3/users/6074527c5b9e4554a5f1182ebf0536be => generated 293 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:39.327395 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1a5c28-b30f-4435-a048-afcaa41f0e9d None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:39.345637 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7912d895-dec6-48b4-98ae-33d41e3b5fd3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:39.347550 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7912d895-dec6-48b4-98ae-33d41e3b5fd3 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:39.490289 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1a5c28-b30f-4435-a048-afcaa41f0e9d None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:39.536619 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed590b4-c6c4-4a37-a64a-9724c25bbdd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:39.537211 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed590b4-c6c4-4a37-a64a-9724c25bbdd0 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:39.537502 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed590b4-c6c4-4a37-a64a-9724c25bbdd0 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:39.537905 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed590b4-c6c4-4a37-a64a-9724c25bbdd0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:39.550305 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 187/372] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:51:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 669 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:51:39.551395 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7912d895-dec6-48b4-98ae-33d41e3b5fd3 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:39.644230 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:39.646255 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:39.676936 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7912d895-dec6-48b4-98ae-33d41e3b5fd3 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:39.715432 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7951c764-d810-4835-a675-f9cb5b88fad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:39.716261 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7951c764-d810-4835-a675-f9cb5b88fad7 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:39.716659 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7951c764-d810-4835-a675-f9cb5b88fad7 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:39.717040 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7951c764-d810-4835-a675-f9cb5b88fad7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:39.718031 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7951c764-d810-4835-a675-f9cb5b88fad7 None admin] RBAC: Adding query filter params (role.id=e56389e17ad24e20bf3bba1ca7684062, user.id=6074527c5b9e4554a5f1182ebf0536be, scope.system=all) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:39.718452 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7951c764-d810-4835-a675-f9cb5b88fad7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:39.720100 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7951c764-d810-4835-a675-f9cb5b88fad7 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:39.728856 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed590b4-c6c4-4a37-a64a-9724c25bbdd0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:39.730368 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed590b4-c6c4-4a37-a64a-9724c25bbdd0 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:39.733905 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 188/373] 10.222.0.57 () {56 vars in 1308 bytes} [Thu May 19 14:51:39 2022] GET /identity/v3/role_assignments?user.id=6074527c5b9e4554a5f1182ebf0536be&scope.system=all&role.id=e56389e17ad24e20bf3bba1ca7684062 => generated 510 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:39.734959 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7951c764-d810-4835-a675-f9cb5b88fad7 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:39.934918 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7951c764-d810-4835-a675-f9cb5b88fad7 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:39.968532 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 186/374] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:51:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 643 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:51:39.975332 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed590b4-c6c4-4a37-a64a-9724c25bbdd0 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:40.013472 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804acb78-74c7-4d93-8179-824a40a247ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:40.014412 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804acb78-74c7-4d93-8179-824a40a247ed None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:40.014701 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804acb78-74c7-4d93-8179-824a40a247ed None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:40.014981 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804acb78-74c7-4d93-8179-824a40a247ed None admin] PATH_INFO: `/v3/projects/caf48e96564645f3b3e590d230116a2d` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:40.016487 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804acb78-74c7-4d93-8179-824a40a247ed None admin] RBAC: Authorizing `identity:get_project(project_id=caf48e96564645f3b3e590d230116a2d)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:40.018704 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804acb78-74c7-4d93-8179-824a40a247ed None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:40.027133 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 189/375] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:51:39 2022] GET /identity/v3/projects/caf48e96564645f3b3e590d230116a2d => generated 302 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:40.032772 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804acb78-74c7-4d93-8179-824a40a247ed None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:40.054259 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:40.144028 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:40.203895 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed590b4-c6c4-4a37-a64a-9724c25bbdd0 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:40.259169 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804acb78-74c7-4d93-8179-824a40a247ed None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:40.272134 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b32e41b-ea41-434e-8d8b-e1e75d50b878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:40.273040 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b32e41b-ea41-434e-8d8b-e1e75d50b878 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:40.273331 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b32e41b-ea41-434e-8d8b-e1e75d50b878 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:40.273614 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b32e41b-ea41-434e-8d8b-e1e75d50b878 None admin] PATH_INFO: `/v3/projects/f00ebb3222e9428fb4f634c779ecb9a3` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:40.275360 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b32e41b-ea41-434e-8d8b-e1e75d50b878 None admin] RBAC: Authorizing `identity:get_project(project_id=f00ebb3222e9428fb4f634c779ecb9a3)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:40.276830 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b32e41b-ea41-434e-8d8b-e1e75d50b878 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:40.278760 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 187/376] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:51:39 2022] GET /identity/v3/projects/f00ebb3222e9428fb4f634c779ecb9a3 => generated 344 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:40.286728 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b32e41b-ea41-434e-8d8b-e1e75d50b878 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:40.333277 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2af0b52-1698-47b1-9b9a-29d6ea4177a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:40.335419 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2af0b52-1698-47b1-9b9a-29d6ea4177a9 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:40.335708 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2af0b52-1698-47b1-9b9a-29d6ea4177a9 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:40.335991 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2af0b52-1698-47b1-9b9a-29d6ea4177a9 None admin] PATH_INFO: `/v3/groups/ee1c47ac2fd14de2abac0ec34d47b7d9` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:40.339895 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2af0b52-1698-47b1-9b9a-29d6ea4177a9 None admin] RBAC: Authorizing `identity:get_group(group_id=ee1c47ac2fd14de2abac0ec34d47b7d9)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:40.341360 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2af0b52-1698-47b1-9b9a-29d6ea4177a9 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:40.348429 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 190/377] 10.222.0.57 () {56 vars in 1180 bytes} [Thu May 19 14:51:40 2022] GET /identity/v3/groups/ee1c47ac2fd14de2abac0ec34d47b7d9 => generated 226 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:51:40.352011 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2af0b52-1698-47b1-9b9a-29d6ea4177a9 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:40.476327 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b32e41b-ea41-434e-8d8b-e1e75d50b878 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:40.521014 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d4e14b-acd5-4d2e-b89e-9c925b7cf27c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:40.521967 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d4e14b-acd5-4d2e-b89e-9c925b7cf27c None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:40.522278 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d4e14b-acd5-4d2e-b89e-9c925b7cf27c None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:40.522567 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d4e14b-acd5-4d2e-b89e-9c925b7cf27c None admin] PATH_INFO: `/v3/groups/32251723ef49480496da49da30172c34` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:40.528184 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d4e14b-acd5-4d2e-b89e-9c925b7cf27c None admin] RBAC: Authorizing `identity:get_group(group_id=32251723ef49480496da49da30172c34)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:40.529803 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d4e14b-acd5-4d2e-b89e-9c925b7cf27c None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:40.532189 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 188/378] 10.222.0.57 () {56 vars in 1180 bytes} [Thu May 19 14:51:40 2022] GET /identity/v3/groups/32251723ef49480496da49da30172c34 => generated 229 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:40.538097 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d4e14b-acd5-4d2e-b89e-9c925b7cf27c None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:40.550680 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:40.557779 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:40.594169 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2af0b52-1698-47b1-9b9a-29d6ea4177a9 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:40.637147 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bec28c5-31f5-4de6-ba19-50896f1ebf47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:40.637914 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bec28c5-31f5-4de6-ba19-50896f1ebf47 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:40.638209 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bec28c5-31f5-4de6-ba19-50896f1ebf47 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:40.638491 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bec28c5-31f5-4de6-ba19-50896f1ebf47 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:40.639336 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bec28c5-31f5-4de6-ba19-50896f1ebf47 None admin] RBAC: Adding query filter params (group.id=ee1c47ac2fd14de2abac0ec34d47b7d9, role.id=3b9c36a010504ba7bc63a8ccf0a9bca0, scope.project.id=caf48e96564645f3b3e590d230116a2d) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:40.639659 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bec28c5-31f5-4de6-ba19-50896f1ebf47 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:40.641109 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bec28c5-31f5-4de6-ba19-50896f1ebf47 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:40.649131 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 191/379] 10.222.0.57 () {56 vars in 1376 bytes} [Thu May 19 14:51:40 2022] GET /identity/v3/role_assignments?group.id=ee1c47ac2fd14de2abac0ec34d47b7d9&scope.project.id=caf48e96564645f3b3e590d230116a2d&role.id=3b9c36a010504ba7bc63a8ccf0a9bca0 => generated 260 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:40.721929 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d4e14b-acd5-4d2e-b89e-9c925b7cf27c None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:40.782811 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44b04ab-7ae7-4923-a8be-9b7790b3c262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:40.783436 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44b04ab-7ae7-4923-a8be-9b7790b3c262 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:40.783822 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44b04ab-7ae7-4923-a8be-9b7790b3c262 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:40.784137 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44b04ab-7ae7-4923-a8be-9b7790b3c262 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:40.785026 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44b04ab-7ae7-4923-a8be-9b7790b3c262 None admin] RBAC: Adding query filter params (group.id=32251723ef49480496da49da30172c34, role.id=028379e01f9245e9b8fc56e8b9e8a6ad, scope.project.id=f00ebb3222e9428fb4f634c779ecb9a3) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:40.785346 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44b04ab-7ae7-4923-a8be-9b7790b3c262 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:40.786860 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44b04ab-7ae7-4923-a8be-9b7790b3c262 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:40.793612 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 189/380] 10.222.0.57 () {56 vars in 1376 bytes} [Thu May 19 14:51:40 2022] GET /identity/v3/role_assignments?group.id=32251723ef49480496da49da30172c34&scope.project.id=f00ebb3222e9428fb4f634c779ecb9a3&role.id=028379e01f9245e9b8fc56e8b9e8a6ad => generated 260 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:41.418656 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287038a6-07b0-4050-919a-19e6c9b31432 None None] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:41.419240 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287038a6-07b0-4050-919a-19e6c9b31432 None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:41.419240 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287038a6-07b0-4050-919a-19e6c9b31432 None None] PATH_INFO: `/` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:41.421003 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 192/381] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:51:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:51:41.426329 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c102c9cb-54f8-47d4-adc8-5ef4ce5ac75d None None] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:41.427235 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c102c9cb-54f8-47d4-adc8-5ef4ce5ac75d None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:41.427761 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c102c9cb-54f8-47d4-adc8-5ef4ce5ac75d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:41.435007 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91f661a-8962-4f44-80cf-c93fe822b04a None None] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:41.435684 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91f661a-8962-4f44-80cf-c93fe822b04a None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:41.436262 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91f661a-8962-4f44-80cf-c93fe822b04a None None] PATH_INFO: `/` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:41.438431 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 193/382] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:51:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:51:41.449741 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a644ce2-5dfc-4d97-b485-11528faff516 None None] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:41.450442 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a644ce2-5dfc-4d97-b485-11528faff516 None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:41.450941 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a644ce2-5dfc-4d97-b485-11528faff516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:41.453787 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.auth.core [None req-c102c9cb-54f8-47d4-adc8-5ef4ce5ac75d None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:51:41.455244 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-c102c9cb-54f8-47d4-adc8-5ef4ce5ac75d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:51:41.477716 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.auth.core [None req-7a644ce2-5dfc-4d97-b485-11528faff516 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:51:41.478993 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.fernet_utils [None req-7a644ce2-5dfc-4d97-b485-11528faff516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:51:41.551033 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 194/383] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:51:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:51:41.574693 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a644ce2-5dfc-4d97-b485-11528faff516 None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:41.605795 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:41.613663 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 190/384] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:51:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:51:41.655562 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c102c9cb-54f8-47d4-adc8-5ef4ce5ac75d None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:41.706415 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:41.753753 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a644ce2-5dfc-4d97-b485-11528faff516 None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:41.789866 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295e1006-c677-4fe9-8fe9-505799b2310e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:41.790708 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295e1006-c677-4fe9-8fe9-505799b2310e None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:41.790997 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295e1006-c677-4fe9-8fe9-505799b2310e None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:41.791399 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295e1006-c677-4fe9-8fe9-505799b2310e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:41.803627 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295e1006-c677-4fe9-8fe9-505799b2310e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:51:41.812480 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-295e1006-c677-4fe9-8fe9-505799b2310e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:51:41.812480 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:51:41.812480 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:51:41.812480 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:51:41.812480 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:51:41.812480 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:51:41.812480 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:51:41.812480 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:51:41.812480 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:51:41.812480 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:51:41.812480 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:51:41.812480 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:51:41.812480 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:51:41.812480 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:51:41.812480 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:51:41.812480 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:51:41.813513 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:51:41.813513 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:51:41.813513 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:51:41.813513 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:51:41.813513 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:51:41.813513 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:51:41.813513 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:51:41.813513 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:51:41.813513 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:51:41.813513 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:51:41.813513 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:51:41.813513 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:51:41.813513 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:51:41.813513 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:51:41.813513 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:51:41.813513 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:51:41.814533 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:51:41.814533 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:51:41.814533 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:51:41.814533 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:51:41.814533 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:51:41.814533 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:51:41.814533 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:51:41.814533 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:51:41.814533 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:51:41.814533 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:51:41.814533 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:51:41.814533 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:51:41.818093 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 195/385] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 19 14:51:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:51:41.819488 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41347a3b-3cee-44d3-87e8-24281a2210ee None None] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:41.819785 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41347a3b-3cee-44d3-87e8-24281a2210ee None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:41.820071 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41347a3b-3cee-44d3-87e8-24281a2210ee None None] PATH_INFO: `/` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:41.825915 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 196/386] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:51:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:51:41.825915 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41347a3b-3cee-44d3-87e8-24281a2210ee None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:41.856480 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c102c9cb-54f8-47d4-adc8-5ef4ce5ac75d None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:41.895195 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c30191d-3d4c-450f-9304-6b5b785adbf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:41.896437 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c30191d-3d4c-450f-9304-6b5b785adbf8 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:41.896983 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c30191d-3d4c-450f-9304-6b5b785adbf8 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:41.897488 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c30191d-3d4c-450f-9304-6b5b785adbf8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:41.909787 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c30191d-3d4c-450f-9304-6b5b785adbf8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:51:41.919133 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-7c30191d-3d4c-450f-9304-6b5b785adbf8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:51:41.919133 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:51:41.919133 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:51:41.919133 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:51:41.919133 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:51:41.919133 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:51:41.919133 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:51:41.919133 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:51:41.919133 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:51:41.919133 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:51:41.919133 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:51:41.919133 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:51:41.919133 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:51:41.919133 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:51:41.919133 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:51:41.919133 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:51:41.920233 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:51:41.920233 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:51:41.920233 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:51:41.920233 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:51:41.920233 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:51:41.920233 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:51:41.920233 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:51:41.920233 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:51:41.920233 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:51:41.920233 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:51:41.920233 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:51:41.920233 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:51:41.920233 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:51:41.920233 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:51:41.920233 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:51:41.920233 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:51:41.921252 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:51:41.921252 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:51:41.921252 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:51:41.921252 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:51:41.921252 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:51:41.921252 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:51:41.921252 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:51:41.921252 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:51:41.921252 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:51:41.921252 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:51:41.921252 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:51:41.921252 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:51:41.924370 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 191/387] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:51:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:51:41.925902 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c359618-f4b6-4792-93cd-7d37ba170d2d None None] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:41.926429 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c359618-f4b6-4792-93cd-7d37ba170d2d None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:41.926941 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c359618-f4b6-4792-93cd-7d37ba170d2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:41.941913 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41347a3b-3cee-44d3-87e8-24281a2210ee None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:41.946367 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.auth.core [None req-8c359618-f4b6-4792-93cd-7d37ba170d2d None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:51:41.947692 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-8c359618-f4b6-4792-93cd-7d37ba170d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:51:41.971761 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:41.978050 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0439ca53-2316-442a-bad7-bd9f7d527513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:41.978705 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0439ca53-2316-442a-bad7-bd9f7d527513 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:41.979039 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0439ca53-2316-442a-bad7-bd9f7d527513 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:41.979327 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0439ca53-2316-442a-bad7-bd9f7d527513 None admin] PATH_INFO: `/v3/roles` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:41.980189 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0439ca53-2316-442a-bad7-bd9f7d527513 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:41.980555 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0439ca53-2316-442a-bad7-bd9f7d527513 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:41.982198 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0439ca53-2316-442a-bad7-bd9f7d527513 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:41.997912 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 197/388] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:51:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:51:41.999027 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0439ca53-2316-442a-bad7-bd9f7d527513 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:42.014165 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:42.027970 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 192/389] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:51:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:51:42.029309 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c359618-f4b6-4792-93cd-7d37ba170d2d None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:42.115610 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0439ca53-2316-442a-bad7-bd9f7d527513 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:42.139541 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c359618-f4b6-4792-93cd-7d37ba170d2d None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:42.153308 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7b8be5-15a0-4458-844d-382be4ac4d3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:42.154139 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7b8be5-15a0-4458-844d-382be4ac4d3f None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:42.154524 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7b8be5-15a0-4458-844d-382be4ac4d3f None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:42.154812 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7b8be5-15a0-4458-844d-382be4ac4d3f None admin] PATH_INFO: `/v3/roles` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:42.155569 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7b8be5-15a0-4458-844d-382be4ac4d3f None admin] RBAC: Adding query filter params (name=member) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:42.155910 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7b8be5-15a0-4458-844d-382be4ac4d3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:42.157396 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7b8be5-15a0-4458-844d-382be4ac4d3f None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:42.168466 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 198/390] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:51:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:42.169458 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7b8be5-15a0-4458-844d-382be4ac4d3f None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:42.181526 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfb7560-9b91-4595-bc03-5df8fa4d0db6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:42.182940 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfb7560-9b91-4595-bc03-5df8fa4d0db6 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:42.183760 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfb7560-9b91-4595-bc03-5df8fa4d0db6 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:42.184078 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfb7560-9b91-4595-bc03-5df8fa4d0db6 None admin] PATH_INFO: `/v3/groups/ee1c47ac2fd14de2abac0ec34d47b7d9` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:42.185940 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfb7560-9b91-4595-bc03-5df8fa4d0db6 None admin] RBAC: Authorizing `identity:get_group(group_id=ee1c47ac2fd14de2abac0ec34d47b7d9)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:42.187790 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfb7560-9b91-4595-bc03-5df8fa4d0db6 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:42.194023 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 193/391] 10.222.0.57 () {56 vars in 1180 bytes} [Thu May 19 14:51:42 2022] GET /identity/v3/groups/ee1c47ac2fd14de2abac0ec34d47b7d9 => generated 226 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:42.195428 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfb7560-9b91-4595-bc03-5df8fa4d0db6 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:42.250479 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:42.267127 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:42.296944 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7b8be5-15a0-4458-844d-382be4ac4d3f None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:42.320851 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfb7560-9b91-4595-bc03-5df8fa4d0db6 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:42.337313 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a31dcb-6c8a-4e6e-8b89-f020c0dba475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:42.338157 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a31dcb-6c8a-4e6e-8b89-f020c0dba475 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:42.338557 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a31dcb-6c8a-4e6e-8b89-f020c0dba475 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:42.338947 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a31dcb-6c8a-4e6e-8b89-f020c0dba475 None admin] PATH_INFO: `/v3/roles/member` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:42.352979 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a31dcb-6c8a-4e6e-8b89-f020c0dba475 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:51:42.361181 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-91a31dcb-6c8a-4e6e-8b89-f020c0dba475 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:51:42.361181 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:51:42.361181 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:51:42.361181 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:51:42.361181 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:51:42.361181 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:51:42.361181 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:51:42.361181 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:51:42.361181 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:51:42.361181 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:51:42.361181 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:51:42.361181 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:51:42.361181 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:51:42.361181 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:51:42.361181 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:51:42.361181 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:51:42.363110 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:51:42.363110 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:51:42.363110 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:51:42.363110 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:51:42.363110 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:51:42.363110 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:51:42.363110 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:51:42.363110 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:51:42.363110 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:51:42.363110 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:51:42.363110 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:51:42.363110 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:51:42.363110 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:51:42.363110 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:51:42.363110 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:51:42.363110 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:51:42.364035 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:51:42.364035 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:51:42.364035 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:51:42.364035 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:51:42.364035 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:51:42.364035 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:51:42.364035 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:51:42.364035 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:51:42.364035 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:51:42.364035 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:51:42.364035 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:51:42.364035 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:51:42.366722 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 199/392] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:51:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:51:42.367784 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a31dcb-6c8a-4e6e-8b89-f020c0dba475 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:42.371411 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb1ef4b-66d9-4c90-a93b-c432e088297d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:42.372429 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb1ef4b-66d9-4c90-a93b-c432e088297d None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:42.372905 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb1ef4b-66d9-4c90-a93b-c432e088297d None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:42.373241 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb1ef4b-66d9-4c90-a93b-c432e088297d None admin] PATH_INFO: `/v3/groups/ee1c47ac2fd14de2abac0ec34d47b7d9` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:42.375277 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb1ef4b-66d9-4c90-a93b-c432e088297d None admin] RBAC: Authorizing `identity:get_group(group_id=ee1c47ac2fd14de2abac0ec34d47b7d9)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:42.376290 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb1ef4b-66d9-4c90-a93b-c432e088297d None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:42.378606 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 194/393] 10.222.0.57 () {56 vars in 1180 bytes} [Thu May 19 14:51:42 2022] GET /identity/v3/groups/ee1c47ac2fd14de2abac0ec34d47b7d9 => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:42.379528 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb1ef4b-66d9-4c90-a93b-c432e088297d None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:42.486324 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a31dcb-6c8a-4e6e-8b89-f020c0dba475 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:42.505309 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb1ef4b-66d9-4c90-a93b-c432e088297d None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:42.506189 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:42.531198 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d48339-2c29-46a1-a5b4-75e8e8fd2118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:42.532937 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d48339-2c29-46a1-a5b4-75e8e8fd2118 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:42.533422 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d48339-2c29-46a1-a5b4-75e8e8fd2118 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:42.534017 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d48339-2c29-46a1-a5b4-75e8e8fd2118 None admin] PATH_INFO: `/v3/projects/aca8075ce986433eae657055d1366699` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:42.534169 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:42.536008 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d48339-2c29-46a1-a5b4-75e8e8fd2118 None admin] RBAC: Authorizing `identity:get_project(project_id=aca8075ce986433eae657055d1366699)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:42.537875 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d48339-2c29-46a1-a5b4-75e8e8fd2118 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:42.544799 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 200/394] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:51:42 2022] GET /identity/v3/projects/aca8075ce986433eae657055d1366699 => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:42.544799 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d48339-2c29-46a1-a5b4-75e8e8fd2118 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:42.551329 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7876d5cf-9846-402e-8cb6-bc07b04b15f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:42.552268 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7876d5cf-9846-402e-8cb6-bc07b04b15f1 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:42.552772 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7876d5cf-9846-402e-8cb6-bc07b04b15f1 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:42.553286 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7876d5cf-9846-402e-8cb6-bc07b04b15f1 None admin] PATH_INFO: `/v3/roles` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:42.554315 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7876d5cf-9846-402e-8cb6-bc07b04b15f1 None admin] RBAC: Adding query filter params (name=member) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:42.554843 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7876d5cf-9846-402e-8cb6-bc07b04b15f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:42.556433 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7876d5cf-9846-402e-8cb6-bc07b04b15f1 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:42.565805 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 195/395] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:51:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:42.567177 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7876d5cf-9846-402e-8cb6-bc07b04b15f1 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:42.677910 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d48339-2c29-46a1-a5b4-75e8e8fd2118 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:42.694387 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7876d5cf-9846-402e-8cb6-bc07b04b15f1 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:42.714507 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46425f7-55d8-4f6b-ab71-ffc4164cf128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:42.715798 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46425f7-55d8-4f6b-ab71-ffc4164cf128 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:42.716300 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46425f7-55d8-4f6b-ab71-ffc4164cf128 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:42.716821 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46425f7-55d8-4f6b-ab71-ffc4164cf128 None admin] PATH_INFO: `/v3/projects/caf48e96564645f3b3e590d230116a2d` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:42.718295 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46425f7-55d8-4f6b-ab71-ffc4164cf128 None admin] RBAC: Authorizing `identity:get_project(project_id=caf48e96564645f3b3e590d230116a2d)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:42.719989 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46425f7-55d8-4f6b-ab71-ffc4164cf128 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:42.723089 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 201/396] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:51:42 2022] GET /identity/v3/projects/caf48e96564645f3b3e590d230116a2d => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:42.724243 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46425f7-55d8-4f6b-ab71-ffc4164cf128 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:42.735898 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006d8cec-2afc-4354-b448-b5d1abe9cc93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:42.738002 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006d8cec-2afc-4354-b448-b5d1abe9cc93 None admin] REQUEST_METHOD: `PUT` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:42.738301 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006d8cec-2afc-4354-b448-b5d1abe9cc93 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:42.738588 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006d8cec-2afc-4354-b448-b5d1abe9cc93 None admin] PATH_INFO: `/v3/projects/aca8075ce986433eae657055d1366699/groups/ee1c47ac2fd14de2abac0ec34d47b7d9/roles/3b9c36a010504ba7bc63a8ccf0a9bca0` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:42.741496 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006d8cec-2afc-4354-b448-b5d1abe9cc93 None admin] RBAC: Authorizing `identity:create_grant(project_id=aca8075ce986433eae657055d1366699, group_id=ee1c47ac2fd14de2abac0ec34d47b7d9, role_id=3b9c36a010504ba7bc63a8ccf0a9bca0)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:42.743157 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006d8cec-2afc-4354-b448-b5d1abe9cc93 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:42.763963 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 196/397] 10.222.0.57 () {58 vars in 1442 bytes} [Thu May 19 14:51:42 2022] PUT /identity/v3/projects/aca8075ce986433eae657055d1366699/groups/ee1c47ac2fd14de2abac0ec34d47b7d9/roles/3b9c36a010504ba7bc63a8ccf0a9bca0 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:51:42.765001 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006d8cec-2afc-4354-b448-b5d1abe9cc93 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:42.787768 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:42.836960 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:42.906219 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006d8cec-2afc-4354-b448-b5d1abe9cc93 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:42.911396 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46425f7-55d8-4f6b-ab71-ffc4164cf128 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:42.961279 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755e5b47-ef58-4e86-9c28-56431ea125ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:42.962713 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755e5b47-ef58-4e86-9c28-56431ea125ab None admin] REQUEST_METHOD: `PUT` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:42.964575 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755e5b47-ef58-4e86-9c28-56431ea125ab None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:42.965707 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755e5b47-ef58-4e86-9c28-56431ea125ab None admin] PATH_INFO: `/v3/projects/caf48e96564645f3b3e590d230116a2d/groups/ee1c47ac2fd14de2abac0ec34d47b7d9/roles/aed9aa9ef7f7448bb3b40c9ad082d545` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:42.969164 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755e5b47-ef58-4e86-9c28-56431ea125ab None admin] RBAC: Authorizing `identity:create_grant(project_id=caf48e96564645f3b3e590d230116a2d, group_id=ee1c47ac2fd14de2abac0ec34d47b7d9, role_id=aed9aa9ef7f7448bb3b40c9ad082d545)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:42.972383 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755e5b47-ef58-4e86-9c28-56431ea125ab None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:42.986584 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3245b8-0f28-49a0-ac8a-0825d651ae4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:42.988508 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3245b8-0f28-49a0-ac8a-0825d651ae4c None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:42.989173 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3245b8-0f28-49a0-ac8a-0825d651ae4c None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:42.989878 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3245b8-0f28-49a0-ac8a-0825d651ae4c None admin] PATH_INFO: `/v3/groups/ee1c47ac2fd14de2abac0ec34d47b7d9` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:42.991185 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 197/398] 10.222.0.57 () {58 vars in 1442 bytes} [Thu May 19 14:51:42 2022] PUT /identity/v3/projects/caf48e96564645f3b3e590d230116a2d/groups/ee1c47ac2fd14de2abac0ec34d47b7d9/roles/aed9aa9ef7f7448bb3b40c9ad082d545 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:51:42.991890 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3245b8-0f28-49a0-ac8a-0825d651ae4c None admin] RBAC: Authorizing `identity:get_group(group_id=ee1c47ac2fd14de2abac0ec34d47b7d9)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:42.994390 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3245b8-0f28-49a0-ac8a-0825d651ae4c None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:42.995470 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 202/399] 10.222.0.57 () {56 vars in 1180 bytes} [Thu May 19 14:51:42 2022] GET /identity/v3/groups/ee1c47ac2fd14de2abac0ec34d47b7d9 => generated 226 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:43.001746 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755e5b47-ef58-4e86-9c28-56431ea125ab None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:43.128610 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755e5b47-ef58-4e86-9c28-56431ea125ab None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:43.139058 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:43.168412 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed99286-3976-49c6-a546-d9d09530364a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:43.169307 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed99286-3976-49c6-a546-d9d09530364a None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:43.169601 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed99286-3976-49c6-a546-d9d09530364a None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:43.169922 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed99286-3976-49c6-a546-d9d09530364a None admin] PATH_INFO: `/v3/projects/aca8075ce986433eae657055d1366699` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:43.171542 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed99286-3976-49c6-a546-d9d09530364a None admin] RBAC: Authorizing `identity:get_project(project_id=aca8075ce986433eae657055d1366699)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:43.173014 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed99286-3976-49c6-a546-d9d09530364a None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:43.176010 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 198/400] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:51:43 2022] GET /identity/v3/projects/aca8075ce986433eae657055d1366699 => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:51:43.182996 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3245b8-0f28-49a0-ac8a-0825d651ae4c None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:43.321258 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3245b8-0f28-49a0-ac8a-0825d651ae4c None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:43.340118 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:43.358847 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cdee2a-dde2-4ef4-8603-0fbf9efbea7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:43.359749 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cdee2a-dde2-4ef4-8603-0fbf9efbea7c None admin] REQUEST_METHOD: `PUT` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:43.360043 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cdee2a-dde2-4ef4-8603-0fbf9efbea7c None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:43.360329 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cdee2a-dde2-4ef4-8603-0fbf9efbea7c None admin] PATH_INFO: `/v3/projects/aca8075ce986433eae657055d1366699/groups/ee1c47ac2fd14de2abac0ec34d47b7d9/roles/aed9aa9ef7f7448bb3b40c9ad082d545` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:43.362506 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cdee2a-dde2-4ef4-8603-0fbf9efbea7c None admin] RBAC: Authorizing `identity:create_grant(project_id=aca8075ce986433eae657055d1366699, group_id=ee1c47ac2fd14de2abac0ec34d47b7d9, role_id=aed9aa9ef7f7448bb3b40c9ad082d545)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:43.364071 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cdee2a-dde2-4ef4-8603-0fbf9efbea7c None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:43.387889 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 203/401] 10.222.0.57 () {58 vars in 1442 bytes} [Thu May 19 14:51:43 2022] PUT /identity/v3/projects/aca8075ce986433eae657055d1366699/groups/ee1c47ac2fd14de2abac0ec34d47b7d9/roles/aed9aa9ef7f7448bb3b40c9ad082d545 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:51:43.400382 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcae200d-373e-4a41-b571-cc826a13569b None None] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:43.400704 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcae200d-373e-4a41-b571-cc826a13569b None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:43.400965 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcae200d-373e-4a41-b571-cc826a13569b None None] PATH_INFO: `/` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:43.402588 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 199/402] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:51:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:51:43.408861 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f95a88-254a-454c-abdd-cdbd0c70c4fe None None] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:43.409261 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f95a88-254a-454c-abdd-cdbd0c70c4fe None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:43.409470 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f95a88-254a-454c-abdd-cdbd0c70c4fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:43.429064 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.auth.core [None req-56f95a88-254a-454c-abdd-cdbd0c70c4fe None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:51:43.430327 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.fernet_utils [None req-56f95a88-254a-454c-abdd-cdbd0c70c4fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:51:43.506670 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 204/403] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:51:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:51:43.529580 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcae200d-373e-4a41-b571-cc826a13569b None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:43.673686 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcae200d-373e-4a41-b571-cc826a13569b None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:43.713871 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76ebcdb-efde-45a4-879e-bf729c99f834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:43.714704 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76ebcdb-efde-45a4-879e-bf729c99f834 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:43.714996 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76ebcdb-efde-45a4-879e-bf729c99f834 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:43.715278 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76ebcdb-efde-45a4-879e-bf729c99f834 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:43.726342 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76ebcdb-efde-45a4-879e-bf729c99f834 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:51:43.734492 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-f76ebcdb-efde-45a4-879e-bf729c99f834 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:51:43.734492 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:51:43.734492 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:51:43.734492 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:51:43.734492 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:51:43.734492 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:51:43.734492 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:51:43.734492 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:51:43.734492 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:51:43.734492 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:51:43.734492 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:51:43.734492 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:51:43.734492 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:51:43.734492 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:51:43.734492 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:51:43.734492 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:51:43.735521 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:51:43.735521 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:51:43.735521 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:51:43.735521 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:51:43.735521 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:51:43.735521 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:51:43.735521 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:51:43.735521 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:51:43.735521 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:51:43.735521 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:51:43.735521 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:51:43.735521 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:51:43.735521 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:51:43.735521 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:51:43.735521 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:51:43.735521 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:51:43.737621 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:51:43.737621 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:51:43.737621 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:51:43.737621 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:51:43.737621 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:51:43.737621 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:51:43.737621 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:51:43.737621 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:51:43.737621 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:51:43.737621 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:51:43.737621 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:51:43.737621 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:51:43.737621 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 200/404] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 19 14:51:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:51:43.742610 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f95a88-254a-454c-abdd-cdbd0c70c4fe None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:43.814557 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146ddeac-60fb-4599-9d2c-a820bc35a128 None None] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:43.815322 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146ddeac-60fb-4599-9d2c-a820bc35a128 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:43.815619 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146ddeac-60fb-4599-9d2c-a820bc35a128 None None] PATH_INFO: `/` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:43.817296 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 201/405] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:51:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:51:43.822827 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0961ee-cbdf-4d43-8612-3f8be0d2d8f2 None None] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:43.823120 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0961ee-cbdf-4d43-8612-3f8be0d2d8f2 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:43.823411 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0961ee-cbdf-4d43-8612-3f8be0d2d8f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:43.841254 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.auth.core [None req-8c0961ee-cbdf-4d43-8612-3f8be0d2d8f2 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:51:43.842501 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-8c0961ee-cbdf-4d43-8612-3f8be0d2d8f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:51:43.875125 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:43.904636 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f95a88-254a-454c-abdd-cdbd0c70c4fe None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:43.926307 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:43.958424 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 202/406] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:51:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:51:43.972378 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be5dde3-e34f-46e0-94e9-f8b23c47c8ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:43.973121 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be5dde3-e34f-46e0-94e9-f8b23c47c8ef None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:43.973412 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be5dde3-e34f-46e0-94e9-f8b23c47c8ef None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:43.973746 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be5dde3-e34f-46e0-94e9-f8b23c47c8ef None admin] PATH_INFO: `/v3/roles` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:43.974593 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be5dde3-e34f-46e0-94e9-f8b23c47c8ef None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:43.974909 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be5dde3-e34f-46e0-94e9-f8b23c47c8ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:43.976324 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be5dde3-e34f-46e0-94e9-f8b23c47c8ef None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:43.990850 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 205/407] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:51:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:51:43.996188 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0961ee-cbdf-4d43-8612-3f8be0d2d8f2 None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:44.032559 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be5dde3-e34f-46e0-94e9-f8b23c47c8ef None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:44.132781 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0961ee-cbdf-4d43-8612-3f8be0d2d8f2 None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:44.171516 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3708ec9f-a609-416b-8f5f-db9c5debcc0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:44.172397 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3708ec9f-a609-416b-8f5f-db9c5debcc0a None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:44.172687 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3708ec9f-a609-416b-8f5f-db9c5debcc0a None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:44.172974 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3708ec9f-a609-416b-8f5f-db9c5debcc0a None admin] PATH_INFO: `/v3/groups/ee1c47ac2fd14de2abac0ec34d47b7d9` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:44.174922 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3708ec9f-a609-416b-8f5f-db9c5debcc0a None admin] RBAC: Authorizing `identity:get_group(group_id=ee1c47ac2fd14de2abac0ec34d47b7d9)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:44.176474 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3708ec9f-a609-416b-8f5f-db9c5debcc0a None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:44.178419 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 203/408] 10.222.0.57 () {56 vars in 1180 bytes} [Thu May 19 14:51:43 2022] GET /identity/v3/groups/ee1c47ac2fd14de2abac0ec34d47b7d9 => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:44.186920 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3708ec9f-a609-416b-8f5f-db9c5debcc0a None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:44.248789 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be5dde3-e34f-46e0-94e9-f8b23c47c8ef None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:44.252770 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:44.318868 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49194f8-1d19-4b75-8caf-785940456c65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:44.319702 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49194f8-1d19-4b75-8caf-785940456c65 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:44.320000 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49194f8-1d19-4b75-8caf-785940456c65 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:44.320283 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49194f8-1d19-4b75-8caf-785940456c65 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:44.344122 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49194f8-1d19-4b75-8caf-785940456c65 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:51:44.352651 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-f49194f8-1d19-4b75-8caf-785940456c65 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:51:44.352651 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:51:44.352651 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:51:44.352651 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:51:44.352651 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:51:44.352651 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:51:44.352651 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:51:44.352651 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:51:44.352651 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:51:44.352651 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:51:44.352651 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:51:44.352651 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:51:44.352651 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:51:44.352651 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:51:44.352651 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:51:44.352651 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:51:44.354997 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:51:44.354997 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:51:44.354997 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:51:44.354997 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:51:44.354997 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:51:44.354997 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:51:44.354997 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:51:44.354997 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:51:44.354997 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:51:44.354997 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:51:44.354997 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:51:44.354997 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:51:44.354997 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:51:44.354997 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:51:44.354997 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:51:44.354997 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:51:44.357129 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:51:44.357129 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:51:44.357129 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:51:44.357129 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:51:44.357129 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:51:44.357129 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:51:44.357129 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:51:44.357129 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:51:44.357129 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:51:44.357129 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:51:44.357129 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:51:44.357129 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:51:44.357129 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 206/409] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:51:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:51:44.362998 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49194f8-1d19-4b75-8caf-785940456c65 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:44.387760 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3708ec9f-a609-416b-8f5f-db9c5debcc0a None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:44.422198 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:44.494691 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec18c9d-7ae0-4fbe-83cd-0601153d1a5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:44.495951 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec18c9d-7ae0-4fbe-83cd-0601153d1a5a None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:44.496480 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec18c9d-7ae0-4fbe-83cd-0601153d1a5a None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:44.496970 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec18c9d-7ae0-4fbe-83cd-0601153d1a5a None admin] PATH_INFO: `/v3/projects/caf48e96564645f3b3e590d230116a2d` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:44.498707 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec18c9d-7ae0-4fbe-83cd-0601153d1a5a None admin] RBAC: Authorizing `identity:get_project(project_id=caf48e96564645f3b3e590d230116a2d)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:44.500419 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec18c9d-7ae0-4fbe-83cd-0601153d1a5a None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:44.501387 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49194f8-1d19-4b75-8caf-785940456c65 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:44.502751 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 204/410] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:51:44 2022] GET /identity/v3/projects/caf48e96564645f3b3e590d230116a2d => generated 302 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:44.510787 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec18c9d-7ae0-4fbe-83cd-0601153d1a5a None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:44.541287 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918054ec-50f6-4db3-bdd8-dc0479739a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:44.542035 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918054ec-50f6-4db3-bdd8-dc0479739a3c None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:44.542392 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918054ec-50f6-4db3-bdd8-dc0479739a3c None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:44.542674 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918054ec-50f6-4db3-bdd8-dc0479739a3c None admin] PATH_INFO: `/v3/roles` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:44.543653 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918054ec-50f6-4db3-bdd8-dc0479739a3c None admin] RBAC: Adding query filter params (name=admin) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:44.544096 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918054ec-50f6-4db3-bdd8-dc0479739a3c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:44.545510 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918054ec-50f6-4db3-bdd8-dc0479739a3c None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:44.556629 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 207/411] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:51:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:51:44.562156 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918054ec-50f6-4db3-bdd8-dc0479739a3c None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:44.718200 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:44.730725 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918054ec-50f6-4db3-bdd8-dc0479739a3c None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:44.768157 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec18c9d-7ae0-4fbe-83cd-0601153d1a5a None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:44.770145 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:44.799856 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21805b29-e18d-4acd-805e-1bf2fc8986d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:44.801278 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21805b29-e18d-4acd-805e-1bf2fc8986d8 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:44.801578 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21805b29-e18d-4acd-805e-1bf2fc8986d8 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:44.801885 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21805b29-e18d-4acd-805e-1bf2fc8986d8 None admin] PATH_INFO: `/v3/groups/32251723ef49480496da49da30172c34` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:44.803124 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21805b29-e18d-4acd-805e-1bf2fc8986d8 None admin] RBAC: Authorizing `identity:get_group(group_id=32251723ef49480496da49da30172c34)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:44.804581 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21805b29-e18d-4acd-805e-1bf2fc8986d8 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:44.806733 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 208/412] 10.222.0.57 () {56 vars in 1180 bytes} [Thu May 19 14:51:44 2022] GET /identity/v3/groups/32251723ef49480496da49da30172c34 => generated 229 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:44.812016 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21805b29-e18d-4acd-805e-1bf2fc8986d8 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:44.852976 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c21aff-6bf9-491d-8654-5c2a31f6adf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:44.854011 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c21aff-6bf9-491d-8654-5c2a31f6adf9 None admin] REQUEST_METHOD: `PUT` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:44.854303 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c21aff-6bf9-491d-8654-5c2a31f6adf9 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:44.854583 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c21aff-6bf9-491d-8654-5c2a31f6adf9 None admin] PATH_INFO: `/v3/projects/caf48e96564645f3b3e590d230116a2d/groups/ee1c47ac2fd14de2abac0ec34d47b7d9/roles/3b9c36a010504ba7bc63a8ccf0a9bca0` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:44.859788 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c21aff-6bf9-491d-8654-5c2a31f6adf9 None admin] RBAC: Authorizing `identity:create_grant(project_id=caf48e96564645f3b3e590d230116a2d, group_id=ee1c47ac2fd14de2abac0ec34d47b7d9, role_id=3b9c36a010504ba7bc63a8ccf0a9bca0)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:44.861342 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c21aff-6bf9-491d-8654-5c2a31f6adf9 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:44.894999 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 205/413] 10.222.0.57 () {58 vars in 1442 bytes} [Thu May 19 14:51:44 2022] PUT /identity/v3/projects/caf48e96564645f3b3e590d230116a2d/groups/ee1c47ac2fd14de2abac0ec34d47b7d9/roles/3b9c36a010504ba7bc63a8ccf0a9bca0 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:51:44.931426 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21805b29-e18d-4acd-805e-1bf2fc8986d8 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:44.969241 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546812f6-4fa7-47a2-b7e9-4ba659d9ed64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:44.970151 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546812f6-4fa7-47a2-b7e9-4ba659d9ed64 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:44.970445 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546812f6-4fa7-47a2-b7e9-4ba659d9ed64 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:44.970825 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546812f6-4fa7-47a2-b7e9-4ba659d9ed64 None admin] PATH_INFO: `/v3/projects/f00ebb3222e9428fb4f634c779ecb9a3` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:44.972861 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546812f6-4fa7-47a2-b7e9-4ba659d9ed64 None admin] RBAC: Authorizing `identity:get_project(project_id=f00ebb3222e9428fb4f634c779ecb9a3)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:44.974466 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546812f6-4fa7-47a2-b7e9-4ba659d9ed64 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:44.976621 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 209/414] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:51:44 2022] GET /identity/v3/projects/f00ebb3222e9428fb4f634c779ecb9a3 => generated 344 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:44.982275 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c21aff-6bf9-491d-8654-5c2a31f6adf9 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:45.151722 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c21aff-6bf9-491d-8654-5c2a31f6adf9 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:45.206893 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43522ac0-214f-4344-9fdb-460b06b690ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:45.208333 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43522ac0-214f-4344-9fdb-460b06b690ef None admin] REQUEST_METHOD: `PUT` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:45.208790 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43522ac0-214f-4344-9fdb-460b06b690ef None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:45.209238 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43522ac0-214f-4344-9fdb-460b06b690ef None admin] PATH_INFO: `/v3/projects/f00ebb3222e9428fb4f634c779ecb9a3/groups/32251723ef49480496da49da30172c34/roles/028379e01f9245e9b8fc56e8b9e8a6ad` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:45.211501 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43522ac0-214f-4344-9fdb-460b06b690ef None admin] RBAC: Authorizing `identity:create_grant(project_id=f00ebb3222e9428fb4f634c779ecb9a3, group_id=32251723ef49480496da49da30172c34, role_id=028379e01f9245e9b8fc56e8b9e8a6ad)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:45.213117 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43522ac0-214f-4344-9fdb-460b06b690ef None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:45.229624 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 206/415] 10.222.0.57 () {58 vars in 1442 bytes} [Thu May 19 14:51:44 2022] PUT /identity/v3/projects/f00ebb3222e9428fb4f634c779ecb9a3/groups/32251723ef49480496da49da30172c34/roles/028379e01f9245e9b8fc56e8b9e8a6ad => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:51:45.640214 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458bd781-4c61-4c5b-b654-437c31a0eb9c None None] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:45.640778 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458bd781-4c61-4c5b-b654-437c31a0eb9c None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:45.641014 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458bd781-4c61-4c5b-b654-437c31a0eb9c None None] PATH_INFO: `/` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:45.642730 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 210/416] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:51:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:51:45.648684 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a63d34c-a1c9-4f27-9133-68b5e6db074f None None] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:45.648984 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a63d34c-a1c9-4f27-9133-68b5e6db074f None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:45.649278 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a63d34c-a1c9-4f27-9133-68b5e6db074f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:45.674312 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.auth.core [None req-8a63d34c-a1c9-4f27-9133-68b5e6db074f None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:51:45.675558 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-8a63d34c-a1c9-4f27-9133-68b5e6db074f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:51:45.694568 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6cacdf-a768-4d2d-b778-4e991c578210 None None] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:45.694866 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6cacdf-a768-4d2d-b778-4e991c578210 None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:45.695177 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6cacdf-a768-4d2d-b778-4e991c578210 None None] PATH_INFO: `/` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:45.697100 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 211/417] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:51:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:51:45.699141 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:45.703008 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458a99a0-975e-43f0-a427-6760f8ee5c08 None None] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:45.703305 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458a99a0-975e-43f0-a427-6760f8ee5c08 None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:45.703593 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458a99a0-975e-43f0-a427-6760f8ee5c08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:45.738893 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.auth.core [None req-458a99a0-975e-43f0-a427-6760f8ee5c08 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:51:45.740124 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.fernet_utils [None req-458a99a0-975e-43f0-a427-6760f8ee5c08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:51:45.760409 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 207/418] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:51:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:51:45.782216 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:45.803287 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a63d34c-a1c9-4f27-9133-68b5e6db074f None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:45.872158 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 212/419] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:51:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:51:45.916840 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458a99a0-975e-43f0-a427-6760f8ee5c08 None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:45.936792 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a63d34c-a1c9-4f27-9133-68b5e6db074f None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:46.008555 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62adcf87-0e35-4537-b319-0028ea33d8e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:46.009387 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62adcf87-0e35-4537-b319-0028ea33d8e1 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:46.009677 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62adcf87-0e35-4537-b319-0028ea33d8e1 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:46.009987 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62adcf87-0e35-4537-b319-0028ea33d8e1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:46.033233 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62adcf87-0e35-4537-b319-0028ea33d8e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:51:46.047982 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-62adcf87-0e35-4537-b319-0028ea33d8e1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:51:46.047982 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:51:46.047982 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:51:46.047982 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:51:46.047982 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:51:46.047982 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:51:46.047982 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:51:46.047982 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:51:46.047982 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:51:46.047982 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:51:46.047982 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:51:46.047982 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:51:46.047982 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:51:46.047982 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:51:46.047982 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:51:46.047982 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:51:46.049741 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:51:46.049741 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:51:46.049741 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:51:46.049741 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:51:46.049741 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:51:46.049741 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:51:46.049741 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:51:46.049741 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:51:46.049741 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:51:46.049741 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:51:46.049741 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:51:46.049741 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:51:46.049741 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:51:46.049741 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:51:46.049741 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:51:46.049741 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:51:46.050738 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:51:46.050738 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:51:46.050738 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:51:46.050738 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:51:46.050738 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:51:46.050738 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:51:46.050738 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:51:46.050738 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:51:46.050738 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:51:46.050738 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:51:46.050738 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:51:46.050738 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:51:46.051446 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 208/420] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:51:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:51:46.067413 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62adcf87-0e35-4537-b319-0028ea33d8e1 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:46.091734 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458a99a0-975e-43f0-a427-6760f8ee5c08 None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:46.096862 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:46.133333 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32786e1-812a-4e46-bad0-1d2e3876d13a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:46.134179 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32786e1-812a-4e46-bad0-1d2e3876d13a None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:46.134474 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32786e1-812a-4e46-bad0-1d2e3876d13a None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:46.134759 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32786e1-812a-4e46-bad0-1d2e3876d13a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:46.146273 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32786e1-812a-4e46-bad0-1d2e3876d13a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:51:46.155033 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-f32786e1-812a-4e46-bad0-1d2e3876d13a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:51:46.155033 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:51:46.155033 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:51:46.155033 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:51:46.155033 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:51:46.155033 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:51:46.155033 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:51:46.155033 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:51:46.155033 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:51:46.155033 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:51:46.155033 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:51:46.155033 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:51:46.155033 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:51:46.155033 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:51:46.155033 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:51:46.155033 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:51:46.157348 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:51:46.157348 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:51:46.157348 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:51:46.157348 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:51:46.157348 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:51:46.157348 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:51:46.157348 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:51:46.157348 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:51:46.157348 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:51:46.157348 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:51:46.157348 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:51:46.157348 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:51:46.157348 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:51:46.157348 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:51:46.157348 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:51:46.157348 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:51:46.162784 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:51:46.162784 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:51:46.162784 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:51:46.162784 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:51:46.162784 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:51:46.162784 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:51:46.162784 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:51:46.162784 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:51:46.162784 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:51:46.162784 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:51:46.162784 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:51:46.162784 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:51:46.162784 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 213/421] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 19 14:51:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:51:46.162784 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32786e1-812a-4e46-bad0-1d2e3876d13a None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:46.184491 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:46.231157 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62adcf87-0e35-4537-b319-0028ea33d8e1 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:46.271864 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e5337a-c6ed-4a84-8e82-1903d168bc4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:46.273055 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e5337a-c6ed-4a84-8e82-1903d168bc4d None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:46.273567 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e5337a-c6ed-4a84-8e82-1903d168bc4d None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:46.274148 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e5337a-c6ed-4a84-8e82-1903d168bc4d None admin] PATH_INFO: `/v3/roles` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:46.275128 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e5337a-c6ed-4a84-8e82-1903d168bc4d None admin] RBAC: Adding query filter params (name=member) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:46.275655 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e5337a-c6ed-4a84-8e82-1903d168bc4d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:46.276656 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32786e1-812a-4e46-bad0-1d2e3876d13a None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:46.277533 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e5337a-c6ed-4a84-8e82-1903d168bc4d None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:46.287617 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 209/422] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:51:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:51:46.289393 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cd350f-b1ba-4531-b291-85f88124e995 None None] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:46.289955 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cd350f-b1ba-4531-b291-85f88124e995 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:46.290556 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cd350f-b1ba-4531-b291-85f88124e995 None None] PATH_INFO: `/` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:46.296611 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 210/423] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:51:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:51:46.297651 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cd350f-b1ba-4531-b291-85f88124e995 None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:46.313470 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431b2de1-4090-48c0-bfd2-cb65832e2a03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:46.314281 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431b2de1-4090-48c0-bfd2-cb65832e2a03 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:46.314716 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431b2de1-4090-48c0-bfd2-cb65832e2a03 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:46.315481 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431b2de1-4090-48c0-bfd2-cb65832e2a03 None admin] PATH_INFO: `/v3/roles` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:46.316358 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431b2de1-4090-48c0-bfd2-cb65832e2a03 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:46.316636 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431b2de1-4090-48c0-bfd2-cb65832e2a03 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:46.318060 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431b2de1-4090-48c0-bfd2-cb65832e2a03 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:46.327865 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 214/424] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:51:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:46.329113 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ec789f-65c5-450c-8a42-a9e75abdd118 None None] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:46.329409 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ec789f-65c5-450c-8a42-a9e75abdd118 None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:46.329692 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ec789f-65c5-450c-8a42-a9e75abdd118 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:46.352929 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.auth.core [None req-f9ec789f-65c5-450c-8a42-a9e75abdd118 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:51:46.354332 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.fernet_utils [None req-f9ec789f-65c5-450c-8a42-a9e75abdd118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:51:46.391446 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:46.422667 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cd350f-b1ba-4531-b291-85f88124e995 None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:46.431188 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 215/425] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:51:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:51:46.432337 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ec789f-65c5-450c-8a42-a9e75abdd118 None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:46.450514 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:46.490815 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5eb0e8-65c3-4339-92c3-939c4e7479e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:46.491483 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5eb0e8-65c3-4339-92c3-939c4e7479e6 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:46.491776 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5eb0e8-65c3-4339-92c3-939c4e7479e6 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:46.492059 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5eb0e8-65c3-4339-92c3-939c4e7479e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:46.570270 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ec789f-65c5-450c-8a42-a9e75abdd118 None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:46.579365 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5eb0e8-65c3-4339-92c3-939c4e7479e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:46.581456 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5eb0e8-65c3-4339-92c3-939c4e7479e6 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:46.620386 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdba29d3-b921-4edd-a635-d80fed712b6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:46.621340 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdba29d3-b921-4edd-a635-d80fed712b6a None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:46.622250 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdba29d3-b921-4edd-a635-d80fed712b6a None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:46.622539 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdba29d3-b921-4edd-a635-d80fed712b6a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:46.673126 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:46.702814 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 211/426] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:51:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:51:46.703908 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5eb0e8-65c3-4339-92c3-939c4e7479e6 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:46.722369 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdba29d3-b921-4edd-a635-d80fed712b6a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:46.724400 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdba29d3-b921-4edd-a635-d80fed712b6a None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:46.744628 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:46.823585 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5eb0e8-65c3-4339-92c3-939c4e7479e6 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:46.871023 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 216/427] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:51:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:51:46.872362 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdba29d3-b921-4edd-a635-d80fed712b6a None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:46.874796 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c819b439-5129-4068-bf3d-60c3ef6f72eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:46.877002 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c819b439-5129-4068-bf3d-60c3ef6f72eb None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:46.877562 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c819b439-5129-4068-bf3d-60c3ef6f72eb None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:46.878037 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c819b439-5129-4068-bf3d-60c3ef6f72eb None admin] PATH_INFO: `/v3/roles/member` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:46.892345 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c819b439-5129-4068-bf3d-60c3ef6f72eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:51:46.903954 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-c819b439-5129-4068-bf3d-60c3ef6f72eb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:51:46.903954 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:51:46.903954 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:51:46.903954 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:51:46.903954 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:51:46.903954 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:51:46.903954 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:51:46.903954 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:51:46.903954 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:51:46.903954 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:51:46.903954 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:51:46.903954 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:51:46.903954 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:51:46.903954 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:51:46.903954 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:51:46.903954 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:51:46.905190 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:51:46.905190 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:51:46.905190 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:51:46.905190 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:51:46.905190 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:51:46.905190 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:51:46.905190 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:51:46.905190 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:51:46.905190 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:51:46.905190 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:51:46.905190 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:51:46.905190 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:51:46.905190 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:51:46.905190 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:51:46.905190 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:51:46.905190 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:51:46.906342 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:51:46.906342 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:51:46.906342 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:51:46.906342 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:51:46.906342 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:51:46.906342 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:51:46.906342 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:51:46.906342 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:51:46.906342 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:51:46.906342 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:51:46.906342 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:51:46.906342 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:51:46.910083 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 212/428] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:51:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:51:46.910828 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c819b439-5129-4068-bf3d-60c3ef6f72eb None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:46.959849 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:46.987069 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdba29d3-b921-4edd-a635-d80fed712b6a None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:46.998770 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:47.028741 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a56f63d-be73-48cb-9942-dcc0a90e596c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:47.030000 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a56f63d-be73-48cb-9942-dcc0a90e596c None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:47.030371 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a56f63d-be73-48cb-9942-dcc0a90e596c None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:47.030707 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a56f63d-be73-48cb-9942-dcc0a90e596c None admin] PATH_INFO: `/v3/projects/caf48e96564645f3b3e590d230116a2d` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:47.032228 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a56f63d-be73-48cb-9942-dcc0a90e596c None admin] RBAC: Authorizing `identity:get_project(project_id=caf48e96564645f3b3e590d230116a2d)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:47.034504 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a56f63d-be73-48cb-9942-dcc0a90e596c None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:47.036659 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 217/429] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:51:46 2022] GET /identity/v3/projects/caf48e96564645f3b3e590d230116a2d => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:47.038255 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a56f63d-be73-48cb-9942-dcc0a90e596c None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:47.053356 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c819b439-5129-4068-bf3d-60c3ef6f72eb None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:47.097425 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3eb11c-c2eb-4ba4-be29-0c681f8663b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:47.098741 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3eb11c-c2eb-4ba4-be29-0c681f8663b4 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:47.099272 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3eb11c-c2eb-4ba4-be29-0c681f8663b4 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:47.099745 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3eb11c-c2eb-4ba4-be29-0c681f8663b4 None admin] PATH_INFO: `/v3/projects/aca8075ce986433eae657055d1366699` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:47.101173 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3eb11c-c2eb-4ba4-be29-0c681f8663b4 None admin] RBAC: Authorizing `identity:get_project(project_id=aca8075ce986433eae657055d1366699)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:47.102932 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3eb11c-c2eb-4ba4-be29-0c681f8663b4 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:47.109497 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 213/430] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:51:46 2022] GET /identity/v3/projects/aca8075ce986433eae657055d1366699 => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:47.110575 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3eb11c-c2eb-4ba4-be29-0c681f8663b4 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:47.175775 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a56f63d-be73-48cb-9942-dcc0a90e596c None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:47.216896 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c55181d-e5f0-4cfa-ae2e-ffd37aa688d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:47.217940 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c55181d-e5f0-4cfa-ae2e-ffd37aa688d4 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:47.218437 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c55181d-e5f0-4cfa-ae2e-ffd37aa688d4 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:47.218906 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c55181d-e5f0-4cfa-ae2e-ffd37aa688d4 None admin] PATH_INFO: `/v3/roles` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:47.219856 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c55181d-e5f0-4cfa-ae2e-ffd37aa688d4 None admin] RBAC: Adding query filter params (name=member) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:47.220362 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c55181d-e5f0-4cfa-ae2e-ffd37aa688d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:47.222686 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c55181d-e5f0-4cfa-ae2e-ffd37aa688d4 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:47.232197 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 218/431] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:51:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:47.233475 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c55181d-e5f0-4cfa-ae2e-ffd37aa688d4 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:47.235859 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3eb11c-c2eb-4ba4-be29-0c681f8663b4 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:47.239702 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:47.284066 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a189de-7f3f-4c6a-aa92-bcaddfdfc188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:47.284987 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a189de-7f3f-4c6a-aa92-bcaddfdfc188 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:47.285919 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:47.286038 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a189de-7f3f-4c6a-aa92-bcaddfdfc188 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:47.286936 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a189de-7f3f-4c6a-aa92-bcaddfdfc188 None admin] PATH_INFO: `/v3/groups/ee1c47ac2fd14de2abac0ec34d47b7d9` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:47.288281 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a189de-7f3f-4c6a-aa92-bcaddfdfc188 None admin] RBAC: Authorizing `identity:get_group(group_id=ee1c47ac2fd14de2abac0ec34d47b7d9)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:47.290416 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a189de-7f3f-4c6a-aa92-bcaddfdfc188 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:47.292465 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 214/432] 10.222.0.57 () {56 vars in 1180 bytes} [Thu May 19 14:51:47 2022] GET /identity/v3/groups/ee1c47ac2fd14de2abac0ec34d47b7d9 => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:47.293050 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a189de-7f3f-4c6a-aa92-bcaddfdfc188 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:47.364936 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c55181d-e5f0-4cfa-ae2e-ffd37aa688d4 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:47.402693 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e49481-562a-4d0b-bda7-5d74df4926d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:47.403961 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e49481-562a-4d0b-bda7-5d74df4926d2 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:47.404467 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e49481-562a-4d0b-bda7-5d74df4926d2 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:47.405022 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e49481-562a-4d0b-bda7-5d74df4926d2 None admin] PATH_INFO: `/v3/groups/ee1c47ac2fd14de2abac0ec34d47b7d9` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:47.406497 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e49481-562a-4d0b-bda7-5d74df4926d2 None admin] RBAC: Authorizing `identity:get_group(group_id=ee1c47ac2fd14de2abac0ec34d47b7d9)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:47.408166 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e49481-562a-4d0b-bda7-5d74df4926d2 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:47.415673 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 219/433] 10.222.0.57 () {56 vars in 1180 bytes} [Thu May 19 14:51:47 2022] GET /identity/v3/groups/ee1c47ac2fd14de2abac0ec34d47b7d9 => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:47.416935 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e49481-562a-4d0b-bda7-5d74df4926d2 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:47.425014 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a189de-7f3f-4c6a-aa92-bcaddfdfc188 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:47.473447 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73e02b8-b836-49da-8f30-1870ee168d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:47.474558 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73e02b8-b836-49da-8f30-1870ee168d1b None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:47.475125 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73e02b8-b836-49da-8f30-1870ee168d1b None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:47.475652 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73e02b8-b836-49da-8f30-1870ee168d1b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:47.522421 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:47.551801 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e49481-562a-4d0b-bda7-5d74df4926d2 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:47.556651 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73e02b8-b836-49da-8f30-1870ee168d1b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:47.558493 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73e02b8-b836-49da-8f30-1870ee168d1b None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:47.563438 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:47.593739 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62f41c6-0171-4303-b316-64099f3d7133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:47.594624 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62f41c6-0171-4303-b316-64099f3d7133 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:47.595098 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62f41c6-0171-4303-b316-64099f3d7133 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:47.595505 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62f41c6-0171-4303-b316-64099f3d7133 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:47.596674 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62f41c6-0171-4303-b316-64099f3d7133 None admin] RBAC: Adding query filter params (group.id=ee1c47ac2fd14de2abac0ec34d47b7d9, role.id=aed9aa9ef7f7448bb3b40c9ad082d545, scope.project.id=caf48e96564645f3b3e590d230116a2d) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:47.597182 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62f41c6-0171-4303-b316-64099f3d7133 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:47.598786 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62f41c6-0171-4303-b316-64099f3d7133 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:47.606058 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 220/434] 10.222.0.57 () {56 vars in 1376 bytes} [Thu May 19 14:51:47 2022] GET /identity/v3/role_assignments?group.id=ee1c47ac2fd14de2abac0ec34d47b7d9&scope.project.id=caf48e96564645f3b3e590d230116a2d&role.id=aed9aa9ef7f7448bb3b40c9ad082d545 => generated 611 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:47.607014 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62f41c6-0171-4303-b316-64099f3d7133 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:47.692936 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 215/435] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:51:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:51:47.698975 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71694610-da35-490f-b7c4-876e9653b661 None None] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:47.699357 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71694610-da35-490f-b7c4-876e9653b661 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:47.699565 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71694610-da35-490f-b7c4-876e9653b661 None None] PATH_INFO: `/` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:47.701146 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 216/436] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:51:47 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:51:47.701887 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71694610-da35-490f-b7c4-876e9653b661 None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:47.756428 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62f41c6-0171-4303-b316-64099f3d7133 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:47.794942 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:47.804015 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab809f3-6b94-41ac-9f88-badfad841aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:47.805679 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab809f3-6b94-41ac-9f88-badfad841aa3 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:47.806179 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab809f3-6b94-41ac-9f88-badfad841aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:47.806620 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab809f3-6b94-41ac-9f88-badfad841aa3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:47.807643 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab809f3-6b94-41ac-9f88-badfad841aa3 None admin] RBAC: Adding query filter params (group.id=ee1c47ac2fd14de2abac0ec34d47b7d9, role.id=3b9c36a010504ba7bc63a8ccf0a9bca0, scope.project.id=aca8075ce986433eae657055d1366699) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:47.808565 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab809f3-6b94-41ac-9f88-badfad841aa3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:47.810945 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab809f3-6b94-41ac-9f88-badfad841aa3 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:47.820444 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 221/437] 10.222.0.57 () {56 vars in 1376 bytes} [Thu May 19 14:51:47 2022] GET /identity/v3/role_assignments?group.id=ee1c47ac2fd14de2abac0ec34d47b7d9&scope.project.id=aca8075ce986433eae657055d1366699&role.id=3b9c36a010504ba7bc63a8ccf0a9bca0 => generated 611 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:47.822344 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda43393-20ed-472a-b23e-8826707c1346 None None] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:47.822855 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda43393-20ed-472a-b23e-8826707c1346 None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:47.823297 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda43393-20ed-472a-b23e-8826707c1346 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:47.843261 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.auth.core [None req-cda43393-20ed-472a-b23e-8826707c1346 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:51:47.844762 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.fernet_utils [None req-cda43393-20ed-472a-b23e-8826707c1346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:51:47.860107 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71694610-da35-490f-b7c4-876e9653b661 None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:47.882292 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:47.904377 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d587bb57-2af3-42d2-a560-176db8d7f39b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:47.904973 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d587bb57-2af3-42d2-a560-176db8d7f39b None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:47.905267 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d587bb57-2af3-42d2-a560-176db8d7f39b None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:47.905550 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d587bb57-2af3-42d2-a560-176db8d7f39b None admin] PATH_INFO: `/v3/projects/aca8075ce986433eae657055d1366699` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:47.907026 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d587bb57-2af3-42d2-a560-176db8d7f39b None admin] RBAC: Authorizing `identity:get_project(project_id=aca8075ce986433eae657055d1366699)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:47.910668 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d587bb57-2af3-42d2-a560-176db8d7f39b None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:47.913149 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 217/438] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:51:47 2022] GET /identity/v3/projects/aca8075ce986433eae657055d1366699 => generated 298 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:47.923621 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d587bb57-2af3-42d2-a560-176db8d7f39b None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:47.944526 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 222/439] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:51:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:51:47.971909 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee24639-de3e-40cb-8505-b65aca5a3608 None None] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:47.972204 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee24639-de3e-40cb-8505-b65aca5a3608 None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:47.972518 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee24639-de3e-40cb-8505-b65aca5a3608 None None] PATH_INFO: `/` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:47.974444 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 223/440] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:51:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:51:47.975069 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee24639-de3e-40cb-8505-b65aca5a3608 None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:48.054087 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d587bb57-2af3-42d2-a560-176db8d7f39b None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:48.095015 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96978462-dde6-4c02-ab1d-554adc44a91a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:48.096087 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96978462-dde6-4c02-ab1d-554adc44a91a None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:48.096330 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96978462-dde6-4c02-ab1d-554adc44a91a None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:48.096616 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96978462-dde6-4c02-ab1d-554adc44a91a None admin] PATH_INFO: `/v3/groups/ee1c47ac2fd14de2abac0ec34d47b7d9` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:48.098557 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96978462-dde6-4c02-ab1d-554adc44a91a None admin] RBAC: Authorizing `identity:get_group(group_id=ee1c47ac2fd14de2abac0ec34d47b7d9)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:48.100041 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96978462-dde6-4c02-ab1d-554adc44a91a None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:48.101365 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee24639-de3e-40cb-8505-b65aca5a3608 None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:48.101743 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 218/441] 10.222.0.57 () {56 vars in 1180 bytes} [Thu May 19 14:51:47 2022] GET /identity/v3/groups/ee1c47ac2fd14de2abac0ec34d47b7d9 => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:48.103156 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faebfacc-4379-4878-b987-a56e42310127 None None] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:48.103452 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faebfacc-4379-4878-b987-a56e42310127 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:48.103741 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faebfacc-4379-4878-b987-a56e42310127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:48.115705 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:48.122323 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.auth.core [None req-faebfacc-4379-4878-b987-a56e42310127 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:51:48.123527 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-faebfacc-4379-4878-b987-a56e42310127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:51:48.138518 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bfad22-fd6c-4033-831f-090e89d4d4fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:48.139334 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bfad22-fd6c-4033-831f-090e89d4d4fb None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:48.139622 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bfad22-fd6c-4033-831f-090e89d4d4fb None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:48.139900 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bfad22-fd6c-4033-831f-090e89d4d4fb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:48.151062 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bfad22-fd6c-4033-831f-090e89d4d4fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:51:48.160071 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-58bfad22-fd6c-4033-831f-090e89d4d4fb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:51:48.160071 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:51:48.160071 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:51:48.160071 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:51:48.160071 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:51:48.160071 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:51:48.160071 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:51:48.160071 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:51:48.160071 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:51:48.160071 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:51:48.160071 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:51:48.160071 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:51:48.160071 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:51:48.160071 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:51:48.160071 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:51:48.160071 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:51:48.161286 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:51:48.161286 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:51:48.161286 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:51:48.161286 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:51:48.161286 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:51:48.161286 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:51:48.161286 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:51:48.161286 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:51:48.161286 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:51:48.161286 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:51:48.161286 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:51:48.161286 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:51:48.161286 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:51:48.161286 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:51:48.161286 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:51:48.161286 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:51:48.162403 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:51:48.162403 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:51:48.162403 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:51:48.162403 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:51:48.162403 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:51:48.162403 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:51:48.162403 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:51:48.162403 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:51:48.162403 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:51:48.162403 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:51:48.162403 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:51:48.162403 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:51:48.162403 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 224/442] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 19 14:51:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:51:48.163194 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bfad22-fd6c-4033-831f-090e89d4d4fb None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:48.191466 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 219/443] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:51:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:51:48.192602 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faebfacc-4379-4878-b987-a56e42310127 None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:48.197149 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:48.276153 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bfad22-fd6c-4033-831f-090e89d4d4fb None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:48.295031 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faebfacc-4379-4878-b987-a56e42310127 None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:48.309309 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e607b7c-7406-4941-94ba-4a45147ab109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:48.309931 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e607b7c-7406-4941-94ba-4a45147ab109 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:48.310220 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e607b7c-7406-4941-94ba-4a45147ab109 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:48.310498 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e607b7c-7406-4941-94ba-4a45147ab109 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:48.311330 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e607b7c-7406-4941-94ba-4a45147ab109 None admin] RBAC: Adding query filter params (group.id=ee1c47ac2fd14de2abac0ec34d47b7d9, role.id=aed9aa9ef7f7448bb3b40c9ad082d545, scope.project.id=aca8075ce986433eae657055d1366699) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:48.311647 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e607b7c-7406-4941-94ba-4a45147ab109 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:48.313101 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e607b7c-7406-4941-94ba-4a45147ab109 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:48.319074 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 225/444] 10.222.0.57 () {56 vars in 1376 bytes} [Thu May 19 14:51:48 2022] GET /identity/v3/role_assignments?group.id=ee1c47ac2fd14de2abac0ec34d47b7d9&scope.project.id=aca8075ce986433eae657055d1366699&role.id=aed9aa9ef7f7448bb3b40c9ad082d545 => generated 611 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:48.319949 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e607b7c-7406-4941-94ba-4a45147ab109 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:48.331200 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ca1354-a904-42ec-a3a6-b41cd85cab2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:48.332230 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ca1354-a904-42ec-a3a6-b41cd85cab2a None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:48.332633 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ca1354-a904-42ec-a3a6-b41cd85cab2a None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:48.332987 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ca1354-a904-42ec-a3a6-b41cd85cab2a None admin] PATH_INFO: `/v3/roles` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:48.334349 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ca1354-a904-42ec-a3a6-b41cd85cab2a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:48.334673 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ca1354-a904-42ec-a3a6-b41cd85cab2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:48.336489 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ca1354-a904-42ec-a3a6-b41cd85cab2a None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:48.339194 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:48.348269 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 220/445] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:51:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:48.353934 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ca1354-a904-42ec-a3a6-b41cd85cab2a None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:48.432172 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:48.443086 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e607b7c-7406-4941-94ba-4a45147ab109 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:48.466504 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ca1354-a904-42ec-a3a6-b41cd85cab2a None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:48.482331 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5abf9b9-d22c-4f15-9358-db942468f8a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:48.483150 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5abf9b9-d22c-4f15-9358-db942468f8a6 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:48.483441 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5abf9b9-d22c-4f15-9358-db942468f8a6 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:48.483722 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5abf9b9-d22c-4f15-9358-db942468f8a6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:48.495894 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5abf9b9-d22c-4f15-9358-db942468f8a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:51:48.503709 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-e5abf9b9-d22c-4f15-9358-db942468f8a6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:51:48.503709 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:51:48.503709 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:51:48.503709 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:51:48.503709 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:51:48.503709 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:51:48.503709 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:51:48.503709 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:51:48.503709 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:51:48.503709 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:51:48.503709 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:51:48.503709 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:51:48.503709 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:51:48.503709 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:51:48.503709 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:51:48.503709 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:51:48.504831 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:51:48.504831 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:51:48.504831 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:51:48.504831 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:51:48.504831 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:51:48.504831 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:51:48.504831 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:51:48.504831 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:51:48.504831 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:51:48.504831 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:51:48.504831 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:51:48.504831 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:51:48.504831 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:51:48.504831 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:51:48.504831 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:51:48.504831 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:51:48.505758 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:51:48.505758 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:51:48.505758 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:51:48.505758 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:51:48.505758 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:51:48.505758 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:51:48.505758 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:51:48.505758 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:51:48.505758 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:51:48.505758 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:51:48.505758 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:51:48.505758 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:51:48.506535 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d85c09d-2f26-452b-9504-514f0f118458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:48.506535 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d85c09d-2f26-452b-9504-514f0f118458 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:48.506535 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d85c09d-2f26-452b-9504-514f0f118458 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:48.506535 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d85c09d-2f26-452b-9504-514f0f118458 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:48.510077 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 226/446] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:51:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:51:48.511043 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5abf9b9-d22c-4f15-9358-db942468f8a6 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:48.589683 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:48.593622 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d85c09d-2f26-452b-9504-514f0f118458 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:48.595394 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d85c09d-2f26-452b-9504-514f0f118458 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:48.645814 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5abf9b9-d22c-4f15-9358-db942468f8a6 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:48.692133 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a08252-c7ca-443c-b684-6660136e710f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:48.692906 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a08252-c7ca-443c-b684-6660136e710f None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:48.693197 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a08252-c7ca-443c-b684-6660136e710f None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:48.693475 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a08252-c7ca-443c-b684-6660136e710f None admin] PATH_INFO: `/v3/roles` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:48.694289 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a08252-c7ca-443c-b684-6660136e710f None admin] RBAC: Adding query filter params (name=admin) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:48.694601 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a08252-c7ca-443c-b684-6660136e710f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:48.696274 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a08252-c7ca-443c-b684-6660136e710f None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:48.705107 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 227/447] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:51:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:48.713108 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a08252-c7ca-443c-b684-6660136e710f None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:48.728156 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 221/448] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:51:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 14:51:48.730683 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:48.733999 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d85c09d-2f26-452b-9504-514f0f118458 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:48.844748 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a08252-c7ca-443c-b684-6660136e710f None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:48.858563 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:48.864112 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d85c09d-2f26-452b-9504-514f0f118458 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:48.888300 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49ff845-47aa-4406-9271-fce34b77111b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:48.888964 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49ff845-47aa-4406-9271-fce34b77111b None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:48.889253 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49ff845-47aa-4406-9271-fce34b77111b None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:48.889541 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49ff845-47aa-4406-9271-fce34b77111b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:48.910829 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c4cff7-6d92-492f-818b-15b40899e598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:48.911824 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c4cff7-6d92-492f-818b-15b40899e598 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:48.913258 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c4cff7-6d92-492f-818b-15b40899e598 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:48.913258 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c4cff7-6d92-492f-818b-15b40899e598 None admin] PATH_INFO: `/v3/projects/caf48e96564645f3b3e590d230116a2d` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:48.914344 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c4cff7-6d92-492f-818b-15b40899e598 None admin] RBAC: Authorizing `identity:get_project(project_id=caf48e96564645f3b3e590d230116a2d)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:48.916307 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c4cff7-6d92-492f-818b-15b40899e598 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:48.918468 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 222/449] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:51:48 2022] GET /identity/v3/projects/caf48e96564645f3b3e590d230116a2d => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:48.923769 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c4cff7-6d92-492f-818b-15b40899e598 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:48.977268 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49ff845-47aa-4406-9271-fce34b77111b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:48.978925 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49ff845-47aa-4406-9271-fce34b77111b None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:48.996574 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:49.047928 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c4cff7-6d92-492f-818b-15b40899e598 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:49.084968 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9820e4-b870-4382-ad1c-0ab597eae3d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:49.085867 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9820e4-b870-4382-ad1c-0ab597eae3d8 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:49.086161 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9820e4-b870-4382-ad1c-0ab597eae3d8 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:49.086449 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9820e4-b870-4382-ad1c-0ab597eae3d8 None admin] PATH_INFO: `/v3/groups/ee1c47ac2fd14de2abac0ec34d47b7d9` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:49.087647 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9820e4-b870-4382-ad1c-0ab597eae3d8 None admin] RBAC: Authorizing `identity:get_group(group_id=ee1c47ac2fd14de2abac0ec34d47b7d9)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:49.089394 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9820e4-b870-4382-ad1c-0ab597eae3d8 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:49.091081 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 223/450] 10.222.0.57 () {56 vars in 1180 bytes} [Thu May 19 14:51:48 2022] GET /identity/v3/groups/ee1c47ac2fd14de2abac0ec34d47b7d9 => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:51:49.092261 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 228/451] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:51:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 14:51:49.097008 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9820e4-b870-4382-ad1c-0ab597eae3d8 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:49.098118 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49ff845-47aa-4406-9271-fce34b77111b None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:49.132220 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:49.200516 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49ff845-47aa-4406-9271-fce34b77111b None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:49.200921 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9820e4-b870-4382-ad1c-0ab597eae3d8 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:49.213643 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:49.234305 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d050c122-2c8f-460d-a407-fe9721d7852c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:49.234711 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b302fa5d-c3a4-4a84-8334-02a2128239d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:49.235164 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d050c122-2c8f-460d-a407-fe9721d7852c None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:49.235366 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b302fa5d-c3a4-4a84-8334-02a2128239d9 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:49.235469 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d050c122-2c8f-460d-a407-fe9721d7852c None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:49.235653 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b302fa5d-c3a4-4a84-8334-02a2128239d9 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:49.235759 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d050c122-2c8f-460d-a407-fe9721d7852c None admin] PATH_INFO: `/v3/projects/f00ebb3222e9428fb4f634c779ecb9a3` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:49.235935 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b302fa5d-c3a4-4a84-8334-02a2128239d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:49.237396 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b302fa5d-c3a4-4a84-8334-02a2128239d9 None admin] RBAC: Adding query filter params (group.id=ee1c47ac2fd14de2abac0ec34d47b7d9, role.id=3b9c36a010504ba7bc63a8ccf0a9bca0, scope.project.id=caf48e96564645f3b3e590d230116a2d) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:49.237743 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d050c122-2c8f-460d-a407-fe9721d7852c None admin] RBAC: Authorizing `identity:get_project(project_id=f00ebb3222e9428fb4f634c779ecb9a3)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:49.238232 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b302fa5d-c3a4-4a84-8334-02a2128239d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:49.238970 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d050c122-2c8f-460d-a407-fe9721d7852c None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:49.239651 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b302fa5d-c3a4-4a84-8334-02a2128239d9 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:49.245129 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 229/452] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:51:49 2022] GET /identity/v3/projects/f00ebb3222e9428fb4f634c779ecb9a3 => generated 344 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:49.261881 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 224/453] 10.222.0.57 () {56 vars in 1376 bytes} [Thu May 19 14:51:49 2022] GET /identity/v3/role_assignments?group.id=ee1c47ac2fd14de2abac0ec34d47b7d9&scope.project.id=caf48e96564645f3b3e590d230116a2d&role.id=3b9c36a010504ba7bc63a8ccf0a9bca0 => generated 611 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:51:49.262152 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d050c122-2c8f-460d-a407-fe9721d7852c None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:49.358467 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d050c122-2c8f-460d-a407-fe9721d7852c None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:49.396850 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc3f69a-be36-4f36-a798-d608afafa017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:49.397727 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc3f69a-be36-4f36-a798-d608afafa017 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:49.398049 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc3f69a-be36-4f36-a798-d608afafa017 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:49.398333 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc3f69a-be36-4f36-a798-d608afafa017 None admin] PATH_INFO: `/v3/groups/32251723ef49480496da49da30172c34` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:49.399596 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc3f69a-be36-4f36-a798-d608afafa017 None admin] RBAC: Authorizing `identity:get_group(group_id=32251723ef49480496da49da30172c34)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:49.401062 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc3f69a-be36-4f36-a798-d608afafa017 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:49.402767 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 230/454] 10.222.0.57 () {56 vars in 1180 bytes} [Thu May 19 14:51:49 2022] GET /identity/v3/groups/32251723ef49480496da49da30172c34 => generated 229 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:49.409074 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b302fa5d-c3a4-4a84-8334-02a2128239d9 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:49.513798 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:49.519777 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b302fa5d-c3a4-4a84-8334-02a2128239d9 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:49.553908 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0193878-5ce1-4fcc-bcf1-23c61ce38596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:49.554587 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0193878-5ce1-4fcc-bcf1-23c61ce38596 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:49.554875 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0193878-5ce1-4fcc-bcf1-23c61ce38596 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:49.555158 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0193878-5ce1-4fcc-bcf1-23c61ce38596 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:49.556002 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0193878-5ce1-4fcc-bcf1-23c61ce38596 None admin] RBAC: Adding query filter params (group.id=32251723ef49480496da49da30172c34, role.id=028379e01f9245e9b8fc56e8b9e8a6ad, scope.project.id=f00ebb3222e9428fb4f634c779ecb9a3) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:49.556323 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0193878-5ce1-4fcc-bcf1-23c61ce38596 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:49.557762 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0193878-5ce1-4fcc-bcf1-23c61ce38596 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:49.563849 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 225/455] 10.222.0.57 () {56 vars in 1376 bytes} [Thu May 19 14:51:49 2022] GET /identity/v3/role_assignments?group.id=32251723ef49480496da49da30172c34&scope.project.id=f00ebb3222e9428fb4f634c779ecb9a3&role.id=028379e01f9245e9b8fc56e8b9e8a6ad => generated 611 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:52.451347 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067faf93-42f8-45cb-878f-678735909fac None None] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:52.451884 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067faf93-42f8-45cb-878f-678735909fac None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:52.452001 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067faf93-42f8-45cb-878f-678735909fac None None] PATH_INFO: `/` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:52.453294 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 231/456] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:51:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:51:52.460332 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3eb6ef-f1ff-42d5-af0c-4c4009e59716 None None] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:52.460332 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3eb6ef-f1ff-42d5-af0c-4c4009e59716 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:52.460332 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3eb6ef-f1ff-42d5-af0c-4c4009e59716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:52.478004 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfbb646-a48e-4286-87f2-69a6dfd6de4c None None] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:52.478635 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfbb646-a48e-4286-87f2-69a6dfd6de4c None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:52.479127 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfbb646-a48e-4286-87f2-69a6dfd6de4c None None] PATH_INFO: `/` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:52.480740 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 232/457] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:51:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:51:52.486150 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cdc05a-c01d-4b7e-bc1c-72ca81a377f4 None None] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:52.486530 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cdc05a-c01d-4b7e-bc1c-72ca81a377f4 None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:52.486822 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cdc05a-c01d-4b7e-bc1c-72ca81a377f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:52.490937 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.auth.core [None req-de3eb6ef-f1ff-42d5-af0c-4c4009e59716 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:51:52.492513 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-de3eb6ef-f1ff-42d5-af0c-4c4009e59716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:51:52.505033 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.auth.core [None req-43cdc05a-c01d-4b7e-bc1c-72ca81a377f4 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:51:52.506345 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.fernet_utils [None req-43cdc05a-c01d-4b7e-bc1c-72ca81a377f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:51:52.562332 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:52.587634 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 233/458] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:51:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:51:52.619064 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 226/459] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:51:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:51:52.628549 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cdc05a-c01d-4b7e-bc1c-72ca81a377f4 None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:52.645393 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3eb6ef-f1ff-42d5-af0c-4c4009e59716 None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:52.760050 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:52.772749 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3eb6ef-f1ff-42d5-af0c-4c4009e59716 None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:52.810672 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e7fc54-dd0b-4975-8f50-137a7f1be9b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:52.811491 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e7fc54-dd0b-4975-8f50-137a7f1be9b6 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:52.811788 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e7fc54-dd0b-4975-8f50-137a7f1be9b6 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:52.812078 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e7fc54-dd0b-4975-8f50-137a7f1be9b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:52.823732 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e7fc54-dd0b-4975-8f50-137a7f1be9b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:52.825283 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e7fc54-dd0b-4975-8f50-137a7f1be9b6 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:52.844433 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-a8e7fc54-dd0b-4975-8f50-137a7f1be9b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:51:52.844433 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:51:52.844433 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:51:52.844433 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:51:52.844433 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:51:52.844433 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:51:52.844433 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:51:52.844433 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:51:52.844433 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:51:52.844433 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:51:52.844433 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:51:52.844433 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:51:52.844433 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:51:52.844433 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:51:52.844433 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:51:52.845502 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:51:52.845502 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:51:52.845502 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:51:52.845502 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:51:52.845502 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:51:52.845502 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:51:52.845502 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:51:52.845502 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:51:52.845502 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:51:52.845502 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:51:52.845502 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:51:52.845502 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:51:52.845502 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:51:52.845502 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:51:52.845502 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:51:52.845502 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:51:52.845502 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:51:52.846826 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:51:52.846826 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:51:52.846826 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:51:52.846826 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:51:52.846826 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 227/460] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:51:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:51:52.853869 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e7fc54-dd0b-4975-8f50-137a7f1be9b6 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:52.867710 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cdc05a-c01d-4b7e-bc1c-72ca81a377f4 None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:52.948632 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54cb28d-905d-4fe8-b59c-10649e50de93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:52.950809 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54cb28d-905d-4fe8-b59c-10649e50de93 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:52.951118 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54cb28d-905d-4fe8-b59c-10649e50de93 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:52.951417 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54cb28d-905d-4fe8-b59c-10649e50de93 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:52.975587 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54cb28d-905d-4fe8-b59c-10649e50de93 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:52.977571 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54cb28d-905d-4fe8-b59c-10649e50de93 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:53.014704 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-d54cb28d-905d-4fe8-b59c-10649e50de93 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:51:53.014704 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:51:53.014704 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:51:53.014704 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:51:53.014704 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:51:53.014704 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:51:53.014704 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:51:53.014704 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:51:53.014704 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:51:53.014704 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:51:53.014704 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:51:53.014704 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:51:53.014704 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:51:53.014704 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:51:53.014704 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:51:53.015731 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:51:53.015731 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:51:53.015731 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:51:53.015731 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:51:53.015731 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:51:53.015731 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:51:53.015731 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:51:53.015731 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:51:53.015731 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:51:53.015731 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:51:53.015731 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:51:53.015731 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:51:53.015731 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:51:53.015731 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:51:53.015731 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:51:53.015731 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:51:53.015731 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:51:53.016745 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:51:53.016745 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:51:53.016745 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:51:53.016745 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:51:53.019844 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 234/461] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:51:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 392 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:51:53.029862 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e7fc54-dd0b-4975-8f50-137a7f1be9b6 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:53.030049 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54cb28d-905d-4fe8-b59c-10649e50de93 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:53.078093 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:53.085637 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89fe73d-dac0-420d-9149-e39edfb80682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:53.086353 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89fe73d-dac0-420d-9149-e39edfb80682 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:53.086638 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89fe73d-dac0-420d-9149-e39edfb80682 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:53.086917 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89fe73d-dac0-420d-9149-e39edfb80682 None admin] PATH_INFO: `/v3/domains` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:53.087655 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89fe73d-dac0-420d-9149-e39edfb80682 None admin] RBAC: Adding query filter params (name=Default) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:53.087965 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89fe73d-dac0-420d-9149-e39edfb80682 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:53.089359 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89fe73d-dac0-420d-9149-e39edfb80682 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:53.104166 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 228/462] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:51:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:53.105564 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bdc3ee-b0d5-4809-9ab9-b6a1c83d57a7 None None] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:53.105880 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bdc3ee-b0d5-4809-9ab9-b6a1c83d57a7 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:53.106284 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bdc3ee-b0d5-4809-9ab9-b6a1c83d57a7 None None] PATH_INFO: `/` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:53.107836 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 229/463] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:51:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:51:53.110494 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bdc3ee-b0d5-4809-9ab9-b6a1c83d57a7 None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:53.115250 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:53.178382 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54cb28d-905d-4fe8-b59c-10649e50de93 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:53.217779 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e7fc36-7884-4013-b001-9239d2028aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:53.218480 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e7fc36-7884-4013-b001-9239d2028aa8 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:53.218808 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e7fc36-7884-4013-b001-9239d2028aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:53.219047 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e7fc36-7884-4013-b001-9239d2028aa8 None admin] PATH_INFO: `/v3/domains` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:53.219782 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e7fc36-7884-4013-b001-9239d2028aa8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:53.220096 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e7fc36-7884-4013-b001-9239d2028aa8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:53.222113 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e7fc36-7884-4013-b001-9239d2028aa8 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:53.223950 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bdc3ee-b0d5-4809-9ab9-b6a1c83d57a7 None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:53.246009 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 235/464] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:51:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:51:53.246009 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26ded7e-c529-465b-8ff8-2d5ceefade62 None None] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:53.246009 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26ded7e-c529-465b-8ff8-2d5ceefade62 None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:53.246009 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26ded7e-c529-465b-8ff8-2d5ceefade62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:53.263179 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.auth.core [None req-e26ded7e-c529-465b-8ff8-2d5ceefade62 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:51:53.264392 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.fernet_utils [None req-e26ded7e-c529-465b-8ff8-2d5ceefade62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:51:53.270528 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626fe965-516d-464a-b6bd-979950927ac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:53.271224 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626fe965-516d-464a-b6bd-979950927ac3 None admin] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:53.271523 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626fe965-516d-464a-b6bd-979950927ac3 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:53.271791 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626fe965-516d-464a-b6bd-979950927ac3 None admin] PATH_INFO: `/v3/users` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:53.272507 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626fe965-516d-464a-b6bd-979950927ac3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:53.273977 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626fe965-516d-464a-b6bd-979950927ac3 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:53.297902 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 230/465] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 19 14:51:53 2022] POST /identity/v3/users => generated 248 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 14:51:53.300251 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3501f760-cf4a-491a-957b-987c96c88e24 None None] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:53.301055 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3501f760-cf4a-491a-957b-987c96c88e24 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:53.301611 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3501f760-cf4a-491a-957b-987c96c88e24 None None] PATH_INFO: `/` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:53.307542 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 231/466] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:51:53 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:51:53.308427 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3501f760-cf4a-491a-957b-987c96c88e24 None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:53.353646 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 236/467] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:51:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:51:53.355337 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6f7b35-1edc-4139-a9cc-4d01ade90ba7 None None] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:53.355756 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6f7b35-1edc-4139-a9cc-4d01ade90ba7 None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:53.356043 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6f7b35-1edc-4139-a9cc-4d01ade90ba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:53.374878 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.auth.core [None req-ae6f7b35-1edc-4139-a9cc-4d01ade90ba7 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:51:53.376091 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.fernet_utils [None req-ae6f7b35-1edc-4139-a9cc-4d01ade90ba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:51:53.390783 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:53.400723 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:53.454511 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3501f760-cf4a-491a-957b-987c96c88e24 None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:53.481885 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 237/468] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:51:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:51:53.481885 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6f7b35-1edc-4139-a9cc-4d01ade90ba7 None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:53.520926 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58a5bd4-7495-4f62-8fea-2c7b4b87d480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:53.522369 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58a5bd4-7495-4f62-8fea-2c7b4b87d480 None admin] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:53.522914 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58a5bd4-7495-4f62-8fea-2c7b4b87d480 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:53.523617 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58a5bd4-7495-4f62-8fea-2c7b4b87d480 None admin] PATH_INFO: `/v3/users` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:53.524682 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58a5bd4-7495-4f62-8fea-2c7b4b87d480 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:53.526602 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58a5bd4-7495-4f62-8fea-2c7b4b87d480 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:53.556220 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 232/469] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 19 14:51:53 2022] POST /identity/v3/users => generated 246 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:51:53.558253 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58a5bd4-7495-4f62-8fea-2c7b4b87d480 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:53.605627 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6f7b35-1edc-4139-a9cc-4d01ade90ba7 None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:53.661255 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e415dba-bc6e-4750-88c8-53763be6c34c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:53.662191 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e415dba-bc6e-4750-88c8-53763be6c34c None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:53.662490 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e415dba-bc6e-4750-88c8-53763be6c34c None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:53.662772 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e415dba-bc6e-4750-88c8-53763be6c34c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:53.682839 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e415dba-bc6e-4750-88c8-53763be6c34c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:53.684348 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e415dba-bc6e-4750-88c8-53763be6c34c None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:53.706640 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-1e415dba-bc6e-4750-88c8-53763be6c34c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:51:53.706640 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:51:53.706640 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:51:53.706640 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:51:53.706640 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:51:53.706640 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:51:53.706640 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:51:53.706640 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:51:53.706640 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:51:53.706640 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:51:53.706640 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:51:53.706640 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:51:53.706640 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:51:53.706640 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:51:53.706640 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:51:53.707702 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:51:53.707702 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:51:53.707702 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:51:53.707702 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:51:53.707702 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:51:53.707702 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:51:53.707702 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:51:53.707702 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:51:53.707702 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:51:53.707702 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:51:53.707702 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:51:53.707702 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:51:53.707702 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:51:53.707702 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:51:53.707702 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:51:53.707702 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:51:53.707702 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:51:53.708703 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:51:53.708703 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:51:53.708703 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:51:53.708703 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:51:53.708703 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 238/470] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:51:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:51:53.714344 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e415dba-bc6e-4750-88c8-53763be6c34c None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:53.715867 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58a5bd4-7495-4f62-8fea-2c7b4b87d480 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:53.717435 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:53.738405 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:53.765044 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9879dc-cf8e-473c-9bc6-36f47a8cbc04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:53.765896 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9879dc-cf8e-473c-9bc6-36f47a8cbc04 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:53.766191 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9879dc-cf8e-473c-9bc6-36f47a8cbc04 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:53.766471 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9879dc-cf8e-473c-9bc6-36f47a8cbc04 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:53.779821 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9879dc-cf8e-473c-9bc6-36f47a8cbc04 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:53.781244 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9879dc-cf8e-473c-9bc6-36f47a8cbc04 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:53.801376 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-0c9879dc-cf8e-473c-9bc6-36f47a8cbc04 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:51:53.801376 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:51:53.801376 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:51:53.801376 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:51:53.801376 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:51:53.801376 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:51:53.801376 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:51:53.801376 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:51:53.801376 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:51:53.801376 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:51:53.801376 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:51:53.801376 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:51:53.801376 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:51:53.801376 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:51:53.801376 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:51:53.803691 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:51:53.803691 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:51:53.803691 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:51:53.803691 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:51:53.803691 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:51:53.803691 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:51:53.803691 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:51:53.803691 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:51:53.803691 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:51:53.803691 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:51:53.803691 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:51:53.803691 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:51:53.803691 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:51:53.803691 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:51:53.803691 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:51:53.803691 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:51:53.803691 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:51:53.805939 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:51:53.805939 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:51:53.805939 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:51:53.805939 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:51:53.810826 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 233/471] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:51:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:51:53.810826 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9879dc-cf8e-473c-9bc6-36f47a8cbc04 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:53.854270 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e415dba-bc6e-4750-88c8-53763be6c34c None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:53.898389 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a070b4c-e7a0-446a-8d11-13ae5fc6e05e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:53.899062 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a070b4c-e7a0-446a-8d11-13ae5fc6e05e None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:53.899349 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a070b4c-e7a0-446a-8d11-13ae5fc6e05e None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:53.899786 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a070b4c-e7a0-446a-8d11-13ae5fc6e05e None admin] PATH_INFO: `/v3/domains` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:53.901061 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a070b4c-e7a0-446a-8d11-13ae5fc6e05e None admin] RBAC: Adding query filter params (name=Default) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:53.901387 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a070b4c-e7a0-446a-8d11-13ae5fc6e05e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:53.903213 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a070b4c-e7a0-446a-8d11-13ae5fc6e05e None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:53.922140 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 239/472] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:51:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:53.928692 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a070b4c-e7a0-446a-8d11-13ae5fc6e05e None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:53.946730 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9879dc-cf8e-473c-9bc6-36f47a8cbc04 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:53.985547 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a645e349-d4ad-40bf-b251-4d8919c82b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:53.986871 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a645e349-d4ad-40bf-b251-4d8919c82b77 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:53.987167 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a645e349-d4ad-40bf-b251-4d8919c82b77 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:53.987448 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a645e349-d4ad-40bf-b251-4d8919c82b77 None admin] PATH_INFO: `/v3/domains` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:53.988767 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a645e349-d4ad-40bf-b251-4d8919c82b77 None admin] RBAC: Adding query filter params (name=Default) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:53.989029 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a645e349-d4ad-40bf-b251-4d8919c82b77 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:53.990466 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a645e349-d4ad-40bf-b251-4d8919c82b77 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:54.003882 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 234/473] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:51:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:51:54.005214 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5540fcab-ed0b-4dbd-b375-181e8835afc2 None None] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:54.005518 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5540fcab-ed0b-4dbd-b375-181e8835afc2 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:54.005806 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5540fcab-ed0b-4dbd-b375-181e8835afc2 None None] PATH_INFO: `/` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:54.009588 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 235/474] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:51:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:51:54.009588 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5540fcab-ed0b-4dbd-b375-181e8835afc2 None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:54.012466 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:54.045704 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:54.069879 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a070b4c-e7a0-446a-8d11-13ae5fc6e05e None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:54.112916 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16384be-e902-4601-8e61-593e5a425084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:54.114253 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16384be-e902-4601-8e61-593e5a425084 None admin] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:54.114539 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16384be-e902-4601-8e61-593e5a425084 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:54.114820 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16384be-e902-4601-8e61-593e5a425084 None admin] PATH_INFO: `/v3/users` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:54.115957 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16384be-e902-4601-8e61-593e5a425084 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:54.117707 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16384be-e902-4601-8e61-593e5a425084 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:54.147032 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 240/475] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 19 14:51:53 2022] POST /identity/v3/users => generated 249 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:51:54.148742 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebf16a6-77c6-44dc-97b6-3dc4bed982a0 None None] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:54.149041 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebf16a6-77c6-44dc-97b6-3dc4bed982a0 None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:54.149325 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebf16a6-77c6-44dc-97b6-3dc4bed982a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:54.155373 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5540fcab-ed0b-4dbd-b375-181e8835afc2 None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:54.187720 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.auth.core [None req-eebf16a6-77c6-44dc-97b6-3dc4bed982a0 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:51:54.190362 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.fernet_utils [None req-eebf16a6-77c6-44dc-97b6-3dc4bed982a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:51:54.255614 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4253876-5601-49fa-89ea-ed0cdf9c8e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:54.256786 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4253876-5601-49fa-89ea-ed0cdf9c8e32 None admin] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:54.257332 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4253876-5601-49fa-89ea-ed0cdf9c8e32 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:54.257962 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4253876-5601-49fa-89ea-ed0cdf9c8e32 None admin] PATH_INFO: `/v3/users` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:54.258964 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4253876-5601-49fa-89ea-ed0cdf9c8e32 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:54.260786 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4253876-5601-49fa-89ea-ed0cdf9c8e32 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:54.316425 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 236/476] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 19 14:51:54 2022] POST /identity/v3/users => generated 248 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:51:54.319684 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 241/477] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:51:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:51:54.347536 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4253876-5601-49fa-89ea-ed0cdf9c8e32 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:54.463316 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:54.483856 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4253876-5601-49fa-89ea-ed0cdf9c8e32 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:54.528518 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a20e253-f415-41d3-9fa6-f690d89a0d57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:54.529214 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a20e253-f415-41d3-9fa6-f690d89a0d57 None admin] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:54.529678 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a20e253-f415-41d3-9fa6-f690d89a0d57 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:54.530168 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a20e253-f415-41d3-9fa6-f690d89a0d57 None admin] PATH_INFO: `/v3/roles` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:54.531102 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a20e253-f415-41d3-9fa6-f690d89a0d57 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:54.532656 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a20e253-f415-41d3-9fa6-f690d89a0d57 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:54.548207 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.sql.core [None req-1a20e253-f415-41d3-9fa6-f690d89a0d57 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 19 14:51:54.548207 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 19 14:51:54.548207 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [parameters: {'id': '1337d033b2024b6b85f82c64d822fa91', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 19 14:51:54.548207 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=281785) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 19 14:51:54.566742 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-1a20e253-f415-41d3-9fa6-f690d89a0d57 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 19 14:51:54.566742 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:51:54.566742 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 14:51:54.566742 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 14:51:54.566742 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 14:51:54.566742 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 14:51:54.566742 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 14:51:54.566742 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application result = self._query(query) May 19 14:51:54.566742 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 14:51:54.566742 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application conn.query(q) May 19 14:51:54.566742 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 14:51:54.566742 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 14:51:54.566742 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 14:51:54.566742 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application result.read() May 19 14:51:54.566742 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 14:51:54.567959 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 14:51:54.567959 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 14:51:54.567959 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 14:51:54.567959 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 14:51:54.567959 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 14:51:54.567959 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 14:51:54.567959 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 14:51:54.567959 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 19 14:51:54.567959 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:51:54.567959 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 19 14:51:54.567959 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:51:54.567959 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:51:54.567959 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 19 14:51:54.567959 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 19 14:51:54.567959 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role May 19 14:51:54.567959 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return ref.to_dict() May 19 14:51:54.567959 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 14:51:54.567959 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application next(self.gen) May 19 14:51:54.569357 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 19 14:51:54.569357 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application yield resource May 19 14:51:54.569357 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 14:51:54.569357 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application next(self.gen) May 19 14:51:54.569357 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 19 14:51:54.569357 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application self.session.rollback() May 19 14:51:54.569357 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 19 14:51:54.569357 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application self.force_reraise() May 19 14:51:54.569357 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 19 14:51:54.569357 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise self.value May 19 14:51:54.569357 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 19 14:51:54.569357 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 19 14:51:54.569357 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 19 14:51:54.569357 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application session.commit() May 19 14:51:54.569357 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 19 14:51:54.569357 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 19 14:51:54.569357 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 19 14:51:54.570548 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application self._prepare_impl() May 19 14:51:54.570548 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 19 14:51:54.570548 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application self.session.flush() May 19 14:51:54.570548 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 19 14:51:54.570548 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application self._flush(objects) May 19 14:51:54.570548 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 19 14:51:54.570548 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 19 14:51:54.570548 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 19 14:51:54.570548 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application compat.raise_( May 19 14:51:54.570548 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 14:51:54.570548 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception May 19 14:51:54.570548 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 19 14:51:54.570548 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application flush_context.execute() May 19 14:51:54.570548 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 19 14:51:54.570548 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rec.execute(self) May 19 14:51:54.570548 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 19 14:51:54.570548 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 19 14:51:54.571901 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 19 14:51:54.571901 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application _emit_insert_statements( May 19 14:51:54.571901 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 19 14:51:54.571901 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application c = connection._execute_20( May 19 14:51:54.571901 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 19 14:51:54.571901 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 19 14:51:54.571901 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 19 14:51:54.571901 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 19 14:51:54.571901 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 19 14:51:54.571901 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application ret = self._execute_context( May 19 14:51:54.571901 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 19 14:51:54.571901 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 19 14:51:54.571901 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 19 14:51:54.571901 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 19 14:51:54.571901 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 14:51:54.571901 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception May 19 14:51:54.572938 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 14:51:54.572938 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 14:51:54.572938 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 14:51:54.572938 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 14:51:54.572938 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 14:51:54.572938 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application result = self._query(query) May 19 14:51:54.572938 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 14:51:54.572938 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application conn.query(q) May 19 14:51:54.572938 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 14:51:54.572938 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 14:51:54.572938 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 14:51:54.572938 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application result.read() May 19 14:51:54.572938 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 14:51:54.572938 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 14:51:54.572938 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 14:51:54.572938 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 14:51:54.574987 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 14:51:54.574987 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 14:51:54.574987 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 14:51:54.574987 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 14:51:54.574987 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 19 14:51:54.574987 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 19 14:51:54.574987 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [parameters: {'id': '1337d033b2024b6b85f82c64d822fa91', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 19 14:51:54.574987 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 19 14:51:54.574987 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:51:54.574987 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:51:54.574987 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:51:54.574987 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:51:54.574987 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:51:54.574987 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:51:54.574987 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:51:54.574987 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:51:54.577206 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:51:54.577206 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:51:54.577206 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:51:54.577206 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:51:54.577206 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:51:54.577206 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:51:54.577206 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post May 19 14:51:54.577206 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( May 19 14:51:54.577206 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:51:54.577206 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:51:54.577206 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role May 19 14:51:54.577206 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) May 19 14:51:54.577206 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 19 14:51:54.577206 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 19 14:51:54.577206 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 19 14:51:54.577206 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:51:54.580336 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 237/478] 10.222.0.57 () {60 vars in 1131 bytes} [Thu May 19 14:51:54 2022] POST /identity/v3/roles => generated 144 bytes in 228 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 19 14:51:54.581381 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebf16a6-77c6-44dc-97b6-3dc4bed982a0 None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:54.666387 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:54.724480 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebf16a6-77c6-44dc-97b6-3dc4bed982a0 None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:54.761736 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c747cd75-4940-4842-8549-469779e3a81e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:54.762978 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c747cd75-4940-4842-8549-469779e3a81e None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:54.763541 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c747cd75-4940-4842-8549-469779e3a81e None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:54.764033 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c747cd75-4940-4842-8549-469779e3a81e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:54.776019 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c747cd75-4940-4842-8549-469779e3a81e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:51:54.784751 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-c747cd75-4940-4842-8549-469779e3a81e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:51:54.784751 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:51:54.784751 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:51:54.784751 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:51:54.784751 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:51:54.784751 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:51:54.784751 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:51:54.784751 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:51:54.784751 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:51:54.784751 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:51:54.784751 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:51:54.784751 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:51:54.784751 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:51:54.784751 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:51:54.784751 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:51:54.784751 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:51:54.785828 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:51:54.785828 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:51:54.785828 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:51:54.785828 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:51:54.785828 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:51:54.785828 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:51:54.785828 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:51:54.785828 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:51:54.785828 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:51:54.785828 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:51:54.785828 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:51:54.785828 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:51:54.785828 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:51:54.785828 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:51:54.785828 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:51:54.785828 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:51:54.786805 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:51:54.786805 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:51:54.786805 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:51:54.786805 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:51:54.786805 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:51:54.786805 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:51:54.786805 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:51:54.786805 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:51:54.786805 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:51:54.786805 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:51:54.786805 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:51:54.786805 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:51:54.789673 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 242/479] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 19 14:51:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:51:54.799031 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a20e253-f415-41d3-9fa6-f690d89a0d57 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:54.961832 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a20e253-f415-41d3-9fa6-f690d89a0d57 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:55.005696 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de2b06c-a73d-4cfd-9f7b-780f5c5cbcb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:55.006878 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de2b06c-a73d-4cfd-9f7b-780f5c5cbcb1 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:55.007405 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de2b06c-a73d-4cfd-9f7b-780f5c5cbcb1 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:55.007946 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de2b06c-a73d-4cfd-9f7b-780f5c5cbcb1 None admin] PATH_INFO: `/v3/roles` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:55.008926 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de2b06c-a73d-4cfd-9f7b-780f5c5cbcb1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:55.009453 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de2b06c-a73d-4cfd-9f7b-780f5c5cbcb1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:55.011204 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de2b06c-a73d-4cfd-9f7b-780f5c5cbcb1 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:55.023653 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 238/480] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:51:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:56.231866 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aaa5dd-beec-437e-9721-d71ea87e2485 None None] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:56.232904 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aaa5dd-beec-437e-9721-d71ea87e2485 None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:56.233408 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aaa5dd-beec-437e-9721-d71ea87e2485 None None] PATH_INFO: `/` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:56.235336 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 243/481] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:51:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:51:56.241325 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79240c81-acdb-4b50-8a43-9205032d5c72 None None] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:56.242248 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79240c81-acdb-4b50-8a43-9205032d5c72 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:56.242745 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79240c81-acdb-4b50-8a43-9205032d5c72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:56.262306 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.auth.core [None req-79240c81-acdb-4b50-8a43-9205032d5c72 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:51:56.264106 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-79240c81-acdb-4b50-8a43-9205032d5c72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:51:56.268031 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:56.343085 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 239/482] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:51:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:51:56.367917 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8aaa5dd-beec-437e-9721-d71ea87e2485 None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:56.478148 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8aaa5dd-beec-437e-9721-d71ea87e2485 None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:56.483848 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f96df9-e917-40d8-b05c-d552aa9f05d5 None None] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:56.484398 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f96df9-e917-40d8-b05c-d552aa9f05d5 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:56.484894 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f96df9-e917-40d8-b05c-d552aa9f05d5 None None] PATH_INFO: `/` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:56.487407 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 240/483] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:51:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:51:56.493895 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:56.494074 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2001a6-4b6e-4ed7-b78c-405e175aa3e4 None None] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:56.494074 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2001a6-4b6e-4ed7-b78c-405e175aa3e4 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:56.494074 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2001a6-4b6e-4ed7-b78c-405e175aa3e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:56.526953 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.auth.core [None req-1d2001a6-4b6e-4ed7-b78c-405e175aa3e4 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:51:56.528168 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-1d2001a6-4b6e-4ed7-b78c-405e175aa3e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:51:56.540449 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a96815e-fe56-4dcc-b7f3-add4e643e734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:56.541571 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a96815e-fe56-4dcc-b7f3-add4e643e734 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:56.542107 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a96815e-fe56-4dcc-b7f3-add4e643e734 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:56.542598 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a96815e-fe56-4dcc-b7f3-add4e643e734 None admin] PATH_INFO: `/v3/roles/service` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:56.561832 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a96815e-fe56-4dcc-b7f3-add4e643e734 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:51:56.577042 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-1a96815e-fe56-4dcc-b7f3-add4e643e734 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:51:56.577042 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:51:56.577042 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:51:56.577042 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:51:56.577042 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:51:56.577042 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:51:56.577042 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:51:56.577042 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:51:56.577042 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:51:56.577042 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:51:56.577042 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:51:56.577042 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:51:56.577042 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:51:56.577042 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:51:56.577042 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:51:56.577042 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:51:56.578125 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:51:56.578125 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:51:56.578125 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:51:56.578125 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:51:56.578125 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:51:56.578125 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:51:56.578125 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:51:56.578125 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:51:56.578125 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:51:56.578125 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:51:56.578125 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:51:56.578125 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:51:56.578125 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:51:56.578125 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:51:56.578125 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:51:56.578125 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:51:56.579071 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:51:56.579071 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:51:56.579071 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:51:56.579071 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:51:56.579071 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:51:56.579071 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:51:56.579071 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:51:56.579071 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:51:56.579071 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:51:56.579071 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:51:56.579071 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:51:56.579071 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:51:56.579786 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 244/484] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 14:51:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:51:56.587808 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a96815e-fe56-4dcc-b7f3-add4e643e734 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:56.647957 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 241/485] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:51:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:51:56.701038 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2001a6-4b6e-4ed7-b78c-405e175aa3e4 None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:56.747603 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a96815e-fe56-4dcc-b7f3-add4e643e734 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:56.813363 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74dd30e-40c3-4d9e-a296-11e81d62b64f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:56.814078 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74dd30e-40c3-4d9e-a296-11e81d62b64f None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:56.814926 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74dd30e-40c3-4d9e-a296-11e81d62b64f None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:56.815228 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74dd30e-40c3-4d9e-a296-11e81d62b64f None admin] PATH_INFO: `/v3/roles` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:56.816186 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74dd30e-40c3-4d9e-a296-11e81d62b64f None admin] RBAC: Adding query filter params (name=service) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:56.816504 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74dd30e-40c3-4d9e-a296-11e81d62b64f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:56.817952 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74dd30e-40c3-4d9e-a296-11e81d62b64f None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:56.824025 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:56.843567 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 245/486] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 19 14:51:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:51:56.851938 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74dd30e-40c3-4d9e-a296-11e81d62b64f None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:56.884997 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2001a6-4b6e-4ed7-b78c-405e175aa3e4 None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:56.928125 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:56.932213 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007fb2c9-a2c1-495d-bda7-ffa8b07582cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:56.933035 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007fb2c9-a2c1-495d-bda7-ffa8b07582cd None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:56.933329 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007fb2c9-a2c1-495d-bda7-ffa8b07582cd None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:56.933612 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007fb2c9-a2c1-495d-bda7-ffa8b07582cd None admin] PATH_INFO: `/v3/roles/service` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:56.963358 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007fb2c9-a2c1-495d-bda7-ffa8b07582cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:51:56.977803 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-007fb2c9-a2c1-495d-bda7-ffa8b07582cd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:51:56.977803 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:51:56.977803 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:51:56.977803 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:51:56.977803 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:51:56.977803 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:51:56.977803 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:51:56.977803 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:51:56.977803 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:51:56.977803 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:51:56.977803 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:51:56.977803 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:51:56.977803 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:51:56.977803 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:51:56.977803 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:51:56.977803 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:51:56.978901 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:51:56.978901 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:51:56.978901 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:51:56.978901 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:51:56.978901 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:51:56.978901 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:51:56.978901 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:51:56.978901 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:51:56.978901 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:51:56.978901 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:51:56.978901 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:51:56.978901 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:51:56.978901 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:51:56.978901 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:51:56.978901 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:51:56.978901 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:51:56.979821 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:51:56.979821 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:51:56.979821 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:51:56.979821 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:51:56.979821 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:51:56.979821 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:51:56.979821 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:51:56.979821 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:51:56.979821 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:51:56.979821 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:51:56.979821 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:51:56.979821 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:51:56.983824 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 242/487] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 14:51:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:51:56.987165 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007fb2c9-a2c1-495d-bda7-ffa8b07582cd None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:57.041780 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74dd30e-40c3-4d9e-a296-11e81d62b64f None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:57.078225 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626e6b6d-6786-4b06-ad70-63007f2be02a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:57.079438 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626e6b6d-6786-4b06-ad70-63007f2be02a None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:57.080101 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626e6b6d-6786-4b06-ad70-63007f2be02a None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:57.080581 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626e6b6d-6786-4b06-ad70-63007f2be02a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:57.093565 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626e6b6d-6786-4b06-ad70-63007f2be02a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:57.095064 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626e6b6d-6786-4b06-ad70-63007f2be02a None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:57.113961 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-626e6b6d-6786-4b06-ad70-63007f2be02a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:51:57.113961 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:51:57.113961 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:51:57.113961 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:51:57.113961 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:51:57.113961 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:51:57.113961 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:51:57.113961 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:51:57.113961 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:51:57.113961 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:51:57.113961 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:51:57.113961 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:51:57.113961 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:51:57.113961 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:51:57.113961 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:51:57.115113 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:51:57.115113 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:51:57.115113 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:51:57.115113 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:51:57.115113 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:51:57.115113 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:51:57.115113 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:51:57.115113 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:51:57.115113 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:51:57.115113 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:51:57.115113 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:51:57.115113 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:51:57.115113 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:51:57.115113 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:51:57.115113 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:51:57.115113 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:51:57.115113 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:51:57.116113 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:51:57.116113 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:51:57.116113 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:51:57.116113 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:51:57.118042 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 246/488] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:51:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:51:57.119524 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b638b746-2e31-45fb-bfc6-f5e531f5c0fd None None] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:57.120042 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b638b746-2e31-45fb-bfc6-f5e531f5c0fd None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:57.120533 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b638b746-2e31-45fb-bfc6-f5e531f5c0fd None None] PATH_INFO: `/` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:57.122326 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 247/489] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:51:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:51:57.124353 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b638b746-2e31-45fb-bfc6-f5e531f5c0fd None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:57.169816 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007fb2c9-a2c1-495d-bda7-ffa8b07582cd None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:57.202246 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:57.211297 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1620d07e-a40b-46f5-944e-c77ad8c4127c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:57.212239 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1620d07e-a40b-46f5-944e-c77ad8c4127c None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:57.212688 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1620d07e-a40b-46f5-944e-c77ad8c4127c None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:57.213129 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1620d07e-a40b-46f5-944e-c77ad8c4127c None admin] PATH_INFO: `/v3/roles` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:57.214115 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1620d07e-a40b-46f5-944e-c77ad8c4127c None admin] RBAC: Adding query filter params (name=service) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:57.214599 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1620d07e-a40b-46f5-944e-c77ad8c4127c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:57.216317 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1620d07e-a40b-46f5-944e-c77ad8c4127c None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:57.224091 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:57.232841 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 243/490] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 19 14:51:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:57.234752 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f42f0f-8a6d-4407-b55f-59f346eedc39 None None] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:57.235307 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f42f0f-8a6d-4407-b55f-59f346eedc39 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:57.235827 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f42f0f-8a6d-4407-b55f-59f346eedc39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:57.247922 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b638b746-2e31-45fb-bfc6-f5e531f5c0fd None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:57.260103 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.auth.core [None req-a5f42f0f-8a6d-4407-b55f-59f346eedc39 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:51:57.262187 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-a5f42f0f-8a6d-4407-b55f-59f346eedc39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:51:57.288137 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c51ef4-05d7-45bb-a216-82cc32646fa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:57.289077 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c51ef4-05d7-45bb-a216-82cc32646fa8 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:57.289327 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c51ef4-05d7-45bb-a216-82cc32646fa8 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:57.289694 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c51ef4-05d7-45bb-a216-82cc32646fa8 None admin] PATH_INFO: `/v3/domains` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:57.290536 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c51ef4-05d7-45bb-a216-82cc32646fa8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:57.290927 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c51ef4-05d7-45bb-a216-82cc32646fa8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:57.292460 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c51ef4-05d7-45bb-a216-82cc32646fa8 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:57.310870 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 248/491] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:51:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:57.310870 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c51ef4-05d7-45bb-a216-82cc32646fa8 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:57.361510 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 244/492] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:51:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:51:57.362572 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f42f0f-8a6d-4407-b55f-59f346eedc39 None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:57.444161 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c51ef4-05d7-45bb-a216-82cc32646fa8 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:57.483542 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f42f0f-8a6d-4407-b55f-59f346eedc39 None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:57.484044 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bff0ff-a355-40a3-a419-182a3a17e6e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:57.484854 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bff0ff-a355-40a3-a419-182a3a17e6e9 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:57.485304 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bff0ff-a355-40a3-a419-182a3a17e6e9 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:57.485597 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bff0ff-a355-40a3-a419-182a3a17e6e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:57.498411 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bff0ff-a355-40a3-a419-182a3a17e6e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:57.500064 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bff0ff-a355-40a3-a419-182a3a17e6e9 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:57.502462 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:57.511263 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:57.519114 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-91bff0ff-a355-40a3-a419-182a3a17e6e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:51:57.519114 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:51:57.519114 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:51:57.519114 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:51:57.519114 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:51:57.519114 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:51:57.519114 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:51:57.519114 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:51:57.519114 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:51:57.519114 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:51:57.519114 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:51:57.519114 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:51:57.519114 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:51:57.519114 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:51:57.519114 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:51:57.520175 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:51:57.520175 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:51:57.520175 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:51:57.520175 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:51:57.520175 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:51:57.520175 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:51:57.520175 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:51:57.520175 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:51:57.520175 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:51:57.520175 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:51:57.520175 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:51:57.520175 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:51:57.520175 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:51:57.520175 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:51:57.520175 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:51:57.520175 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:51:57.520175 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:51:57.521344 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:51:57.521344 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:51:57.521344 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:51:57.521344 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:51:57.521344 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 249/493] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:51:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:51:57.521344 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bff0ff-a355-40a3-a419-182a3a17e6e9 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:57.530418 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3f47f0-c0d1-41cc-8cc1-1d982cf7fa1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:57.531240 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3f47f0-c0d1-41cc-8cc1-1d982cf7fa1b None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:57.531534 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3f47f0-c0d1-41cc-8cc1-1d982cf7fa1b None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:57.531819 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3f47f0-c0d1-41cc-8cc1-1d982cf7fa1b None admin] PATH_INFO: `/v3/users/glance` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:57.539987 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3f47f0-c0d1-41cc-8cc1-1d982cf7fa1b None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:57.541443 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3f47f0-c0d1-41cc-8cc1-1d982cf7fa1b None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:57.558786 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-7a3f47f0-c0d1-41cc-8cc1-1d982cf7fa1b None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 19 14:51:57.558786 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:51:57.558786 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:51:57.558786 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:51:57.558786 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:51:57.558786 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:51:57.558786 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:51:57.558786 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:51:57.558786 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:51:57.558786 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:51:57.558786 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:51:57.558786 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:51:57.558786 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:51:57.558786 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:51:57.558786 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:51:57.558786 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:51:57.559899 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:51:57.559899 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:51:57.559899 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:51:57.559899 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:51:57.559899 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:51:57.559899 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:51:57.559899 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:51:57.559899 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:51:57.559899 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:51:57.559899 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:51:57.559899 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:51:57.559899 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:51:57.559899 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:51:57.559899 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:51:57.559899 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:51:57.559899 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:51:57.559899 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:51:57.559899 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:51:57.561581 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:51:57.561581 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:51:57.561581 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:51:57.561581 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:51:57.561581 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:51:57.561581 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:51:57.561581 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:51:57.561581 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:51:57.561581 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 19 14:51:57.561581 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:51:57.565436 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 245/494] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:51:57 2022] GET /identity/v3/users/glance => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:51:57.566865 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8beef0-2bd1-4240-b543-d87237796ded None None] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:57.567198 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8beef0-2bd1-4240-b543-d87237796ded None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:57.567457 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8beef0-2bd1-4240-b543-d87237796ded None None] PATH_INFO: `/` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:57.569094 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 246/495] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:51:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:51:57.569495 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8beef0-2bd1-4240-b543-d87237796ded None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:57.642292 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bff0ff-a355-40a3-a419-182a3a17e6e9 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:57.682761 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8beef0-2bd1-4240-b543-d87237796ded None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:57.685710 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e88aec-dc8b-4a51-8514-5049636d4ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:57.686632 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e88aec-dc8b-4a51-8514-5049636d4ee1 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:57.686950 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e88aec-dc8b-4a51-8514-5049636d4ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:57.687239 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e88aec-dc8b-4a51-8514-5049636d4ee1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:57.699074 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e88aec-dc8b-4a51-8514-5049636d4ee1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:51:57.707556 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-e8e88aec-dc8b-4a51-8514-5049636d4ee1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:51:57.707556 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:51:57.707556 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:51:57.707556 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:51:57.707556 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:51:57.707556 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:51:57.707556 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:51:57.707556 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:51:57.707556 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:51:57.707556 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:51:57.707556 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:51:57.707556 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:51:57.707556 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:51:57.707556 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:51:57.707556 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:51:57.707556 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:51:57.708615 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:51:57.708615 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:51:57.708615 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:51:57.708615 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:51:57.708615 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:51:57.708615 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:51:57.708615 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:51:57.708615 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:51:57.708615 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:51:57.708615 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:51:57.708615 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:51:57.708615 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:51:57.708615 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:51:57.708615 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:51:57.708615 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:51:57.708615 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:51:57.709564 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:51:57.709564 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:51:57.709564 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:51:57.709564 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:51:57.709564 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:51:57.709564 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:51:57.709564 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:51:57.709564 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:51:57.709564 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:51:57.709564 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:51:57.709564 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:51:57.709564 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:51:57.709564 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 250/496] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 14:51:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:51:57.709564 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e88aec-dc8b-4a51-8514-5049636d4ee1 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:57.726033 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6a5517-fabb-4f62-af8b-164b7bec91af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:57.726849 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6a5517-fabb-4f62-af8b-164b7bec91af None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:57.727144 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6a5517-fabb-4f62-af8b-164b7bec91af None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:57.727426 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6a5517-fabb-4f62-af8b-164b7bec91af None admin] PATH_INFO: `/v3/domains` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:57.728163 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6a5517-fabb-4f62-af8b-164b7bec91af None admin] RBAC: Adding query filter params (name=Default) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:57.728476 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6a5517-fabb-4f62-af8b-164b7bec91af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:57.729883 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6a5517-fabb-4f62-af8b-164b7bec91af None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:57.742156 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 247/497] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:51:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:57.743577 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d71909f-0e00-4b63-9b9e-010bef20224b None None] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:57.743876 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d71909f-0e00-4b63-9b9e-010bef20224b None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:57.744156 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d71909f-0e00-4b63-9b9e-010bef20224b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:57.761209 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.auth.core [None req-6d71909f-0e00-4b63-9b9e-010bef20224b None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:51:57.762422 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-6d71909f-0e00-4b63-9b9e-010bef20224b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:51:57.783983 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:57.803677 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:57.833877 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e88aec-dc8b-4a51-8514-5049636d4ee1 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:57.847260 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 248/498] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:51:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:51:57.848834 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d71909f-0e00-4b63-9b9e-010bef20224b None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:57.877481 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f272344e-6548-4ef4-8f4a-5100251dcbd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:57.878208 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f272344e-6548-4ef4-8f4a-5100251dcbd6 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:57.878534 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f272344e-6548-4ef4-8f4a-5100251dcbd6 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:57.878774 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f272344e-6548-4ef4-8f4a-5100251dcbd6 None admin] PATH_INFO: `/v3/users` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:57.879629 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f272344e-6548-4ef4-8f4a-5100251dcbd6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:57.879972 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f272344e-6548-4ef4-8f4a-5100251dcbd6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:57.881412 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f272344e-6548-4ef4-8f4a-5100251dcbd6 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:57.920568 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 251/499] 10.222.0.57 () {56 vars in 1137 bytes} [Thu May 19 14:51:57 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:57.923100 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f272344e-6548-4ef4-8f4a-5100251dcbd6 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:57.965256 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d71909f-0e00-4b63-9b9e-010bef20224b None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:58.005282 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8defa0be-90e2-4e9f-9416-d6995870740e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:58.006013 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8defa0be-90e2-4e9f-9416-d6995870740e None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:58.006306 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8defa0be-90e2-4e9f-9416-d6995870740e None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:58.006956 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8defa0be-90e2-4e9f-9416-d6995870740e None admin] PATH_INFO: `/v3/roles` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:58.007722 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8defa0be-90e2-4e9f-9416-d6995870740e None admin] RBAC: Adding query filter params (name=service) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:58.008360 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8defa0be-90e2-4e9f-9416-d6995870740e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:58.010081 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8defa0be-90e2-4e9f-9416-d6995870740e None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:58.020941 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 249/500] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 19 14:51:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:58.022126 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8defa0be-90e2-4e9f-9416-d6995870740e None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:58.035840 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f272344e-6548-4ef4-8f4a-5100251dcbd6 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:58.050247 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:58.059951 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:58.073982 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f1f26f-ae79-439d-bac0-0d79d29c9303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:58.075158 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f1f26f-ae79-439d-bac0-0d79d29c9303 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:58.075449 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f1f26f-ae79-439d-bac0-0d79d29c9303 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:58.075731 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f1f26f-ae79-439d-bac0-0d79d29c9303 None admin] PATH_INFO: `/v3/users/nova` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:58.083107 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f1f26f-ae79-439d-bac0-0d79d29c9303 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:58.084647 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f1f26f-ae79-439d-bac0-0d79d29c9303 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:58.100433 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-f6f1f26f-ae79-439d-bac0-0d79d29c9303 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 14:51:58.100433 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:51:58.100433 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:51:58.100433 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:51:58.100433 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:51:58.100433 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:51:58.100433 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:51:58.100433 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:51:58.100433 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:51:58.100433 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:51:58.100433 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:51:58.100433 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:51:58.100433 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:51:58.100433 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:51:58.100433 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:51:58.100433 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:51:58.101648 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:51:58.101648 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:51:58.101648 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:51:58.101648 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:51:58.101648 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:51:58.101648 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:51:58.101648 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:51:58.101648 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:51:58.101648 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:51:58.101648 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:51:58.101648 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:51:58.101648 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:51:58.101648 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:51:58.101648 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:51:58.101648 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:51:58.101648 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:51:58.101648 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:51:58.101648 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:51:58.102714 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:51:58.102714 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:51:58.102714 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:51:58.102714 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:51:58.102714 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:51:58.102714 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:51:58.102714 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:51:58.102714 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:51:58.102714 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 14:51:58.102714 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:51:58.102714 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 252/501] 10.222.0.57 () {56 vars in 1093 bytes} [Thu May 19 14:51:57 2022] GET /identity/v3/users/nova => generated 82 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:51:58.103375 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f1f26f-ae79-439d-bac0-0d79d29c9303 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:58.130345 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8defa0be-90e2-4e9f-9416-d6995870740e None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:58.165696 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f14d4c-947b-4c4b-8bfb-8a18e47e006b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:58.166539 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f14d4c-947b-4c4b-8bfb-8a18e47e006b None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:58.166835 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f14d4c-947b-4c4b-8bfb-8a18e47e006b None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:58.167389 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f14d4c-947b-4c4b-8bfb-8a18e47e006b None admin] PATH_INFO: `/v3/roles/service` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:58.178053 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f14d4c-947b-4c4b-8bfb-8a18e47e006b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:51:58.185912 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-19f14d4c-947b-4c4b-8bfb-8a18e47e006b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:51:58.185912 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:51:58.185912 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:51:58.185912 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:51:58.185912 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:51:58.185912 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:51:58.185912 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:51:58.185912 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:51:58.185912 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:51:58.185912 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:51:58.185912 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:51:58.185912 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:51:58.185912 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:51:58.185912 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:51:58.185912 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:51:58.185912 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:51:58.186961 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:51:58.186961 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:51:58.186961 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:51:58.186961 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:51:58.186961 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:51:58.186961 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:51:58.186961 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:51:58.186961 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:51:58.186961 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:51:58.186961 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:51:58.186961 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:51:58.186961 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:51:58.186961 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:51:58.186961 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:51:58.186961 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:51:58.186961 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:51:58.187899 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:51:58.187899 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:51:58.187899 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:51:58.187899 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:51:58.187899 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:51:58.187899 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:51:58.187899 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:51:58.187899 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:51:58.187899 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:51:58.187899 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:51:58.187899 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:51:58.187899 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:51:58.187899 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 250/502] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 14:51:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:51:58.188696 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f14d4c-947b-4c4b-8bfb-8a18e47e006b None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:58.210129 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f1f26f-ae79-439d-bac0-0d79d29c9303 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:58.244839 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35aa731a-ba4d-4cc6-8e22-2640a8b9e183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:58.245522 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aa731a-ba4d-4cc6-8e22-2640a8b9e183 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:58.245831 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aa731a-ba4d-4cc6-8e22-2640a8b9e183 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:58.246137 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aa731a-ba4d-4cc6-8e22-2640a8b9e183 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:58.284151 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:58.291213 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:58.292546 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f14d4c-947b-4c4b-8bfb-8a18e47e006b None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:58.319437 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35aa731a-ba4d-4cc6-8e22-2640a8b9e183 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:58.321342 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35aa731a-ba4d-4cc6-8e22-2640a8b9e183 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:58.327563 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ee23b6-daa3-49c7-bf14-389191c25370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:58.328363 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ee23b6-daa3-49c7-bf14-389191c25370 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:58.328656 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ee23b6-daa3-49c7-bf14-389191c25370 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:58.328949 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ee23b6-daa3-49c7-bf14-389191c25370 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:58.339616 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ee23b6-daa3-49c7-bf14-389191c25370 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:58.340997 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ee23b6-daa3-49c7-bf14-389191c25370 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:58.358422 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-74ee23b6-daa3-49c7-bf14-389191c25370 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:51:58.358422 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:51:58.358422 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:51:58.358422 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:51:58.358422 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:51:58.358422 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:51:58.358422 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:51:58.358422 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:51:58.358422 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:51:58.358422 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:51:58.358422 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:51:58.358422 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:51:58.358422 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:51:58.358422 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:51:58.358422 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:51:58.359617 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:51:58.359617 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:51:58.359617 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:51:58.359617 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:51:58.359617 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:51:58.359617 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:51:58.359617 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:51:58.359617 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:51:58.359617 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:51:58.359617 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:51:58.359617 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:51:58.359617 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:51:58.359617 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:51:58.359617 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:51:58.359617 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:51:58.359617 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:51:58.359617 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:51:58.360598 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:51:58.360598 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:51:58.360598 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:51:58.360598 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:51:58.360598 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 251/503] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:51:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:51:58.360926 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35313264-b960-40eb-a3dd-7a90c527d1df None None] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:58.361744 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35313264-b960-40eb-a3dd-7a90c527d1df None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:58.362120 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35313264-b960-40eb-a3dd-7a90c527d1df None None] PATH_INFO: `/` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:58.363349 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 252/504] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:51:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:51:58.364055 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35313264-b960-40eb-a3dd-7a90c527d1df None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:58.428542 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 253/505] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:51:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:51:58.429395 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35aa731a-ba4d-4cc6-8e22-2640a8b9e183 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:58.468933 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35313264-b960-40eb-a3dd-7a90c527d1df None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:58.499761 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:58.503467 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edbb23f-500c-4424-8970-fd7b34e24e7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:58.504144 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edbb23f-500c-4424-8970-fd7b34e24e7a None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:58.504442 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edbb23f-500c-4424-8970-fd7b34e24e7a None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:58.504735 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edbb23f-500c-4424-8970-fd7b34e24e7a None admin] PATH_INFO: `/v3/users` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:58.505617 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edbb23f-500c-4424-8970-fd7b34e24e7a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:58.505942 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edbb23f-500c-4424-8970-fd7b34e24e7a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:58.507398 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edbb23f-500c-4424-8970-fd7b34e24e7a None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:58.531225 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35aa731a-ba4d-4cc6-8e22-2640a8b9e183 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:58.545409 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 253/506] 10.222.0.57 () {56 vars in 1133 bytes} [Thu May 19 14:51:58 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:58.546195 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edbb23f-500c-4424-8970-fd7b34e24e7a None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:58.550513 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:58.572131 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fb3960-40ba-41ad-a16e-7fdc84b9b17c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:58.572808 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fb3960-40ba-41ad-a16e-7fdc84b9b17c None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:58.573097 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fb3960-40ba-41ad-a16e-7fdc84b9b17c None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:58.573593 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fb3960-40ba-41ad-a16e-7fdc84b9b17c None admin] PATH_INFO: `/v3/roles` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:58.574145 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fb3960-40ba-41ad-a16e-7fdc84b9b17c None admin] RBAC: Adding query filter params (name=service) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:58.574458 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fb3960-40ba-41ad-a16e-7fdc84b9b17c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:58.575844 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fb3960-40ba-41ad-a16e-7fdc84b9b17c None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:58.585364 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 254/507] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 19 14:51:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:58.586595 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5fb1fc-5be2-428b-b443-079fdc8d921e None None] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:58.586891 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5fb1fc-5be2-428b-b443-079fdc8d921e None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:58.587174 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5fb1fc-5be2-428b-b443-079fdc8d921e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:58.604898 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.auth.core [None req-ab5fb1fc-5be2-428b-b443-079fdc8d921e None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:51:58.606075 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.fernet_utils [None req-ab5fb1fc-5be2-428b-b443-079fdc8d921e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:51:58.649895 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edbb23f-500c-4424-8970-fd7b34e24e7a None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:58.673658 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 255/508] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:51:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:51:58.675121 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5fb1fc-5be2-428b-b443-079fdc8d921e None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:58.701930 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cce322e-bcd5-44ff-a6ac-6c8d19eb3258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:58.702601 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cce322e-bcd5-44ff-a6ac-6c8d19eb3258 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:58.702888 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cce322e-bcd5-44ff-a6ac-6c8d19eb3258 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:58.703168 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cce322e-bcd5-44ff-a6ac-6c8d19eb3258 None admin] PATH_INFO: `/v3/domains` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:58.703905 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cce322e-bcd5-44ff-a6ac-6c8d19eb3258 None admin] RBAC: Adding query filter params (name=Default) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:58.704216 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cce322e-bcd5-44ff-a6ac-6c8d19eb3258 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:58.706534 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cce322e-bcd5-44ff-a6ac-6c8d19eb3258 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:58.717952 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 254/509] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:51:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:58.718766 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cce322e-bcd5-44ff-a6ac-6c8d19eb3258 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:58.742424 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:58.781962 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5fb1fc-5be2-428b-b443-079fdc8d921e None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:58.782793 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:58.817056 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7298b05-29d3-47a9-abb9-8bc3a5ac1321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:58.817872 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7298b05-29d3-47a9-abb9-8bc3a5ac1321 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:58.818167 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7298b05-29d3-47a9-abb9-8bc3a5ac1321 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:58.818456 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7298b05-29d3-47a9-abb9-8bc3a5ac1321 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:58.821674 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cce322e-bcd5-44ff-a6ac-6c8d19eb3258 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:58.828663 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7298b05-29d3-47a9-abb9-8bc3a5ac1321 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:58.830953 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7298b05-29d3-47a9-abb9-8bc3a5ac1321 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:58.849349 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-d7298b05-29d3-47a9-abb9-8bc3a5ac1321 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:51:58.849349 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:51:58.849349 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:51:58.849349 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:51:58.849349 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:51:58.849349 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:51:58.849349 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:51:58.849349 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:51:58.849349 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:51:58.849349 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:51:58.849349 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:51:58.849349 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:51:58.849349 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:51:58.849349 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:51:58.849349 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:51:58.850568 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:51:58.850568 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:51:58.850568 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:51:58.850568 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:51:58.850568 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:51:58.850568 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:51:58.850568 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:51:58.850568 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:51:58.850568 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:51:58.850568 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:51:58.850568 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:51:58.850568 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:51:58.850568 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:51:58.850568 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:51:58.850568 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:51:58.850568 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:51:58.850568 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:51:58.851979 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:51:58.851979 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:51:58.851979 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:51:58.851979 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:51:58.851979 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 256/510] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:51:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:51:58.851979 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7298b05-29d3-47a9-abb9-8bc3a5ac1321 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:58.860753 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6406ab2b-41c5-4df0-87ee-0c1cf2499681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:58.861460 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6406ab2b-41c5-4df0-87ee-0c1cf2499681 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:58.861802 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6406ab2b-41c5-4df0-87ee-0c1cf2499681 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:58.862155 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6406ab2b-41c5-4df0-87ee-0c1cf2499681 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:58.930050 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6406ab2b-41c5-4df0-87ee-0c1cf2499681 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:58.931525 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6406ab2b-41c5-4df0-87ee-0c1cf2499681 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:58.952344 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7298b05-29d3-47a9-abb9-8bc3a5ac1321 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:58.969224 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:58.988281 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9188ecf8-c405-4ab3-a5bb-019227c42bf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:58.989076 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9188ecf8-c405-4ab3-a5bb-019227c42bf3 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:58.989364 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9188ecf8-c405-4ab3-a5bb-019227c42bf3 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:58.989641 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9188ecf8-c405-4ab3-a5bb-019227c42bf3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:58.992976 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:58.999944 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9188ecf8-c405-4ab3-a5bb-019227c42bf3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:59.001403 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9188ecf8-c405-4ab3-a5bb-019227c42bf3 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:59.019599 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-9188ecf8-c405-4ab3-a5bb-019227c42bf3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:51:59.019599 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:51:59.019599 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:51:59.019599 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:51:59.019599 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:51:59.019599 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:51:59.019599 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:51:59.019599 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:51:59.019599 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:51:59.019599 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:51:59.019599 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:51:59.019599 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:51:59.019599 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:51:59.019599 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:51:59.019599 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:51:59.020761 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:51:59.020761 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:51:59.020761 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:51:59.020761 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:51:59.020761 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:51:59.020761 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:51:59.020761 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:51:59.020761 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:51:59.020761 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:51:59.020761 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:51:59.020761 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:51:59.020761 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:51:59.020761 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:51:59.020761 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:51:59.020761 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:51:59.020761 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:51:59.020761 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:51:59.021754 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:51:59.021754 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:51:59.021754 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:51:59.021754 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:51:59.021754 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 257/511] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:51:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:51:59.021754 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9188ecf8-c405-4ab3-a5bb-019227c42bf3 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:59.036727 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 255/512] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:51:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:51:59.037584 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6406ab2b-41c5-4df0-87ee-0c1cf2499681 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:59.127620 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9188ecf8-c405-4ab3-a5bb-019227c42bf3 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:59.143031 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6406ab2b-41c5-4df0-87ee-0c1cf2499681 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:59.161645 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0337c365-668e-4919-a2b8-e4b8ba0568b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:59.162473 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0337c365-668e-4919-a2b8-e4b8ba0568b0 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:59.162763 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0337c365-668e-4919-a2b8-e4b8ba0568b0 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:59.163047 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0337c365-668e-4919-a2b8-e4b8ba0568b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:59.173263 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0337c365-668e-4919-a2b8-e4b8ba0568b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:59.175436 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0337c365-668e-4919-a2b8-e4b8ba0568b0 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:59.177985 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d09ed4-1c4d-4c9b-89b2-f9233e027715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:59.178793 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d09ed4-1c4d-4c9b-89b2-f9233e027715 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:59.179081 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d09ed4-1c4d-4c9b-89b2-f9233e027715 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:59.179365 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d09ed4-1c4d-4c9b-89b2-f9233e027715 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:59.186639 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d09ed4-1c4d-4c9b-89b2-f9233e027715 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:59.188074 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d09ed4-1c4d-4c9b-89b2-f9233e027715 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:59.193116 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-0337c365-668e-4919-a2b8-e4b8ba0568b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:51:59.193116 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:51:59.193116 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:51:59.193116 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:51:59.193116 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:51:59.193116 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:51:59.193116 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:51:59.193116 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:51:59.193116 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:51:59.193116 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:51:59.193116 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:51:59.193116 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:51:59.193116 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:51:59.193116 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:51:59.193116 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:51:59.194342 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:51:59.194342 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:51:59.194342 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:51:59.194342 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:51:59.194342 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:51:59.194342 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:51:59.194342 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:51:59.194342 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:51:59.194342 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:51:59.194342 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:51:59.194342 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:51:59.194342 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:51:59.194342 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:51:59.194342 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:51:59.194342 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:51:59.194342 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:51:59.194342 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:51:59.200853 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:51:59.200853 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:51:59.200853 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:51:59.200853 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:51:59.200853 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 258/513] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:51:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:51:59.200853 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0337c365-668e-4919-a2b8-e4b8ba0568b0 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:59.209826 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-a6d09ed4-1c4d-4c9b-89b2-f9233e027715 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 19 14:51:59.209826 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:51:59.209826 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:51:59.209826 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:51:59.209826 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:51:59.209826 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:51:59.209826 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:51:59.209826 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:51:59.209826 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:51:59.209826 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:51:59.209826 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:51:59.209826 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:51:59.209826 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:51:59.209826 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:51:59.209826 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:51:59.209826 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:51:59.211026 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:51:59.211026 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:51:59.211026 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:51:59.211026 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:51:59.211026 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:51:59.211026 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:51:59.211026 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:51:59.211026 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:51:59.211026 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:51:59.211026 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:51:59.211026 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:51:59.211026 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:51:59.211026 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:51:59.211026 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:51:59.211026 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:51:59.211026 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:51:59.211026 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:51:59.211026 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:51:59.212487 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:51:59.212487 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:51:59.212487 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:51:59.212487 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:51:59.212487 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:51:59.212487 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:51:59.212487 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:51:59.212487 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:51:59.212487 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 19 14:51:59.212487 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:51:59.212487 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 256/514] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 14:51:59 2022] GET /identity/v3/users/neutron => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:51:59.212487 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d09ed4-1c4d-4c9b-89b2-f9233e027715 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:59.221296 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:59.239349 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:59.303373 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0337c365-668e-4919-a2b8-e4b8ba0568b0 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:59.318509 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d09ed4-1c4d-4c9b-89b2-f9233e027715 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:59.339939 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6c440c-056d-42cb-8d2a-462362b0fd1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:59.340602 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6c440c-056d-42cb-8d2a-462362b0fd1b None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:59.340890 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6c440c-056d-42cb-8d2a-462362b0fd1b None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:59.341169 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6c440c-056d-42cb-8d2a-462362b0fd1b None admin] PATH_INFO: `/v3/domains` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:59.341918 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6c440c-056d-42cb-8d2a-462362b0fd1b None admin] RBAC: Adding query filter params (name=Default) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:59.342232 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6c440c-056d-42cb-8d2a-462362b0fd1b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:59.343609 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6c440c-056d-42cb-8d2a-462362b0fd1b None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:59.353946 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a420734-d950-4527-9c18-34dd52b47975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:59.354615 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a420734-d950-4527-9c18-34dd52b47975 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:59.354911 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a420734-d950-4527-9c18-34dd52b47975 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:59.355194 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a420734-d950-4527-9c18-34dd52b47975 None admin] PATH_INFO: `/v3/domains` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:59.355851 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 259/515] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:51:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:59.355961 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a420734-d950-4527-9c18-34dd52b47975 None admin] RBAC: Adding query filter params (name=Default) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:59.356237 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a420734-d950-4527-9c18-34dd52b47975 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:59.356649 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6c440c-056d-42cb-8d2a-462362b0fd1b None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:59.357704 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a420734-d950-4527-9c18-34dd52b47975 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:59.371225 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 257/516] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:51:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:59.372046 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a420734-d950-4527-9c18-34dd52b47975 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:59.448133 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:59.464761 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:59.469891 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6c440c-056d-42cb-8d2a-462362b0fd1b None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:59.480887 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a420734-d950-4527-9c18-34dd52b47975 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:59.506884 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31bc8e8-6aa7-48f6-974e-c10742702bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:59.507768 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31bc8e8-6aa7-48f6-974e-c10742702bc7 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:59.508128 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31bc8e8-6aa7-48f6-974e-c10742702bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:59.508421 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31bc8e8-6aa7-48f6-974e-c10742702bc7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:59.515246 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251540ca-f5b5-48d1-9606-58cd8bb67d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:59.515903 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251540ca-f5b5-48d1-9606-58cd8bb67d3c None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:59.516190 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251540ca-f5b5-48d1-9606-58cd8bb67d3c None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:59.516470 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251540ca-f5b5-48d1-9606-58cd8bb67d3c None admin] PATH_INFO: `/v3/domains` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:59.517194 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251540ca-f5b5-48d1-9606-58cd8bb67d3c None admin] RBAC: Adding query filter params (name=Default) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:59.517833 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251540ca-f5b5-48d1-9606-58cd8bb67d3c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:59.519192 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251540ca-f5b5-48d1-9606-58cd8bb67d3c None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:59.519420 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31bc8e8-6aa7-48f6-974e-c10742702bc7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:59.520883 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31bc8e8-6aa7-48f6-974e-c10742702bc7 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:59.531595 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 258/517] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:51:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:59.532405 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251540ca-f5b5-48d1-9606-58cd8bb67d3c None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:59.538514 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-f31bc8e8-6aa7-48f6-974e-c10742702bc7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:51:59.538514 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:51:59.538514 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:51:59.538514 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:51:59.538514 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:51:59.538514 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:51:59.538514 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:51:59.538514 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:51:59.538514 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:51:59.538514 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:51:59.538514 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:51:59.538514 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:51:59.538514 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:51:59.538514 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:51:59.538514 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:51:59.540518 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:51:59.540518 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:51:59.540518 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:51:59.540518 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:51:59.540518 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:51:59.540518 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:51:59.540518 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:51:59.540518 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:51:59.540518 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:51:59.540518 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:51:59.540518 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:51:59.540518 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:51:59.540518 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:51:59.540518 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:51:59.540518 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:51:59.540518 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:51:59.540518 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:51:59.541499 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:51:59.541499 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:51:59.541499 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:51:59.541499 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:51:59.541499 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 260/518] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:51:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:51:59.541499 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31bc8e8-6aa7-48f6-974e-c10742702bc7 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:59.640382 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251540ca-f5b5-48d1-9606-58cd8bb67d3c None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:59.640630 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31bc8e8-6aa7-48f6-974e-c10742702bc7 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:59.669780 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:59.675646 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357389af-c5d7-441b-a238-cf5a40c567c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:59.676599 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357389af-c5d7-441b-a238-cf5a40c567c3 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:59.676885 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357389af-c5d7-441b-a238-cf5a40c567c3 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:59.677080 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fade66f0-f060-4376-a86c-24986530c1ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:59.677214 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357389af-c5d7-441b-a238-cf5a40c567c3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:59.677764 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fade66f0-f060-4376-a86c-24986530c1ac None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:59.678091 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fade66f0-f060-4376-a86c-24986530c1ac None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:59.678378 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fade66f0-f060-4376-a86c-24986530c1ac None admin] PATH_INFO: `/v3/users` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:59.679234 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fade66f0-f060-4376-a86c-24986530c1ac None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:59.679556 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fade66f0-f060-4376-a86c-24986530c1ac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:59.681014 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fade66f0-f060-4376-a86c-24986530c1ac None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:59.684149 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357389af-c5d7-441b-a238-cf5a40c567c3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:59.685576 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357389af-c5d7-441b-a238-cf5a40c567c3 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:59.699645 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 259/519] 10.222.0.57 () {56 vars in 1139 bytes} [Thu May 19 14:51:59 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:59.700407 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fade66f0-f060-4376-a86c-24986530c1ac None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:59.702235 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-357389af-c5d7-441b-a238-cf5a40c567c3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:51:59.702235 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:51:59.702235 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:51:59.702235 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:51:59.702235 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:51:59.702235 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:51:59.702235 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:51:59.702235 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:51:59.702235 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:51:59.702235 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:51:59.702235 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:51:59.702235 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:51:59.702235 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:51:59.702235 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:51:59.702235 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:51:59.703203 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:51:59.703203 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:51:59.703203 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:51:59.703203 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:51:59.703203 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:51:59.703203 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:51:59.703203 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:51:59.703203 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:51:59.703203 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:51:59.703203 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:51:59.703203 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:51:59.703203 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:51:59.703203 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:51:59.703203 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:51:59.703203 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:51:59.703203 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:51:59.703203 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:51:59.704206 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:51:59.704206 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:51:59.704206 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:51:59.704206 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:51:59.704206 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:51:59.704206 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:51:59.704206 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:51:59.704206 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:51:59.704206 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 261/520] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:51:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:51:59.704771 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357389af-c5d7-441b-a238-cf5a40c567c3 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:59.706149 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:59.804633 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fade66f0-f060-4376-a86c-24986530c1ac None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:59.805021 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357389af-c5d7-441b-a238-cf5a40c567c3 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:59.839425 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710aab32-bc98-4c57-b3ba-b7fe64b918b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:59.839624 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed094fa-a8b2-469e-8e36-c5df50f12b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:59.840108 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710aab32-bc98-4c57-b3ba-b7fe64b918b8 None admin] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:59.840250 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed094fa-a8b2-469e-8e36-c5df50f12b95 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:59.840386 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710aab32-bc98-4c57-b3ba-b7fe64b918b8 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:59.840544 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed094fa-a8b2-469e-8e36-c5df50f12b95 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:59.840666 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710aab32-bc98-4c57-b3ba-b7fe64b918b8 None admin] PATH_INFO: `/v3/users` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:59.840828 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed094fa-a8b2-469e-8e36-c5df50f12b95 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:59.841383 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710aab32-bc98-4c57-b3ba-b7fe64b918b8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:59.842849 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710aab32-bc98-4c57-b3ba-b7fe64b918b8 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:59.848512 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed094fa-a8b2-469e-8e36-c5df50f12b95 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:59.850133 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed094fa-a8b2-469e-8e36-c5df50f12b95 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:59.864854 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 262/521] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 19 14:51:59 2022] POST /identity/v3/users => generated 247 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:51:59.865689 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710aab32-bc98-4c57-b3ba-b7fe64b918b8 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:59.865816 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-0ed094fa-a8b2-469e-8e36-c5df50f12b95 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 14:51:59.865816 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:51:59.865816 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:51:59.865816 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:51:59.865816 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:51:59.865816 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:51:59.865816 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:51:59.865816 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:51:59.865816 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:51:59.865816 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:51:59.865816 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:51:59.865816 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:51:59.865816 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:51:59.865816 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:51:59.865816 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:51:59.865816 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:51:59.866879 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:51:59.866879 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:51:59.866879 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:51:59.866879 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:51:59.866879 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:51:59.866879 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:51:59.866879 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:51:59.866879 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:51:59.866879 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:51:59.866879 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:51:59.866879 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:51:59.866879 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:51:59.866879 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:51:59.866879 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:51:59.866879 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:51:59.866879 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:51:59.866879 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:51:59.866879 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:51:59.867976 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:51:59.867976 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:51:59.867976 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:51:59.867976 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:51:59.867976 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:51:59.867976 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:51:59.867976 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:51:59.867976 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:51:59.867976 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 14:51:59.867976 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:51:59.867976 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 260/522] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:51:59 2022] GET /identity/v3/users/cinder => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:51:59.867976 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed094fa-a8b2-469e-8e36-c5df50f12b95 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:59.942336 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:59.951638 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:59.990807 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed094fa-a8b2-469e-8e36-c5df50f12b95 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:00.015460 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710aab32-bc98-4c57-b3ba-b7fe64b918b8 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:00.030537 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef044a2-4d64-4a4f-9c12-34b37e9efd36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:00.031450 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef044a2-4d64-4a4f-9c12-34b37e9efd36 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:00.031664 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef044a2-4d64-4a4f-9c12-34b37e9efd36 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:00.031914 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef044a2-4d64-4a4f-9c12-34b37e9efd36 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:00.055276 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9ca4dc-7657-42f9-be57-50e616e50867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:00.055962 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9ca4dc-7657-42f9-be57-50e616e50867 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:00.056256 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9ca4dc-7657-42f9-be57-50e616e50867 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:00.056545 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9ca4dc-7657-42f9-be57-50e616e50867 None admin] PATH_INFO: `/v3/domains` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:00.057290 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9ca4dc-7657-42f9-be57-50e616e50867 None admin] RBAC: Adding query filter params (name=Default) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:00.057604 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9ca4dc-7657-42f9-be57-50e616e50867 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:00.059164 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9ca4dc-7657-42f9-be57-50e616e50867 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:00.072804 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 263/523] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:51:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:00.073647 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9ca4dc-7657-42f9-be57-50e616e50867 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:00.116315 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef044a2-4d64-4a4f-9c12-34b37e9efd36 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:00.117804 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef044a2-4d64-4a4f-9c12-34b37e9efd36 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:00.182648 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:00.202182 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9ca4dc-7657-42f9-be57-50e616e50867 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:00.212960 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:00.227611 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 261/524] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:51:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:52:00.232632 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef044a2-4d64-4a4f-9c12-34b37e9efd36 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:00.241478 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b13658e-c995-498a-8f3a-bab5444a3dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:00.242086 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b13658e-c995-498a-8f3a-bab5444a3dc5 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:00.242374 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b13658e-c995-498a-8f3a-bab5444a3dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:00.242657 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b13658e-c995-498a-8f3a-bab5444a3dc5 None admin] PATH_INFO: `/v3/projects` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:00.243452 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b13658e-c995-498a-8f3a-bab5444a3dc5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:00.243933 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b13658e-c995-498a-8f3a-bab5444a3dc5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:00.245367 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b13658e-c995-498a-8f3a-bab5444a3dc5 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:00.269493 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 264/525] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:52:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:00.269493 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b13658e-c995-498a-8f3a-bab5444a3dc5 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:00.347447 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef044a2-4d64-4a4f-9c12-34b37e9efd36 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:00.379287 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b13658e-c995-498a-8f3a-bab5444a3dc5 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:00.384730 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30831755-64f8-4519-b0d2-5e36f32a1b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:00.385408 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30831755-64f8-4519-b0d2-5e36f32a1b05 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:00.385693 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30831755-64f8-4519-b0d2-5e36f32a1b05 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:00.386005 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30831755-64f8-4519-b0d2-5e36f32a1b05 None admin] PATH_INFO: `/v3/users` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:00.387182 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30831755-64f8-4519-b0d2-5e36f32a1b05 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:00.387182 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30831755-64f8-4519-b0d2-5e36f32a1b05 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:00.388578 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30831755-64f8-4519-b0d2-5e36f32a1b05 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:00.414043 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 262/526] 10.222.0.57 () {56 vars in 1137 bytes} [Thu May 19 14:52:00 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:00.414043 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30831755-64f8-4519-b0d2-5e36f32a1b05 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:00.423518 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bf99fd-783d-49e3-b063-cbee883769bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:00.424946 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bf99fd-783d-49e3-b063-cbee883769bf None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:00.425234 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bf99fd-783d-49e3-b063-cbee883769bf None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:00.425509 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bf99fd-783d-49e3-b063-cbee883769bf None admin] PATH_INFO: `/v3/projects/service` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:00.433221 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bf99fd-783d-49e3-b063-cbee883769bf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:00.434772 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bf99fd-783d-49e3-b063-cbee883769bf None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:00.440104 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:00.449611 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-52bf99fd-783d-49e3-b063-cbee883769bf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:52:00.449611 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:00.449611 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:00.449611 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:00.449611 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:00.449611 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:00.449611 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:00.449611 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:00.449611 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:00.449611 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:00.449611 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:00.449611 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:00.449611 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:52:00.449611 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:52:00.449611 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:52:00.450671 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:52:00.450671 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:00.450671 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:00.450671 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:00.450671 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:00.450671 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:00.450671 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:00.450671 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:00.450671 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:52:00.450671 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:00.450671 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:00.450671 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:00.450671 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:00.450671 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:00.450671 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:00.450671 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:00.450671 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:00.451693 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:52:00.451693 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:52:00.451693 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:52:00.451693 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:52:00.451693 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:52:00.451693 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:52:00.451693 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:52:00.451693 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:52:00.451693 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 265/527] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:52:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:00.451693 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bf99fd-783d-49e3-b063-cbee883769bf None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:00.484557 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:00.540497 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30831755-64f8-4519-b0d2-5e36f32a1b05 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:00.563679 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bf99fd-783d-49e3-b063-cbee883769bf None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:00.578325 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27abcae4-41d0-4298-85af-89386b5e5bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:00.579135 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27abcae4-41d0-4298-85af-89386b5e5bfc None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:00.579425 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27abcae4-41d0-4298-85af-89386b5e5bfc None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:00.580232 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27abcae4-41d0-4298-85af-89386b5e5bfc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:00.592211 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27abcae4-41d0-4298-85af-89386b5e5bfc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:00.593782 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27abcae4-41d0-4298-85af-89386b5e5bfc None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:00.602261 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90368275-56a6-4c17-b2db-95e4d441aee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:00.602927 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90368275-56a6-4c17-b2db-95e4d441aee6 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:00.603218 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90368275-56a6-4c17-b2db-95e4d441aee6 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:00.603501 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90368275-56a6-4c17-b2db-95e4d441aee6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:00.604735 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90368275-56a6-4c17-b2db-95e4d441aee6 None admin] RBAC: Adding query filter params (role.id=08f16502894349708527e30032247468, scope.project.id=fc99d742ae004954b6d841293b5fa79c, user.id=27d9bc13809245299fa5d401f0696215) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:00.604735 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90368275-56a6-4c17-b2db-95e4d441aee6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:00.606194 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90368275-56a6-4c17-b2db-95e4d441aee6 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:00.622208 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 266/528] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:52:00 2022] GET /identity/v3/role_assignments?user.id=27d9bc13809245299fa5d401f0696215&scope.project.id=fc99d742ae004954b6d841293b5fa79c&role.id=08f16502894349708527e30032247468 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:00.622208 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90368275-56a6-4c17-b2db-95e4d441aee6 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:00.622501 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-27abcae4-41d0-4298-85af-89386b5e5bfc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:52:00.622501 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:00.622501 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:00.622501 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:00.622501 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:00.622501 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:00.622501 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:00.622501 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:00.622501 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:00.622501 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:00.622501 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:00.622501 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:00.622501 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:52:00.622501 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:52:00.622501 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:52:00.623610 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:52:00.623610 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:00.623610 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:00.623610 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:00.623610 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:00.623610 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:00.623610 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:00.623610 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:00.623610 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:52:00.623610 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:00.623610 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:00.623610 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:00.623610 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:00.623610 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:00.623610 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:00.623610 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:00.623610 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:00.624793 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:52:00.624793 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:52:00.624793 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:52:00.624793 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:52:00.624793 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 263/529] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:52:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:00.624793 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27abcae4-41d0-4298-85af-89386b5e5bfc None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:00.717672 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:00.747453 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:00.759892 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90368275-56a6-4c17-b2db-95e4d441aee6 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:00.768720 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27abcae4-41d0-4298-85af-89386b5e5bfc None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:00.801433 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a159f1-8177-40cc-a3be-40632a56dad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:00.802303 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a159f1-8177-40cc-a3be-40632a56dad0 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:00.802687 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a159f1-8177-40cc-a3be-40632a56dad0 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:00.803063 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a159f1-8177-40cc-a3be-40632a56dad0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:00.820461 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c32d7c4-5b2b-4ea0-a11e-53817976850e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:00.820461 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c32d7c4-5b2b-4ea0-a11e-53817976850e None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:00.821032 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c32d7c4-5b2b-4ea0-a11e-53817976850e None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:00.823416 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c32d7c4-5b2b-4ea0-a11e-53817976850e None admin] PATH_INFO: `/v3/projects` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:00.823416 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c32d7c4-5b2b-4ea0-a11e-53817976850e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:00.823416 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c32d7c4-5b2b-4ea0-a11e-53817976850e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:00.825077 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c32d7c4-5b2b-4ea0-a11e-53817976850e None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:00.856122 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 264/530] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:52:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:00.857262 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c32d7c4-5b2b-4ea0-a11e-53817976850e None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:00.885892 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a159f1-8177-40cc-a3be-40632a56dad0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:00.887363 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a159f1-8177-40cc-a3be-40632a56dad0 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:01.001068 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:01.001542 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c32d7c4-5b2b-4ea0-a11e-53817976850e None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:01.017989 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 267/531] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:52:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:52:01.017989 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a159f1-8177-40cc-a3be-40632a56dad0 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:01.042352 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:01.049320 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003634c8-78b7-49fa-91b5-484a8d9a9f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:01.050444 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003634c8-78b7-49fa-91b5-484a8d9a9f8f None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:01.051923 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003634c8-78b7-49fa-91b5-484a8d9a9f8f None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:01.052718 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003634c8-78b7-49fa-91b5-484a8d9a9f8f None admin] PATH_INFO: `/v3/domains` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:01.053944 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003634c8-78b7-49fa-91b5-484a8d9a9f8f None admin] RBAC: Adding query filter params (name=Default) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:01.054374 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003634c8-78b7-49fa-91b5-484a8d9a9f8f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:01.055928 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003634c8-78b7-49fa-91b5-484a8d9a9f8f None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:01.071781 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 265/532] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:52:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:01.072962 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003634c8-78b7-49fa-91b5-484a8d9a9f8f None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:01.152257 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a159f1-8177-40cc-a3be-40632a56dad0 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:01.200400 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003634c8-78b7-49fa-91b5-484a8d9a9f8f None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:01.200676 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df5aaad-96c9-4f15-8ba6-861e4b1fe0cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:01.201512 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df5aaad-96c9-4f15-8ba6-861e4b1fe0cc None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:01.202001 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df5aaad-96c9-4f15-8ba6-861e4b1fe0cc None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:01.202686 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df5aaad-96c9-4f15-8ba6-861e4b1fe0cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:01.204605 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df5aaad-96c9-4f15-8ba6-861e4b1fe0cc None admin] RBAC: Adding query filter params (role.id=08f16502894349708527e30032247468, scope.project.id=fc99d742ae004954b6d841293b5fa79c, user.id=5d2aa8c8870a49d98f23f26e036695ef) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:01.205614 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df5aaad-96c9-4f15-8ba6-861e4b1fe0cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:01.207272 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df5aaad-96c9-4f15-8ba6-861e4b1fe0cc None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:01.223785 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 268/533] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:52:01 2022] GET /identity/v3/role_assignments?user.id=5d2aa8c8870a49d98f23f26e036695ef&scope.project.id=fc99d742ae004954b6d841293b5fa79c&role.id=08f16502894349708527e30032247468 => generated 259 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:01.224451 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df5aaad-96c9-4f15-8ba6-861e4b1fe0cc None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:01.269887 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0d241f-6784-4a17-a49d-1d6d89884c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:01.270640 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0d241f-6784-4a17-a49d-1d6d89884c41 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:01.272272 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0d241f-6784-4a17-a49d-1d6d89884c41 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:01.272563 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0d241f-6784-4a17-a49d-1d6d89884c41 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:01.297638 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0d241f-6784-4a17-a49d-1d6d89884c41 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:01.306388 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0d241f-6784-4a17-a49d-1d6d89884c41 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:01.347356 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-2c0d241f-6784-4a17-a49d-1d6d89884c41 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:52:01.347356 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:01.347356 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:01.347356 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:01.347356 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:01.347356 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:01.347356 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:01.347356 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:01.347356 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:01.347356 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:01.347356 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:01.347356 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:01.347356 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:52:01.347356 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:52:01.347356 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:52:01.348960 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:52:01.348960 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:01.348960 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:01.348960 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:01.348960 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:01.348960 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:01.348960 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:01.348960 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:01.348960 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:52:01.348960 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:01.348960 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:01.348960 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:01.348960 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:01.348960 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:01.348960 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:01.348960 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:01.348960 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:01.350533 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:52:01.350533 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:52:01.350533 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:52:01.350533 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:52:01.350820 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:01.352349 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 266/534] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:52:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:01.358183 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0d241f-6784-4a17-a49d-1d6d89884c41 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:01.451788 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df5aaad-96c9-4f15-8ba6-861e4b1fe0cc None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:01.453217 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:01.515032 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5005bef9-b8c0-4556-b889-ae3440a6171b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:01.516276 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5005bef9-b8c0-4556-b889-ae3440a6171b None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:01.516786 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5005bef9-b8c0-4556-b889-ae3440a6171b None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:01.524342 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0d241f-6784-4a17-a49d-1d6d89884c41 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:01.524698 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5005bef9-b8c0-4556-b889-ae3440a6171b None admin] PATH_INFO: `/v3/projects/service` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:01.543414 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5005bef9-b8c0-4556-b889-ae3440a6171b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:01.545270 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5005bef9-b8c0-4556-b889-ae3440a6171b None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:01.563808 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ae555f-a86c-4e54-a448-b52c8223ec88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:01.565172 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ae555f-a86c-4e54-a448-b52c8223ec88 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:01.565465 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ae555f-a86c-4e54-a448-b52c8223ec88 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:01.565746 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ae555f-a86c-4e54-a448-b52c8223ec88 None admin] PATH_INFO: `/v3/domains` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:01.566549 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ae555f-a86c-4e54-a448-b52c8223ec88 None admin] RBAC: Adding query filter params (name=Default) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:01.566867 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ae555f-a86c-4e54-a448-b52c8223ec88 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:01.568253 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ae555f-a86c-4e54-a448-b52c8223ec88 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:01.573358 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-5005bef9-b8c0-4556-b889-ae3440a6171b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:52:01.573358 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:01.573358 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:01.573358 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:01.573358 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:01.573358 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:01.573358 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:01.573358 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:01.573358 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:01.573358 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:01.573358 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:01.573358 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:01.573358 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:52:01.573358 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:52:01.573358 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:52:01.574395 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:52:01.574395 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:01.574395 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:01.574395 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:01.574395 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:01.574395 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:01.574395 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:01.574395 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:01.574395 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:52:01.574395 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:01.574395 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:01.574395 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:01.574395 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:01.574395 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:01.574395 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:01.574395 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:01.574395 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:01.575411 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:52:01.575411 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:52:01.575411 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:52:01.575411 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:52:01.575411 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:52:01.575411 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:52:01.575411 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:52:01.575411 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:52:01.577654 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 269/535] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:52:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:01.586632 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 267/536] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:52:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:01.588714 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5005bef9-b8c0-4556-b889-ae3440a6171b None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:01.591636 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ae555f-a86c-4e54-a448-b52c8223ec88 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:01.753865 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ae555f-a86c-4e54-a448-b52c8223ec88 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:01.771514 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:01.791737 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e19def3-7141-49f8-9cd7-6a483e48c554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:01.792473 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e19def3-7141-49f8-9cd7-6a483e48c554 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:01.792774 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e19def3-7141-49f8-9cd7-6a483e48c554 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:01.793069 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e19def3-7141-49f8-9cd7-6a483e48c554 None admin] PATH_INFO: `/v3/projects/service` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:01.801306 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e19def3-7141-49f8-9cd7-6a483e48c554 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:01.802834 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e19def3-7141-49f8-9cd7-6a483e48c554 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:01.824471 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-5e19def3-7141-49f8-9cd7-6a483e48c554 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:52:01.824471 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:01.824471 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:01.824471 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:01.824471 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:01.824471 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:01.824471 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:01.824471 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:01.824471 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:01.824471 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:01.824471 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:01.824471 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:01.824471 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:52:01.824471 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:52:01.824471 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:52:01.825535 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:52:01.825535 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:01.825535 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:01.825535 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:01.825535 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:01.825535 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:01.825535 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:01.825535 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:01.825535 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:52:01.825535 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:01.825535 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:01.825535 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:01.825535 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:01.825535 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:01.825535 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:01.825535 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:01.825535 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:01.826600 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:52:01.826600 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:52:01.826600 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:52:01.826600 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:52:01.826600 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:52:01.826600 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:52:01.826600 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:52:01.826600 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:52:01.826600 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 268/537] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:52:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:01.833330 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e19def3-7141-49f8-9cd7-6a483e48c554 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:01.854507 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5005bef9-b8c0-4556-b889-ae3440a6171b None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:01.874055 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:01.896150 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af63bb5c-4519-4839-8d5f-7847fd5ff92b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:01.896909 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af63bb5c-4519-4839-8d5f-7847fd5ff92b None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:01.897195 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af63bb5c-4519-4839-8d5f-7847fd5ff92b None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:01.897472 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af63bb5c-4519-4839-8d5f-7847fd5ff92b None admin] PATH_INFO: `/v3/projects` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:01.898300 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af63bb5c-4519-4839-8d5f-7847fd5ff92b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:01.898613 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af63bb5c-4519-4839-8d5f-7847fd5ff92b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:01.900136 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af63bb5c-4519-4839-8d5f-7847fd5ff92b None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:01.914001 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 270/538] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:52:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:01.919819 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af63bb5c-4519-4839-8d5f-7847fd5ff92b None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:02.009338 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e19def3-7141-49f8-9cd7-6a483e48c554 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:02.045325 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b533fc6a-b9b9-4d1b-b951-c6f3838de371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:02.045966 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b533fc6a-b9b9-4d1b-b951-c6f3838de371 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:02.046269 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b533fc6a-b9b9-4d1b-b951-c6f3838de371 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:02.046557 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b533fc6a-b9b9-4d1b-b951-c6f3838de371 None admin] PATH_INFO: `/v3/projects` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:02.047369 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b533fc6a-b9b9-4d1b-b951-c6f3838de371 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:02.047689 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b533fc6a-b9b9-4d1b-b951-c6f3838de371 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:02.049808 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b533fc6a-b9b9-4d1b-b951-c6f3838de371 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:02.063775 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 269/539] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:52:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:02.070313 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b533fc6a-b9b9-4d1b-b951-c6f3838de371 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:02.106222 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:02.138872 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af63bb5c-4519-4839-8d5f-7847fd5ff92b None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:02.185925 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b533fc6a-b9b9-4d1b-b951-c6f3838de371 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:02.222608 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14fa4cc-7cd1-493e-bd91-3f49c4581cfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:02.223306 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14fa4cc-7cd1-493e-bd91-3f49c4581cfb None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:02.223560 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7415f34-a872-490b-99ba-c32d9e790eea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:02.223712 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14fa4cc-7cd1-493e-bd91-3f49c4581cfb None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:02.223877 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14fa4cc-7cd1-493e-bd91-3f49c4581cfb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:02.224513 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7415f34-a872-490b-99ba-c32d9e790eea None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:02.224728 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14fa4cc-7cd1-493e-bd91-3f49c4581cfb None admin] RBAC: Adding query filter params (role.id=08f16502894349708527e30032247468, scope.project.id=fc99d742ae004954b6d841293b5fa79c, user.id=8de553016ea641cdbf9c4a8e08c96a67) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:02.224996 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7415f34-a872-490b-99ba-c32d9e790eea None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:02.225133 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14fa4cc-7cd1-493e-bd91-3f49c4581cfb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:02.225462 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7415f34-a872-490b-99ba-c32d9e790eea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:02.226530 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7415f34-a872-490b-99ba-c32d9e790eea None admin] RBAC: Adding query filter params (role.id=08f16502894349708527e30032247468, scope.project.id=fc99d742ae004954b6d841293b5fa79c, user.id=7f75f8f59dcd42c480aac3eca18fc8db) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:02.226674 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14fa4cc-7cd1-493e-bd91-3f49c4581cfb None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:02.227049 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7415f34-a872-490b-99ba-c32d9e790eea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:02.228700 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7415f34-a872-490b-99ba-c32d9e790eea None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:02.233230 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 271/540] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:52:01 2022] GET /identity/v3/role_assignments?user.id=8de553016ea641cdbf9c4a8e08c96a67&scope.project.id=fc99d742ae004954b6d841293b5fa79c&role.id=08f16502894349708527e30032247468 => generated 259 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:02.235343 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 270/541] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:52:02 2022] GET /identity/v3/role_assignments?user.id=7f75f8f59dcd42c480aac3eca18fc8db&scope.project.id=fc99d742ae004954b6d841293b5fa79c&role.id=08f16502894349708527e30032247468 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:52:02.312517 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3a5f66-cb33-4dee-aefb-dbb24b5cbddf None None] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:02.312732 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3a5f66-cb33-4dee-aefb-dbb24b5cbddf None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:02.312995 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3a5f66-cb33-4dee-aefb-dbb24b5cbddf None None] PATH_INFO: `/` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:02.314621 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 272/542] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:52:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:52:02.321037 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faca9974-932f-4305-a284-2a7a366099ee None None] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:02.321389 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faca9974-932f-4305-a284-2a7a366099ee None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:02.321615 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faca9974-932f-4305-a284-2a7a366099ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:02.343019 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.auth.core [None req-faca9974-932f-4305-a284-2a7a366099ee None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:52:02.344271 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-faca9974-932f-4305-a284-2a7a366099ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:52:02.453017 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 271/543] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:52:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:52:02.490567 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3a5f66-cb33-4dee-aefb-dbb24b5cbddf None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:02.545130 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:02.611770 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3a5f66-cb33-4dee-aefb-dbb24b5cbddf None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:02.656583 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adae7116-dc5d-4624-84e0-81ae82f6f606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:02.657431 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adae7116-dc5d-4624-84e0-81ae82f6f606 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:02.657757 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adae7116-dc5d-4624-84e0-81ae82f6f606 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:02.658620 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adae7116-dc5d-4624-84e0-81ae82f6f606 None admin] PATH_INFO: `/v3/roles/service` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:02.675949 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adae7116-dc5d-4624-84e0-81ae82f6f606 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:52:02.683876 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-adae7116-dc5d-4624-84e0-81ae82f6f606 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:52:02.683876 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:02.683876 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:02.683876 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:02.683876 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:02.683876 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:02.683876 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:02.683876 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:02.683876 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:02.683876 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:02.683876 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:02.683876 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:02.683876 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:52:02.683876 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:52:02.683876 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:52:02.683876 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:52:02.684925 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:52:02.684925 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:52:02.684925 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:52:02.684925 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:52:02.684925 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:02.684925 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:02.684925 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:02.684925 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:02.684925 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:02.684925 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:02.684925 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:02.684925 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:52:02.684925 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:02.684925 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:02.684925 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:02.684925 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:02.685998 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:02.685998 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:02.685998 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:02.685998 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:02.685998 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:52:02.685998 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:52:02.685998 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:52:02.685998 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:52:02.685998 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:52:02.685998 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:52:02.685998 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:52:02.685998 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:52:02.685998 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 273/544] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 14:52:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:02.693150 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faca9974-932f-4305-a284-2a7a366099ee None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:02.715961 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:02.834278 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faca9974-932f-4305-a284-2a7a366099ee None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:02.879880 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef362291-16a2-4874-9560-8a90dc952d9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:02.880624 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef362291-16a2-4874-9560-8a90dc952d9b None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:02.880916 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef362291-16a2-4874-9560-8a90dc952d9b None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:02.881200 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef362291-16a2-4874-9560-8a90dc952d9b None admin] PATH_INFO: `/v3/roles` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:02.882005 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef362291-16a2-4874-9560-8a90dc952d9b None admin] RBAC: Adding query filter params (name=service) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:02.882323 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef362291-16a2-4874-9560-8a90dc952d9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:02.883716 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef362291-16a2-4874-9560-8a90dc952d9b None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:02.898081 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 272/545] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 19 14:52:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:02.918707 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adae7116-dc5d-4624-84e0-81ae82f6f606 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:03.056474 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:03.060165 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adae7116-dc5d-4624-84e0-81ae82f6f606 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:03.101728 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01d7326-17de-4c31-8a9b-ebd14c532e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:03.102568 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01d7326-17de-4c31-8a9b-ebd14c532e71 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:03.102859 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01d7326-17de-4c31-8a9b-ebd14c532e71 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:03.103139 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01d7326-17de-4c31-8a9b-ebd14c532e71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:03.115674 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01d7326-17de-4c31-8a9b-ebd14c532e71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:03.117216 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01d7326-17de-4c31-8a9b-ebd14c532e71 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:03.135155 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-d01d7326-17de-4c31-8a9b-ebd14c532e71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:52:03.135155 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:03.135155 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:03.135155 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:03.135155 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:03.135155 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:03.135155 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:03.135155 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:03.135155 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:03.135155 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:03.135155 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:03.135155 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:03.135155 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:52:03.135155 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:52:03.135155 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:52:03.136152 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:52:03.136152 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:03.136152 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:03.136152 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:03.136152 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:03.136152 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:03.136152 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:03.136152 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:03.136152 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:52:03.136152 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:03.136152 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:03.136152 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:03.136152 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:03.136152 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:03.136152 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:03.136152 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:03.136152 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:03.137136 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:52:03.137136 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:52:03.137136 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:52:03.137136 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:52:03.137136 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 274/546] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:52:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:03.143283 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef362291-16a2-4874-9560-8a90dc952d9b None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:03.213102 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e3540c-8a50-466b-8c0a-87945bd2199b None None] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:03.213450 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e3540c-8a50-466b-8c0a-87945bd2199b None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:03.213678 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e3540c-8a50-466b-8c0a-87945bd2199b None None] PATH_INFO: `/` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:03.215403 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 275/547] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:52:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:52:03.221021 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bef3a7-f4d3-4086-bcb9-f626cb577998 None None] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:03.221387 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bef3a7-f4d3-4086-bcb9-f626cb577998 None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:03.221685 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bef3a7-f4d3-4086-bcb9-f626cb577998 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:03.244219 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.auth.core [None req-97bef3a7-f4d3-4086-bcb9-f626cb577998 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:52:03.246946 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.fernet_utils [None req-97bef3a7-f4d3-4086-bcb9-f626cb577998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:52:03.285230 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:03.319877 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef362291-16a2-4874-9560-8a90dc952d9b None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:03.380734 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 276/548] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:52:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:52:03.386021 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2175fd08-041f-4b21-b045-c234513908f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:03.386985 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2175fd08-041f-4b21-b045-c234513908f9 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:03.387501 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2175fd08-041f-4b21-b045-c234513908f9 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:03.387997 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2175fd08-041f-4b21-b045-c234513908f9 None admin] PATH_INFO: `/v3/domains` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:03.388960 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2175fd08-041f-4b21-b045-c234513908f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:03.389496 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2175fd08-041f-4b21-b045-c234513908f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:03.391200 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2175fd08-041f-4b21-b045-c234513908f9 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:03.409573 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 273/549] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:52:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:03.414815 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bef3a7-f4d3-4086-bcb9-f626cb577998 None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:03.414977 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2175fd08-041f-4b21-b045-c234513908f9 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:03.535954 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2175fd08-041f-4b21-b045-c234513908f9 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:03.555651 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:03.587969 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bef3a7-f4d3-4086-bcb9-f626cb577998 None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:03.606557 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e6f0af-dcd7-4f30-af26-5405159a27b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:03.607385 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e6f0af-dcd7-4f30-af26-5405159a27b7 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:03.607681 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e6f0af-dcd7-4f30-af26-5405159a27b7 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:03.607969 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e6f0af-dcd7-4f30-af26-5405159a27b7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:03.621644 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e6f0af-dcd7-4f30-af26-5405159a27b7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:03.623137 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e6f0af-dcd7-4f30-af26-5405159a27b7 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:03.624070 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a785b2c-887a-4e93-9d4d-302b9dba5aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:03.624879 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a785b2c-887a-4e93-9d4d-302b9dba5aa2 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:03.625167 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a785b2c-887a-4e93-9d4d-302b9dba5aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:03.625448 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a785b2c-887a-4e93-9d4d-302b9dba5aa2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:03.636640 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:03.636875 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a785b2c-887a-4e93-9d4d-302b9dba5aa2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:52:03.646295 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-3a785b2c-887a-4e93-9d4d-302b9dba5aa2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:52:03.646295 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:03.646295 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:03.646295 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:03.646295 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:03.646295 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:03.646295 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:03.646295 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:03.646295 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:03.646295 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:03.646295 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:03.646295 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:03.646295 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:52:03.646295 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:52:03.646295 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:52:03.646295 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:52:03.647349 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:52:03.647349 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:52:03.647349 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:52:03.647349 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:52:03.647349 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:03.647349 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:03.647349 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:03.647349 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:03.647349 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:03.647349 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:03.647349 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:03.647349 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:52:03.647349 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:03.647349 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:03.647349 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:03.647349 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:03.648330 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:03.648330 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:03.648330 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:03.648330 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:03.648330 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:52:03.648330 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:52:03.648330 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:52:03.648330 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:52:03.648330 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:52:03.648330 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:52:03.648330 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:52:03.648330 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:52:03.648330 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 277/550] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 14:52:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:03.659582 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a785b2c-887a-4e93-9d4d-302b9dba5aa2 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:03.659755 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-67e6f0af-dcd7-4f30-af26-5405159a27b7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 14:52:03.659755 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:03.659755 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:03.659755 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:03.659755 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:03.659755 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:03.659755 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:03.659755 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:03.659755 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:03.659755 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:03.659755 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:03.659755 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:03.659755 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:52:03.659755 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:52:03.659755 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:52:03.659755 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:52:03.660769 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:03.660769 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:03.660769 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:52:03.660769 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:52:03.660769 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:52:03.660769 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:52:03.660769 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:03.660769 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:03.660769 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:03.660769 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:03.660769 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:03.660769 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:52:03.660769 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:03.660769 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:03.660769 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:03.660769 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:03.660769 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:03.660769 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:03.661890 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:03.661890 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:03.661890 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:52:03.661890 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:52:03.661890 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:52:03.661890 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:52:03.661890 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:52:03.661890 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:52:03.661890 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 14:52:03.661890 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:52:03.661890 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 274/551] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:52:03 2022] GET /identity/v3/users/swift => generated 83 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:03.665878 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e6f0af-dcd7-4f30-af26-5405159a27b7 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:03.838808 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a785b2c-887a-4e93-9d4d-302b9dba5aa2 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:03.871827 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e6f0af-dcd7-4f30-af26-5405159a27b7 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:03.921955 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a532ed-9da5-4f82-b163-1eec2de5a6b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:03.922553 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a532ed-9da5-4f82-b163-1eec2de5a6b6 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:03.922844 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a532ed-9da5-4f82-b163-1eec2de5a6b6 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:03.923127 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a532ed-9da5-4f82-b163-1eec2de5a6b6 None admin] PATH_INFO: `/v3/roles` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:03.924286 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a532ed-9da5-4f82-b163-1eec2de5a6b6 None admin] RBAC: Adding query filter params (name=service) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:03.924609 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a532ed-9da5-4f82-b163-1eec2de5a6b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:03.926031 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a532ed-9da5-4f82-b163-1eec2de5a6b6 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:03.943217 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 278/552] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 19 14:52:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:03.949775 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a532ed-9da5-4f82-b163-1eec2de5a6b6 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:03.951379 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af67e10d-8543-4b0e-95f4-91636262a2dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:03.952078 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af67e10d-8543-4b0e-95f4-91636262a2dc None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:03.952371 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af67e10d-8543-4b0e-95f4-91636262a2dc None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:03.952656 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af67e10d-8543-4b0e-95f4-91636262a2dc None admin] PATH_INFO: `/v3/users` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:03.953523 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af67e10d-8543-4b0e-95f4-91636262a2dc None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:03.953868 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af67e10d-8543-4b0e-95f4-91636262a2dc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:03.955682 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af67e10d-8543-4b0e-95f4-91636262a2dc None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:03.966212 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:03.992592 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 275/553] 10.222.0.57 () {56 vars in 1135 bytes} [Thu May 19 14:52:03 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:03.999564 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af67e10d-8543-4b0e-95f4-91636262a2dc None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:04.085050 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:04.133326 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af67e10d-8543-4b0e-95f4-91636262a2dc None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:04.135956 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a532ed-9da5-4f82-b163-1eec2de5a6b6 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:04.186355 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2050c047-9f10-44cd-b8dc-a8ce65dba315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:04.187486 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2050c047-9f10-44cd-b8dc-a8ce65dba315 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:04.187799 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b449b1bc-49ac-491d-a3b7-888ba94e9b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:04.188274 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2050c047-9f10-44cd-b8dc-a8ce65dba315 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:04.188605 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b449b1bc-49ac-491d-a3b7-888ba94e9b1d None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:04.188920 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b449b1bc-49ac-491d-a3b7-888ba94e9b1d None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:04.189203 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b449b1bc-49ac-491d-a3b7-888ba94e9b1d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:04.189521 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2050c047-9f10-44cd-b8dc-a8ce65dba315 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:04.202596 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b449b1bc-49ac-491d-a3b7-888ba94e9b1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:04.205126 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b449b1bc-49ac-491d-a3b7-888ba94e9b1d None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:04.229557 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-b449b1bc-49ac-491d-a3b7-888ba94e9b1d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:52:04.229557 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:04.229557 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:04.229557 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:04.229557 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:04.229557 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:04.229557 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:04.229557 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:04.229557 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:04.229557 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:04.229557 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:04.229557 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:04.229557 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:52:04.229557 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:52:04.229557 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:52:04.230704 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:52:04.230704 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:04.230704 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:04.230704 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:04.230704 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:04.230704 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:04.230704 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:04.230704 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:04.230704 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:52:04.230704 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:04.230704 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:04.230704 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:04.230704 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:04.230704 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:04.230704 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:04.230704 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:04.230704 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:04.231710 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:52:04.231710 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:52:04.231710 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:52:04.231710 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:52:04.231710 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 279/554] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:52:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:04.232506 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04245ee0-3ba8-426c-9aeb-0feb8191a3dd None None] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:04.232696 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04245ee0-3ba8-426c-9aeb-0feb8191a3dd None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:04.232972 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04245ee0-3ba8-426c-9aeb-0feb8191a3dd None None] PATH_INFO: `/` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:04.234290 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 280/555] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:52:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:52:04.236460 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04245ee0-3ba8-426c-9aeb-0feb8191a3dd None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:04.294463 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2050c047-9f10-44cd-b8dc-a8ce65dba315 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:04.296403 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2050c047-9f10-44cd-b8dc-a8ce65dba315 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:04.325339 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:04.362130 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:04.392510 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04245ee0-3ba8-426c-9aeb-0feb8191a3dd None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:04.421019 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 276/556] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:52:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:52:04.423156 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf14355-196e-48a3-a4c8-4920c608b678 None None] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:04.424623 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf14355-196e-48a3-a4c8-4920c608b678 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:04.425479 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf14355-196e-48a3-a4c8-4920c608b678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:04.429232 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53b9365-79bc-4d9a-a6aa-8e512b7cbccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:04.429945 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53b9365-79bc-4d9a-a6aa-8e512b7cbccd None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:04.430237 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53b9365-79bc-4d9a-a6aa-8e512b7cbccd None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:04.430521 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53b9365-79bc-4d9a-a6aa-8e512b7cbccd None admin] PATH_INFO: `/v3/domains` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:04.431254 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53b9365-79bc-4d9a-a6aa-8e512b7cbccd None admin] RBAC: Adding query filter params (name=Default) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:04.431568 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53b9365-79bc-4d9a-a6aa-8e512b7cbccd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:04.432967 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53b9365-79bc-4d9a-a6aa-8e512b7cbccd None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:04.447429 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 281/557] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:52:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:52:04.448230 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53b9365-79bc-4d9a-a6aa-8e512b7cbccd None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:04.449231 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.auth.core [None req-3bf14355-196e-48a3-a4c8-4920c608b678 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:52:04.450809 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-3bf14355-196e-48a3-a4c8-4920c608b678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:52:04.545223 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 277/558] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:52:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:52:04.546974 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf14355-196e-48a3-a4c8-4920c608b678 None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:04.580751 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53b9365-79bc-4d9a-a6aa-8e512b7cbccd None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:04.602112 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:04.622743 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bcf528-6101-4a4d-920c-7a82ad80e538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:04.623551 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bcf528-6101-4a4d-920c-7a82ad80e538 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:04.623839 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bcf528-6101-4a4d-920c-7a82ad80e538 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:04.624121 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bcf528-6101-4a4d-920c-7a82ad80e538 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:04.635588 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bcf528-6101-4a4d-920c-7a82ad80e538 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:04.637062 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bcf528-6101-4a4d-920c-7a82ad80e538 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:04.658753 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-61bcf528-6101-4a4d-920c-7a82ad80e538 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:52:04.658753 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:04.658753 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:04.658753 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:04.658753 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:04.658753 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:04.658753 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:04.658753 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:04.658753 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:04.658753 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:04.658753 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:04.658753 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:04.658753 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:52:04.658753 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:52:04.658753 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:52:04.659936 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:52:04.659936 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:04.659936 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:04.659936 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:04.659936 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:04.659936 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:04.659936 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:04.659936 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:04.659936 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:52:04.659936 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:04.659936 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:04.659936 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:04.659936 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:04.659936 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:04.659936 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:04.659936 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:04.659936 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:04.660938 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:52:04.660938 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:52:04.660938 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:52:04.660938 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:52:04.660938 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 282/559] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:52:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:04.663169 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bcf528-6101-4a4d-920c-7a82ad80e538 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:04.671172 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:04.690543 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf14355-196e-48a3-a4c8-4920c608b678 None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:04.732561 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03341b0-0355-474d-9beb-56ef90c5ee23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:04.733344 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03341b0-0355-474d-9beb-56ef90c5ee23 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:04.733570 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03341b0-0355-474d-9beb-56ef90c5ee23 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:04.733866 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03341b0-0355-474d-9beb-56ef90c5ee23 None admin] PATH_INFO: `/v3/users/glance` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:04.742861 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03341b0-0355-474d-9beb-56ef90c5ee23 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:04.744338 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03341b0-0355-474d-9beb-56ef90c5ee23 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:04.761352 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-b03341b0-0355-474d-9beb-56ef90c5ee23 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 19 14:52:04.761352 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:04.761352 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:04.761352 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:04.761352 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:04.761352 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:04.761352 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:04.761352 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:04.761352 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:04.761352 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:04.761352 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:04.761352 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:04.761352 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:52:04.761352 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:52:04.761352 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:52:04.761352 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:52:04.762544 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:04.762544 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:04.762544 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:52:04.762544 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:52:04.762544 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:52:04.762544 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:52:04.762544 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:04.762544 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:04.762544 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:04.762544 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:04.762544 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:04.762544 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:52:04.762544 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:04.762544 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:04.762544 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:04.762544 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:04.762544 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:04.762544 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:04.763703 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:04.763703 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:04.763703 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:52:04.763703 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:52:04.763703 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:52:04.763703 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:52:04.763703 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:52:04.763703 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:52:04.763703 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 19 14:52:04.763703 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:52:04.763703 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 278/560] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:52:04 2022] GET /identity/v3/users/glance => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:04.764400 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03341b0-0355-474d-9beb-56ef90c5ee23 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:04.803737 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bcf528-6101-4a4d-920c-7a82ad80e538 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:04.847436 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d308e081-4345-43a3-a6a0-edc0e54aa535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:04.848343 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d308e081-4345-43a3-a6a0-edc0e54aa535 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:04.848635 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d308e081-4345-43a3-a6a0-edc0e54aa535 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:04.848913 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d308e081-4345-43a3-a6a0-edc0e54aa535 None admin] PATH_INFO: `/v3/roles/service` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:04.862575 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d308e081-4345-43a3-a6a0-edc0e54aa535 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:52:04.870697 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-d308e081-4345-43a3-a6a0-edc0e54aa535 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:52:04.870697 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:04.870697 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:04.870697 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:04.870697 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:04.870697 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:04.870697 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:04.870697 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:04.870697 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:04.870697 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:04.870697 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:04.870697 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:04.870697 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:52:04.870697 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:52:04.870697 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:52:04.870697 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:52:04.871854 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:52:04.871854 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:52:04.871854 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:52:04.871854 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:52:04.871854 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:04.871854 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:04.871854 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:04.871854 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:04.871854 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:04.871854 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:04.871854 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:04.871854 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:52:04.871854 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:04.871854 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:04.871854 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:04.871854 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:04.872823 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:04.872823 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:04.872823 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:04.872823 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:04.872823 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:52:04.872823 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:52:04.872823 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:52:04.872823 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:52:04.872823 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:52:04.872823 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:52:04.872823 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:52:04.872823 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:52:04.877074 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 283/561] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 14:52:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:04.878470 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d308e081-4345-43a3-a6a0-edc0e54aa535 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:04.911510 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:04.914280 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03341b0-0355-474d-9beb-56ef90c5ee23 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:04.957180 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8b767a-c7ed-48c8-95d0-06a0862549ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:04.957866 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8b767a-c7ed-48c8-95d0-06a0862549ee None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:04.958171 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8b767a-c7ed-48c8-95d0-06a0862549ee None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:04.958661 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8b767a-c7ed-48c8-95d0-06a0862549ee None admin] PATH_INFO: `/v3/domains` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:04.959252 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8b767a-c7ed-48c8-95d0-06a0862549ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:04.959499 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8b767a-c7ed-48c8-95d0-06a0862549ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:04.960893 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8b767a-c7ed-48c8-95d0-06a0862549ee None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:04.973377 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 279/562] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:52:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:04.978096 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8b767a-c7ed-48c8-95d0-06a0862549ee None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:04.989985 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:05.006640 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d308e081-4345-43a3-a6a0-edc0e54aa535 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:05.044888 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2971f64-ef56-4180-a207-9ef9be0b3b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:05.045563 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2971f64-ef56-4180-a207-9ef9be0b3b13 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:05.045869 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2971f64-ef56-4180-a207-9ef9be0b3b13 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:05.046155 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2971f64-ef56-4180-a207-9ef9be0b3b13 None admin] PATH_INFO: `/v3/users` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:05.046999 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2971f64-ef56-4180-a207-9ef9be0b3b13 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:05.047313 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2971f64-ef56-4180-a207-9ef9be0b3b13 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:05.048959 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2971f64-ef56-4180-a207-9ef9be0b3b13 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:05.066812 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 284/563] 10.222.0.57 () {56 vars in 1137 bytes} [Thu May 19 14:52:04 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:05.068097 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a448d83d-513c-4944-ba24-f746077f7711 None None] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:05.068390 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a448d83d-513c-4944-ba24-f746077f7711 None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:05.068668 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a448d83d-513c-4944-ba24-f746077f7711 None None] PATH_INFO: `/` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:05.069908 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 285/564] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:52:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:52:05.071062 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a448d83d-513c-4944-ba24-f746077f7711 None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:05.093620 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8b767a-c7ed-48c8-95d0-06a0862549ee None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:05.129696 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde2164c-18a7-4260-b5b1-b6a594a36a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:05.130391 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde2164c-18a7-4260-b5b1-b6a594a36a3d None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:05.130681 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde2164c-18a7-4260-b5b1-b6a594a36a3d None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:05.130964 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde2164c-18a7-4260-b5b1-b6a594a36a3d None admin] PATH_INFO: `/v3/roles` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:05.131717 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde2164c-18a7-4260-b5b1-b6a594a36a3d None admin] RBAC: Adding query filter params (name=service) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:05.132030 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde2164c-18a7-4260-b5b1-b6a594a36a3d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:05.133415 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde2164c-18a7-4260-b5b1-b6a594a36a3d None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:05.141534 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 280/565] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 19 14:52:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:05.144038 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250fa6ac-3765-4949-bfe5-1ef45427d265 None None] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:05.144334 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250fa6ac-3765-4949-bfe5-1ef45427d265 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:05.144621 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250fa6ac-3765-4949-bfe5-1ef45427d265 None None] PATH_INFO: `/` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:05.146051 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 281/566] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:52:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:52:05.146592 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250fa6ac-3765-4949-bfe5-1ef45427d265 None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:05.160983 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:05.180301 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a448d83d-513c-4944-ba24-f746077f7711 None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:05.217148 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:05.218230 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69e9508-965d-4c66-b43b-432d4fb3f912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:05.219064 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69e9508-965d-4c66-b43b-432d4fb3f912 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:05.219375 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69e9508-965d-4c66-b43b-432d4fb3f912 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:05.220112 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69e9508-965d-4c66-b43b-432d4fb3f912 None admin] PATH_INFO: `/v3/projects/service` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:05.227281 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69e9508-965d-4c66-b43b-432d4fb3f912 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:05.228756 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69e9508-965d-4c66-b43b-432d4fb3f912 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:05.242572 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-e69e9508-965d-4c66-b43b-432d4fb3f912 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:52:05.242572 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:05.242572 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:05.242572 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:05.242572 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:05.242572 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:05.242572 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:05.242572 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:05.242572 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:05.242572 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:05.242572 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:05.242572 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:05.242572 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:52:05.242572 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:52:05.242572 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:52:05.243700 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:52:05.243700 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:05.243700 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:05.243700 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:05.243700 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:05.243700 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:05.243700 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:05.243700 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:05.243700 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:52:05.243700 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:05.243700 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:05.243700 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:05.243700 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:05.243700 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:05.243700 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:05.243700 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:05.243700 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:05.244682 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:52:05.244682 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:52:05.244682 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:52:05.244682 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:52:05.244682 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:52:05.244682 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:52:05.244682 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:52:05.244682 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:52:05.244682 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 286/567] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:52:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:05.245233 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d838fd-4077-4438-b1f0-e215205414e7 None None] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:05.245314 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d838fd-4077-4438-b1f0-e215205414e7 None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:05.245583 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d838fd-4077-4438-b1f0-e215205414e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:05.253516 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250fa6ac-3765-4949-bfe5-1ef45427d265 None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:05.265124 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.auth.core [None req-c5d838fd-4077-4438-b1f0-e215205414e7 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:52:05.266652 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.fernet_utils [None req-c5d838fd-4077-4438-b1f0-e215205414e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:52:05.289632 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0d4c39-1745-4806-85b4-9d5defc11da1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:05.290469 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0d4c39-1745-4806-85b4-9d5defc11da1 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:05.290759 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0d4c39-1745-4806-85b4-9d5defc11da1 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:05.291040 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0d4c39-1745-4806-85b4-9d5defc11da1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:05.301768 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0d4c39-1745-4806-85b4-9d5defc11da1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:05.303243 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0d4c39-1745-4806-85b4-9d5defc11da1 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:05.320890 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-8f0d4c39-1745-4806-85b4-9d5defc11da1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:52:05.320890 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:05.320890 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:05.320890 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:05.320890 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:05.320890 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:05.320890 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:05.320890 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:05.320890 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:05.320890 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:05.320890 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:05.320890 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:05.320890 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:52:05.320890 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:52:05.320890 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:52:05.322197 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:52:05.322197 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:05.322197 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:05.322197 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:05.322197 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:05.322197 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:05.322197 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:05.322197 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:05.322197 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:52:05.322197 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:05.322197 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:05.322197 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:05.322197 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:05.322197 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:05.322197 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:05.322197 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:05.322197 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:05.323447 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:52:05.323447 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:52:05.323447 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:52:05.323447 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:52:05.323447 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 282/568] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:52:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:05.323761 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bc9ca9-ee74-4c76-a5e3-78c7c484c9eb None None] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:05.323879 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bc9ca9-ee74-4c76-a5e3-78c7c484c9eb None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:05.324162 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bc9ca9-ee74-4c76-a5e3-78c7c484c9eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:05.340885 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.auth.core [None req-81bc9ca9-ee74-4c76-a5e3-78c7c484c9eb None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:52:05.341174 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 287/569] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:52:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:52:05.342052 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d838fd-4077-4438-b1f0-e215205414e7 None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:05.342179 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-81bc9ca9-ee74-4c76-a5e3-78c7c484c9eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:52:05.410798 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 283/570] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:52:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:52:05.411649 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bc9ca9-ee74-4c76-a5e3-78c7c484c9eb None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:05.438462 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:05.463507 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d838fd-4077-4438-b1f0-e215205414e7 None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:05.468757 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:05.499871 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc82aeb-8b72-4c8e-98e6-274c18c0ea14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:05.500623 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc82aeb-8b72-4c8e-98e6-274c18c0ea14 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:05.500924 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc82aeb-8b72-4c8e-98e6-274c18c0ea14 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:05.501209 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc82aeb-8b72-4c8e-98e6-274c18c0ea14 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:05.512083 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc82aeb-8b72-4c8e-98e6-274c18c0ea14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:05.513555 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc82aeb-8b72-4c8e-98e6-274c18c0ea14 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:05.522824 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bc9ca9-ee74-4c76-a5e3-78c7c484c9eb None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:05.530935 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-4cc82aeb-8b72-4c8e-98e6-274c18c0ea14 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:52:05.530935 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:05.530935 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:05.530935 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:05.530935 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:05.530935 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:05.530935 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:05.530935 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:05.530935 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:05.530935 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:05.530935 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:05.530935 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:05.530935 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:52:05.530935 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:52:05.530935 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:52:05.531944 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:52:05.531944 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:05.531944 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:05.531944 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:05.531944 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:05.531944 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:05.531944 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:05.531944 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:05.531944 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:52:05.531944 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:05.531944 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:05.531944 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:05.531944 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:05.531944 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:05.531944 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:05.531944 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:05.531944 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:05.533006 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:52:05.533006 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:52:05.533006 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:52:05.533006 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:52:05.533006 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 288/571] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:52:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:05.533006 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc82aeb-8b72-4c8e-98e6-274c18c0ea14 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:05.559954 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3901000-4a26-4202-a708-873e59c516dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:05.560627 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3901000-4a26-4202-a708-873e59c516dc None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:05.560919 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3901000-4a26-4202-a708-873e59c516dc None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:05.561202 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3901000-4a26-4202-a708-873e59c516dc None admin] PATH_INFO: `/v3/projects` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:05.562027 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3901000-4a26-4202-a708-873e59c516dc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:05.562349 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3901000-4a26-4202-a708-873e59c516dc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:05.563780 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3901000-4a26-4202-a708-873e59c516dc None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:05.576397 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 284/572] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:52:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:05.577206 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3901000-4a26-4202-a708-873e59c516dc None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:05.643644 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc82aeb-8b72-4c8e-98e6-274c18c0ea14 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:05.672849 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:05.681065 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3901000-4a26-4202-a708-873e59c516dc None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:05.682390 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b20677-b3fd-4da2-8d26-1f9932d18ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:05.683067 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b20677-b3fd-4da2-8d26-1f9932d18ee9 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:05.683361 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b20677-b3fd-4da2-8d26-1f9932d18ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:05.683642 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b20677-b3fd-4da2-8d26-1f9932d18ee9 None admin] PATH_INFO: `/v3/domains` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:05.684401 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b20677-b3fd-4da2-8d26-1f9932d18ee9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:05.684714 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b20677-b3fd-4da2-8d26-1f9932d18ee9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:05.686134 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b20677-b3fd-4da2-8d26-1f9932d18ee9 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:05.694792 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:05.698823 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 289/573] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:52:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:05.699649 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b20677-b3fd-4da2-8d26-1f9932d18ee9 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:05.717448 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c116074-feed-49c5-83ee-eca74dd855e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:05.718281 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c116074-feed-49c5-83ee-eca74dd855e2 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:05.718575 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c116074-feed-49c5-83ee-eca74dd855e2 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:05.718861 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c116074-feed-49c5-83ee-eca74dd855e2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:05.729166 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c116074-feed-49c5-83ee-eca74dd855e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:52:05.737421 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-7c116074-feed-49c5-83ee-eca74dd855e2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:52:05.737421 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:05.737421 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:05.737421 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:05.737421 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:05.737421 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:05.737421 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:05.737421 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:05.737421 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:05.737421 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:05.737421 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:05.737421 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:05.737421 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:52:05.737421 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:52:05.737421 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:52:05.737421 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:52:05.738673 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:52:05.738673 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:52:05.738673 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:52:05.738673 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:52:05.738673 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:05.738673 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:05.738673 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:05.738673 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:05.738673 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:05.738673 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:05.738673 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:05.738673 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:52:05.738673 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:05.738673 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:05.738673 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:05.738673 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:05.739624 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:05.739624 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:05.739624 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:05.739624 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:05.739624 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:52:05.739624 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:52:05.739624 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:52:05.739624 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:52:05.739624 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:52:05.739624 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:52:05.739624 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:52:05.739624 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:52:05.739624 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 285/574] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 14:52:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:05.739624 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c116074-feed-49c5-83ee-eca74dd855e2 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:05.806395 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b20677-b3fd-4da2-8d26-1f9932d18ee9 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:05.840625 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c116074-feed-49c5-83ee-eca74dd855e2 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:05.840841 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d783c630-a9c1-41f9-b4fe-132091e67d43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:05.841464 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d783c630-a9c1-41f9-b4fe-132091e67d43 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:05.841756 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d783c630-a9c1-41f9-b4fe-132091e67d43 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:05.842068 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d783c630-a9c1-41f9-b4fe-132091e67d43 None admin] PATH_INFO: `/v3/roles/service` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:05.852921 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d783c630-a9c1-41f9-b4fe-132091e67d43 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:52:05.861328 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-d783c630-a9c1-41f9-b4fe-132091e67d43 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:52:05.861328 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:05.861328 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:05.861328 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:05.861328 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:05.861328 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:05.861328 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:05.861328 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:05.861328 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:05.861328 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:05.861328 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:05.861328 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:05.861328 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:52:05.861328 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:52:05.861328 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:52:05.861328 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:52:05.862587 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:52:05.862587 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:52:05.862587 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:52:05.862587 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:52:05.862587 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:05.862587 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:05.862587 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:05.862587 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:05.862587 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:05.862587 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:05.862587 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:05.862587 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:52:05.862587 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:05.862587 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:05.862587 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:05.862587 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:05.863547 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:05.863547 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:05.863547 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:05.863547 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:05.863547 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:52:05.863547 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:52:05.863547 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:52:05.863547 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:52:05.863547 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:52:05.863547 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:52:05.863547 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:52:05.863547 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:52:05.863547 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 290/575] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 14:52:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:05.864479 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d783c630-a9c1-41f9-b4fe-132091e67d43 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:05.878931 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62935740-6172-4e48-9c4c-2888b29d1a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:05.879613 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62935740-6172-4e48-9c4c-2888b29d1a84 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:05.879907 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62935740-6172-4e48-9c4c-2888b29d1a84 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:05.880198 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62935740-6172-4e48-9c4c-2888b29d1a84 None admin] PATH_INFO: `/v3/domains` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:05.880929 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62935740-6172-4e48-9c4c-2888b29d1a84 None admin] RBAC: Adding query filter params (name=Default) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:05.881244 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62935740-6172-4e48-9c4c-2888b29d1a84 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:05.882664 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62935740-6172-4e48-9c4c-2888b29d1a84 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:05.894543 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 286/576] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:52:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:05.895403 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62935740-6172-4e48-9c4c-2888b29d1a84 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:05.916520 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:05.929103 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:05.967684 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d783c630-a9c1-41f9-b4fe-132091e67d43 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:05.999338 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62935740-6172-4e48-9c4c-2888b29d1a84 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:06.002007 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d5679d-0efc-4ab2-aab0-d3f05013b6c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:06.002665 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d5679d-0efc-4ab2-aab0-d3f05013b6c1 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:06.002956 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d5679d-0efc-4ab2-aab0-d3f05013b6c1 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:06.003245 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d5679d-0efc-4ab2-aab0-d3f05013b6c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:06.004079 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d5679d-0efc-4ab2-aab0-d3f05013b6c1 None admin] RBAC: Adding query filter params (role.id=08f16502894349708527e30032247468, scope.project.id=fc99d742ae004954b6d841293b5fa79c, user.id=17afb27b2bbe481e959cf31db4470593) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:06.004400 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d5679d-0efc-4ab2-aab0-d3f05013b6c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:06.005860 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d5679d-0efc-4ab2-aab0-d3f05013b6c1 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:06.011699 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 291/577] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:52:05 2022] GET /identity/v3/role_assignments?user.id=17afb27b2bbe481e959cf31db4470593&scope.project.id=fc99d742ae004954b6d841293b5fa79c&role.id=08f16502894349708527e30032247468 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:06.012490 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d5679d-0efc-4ab2-aab0-d3f05013b6c1 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:06.038589 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1958567-dc49-4363-8c62-34a9354a1aa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:06.039521 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1958567-dc49-4363-8c62-34a9354a1aa9 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:06.039906 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1958567-dc49-4363-8c62-34a9354a1aa9 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:06.040201 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1958567-dc49-4363-8c62-34a9354a1aa9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:06.047915 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1958567-dc49-4363-8c62-34a9354a1aa9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:06.050139 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1958567-dc49-4363-8c62-34a9354a1aa9 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:06.065365 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-f1958567-dc49-4363-8c62-34a9354a1aa9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:52:06.065365 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:06.065365 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:06.065365 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:06.065365 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:06.065365 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:06.065365 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:06.065365 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:06.065365 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:06.065365 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:06.065365 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:06.065365 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:06.065365 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:52:06.065365 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:52:06.065365 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:52:06.066458 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:52:06.066458 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:06.066458 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:06.066458 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:06.066458 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:06.066458 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:06.066458 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:06.066458 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:06.066458 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:52:06.066458 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:06.066458 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:06.066458 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:06.066458 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:06.066458 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:06.066458 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:06.066458 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:06.066458 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:06.067492 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:52:06.067492 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:52:06.067492 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:52:06.067492 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:52:06.067492 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:52:06.067492 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:52:06.067492 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:52:06.067492 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:52:06.067492 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 287/578] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:52:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:06.067492 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1958567-dc49-4363-8c62-34a9354a1aa9 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:06.137882 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d5679d-0efc-4ab2-aab0-d3f05013b6c1 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:06.153199 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:06.175646 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:06.182579 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703f8c29-2bc6-4414-be09-972c86976f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:06.183373 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703f8c29-2bc6-4414-be09-972c86976f9e None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:06.183700 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703f8c29-2bc6-4414-be09-972c86976f9e None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:06.183977 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703f8c29-2bc6-4414-be09-972c86976f9e None admin] PATH_INFO: `/v3/roles` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:06.184743 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703f8c29-2bc6-4414-be09-972c86976f9e None admin] RBAC: Adding query filter params (name=service) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:06.185054 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703f8c29-2bc6-4414-be09-972c86976f9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:06.186541 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1958567-dc49-4363-8c62-34a9354a1aa9 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:06.186679 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703f8c29-2bc6-4414-be09-972c86976f9e None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:06.196634 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 292/579] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 19 14:52:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:06.197632 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703f8c29-2bc6-4414-be09-972c86976f9e None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:06.233014 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5170a6bd-72e4-4fb2-acfb-6100b8dd4c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:06.233699 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5170a6bd-72e4-4fb2-acfb-6100b8dd4c1b None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:06.234039 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5170a6bd-72e4-4fb2-acfb-6100b8dd4c1b None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:06.234329 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5170a6bd-72e4-4fb2-acfb-6100b8dd4c1b None admin] PATH_INFO: `/v3/roles` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:06.235097 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5170a6bd-72e4-4fb2-acfb-6100b8dd4c1b None admin] RBAC: Adding query filter params (name=service) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:06.235422 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5170a6bd-72e4-4fb2-acfb-6100b8dd4c1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:06.237342 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5170a6bd-72e4-4fb2-acfb-6100b8dd4c1b None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:06.246230 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 288/580] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 19 14:52:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:06.247055 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5170a6bd-72e4-4fb2-acfb-6100b8dd4c1b None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:06.314503 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703f8c29-2bc6-4414-be09-972c86976f9e None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:06.350316 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524dcb8d-0ed0-4bc0-8b67-301d47a7c269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:06.351177 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524dcb8d-0ed0-4bc0-8b67-301d47a7c269 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:06.351567 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524dcb8d-0ed0-4bc0-8b67-301d47a7c269 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:06.351947 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524dcb8d-0ed0-4bc0-8b67-301d47a7c269 None admin] PATH_INFO: `/v3/users/nova` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:06.352062 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5170a6bd-72e4-4fb2-acfb-6100b8dd4c1b None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:06.360219 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524dcb8d-0ed0-4bc0-8b67-301d47a7c269 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:06.362505 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524dcb8d-0ed0-4bc0-8b67-301d47a7c269 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:06.378991 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-524dcb8d-0ed0-4bc0-8b67-301d47a7c269 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 14:52:06.378991 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:06.378991 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:06.378991 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:06.378991 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:06.378991 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:06.378991 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:06.378991 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:06.378991 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:06.378991 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:06.378991 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:06.378991 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:06.378991 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:52:06.378991 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:52:06.378991 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:52:06.378991 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:52:06.380285 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:06.380285 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:06.380285 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:52:06.380285 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:52:06.380285 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:52:06.380285 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:52:06.380285 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:06.380285 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:06.380285 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:06.380285 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:06.380285 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:06.380285 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:52:06.380285 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:06.380285 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:06.380285 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:06.380285 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:06.380285 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:06.380285 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:06.381639 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:06.381639 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:06.381639 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:52:06.381639 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:52:06.381639 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:52:06.381639 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:52:06.381639 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:52:06.381639 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:52:06.381639 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 14:52:06.381639 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:52:06.387541 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 293/581] 10.222.0.57 () {56 vars in 1093 bytes} [Thu May 19 14:52:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:06.388428 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524dcb8d-0ed0-4bc0-8b67-301d47a7c269 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:06.391074 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b1bb73-33ee-4726-940f-a2bc48deec1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:06.391826 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b1bb73-33ee-4726-940f-a2bc48deec1a None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:06.392228 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b1bb73-33ee-4726-940f-a2bc48deec1a None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:06.392584 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b1bb73-33ee-4726-940f-a2bc48deec1a None admin] PATH_INFO: `/v3/projects` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:06.393458 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b1bb73-33ee-4726-940f-a2bc48deec1a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:06.393909 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b1bb73-33ee-4726-940f-a2bc48deec1a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:06.395467 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b1bb73-33ee-4726-940f-a2bc48deec1a None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:06.409185 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 289/582] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:52:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:06.410203 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b1bb73-33ee-4726-940f-a2bc48deec1a None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:06.430412 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:06.430967 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:06.505171 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524dcb8d-0ed0-4bc0-8b67-301d47a7c269 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:06.524069 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b1bb73-33ee-4726-940f-a2bc48deec1a None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:06.544475 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f656ae51-8dfc-4bbc-8854-ae7d14d4afe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:06.545419 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f656ae51-8dfc-4bbc-8854-ae7d14d4afe7 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:06.545810 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f656ae51-8dfc-4bbc-8854-ae7d14d4afe7 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:06.546225 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f656ae51-8dfc-4bbc-8854-ae7d14d4afe7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:06.557704 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f656ae51-8dfc-4bbc-8854-ae7d14d4afe7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:06.559213 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f656ae51-8dfc-4bbc-8854-ae7d14d4afe7 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:06.560280 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14b5235-95f5-4419-be95-3d1a1a528410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:06.561093 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14b5235-95f5-4419-be95-3d1a1a528410 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:06.561383 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14b5235-95f5-4419-be95-3d1a1a528410 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:06.561719 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14b5235-95f5-4419-be95-3d1a1a528410 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:06.572698 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14b5235-95f5-4419-be95-3d1a1a528410 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:06.574188 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14b5235-95f5-4419-be95-3d1a1a528410 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:06.577486 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-f656ae51-8dfc-4bbc-8854-ae7d14d4afe7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:52:06.577486 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:06.577486 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:06.577486 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:06.577486 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:06.577486 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:06.577486 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:06.577486 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:06.577486 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:06.577486 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:06.577486 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:06.577486 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:06.577486 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:52:06.577486 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:52:06.577486 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:52:06.578658 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:52:06.578658 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:06.578658 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:06.578658 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:06.578658 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:06.578658 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:06.578658 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:06.578658 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:06.578658 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:52:06.578658 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:06.578658 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:06.578658 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:06.578658 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:06.578658 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:06.578658 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:06.578658 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:06.578658 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:06.585151 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:52:06.585151 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:52:06.585151 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:52:06.585151 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:52:06.585151 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 294/583] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:52:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:06.585151 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f656ae51-8dfc-4bbc-8854-ae7d14d4afe7 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:06.598361 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-b14b5235-95f5-4419-be95-3d1a1a528410 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:52:06.598361 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:06.598361 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:06.598361 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:06.598361 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:06.598361 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:06.598361 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:06.598361 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:06.598361 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:06.598361 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:06.598361 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:06.598361 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:06.598361 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:52:06.598361 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:52:06.598361 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:52:06.599559 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:52:06.599559 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:06.599559 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:06.599559 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:06.599559 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:06.599559 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:06.599559 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:06.599559 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:06.599559 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:52:06.599559 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:06.599559 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:06.599559 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:06.599559 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:06.599559 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:06.599559 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:06.599559 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:06.599559 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:06.600793 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:52:06.600793 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:52:06.600793 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:52:06.600793 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:52:06.601405 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 290/584] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:52:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:06.603428 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14b5235-95f5-4419-be95-3d1a1a528410 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:06.683211 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:06.685088 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:06.701323 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f656ae51-8dfc-4bbc-8854-ae7d14d4afe7 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:06.726240 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14b5235-95f5-4419-be95-3d1a1a528410 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:06.738120 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a6c297-080b-4b21-a25d-b73f353b8655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:06.738796 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a6c297-080b-4b21-a25d-b73f353b8655 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:06.739082 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a6c297-080b-4b21-a25d-b73f353b8655 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:06.739362 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a6c297-080b-4b21-a25d-b73f353b8655 None admin] PATH_INFO: `/v3/users` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:06.740195 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a6c297-080b-4b21-a25d-b73f353b8655 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:06.740509 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a6c297-080b-4b21-a25d-b73f353b8655 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:06.741981 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a6c297-080b-4b21-a25d-b73f353b8655 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:06.765216 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 295/585] 10.222.0.57 () {56 vars in 1133 bytes} [Thu May 19 14:52:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:06.765216 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a6c297-080b-4b21-a25d-b73f353b8655 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:06.770635 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90064222-3e96-467e-9d38-523a708f1369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:06.771645 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90064222-3e96-467e-9d38-523a708f1369 None admin] REQUEST_METHOD: `PUT` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:06.772007 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90064222-3e96-467e-9d38-523a708f1369 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:06.772360 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90064222-3e96-467e-9d38-523a708f1369 None admin] PATH_INFO: `/v3/projects/fc99d742ae004954b6d841293b5fa79c/users/27d9bc13809245299fa5d401f0696215/roles/08f16502894349708527e30032247468` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:06.788119 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90064222-3e96-467e-9d38-523a708f1369 None admin] RBAC: Authorizing `identity:create_grant(project_id=fc99d742ae004954b6d841293b5fa79c, user_id=27d9bc13809245299fa5d401f0696215, role_id=08f16502894349708527e30032247468)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:06.789654 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90064222-3e96-467e-9d38-523a708f1369 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:06.814331 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 291/586] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 14:52:06 2022] PUT /identity/v3/projects/fc99d742ae004954b6d841293b5fa79c/users/27d9bc13809245299fa5d401f0696215/roles/08f16502894349708527e30032247468 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:52:06.814331 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90064222-3e96-467e-9d38-523a708f1369 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:06.886505 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a6c297-080b-4b21-a25d-b73f353b8655 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:06.932868 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90064222-3e96-467e-9d38-523a708f1369 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:06.934891 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7c2ec6-c283-4157-a833-67958ad59a48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:06.936691 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7c2ec6-c283-4157-a833-67958ad59a48 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:06.937163 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7c2ec6-c283-4157-a833-67958ad59a48 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:06.937678 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7c2ec6-c283-4157-a833-67958ad59a48 None admin] PATH_INFO: `/v3/domains` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:06.938733 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7c2ec6-c283-4157-a833-67958ad59a48 None admin] RBAC: Adding query filter params (name=Default) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:06.939978 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7c2ec6-c283-4157-a833-67958ad59a48 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:06.941792 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7c2ec6-c283-4157-a833-67958ad59a48 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:06.956472 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:06.958943 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 296/587] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:52:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:06.960084 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7c2ec6-c283-4157-a833-67958ad59a48 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:06.964477 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:06.983550 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ca5f71-a854-4659-a6fa-703b10b83b8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:06.984461 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ca5f71-a854-4659-a6fa-703b10b83b8f None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:06.984898 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ca5f71-a854-4659-a6fa-703b10b83b8f None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:06.985277 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ca5f71-a854-4659-a6fa-703b10b83b8f None admin] PATH_INFO: `/v3/domains` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:06.986902 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ca5f71-a854-4659-a6fa-703b10b83b8f None admin] RBAC: Adding query filter params (name=Default) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:06.987325 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ca5f71-a854-4659-a6fa-703b10b83b8f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:06.988827 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ca5f71-a854-4659-a6fa-703b10b83b8f None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:07.006249 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 292/588] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:52:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:07.007350 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ca5f71-a854-4659-a6fa-703b10b83b8f None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:07.124988 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7c2ec6-c283-4157-a833-67958ad59a48 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:07.148652 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ca5f71-a854-4659-a6fa-703b10b83b8f None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:07.186273 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55eff153-bbb6-4b3d-8e37-61417980a60e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:07.187798 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eff153-bbb6-4b3d-8e37-61417980a60e None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:07.188450 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eff153-bbb6-4b3d-8e37-61417980a60e None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:07.189772 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eff153-bbb6-4b3d-8e37-61417980a60e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:07.192626 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbf63fa-0518-42d8-8145-20878ae44128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:07.193731 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbf63fa-0518-42d8-8145-20878ae44128 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:07.194050 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbf63fa-0518-42d8-8145-20878ae44128 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:07.194338 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbf63fa-0518-42d8-8145-20878ae44128 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:07.203702 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbf63fa-0518-42d8-8145-20878ae44128 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:07.205578 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbf63fa-0518-42d8-8145-20878ae44128 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:07.210982 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55eff153-bbb6-4b3d-8e37-61417980a60e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:07.213816 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55eff153-bbb6-4b3d-8e37-61417980a60e None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:07.222403 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-9dbf63fa-0518-42d8-8145-20878ae44128 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 19 14:52:07.222403 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:07.222403 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:07.222403 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:07.222403 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:07.222403 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:07.222403 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:07.222403 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:07.222403 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:07.222403 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:07.222403 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:07.222403 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:07.222403 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:52:07.222403 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:52:07.222403 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:52:07.222403 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:52:07.224666 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:07.224666 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:07.224666 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:52:07.224666 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:52:07.224666 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:52:07.224666 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:52:07.224666 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:07.224666 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:07.224666 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:07.224666 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:07.224666 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:07.224666 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:52:07.224666 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:07.224666 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:07.224666 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:07.224666 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:07.224666 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:07.224666 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:07.226934 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:07.226934 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:07.226934 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:52:07.226934 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:52:07.226934 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:52:07.226934 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:52:07.226934 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:52:07.226934 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:52:07.226934 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 19 14:52:07.226934 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:52:07.226934 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 293/589] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 14:52:07 2022] GET /identity/v3/users/neutron => generated 85 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:07.226934 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbf63fa-0518-42d8-8145-20878ae44128 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:07.256964 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-55eff153-bbb6-4b3d-8e37-61417980a60e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:52:07.256964 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:07.256964 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:07.256964 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:07.256964 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:07.256964 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:07.256964 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:07.256964 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:07.256964 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:07.256964 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:07.256964 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:07.256964 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:07.256964 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:52:07.256964 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:52:07.256964 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:52:07.259277 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:52:07.259277 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:07.259277 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:07.259277 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:07.259277 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:07.259277 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:07.259277 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:07.259277 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:07.259277 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:52:07.259277 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:07.259277 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:07.259277 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:07.259277 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:07.259277 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:07.259277 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:07.259277 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:07.259277 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:07.261468 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:52:07.261468 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:52:07.261468 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:52:07.261468 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:52:07.264444 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 297/590] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:52:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:07.264444 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55eff153-bbb6-4b3d-8e37-61417980a60e None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:07.269520 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:07.343930 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:07.353044 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbf63fa-0518-42d8-8145-20878ae44128 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:07.396285 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55eff153-bbb6-4b3d-8e37-61417980a60e None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:07.400138 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53c4cc2-bd6c-4095-8ff7-875868b203ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:07.401125 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53c4cc2-bd6c-4095-8ff7-875868b203ed None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:07.401570 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53c4cc2-bd6c-4095-8ff7-875868b203ed None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:07.402052 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53c4cc2-bd6c-4095-8ff7-875868b203ed None admin] PATH_INFO: `/v3/users/cinder` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:07.413351 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53c4cc2-bd6c-4095-8ff7-875868b203ed None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:07.415363 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53c4cc2-bd6c-4095-8ff7-875868b203ed None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:07.434356 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-d53c4cc2-bd6c-4095-8ff7-875868b203ed None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 14:52:07.434356 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:07.434356 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:07.434356 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:07.434356 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:07.434356 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:07.434356 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:07.434356 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:07.434356 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:07.434356 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:07.434356 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:07.434356 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:07.434356 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:52:07.434356 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:52:07.434356 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:52:07.434356 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:52:07.435663 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:07.435663 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:07.435663 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:52:07.435663 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:52:07.435663 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:52:07.435663 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:52:07.435663 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:07.435663 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:07.435663 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:07.435663 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:07.435663 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:07.435663 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:52:07.435663 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:07.435663 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:07.435663 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:07.435663 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:07.435663 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:07.435663 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:07.436763 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:07.436763 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:07.436763 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:52:07.436763 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:52:07.436763 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:52:07.436763 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:52:07.436763 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:52:07.436763 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:52:07.436763 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 14:52:07.436763 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:52:07.441915 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577a2e7f-c7ee-4064-8a60-66f1230875f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:07.442782 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577a2e7f-c7ee-4064-8a60-66f1230875f7 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:07.443314 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577a2e7f-c7ee-4064-8a60-66f1230875f7 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:07.443446 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 294/591] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:52:07 2022] GET /identity/v3/users/cinder => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:07.443857 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577a2e7f-c7ee-4064-8a60-66f1230875f7 None admin] PATH_INFO: `/v3/users` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:07.444492 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53c4cc2-bd6c-4095-8ff7-875868b203ed None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:07.445143 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577a2e7f-c7ee-4064-8a60-66f1230875f7 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:07.445963 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577a2e7f-c7ee-4064-8a60-66f1230875f7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:07.447692 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577a2e7f-c7ee-4064-8a60-66f1230875f7 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:07.471836 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 298/592] 10.222.0.57 () {56 vars in 1139 bytes} [Thu May 19 14:52:07 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:07.473135 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577a2e7f-c7ee-4064-8a60-66f1230875f7 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:07.574439 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:07.585031 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53c4cc2-bd6c-4095-8ff7-875868b203ed None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:07.597998 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577a2e7f-c7ee-4064-8a60-66f1230875f7 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:07.620734 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:07.623244 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769d669e-2136-481a-8b88-76ca7c090869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:07.623979 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769d669e-2136-481a-8b88-76ca7c090869 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:07.624275 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769d669e-2136-481a-8b88-76ca7c090869 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:07.624615 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769d669e-2136-481a-8b88-76ca7c090869 None admin] PATH_INFO: `/v3/domains` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:07.625369 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769d669e-2136-481a-8b88-76ca7c090869 None admin] RBAC: Adding query filter params (name=Default) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:07.625687 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769d669e-2136-481a-8b88-76ca7c090869 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:07.627148 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769d669e-2136-481a-8b88-76ca7c090869 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:07.644390 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 295/593] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:52:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:07.645599 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769d669e-2136-481a-8b88-76ca7c090869 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:07.648544 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7500a5f6-8ead-44a9-bbe7-4746bff84a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:07.650861 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7500a5f6-8ead-44a9-bbe7-4746bff84a27 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:07.651394 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7500a5f6-8ead-44a9-bbe7-4746bff84a27 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:07.651892 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7500a5f6-8ead-44a9-bbe7-4746bff84a27 None admin] PATH_INFO: `/v3/users` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:07.652960 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7500a5f6-8ead-44a9-bbe7-4746bff84a27 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:07.653500 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7500a5f6-8ead-44a9-bbe7-4746bff84a27 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:07.655755 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7500a5f6-8ead-44a9-bbe7-4746bff84a27 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:07.682193 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 299/594] 10.222.0.57 () {56 vars in 1137 bytes} [Thu May 19 14:52:07 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:07.686767 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7500a5f6-8ead-44a9-bbe7-4746bff84a27 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:07.768467 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769d669e-2136-481a-8b88-76ca7c090869 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:07.807931 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f77b3e-6203-4efe-8744-83b813c704ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:07.809043 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f77b3e-6203-4efe-8744-83b813c704ce None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:07.809513 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f77b3e-6203-4efe-8744-83b813c704ce None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:07.810007 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f77b3e-6203-4efe-8744-83b813c704ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:07.822451 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f77b3e-6203-4efe-8744-83b813c704ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:07.824254 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f77b3e-6203-4efe-8744-83b813c704ce None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:07.829903 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7500a5f6-8ead-44a9-bbe7-4746bff84a27 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:07.844543 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-c7f77b3e-6203-4efe-8744-83b813c704ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:52:07.844543 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:07.844543 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:07.844543 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:07.844543 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:07.844543 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:07.844543 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:07.844543 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:07.844543 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:07.844543 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:07.844543 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:07.844543 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:07.844543 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:52:07.844543 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:52:07.844543 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:52:07.845653 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:52:07.845653 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:07.845653 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:07.845653 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:07.845653 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:07.845653 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:07.845653 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:07.845653 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:07.845653 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:52:07.845653 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:07.845653 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:07.845653 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:07.845653 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:07.845653 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:07.845653 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:07.845653 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:07.845653 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:07.846713 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:52:07.846713 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:52:07.846713 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:52:07.846713 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:52:07.849053 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 296/595] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:52:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:07.850178 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f77b3e-6203-4efe-8744-83b813c704ce None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:07.862741 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:07.879475 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce588d5e-33c0-418c-ba7d-46e42101e5ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:07.880635 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce588d5e-33c0-418c-ba7d-46e42101e5ba None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:07.880926 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce588d5e-33c0-418c-ba7d-46e42101e5ba None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:07.881209 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce588d5e-33c0-418c-ba7d-46e42101e5ba None admin] PATH_INFO: `/v3/projects/service` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:07.892590 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce588d5e-33c0-418c-ba7d-46e42101e5ba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:07.894488 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce588d5e-33c0-418c-ba7d-46e42101e5ba None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:07.909230 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-ce588d5e-33c0-418c-ba7d-46e42101e5ba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:52:07.909230 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:07.909230 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:07.909230 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:07.909230 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:07.909230 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:07.909230 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:07.909230 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:07.909230 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:07.909230 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:07.909230 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:07.909230 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:07.909230 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:52:07.909230 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:52:07.909230 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:52:07.910284 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:52:07.910284 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:07.910284 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:07.910284 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:07.910284 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:07.910284 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:07.910284 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:07.910284 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:07.910284 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:52:07.910284 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:07.910284 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:07.910284 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:07.910284 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:07.910284 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:07.910284 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:07.910284 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:07.910284 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:07.911267 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:52:07.911267 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:52:07.911267 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:52:07.911267 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:52:07.911267 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:52:07.911267 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:52:07.911267 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:52:07.911267 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:52:07.918662 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 300/596] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:52:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:07.919839 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce588d5e-33c0-418c-ba7d-46e42101e5ba None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:07.955084 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:07.984845 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f77b3e-6203-4efe-8744-83b813c704ce None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:08.023737 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fde2555-fdb3-4fb6-afee-0ab144232f67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:08.024938 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fde2555-fdb3-4fb6-afee-0ab144232f67 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:08.025451 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fde2555-fdb3-4fb6-afee-0ab144232f67 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:08.025981 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fde2555-fdb3-4fb6-afee-0ab144232f67 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:08.038287 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fde2555-fdb3-4fb6-afee-0ab144232f67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:08.040056 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fde2555-fdb3-4fb6-afee-0ab144232f67 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:08.047684 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce588d5e-33c0-418c-ba7d-46e42101e5ba None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:08.062269 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-0fde2555-fdb3-4fb6-afee-0ab144232f67 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:52:08.062269 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:08.062269 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:08.062269 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:08.062269 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:08.062269 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:08.062269 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:08.062269 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:08.062269 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:08.062269 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:08.062269 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:08.062269 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:08.062269 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:52:08.062269 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:52:08.062269 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:52:08.063336 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:52:08.063336 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:08.063336 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:08.063336 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:08.063336 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:08.063336 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:08.063336 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:08.063336 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:08.063336 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:52:08.063336 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:08.063336 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:08.063336 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:08.063336 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:08.063336 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:08.063336 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:08.063336 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:08.063336 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:08.065003 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:52:08.065003 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:52:08.065003 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:52:08.065003 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:52:08.069212 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 297/597] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:52:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:08.071083 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fde2555-fdb3-4fb6-afee-0ab144232f67 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:08.093063 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab553737-4aec-4d8f-9c3a-4abe5fe49a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:08.094268 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab553737-4aec-4d8f-9c3a-4abe5fe49a4b None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:08.094806 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab553737-4aec-4d8f-9c3a-4abe5fe49a4b None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:08.095861 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab553737-4aec-4d8f-9c3a-4abe5fe49a4b None admin] PATH_INFO: `/v3/domains` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:08.096876 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab553737-4aec-4d8f-9c3a-4abe5fe49a4b None admin] RBAC: Adding query filter params (name=Default) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:08.097494 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab553737-4aec-4d8f-9c3a-4abe5fe49a4b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:08.099313 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab553737-4aec-4d8f-9c3a-4abe5fe49a4b None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:08.113686 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 301/598] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:52:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:08.115184 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab553737-4aec-4d8f-9c3a-4abe5fe49a4b None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:08.153723 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:08.202943 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fde2555-fdb3-4fb6-afee-0ab144232f67 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:08.249599 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:08.250731 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f7e08f-9fa0-483a-98a0-6ca3db7c0f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:08.251966 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f7e08f-9fa0-483a-98a0-6ca3db7c0f8a None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:08.252155 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f7e08f-9fa0-483a-98a0-6ca3db7c0f8a None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:08.252465 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f7e08f-9fa0-483a-98a0-6ca3db7c0f8a None admin] PATH_INFO: `/v3/projects` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:08.254159 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f7e08f-9fa0-483a-98a0-6ca3db7c0f8a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:08.254574 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f7e08f-9fa0-483a-98a0-6ca3db7c0f8a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:08.255949 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f7e08f-9fa0-483a-98a0-6ca3db7c0f8a None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:08.266283 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab553737-4aec-4d8f-9c3a-4abe5fe49a4b None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:08.277954 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 298/599] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:52:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:08.279637 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f7e08f-9fa0-483a-98a0-6ca3db7c0f8a None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:08.311102 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8ca358-6dbd-4099-8b6f-eaa469e91a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:08.311789 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8ca358-6dbd-4099-8b6f-eaa469e91a88 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:08.312246 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8ca358-6dbd-4099-8b6f-eaa469e91a88 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:08.312541 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8ca358-6dbd-4099-8b6f-eaa469e91a88 None admin] PATH_INFO: `/v3/domains` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:08.313793 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8ca358-6dbd-4099-8b6f-eaa469e91a88 None admin] RBAC: Adding query filter params (name=Default) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:08.314275 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8ca358-6dbd-4099-8b6f-eaa469e91a88 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:08.315923 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8ca358-6dbd-4099-8b6f-eaa469e91a88 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:08.329401 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 302/600] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:52:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:08.334004 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8ca358-6dbd-4099-8b6f-eaa469e91a88 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:08.399754 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f7e08f-9fa0-483a-98a0-6ca3db7c0f8a None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:08.421038 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:08.436281 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed20891-af2c-44d4-acb5-777dda58ef74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:08.437090 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed20891-af2c-44d4-acb5-777dda58ef74 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:08.437384 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed20891-af2c-44d4-acb5-777dda58ef74 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:08.437671 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed20891-af2c-44d4-acb5-777dda58ef74 None admin] PATH_INFO: `/v3/projects/service` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:08.442390 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8ca358-6dbd-4099-8b6f-eaa469e91a88 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:08.445830 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed20891-af2c-44d4-acb5-777dda58ef74 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:08.447325 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed20891-af2c-44d4-acb5-777dda58ef74 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:08.461905 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-8ed20891-af2c-44d4-acb5-777dda58ef74 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:52:08.461905 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:08.461905 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:08.461905 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:08.461905 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:08.461905 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:08.461905 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:08.461905 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:08.461905 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:08.461905 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:08.461905 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:08.461905 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:08.461905 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:52:08.461905 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:52:08.461905 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:52:08.468581 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:52:08.468581 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:08.468581 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:08.468581 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:08.468581 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:08.468581 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:08.468581 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:08.468581 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:08.468581 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:52:08.468581 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:08.468581 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:08.468581 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:08.468581 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:08.468581 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:08.468581 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:08.468581 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:08.468581 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:08.469667 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:52:08.469667 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:52:08.469667 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:52:08.469667 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:52:08.469667 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:52:08.469667 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:52:08.469667 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:52:08.469667 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:52:08.469667 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 299/601] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:52:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:08.469667 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f148f5-5bbd-4cf8-9c25-5a8168128336 None None] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:08.469667 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f148f5-5bbd-4cf8-9c25-5a8168128336 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:08.470420 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f148f5-5bbd-4cf8-9c25-5a8168128336 None None] PATH_INFO: `/` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:08.470420 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 300/602] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:52:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:52:08.470420 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f148f5-5bbd-4cf8-9c25-5a8168128336 None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:08.488161 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72dc9d9-7f26-41d1-97b9-e2d85e419dea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:08.489194 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72dc9d9-7f26-41d1-97b9-e2d85e419dea None admin] REQUEST_METHOD: `PUT` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:08.489646 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72dc9d9-7f26-41d1-97b9-e2d85e419dea None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:08.489911 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72dc9d9-7f26-41d1-97b9-e2d85e419dea None admin] PATH_INFO: `/v3/projects/fc99d742ae004954b6d841293b5fa79c/users/5d2aa8c8870a49d98f23f26e036695ef/roles/08f16502894349708527e30032247468` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:08.506116 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72dc9d9-7f26-41d1-97b9-e2d85e419dea None admin] RBAC: Authorizing `identity:create_grant(project_id=fc99d742ae004954b6d841293b5fa79c, user_id=5d2aa8c8870a49d98f23f26e036695ef, role_id=08f16502894349708527e30032247468)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:08.507717 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72dc9d9-7f26-41d1-97b9-e2d85e419dea None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:08.533951 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 303/603] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 14:52:08 2022] PUT /identity/v3/projects/fc99d742ae004954b6d841293b5fa79c/users/5d2aa8c8870a49d98f23f26e036695ef/roles/08f16502894349708527e30032247468 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:52:08.536011 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f56f361-964f-4299-9071-d46f39a9beb8 None None] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:08.536318 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f56f361-964f-4299-9071-d46f39a9beb8 None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:08.536655 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f56f361-964f-4299-9071-d46f39a9beb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:08.544358 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:08.567119 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.auth.core [None req-8f56f361-964f-4299-9071-d46f39a9beb8 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:52:08.568382 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.fernet_utils [None req-8f56f361-964f-4299-9071-d46f39a9beb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:52:08.606101 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f148f5-5bbd-4cf8-9c25-5a8168128336 None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:08.654351 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 304/604] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:52:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:52:08.655786 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f56f361-964f-4299-9071-d46f39a9beb8 None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:08.658542 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b15236-1f63-4198-8494-434c57170ce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:08.665540 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b15236-1f63-4198-8494-434c57170ce2 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:08.665540 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b15236-1f63-4198-8494-434c57170ce2 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:08.665540 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b15236-1f63-4198-8494-434c57170ce2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:08.676314 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b15236-1f63-4198-8494-434c57170ce2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:08.683286 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b15236-1f63-4198-8494-434c57170ce2 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:08.701793 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-c8b15236-1f63-4198-8494-434c57170ce2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:52:08.701793 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:08.701793 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:08.701793 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:08.701793 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:08.701793 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:08.701793 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:08.701793 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:08.701793 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:08.701793 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:08.701793 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:08.701793 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:08.701793 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:52:08.701793 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:52:08.701793 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:52:08.702880 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:52:08.702880 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:08.702880 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:08.702880 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:08.702880 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:08.702880 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:08.702880 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:08.702880 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:08.702880 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:52:08.702880 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:08.702880 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:08.702880 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:08.702880 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:08.702880 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:08.702880 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:08.702880 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:08.702880 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:08.703885 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:52:08.703885 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:52:08.703885 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:52:08.703885 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:52:08.703885 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:52:08.703885 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:52:08.703885 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:52:08.703885 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:52:08.707552 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 301/605] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:52:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:08.708970 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b15236-1f63-4198-8494-434c57170ce2 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:08.759910 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:08.799565 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f56f361-964f-4299-9071-d46f39a9beb8 None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:08.842639 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b15236-1f63-4198-8494-434c57170ce2 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:08.844167 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:08.850842 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38dbb0a-59e3-4a9d-9bdf-9b6018d70d42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:08.851523 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38dbb0a-59e3-4a9d-9bdf-9b6018d70d42 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:08.851814 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38dbb0a-59e3-4a9d-9bdf-9b6018d70d42 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:08.852097 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38dbb0a-59e3-4a9d-9bdf-9b6018d70d42 None admin] PATH_INFO: `/v3/projects` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:08.852901 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38dbb0a-59e3-4a9d-9bdf-9b6018d70d42 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:08.853220 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38dbb0a-59e3-4a9d-9bdf-9b6018d70d42 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:08.854720 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38dbb0a-59e3-4a9d-9bdf-9b6018d70d42 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:08.867300 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 305/606] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:52:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:08.868142 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38dbb0a-59e3-4a9d-9bdf-9b6018d70d42 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:08.887907 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5451dd8d-798a-4e24-a5b7-6524f64edf3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:08.889149 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5451dd8d-798a-4e24-a5b7-6524f64edf3a None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:08.889776 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5451dd8d-798a-4e24-a5b7-6524f64edf3a None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:08.890306 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5451dd8d-798a-4e24-a5b7-6524f64edf3a None admin] PATH_INFO: `/v3/roles/service` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:08.903429 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5451dd8d-798a-4e24-a5b7-6524f64edf3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:52:08.912998 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-5451dd8d-798a-4e24-a5b7-6524f64edf3a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:52:08.912998 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:08.912998 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:08.912998 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:08.912998 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:08.912998 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:08.912998 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:08.912998 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:08.912998 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:08.912998 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:08.912998 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:08.912998 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:08.912998 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:52:08.912998 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:52:08.912998 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:52:08.912998 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:52:08.914090 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:52:08.914090 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:52:08.914090 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:52:08.914090 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:52:08.914090 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:08.914090 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:08.914090 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:08.914090 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:08.914090 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:08.914090 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:08.914090 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:08.914090 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:52:08.914090 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:08.914090 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:08.914090 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:08.914090 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:08.915051 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:08.915051 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:08.915051 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:08.915051 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:08.915051 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:52:08.915051 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:52:08.915051 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:52:08.915051 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:52:08.915051 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:52:08.915051 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:52:08.915051 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:52:08.915051 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:52:08.922565 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 302/607] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 14:52:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:08.924197 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5451dd8d-798a-4e24-a5b7-6524f64edf3a None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:08.980465 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38dbb0a-59e3-4a9d-9bdf-9b6018d70d42 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:09.017332 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0123240c-368e-4301-9670-2b8b8ef1fe1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:09.018116 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0123240c-368e-4301-9670-2b8b8ef1fe1f None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:09.018524 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0123240c-368e-4301-9670-2b8b8ef1fe1f None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:09.018814 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0123240c-368e-4301-9670-2b8b8ef1fe1f None admin] PATH_INFO: `/v3/projects` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:09.019619 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0123240c-368e-4301-9670-2b8b8ef1fe1f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:09.019941 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0123240c-368e-4301-9670-2b8b8ef1fe1f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:09.022194 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0123240c-368e-4301-9670-2b8b8ef1fe1f None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:09.040084 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:09.041982 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 306/608] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:52:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:09.043101 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0123240c-368e-4301-9670-2b8b8ef1fe1f None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:09.050786 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5451dd8d-798a-4e24-a5b7-6524f64edf3a None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:09.093170 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1cbeb1-703e-4c0a-9cf6-ffd36782765f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:09.094910 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1cbeb1-703e-4c0a-9cf6-ffd36782765f None admin] REQUEST_METHOD: `PUT` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:09.096092 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1cbeb1-703e-4c0a-9cf6-ffd36782765f None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:09.096640 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1cbeb1-703e-4c0a-9cf6-ffd36782765f None admin] PATH_INFO: `/v3/projects/fc99d742ae004954b6d841293b5fa79c/users/8de553016ea641cdbf9c4a8e08c96a67/roles/08f16502894349708527e30032247468` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:09.116719 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1cbeb1-703e-4c0a-9cf6-ffd36782765f None admin] RBAC: Authorizing `identity:create_grant(project_id=fc99d742ae004954b6d841293b5fa79c, user_id=8de553016ea641cdbf9c4a8e08c96a67, role_id=08f16502894349708527e30032247468)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:09.118831 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1cbeb1-703e-4c0a-9cf6-ffd36782765f None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:09.121219 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:09.140469 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 303/609] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 14:52:08 2022] PUT /identity/v3/projects/fc99d742ae004954b6d841293b5fa79c/users/8de553016ea641cdbf9c4a8e08c96a67/roles/08f16502894349708527e30032247468 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:52:09.141919 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1cbeb1-703e-4c0a-9cf6-ffd36782765f None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:09.239931 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0123240c-368e-4301-9670-2b8b8ef1fe1f None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:09.297611 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1cbeb1-703e-4c0a-9cf6-ffd36782765f None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:09.314577 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c332c6-5c11-40c1-8161-0896ab0ad3b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:09.315352 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c332c6-5c11-40c1-8161-0896ab0ad3b8 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:09.315992 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c332c6-5c11-40c1-8161-0896ab0ad3b8 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:09.316351 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c332c6-5c11-40c1-8161-0896ab0ad3b8 None admin] PATH_INFO: `/v3/roles` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:09.317787 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c332c6-5c11-40c1-8161-0896ab0ad3b8 None admin] RBAC: Adding query filter params (name=service) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:09.318208 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c332c6-5c11-40c1-8161-0896ab0ad3b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:09.320023 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c332c6-5c11-40c1-8161-0896ab0ad3b8 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:09.337883 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 307/610] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 19 14:52:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:09.337883 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4398c426-ddd1-4570-9a9d-daf41148d04c None None] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:09.337883 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4398c426-ddd1-4570-9a9d-daf41148d04c None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:09.337883 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4398c426-ddd1-4570-9a9d-daf41148d04c None None] PATH_INFO: `/` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:09.341716 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 308/611] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:52:09 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:52:09.342662 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4398c426-ddd1-4570-9a9d-daf41148d04c None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:09.350243 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff71859-41d5-47f7-b862-a8f601ae5675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:09.351617 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff71859-41d5-47f7-b862-a8f601ae5675 None admin] REQUEST_METHOD: `PUT` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:09.352116 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff71859-41d5-47f7-b862-a8f601ae5675 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:09.352603 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff71859-41d5-47f7-b862-a8f601ae5675 None admin] PATH_INFO: `/v3/projects/fc99d742ae004954b6d841293b5fa79c/users/7f75f8f59dcd42c480aac3eca18fc8db/roles/08f16502894349708527e30032247468` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:09.375316 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff71859-41d5-47f7-b862-a8f601ae5675 None admin] RBAC: Authorizing `identity:create_grant(project_id=fc99d742ae004954b6d841293b5fa79c, user_id=7f75f8f59dcd42c480aac3eca18fc8db, role_id=08f16502894349708527e30032247468)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:09.377750 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff71859-41d5-47f7-b862-a8f601ae5675 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:09.392751 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:09.401009 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 304/612] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 14:52:09 2022] PUT /identity/v3/projects/fc99d742ae004954b6d841293b5fa79c/users/7f75f8f59dcd42c480aac3eca18fc8db/roles/08f16502894349708527e30032247468 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:52:09.402413 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab27df6b-952b-4d83-89ca-0a1972883da3 None None] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:09.402774 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab27df6b-952b-4d83-89ca-0a1972883da3 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:09.403390 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab27df6b-952b-4d83-89ca-0a1972883da3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:09.427095 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.auth.core [None req-ab27df6b-952b-4d83-89ca-0a1972883da3 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:52:09.428983 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-ab27df6b-952b-4d83-89ca-0a1972883da3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:52:09.490086 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4398c426-ddd1-4570-9a9d-daf41148d04c None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:09.516619 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:09.531851 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 305/613] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:52:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:52:09.543769 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecb0319-7580-446b-ab47-a9c618890aeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:09.544946 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecb0319-7580-446b-ab47-a9c618890aeb None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:09.545447 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecb0319-7580-446b-ab47-a9c618890aeb None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:09.545969 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecb0319-7580-446b-ab47-a9c618890aeb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:09.557109 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab27df6b-952b-4d83-89ca-0a1972883da3 None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:09.559018 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecb0319-7580-446b-ab47-a9c618890aeb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:09.560767 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecb0319-7580-446b-ab47-a9c618890aeb None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:09.580638 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-0ecb0319-7580-446b-ab47-a9c618890aeb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:52:09.580638 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:09.580638 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:09.580638 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:09.580638 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:09.580638 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:09.580638 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:09.580638 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:09.580638 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:09.580638 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:09.580638 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:09.580638 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:09.580638 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:52:09.580638 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:52:09.580638 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:52:09.581893 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:52:09.581893 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:09.581893 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:09.581893 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:09.581893 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:09.581893 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:09.581893 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:09.581893 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:09.581893 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:52:09.581893 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:09.581893 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:09.581893 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:09.581893 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:09.581893 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:09.581893 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:09.581893 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:09.581893 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:09.583072 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:52:09.583072 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:52:09.583072 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:52:09.583072 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:52:09.584275 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 309/614] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:52:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:09.590284 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecb0319-7580-446b-ab47-a9c618890aeb None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:09.700798 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab27df6b-952b-4d83-89ca-0a1972883da3 None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:09.744876 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:09.748154 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecb0319-7580-446b-ab47-a9c618890aeb None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:09.752377 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99785927-0152-4749-8776-9057c31c67b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:09.752695 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99785927-0152-4749-8776-9057c31c67b8 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:09.753347 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99785927-0152-4749-8776-9057c31c67b8 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:09.754009 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99785927-0152-4749-8776-9057c31c67b8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:09.766513 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99785927-0152-4749-8776-9057c31c67b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:52:09.775657 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-99785927-0152-4749-8776-9057c31c67b8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:52:09.775657 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:09.775657 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:09.775657 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:09.775657 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:09.775657 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:09.775657 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:09.775657 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:09.775657 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:09.775657 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:09.775657 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:09.775657 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:09.775657 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:52:09.775657 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:52:09.775657 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:52:09.775657 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:52:09.776886 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:52:09.776886 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:52:09.776886 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:52:09.776886 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:52:09.776886 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:09.776886 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:09.776886 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:09.776886 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:09.776886 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:09.776886 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:09.776886 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:09.776886 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:52:09.776886 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:09.776886 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:09.776886 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:09.776886 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:09.778022 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:09.778022 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:09.778022 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:09.778022 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:09.778022 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:52:09.778022 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:52:09.778022 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:52:09.778022 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:52:09.778022 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:52:09.778022 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:52:09.778022 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:52:09.778022 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:52:09.780076 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 306/615] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 14:52:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:09.792611 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99785927-0152-4749-8776-9057c31c67b8 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:09.797590 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b5da92-956e-4904-8e24-602026dc8566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:09.799222 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b5da92-956e-4904-8e24-602026dc8566 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:09.799748 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b5da92-956e-4904-8e24-602026dc8566 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:09.800627 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b5da92-956e-4904-8e24-602026dc8566 None admin] PATH_INFO: `/v3/domains` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:09.801581 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b5da92-956e-4904-8e24-602026dc8566 None admin] RBAC: Adding query filter params (name=Default) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:09.802124 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b5da92-956e-4904-8e24-602026dc8566 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:09.803740 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b5da92-956e-4904-8e24-602026dc8566 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:09.819754 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 310/616] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:52:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:52:09.826043 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b5da92-956e-4904-8e24-602026dc8566 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:09.881938 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:09.986038 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b5da92-956e-4904-8e24-602026dc8566 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:09.986290 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99785927-0152-4749-8776-9057c31c67b8 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:10.014862 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9926cf5a-73fe-4e55-8feb-5d3b7b70b1d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:10.015806 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9926cf5a-73fe-4e55-8feb-5d3b7b70b1d4 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:10.016202 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9926cf5a-73fe-4e55-8feb-5d3b7b70b1d4 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:10.016585 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9926cf5a-73fe-4e55-8feb-5d3b7b70b1d4 None admin] PATH_INFO: `/v3/roles` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:10.017758 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9926cf5a-73fe-4e55-8feb-5d3b7b70b1d4 None admin] RBAC: Adding query filter params (name=service) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:10.018443 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9926cf5a-73fe-4e55-8feb-5d3b7b70b1d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:10.019980 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9926cf5a-73fe-4e55-8feb-5d3b7b70b1d4 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:10.033470 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 307/617] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 19 14:52:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:52:10.034752 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9926cf5a-73fe-4e55-8feb-5d3b7b70b1d4 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:10.051764 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6164021c-f09e-4be1-b85a-7549f4d138e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:10.052858 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6164021c-f09e-4be1-b85a-7549f4d138e6 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:10.053307 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6164021c-f09e-4be1-b85a-7549f4d138e6 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:10.053738 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6164021c-f09e-4be1-b85a-7549f4d138e6 None admin] PATH_INFO: `/v3/users/swift` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:10.072730 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6164021c-f09e-4be1-b85a-7549f4d138e6 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:10.074630 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6164021c-f09e-4be1-b85a-7549f4d138e6 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:10.107517 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-6164021c-f09e-4be1-b85a-7549f4d138e6 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 14:52:10.107517 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:10.107517 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:10.107517 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:10.107517 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:10.107517 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:10.107517 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:10.107517 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:10.107517 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:10.107517 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:10.107517 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:10.107517 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:10.107517 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:52:10.107517 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:52:10.107517 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:52:10.107517 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:52:10.108822 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:10.108822 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:10.108822 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:52:10.108822 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:52:10.108822 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:52:10.108822 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:52:10.108822 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:10.108822 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:10.108822 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:10.108822 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:10.108822 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:10.108822 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:52:10.108822 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:10.108822 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:10.108822 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:10.108822 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:10.108822 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:10.108822 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:10.110079 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:10.110079 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:10.110079 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:52:10.110079 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:52:10.110079 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:52:10.110079 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:52:10.110079 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:52:10.110079 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:52:10.110079 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 14:52:10.110079 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:52:10.115935 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 311/618] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:52:09 2022] GET /identity/v3/users/swift => generated 83 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:10.121863 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6164021c-f09e-4be1-b85a-7549f4d138e6 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:10.138190 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:10.232357 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9926cf5a-73fe-4e55-8feb-5d3b7b70b1d4 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:10.294225 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:10.301796 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d17e01f-af93-40b9-8bfa-a3898e7413ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:10.302643 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d17e01f-af93-40b9-8bfa-a3898e7413ef None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:10.302931 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d17e01f-af93-40b9-8bfa-a3898e7413ef None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:10.303214 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d17e01f-af93-40b9-8bfa-a3898e7413ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:10.321306 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6164021c-f09e-4be1-b85a-7549f4d138e6 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:10.325339 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d17e01f-af93-40b9-8bfa-a3898e7413ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:10.326760 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d17e01f-af93-40b9-8bfa-a3898e7413ef None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:10.355895 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-7d17e01f-af93-40b9-8bfa-a3898e7413ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:52:10.355895 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:10.355895 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:10.355895 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:10.355895 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:10.355895 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:10.355895 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:10.355895 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:10.355895 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:10.355895 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:10.355895 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:10.355895 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:10.355895 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:52:10.355895 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:52:10.355895 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:52:10.356946 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:52:10.356946 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:10.356946 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:10.356946 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:10.356946 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:10.356946 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:10.356946 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:10.356946 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:10.356946 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:52:10.356946 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:10.356946 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:10.356946 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:10.356946 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:10.356946 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:10.356946 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:10.356946 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:10.356946 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:10.357981 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:52:10.357981 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:52:10.357981 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:52:10.357981 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:52:10.362427 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 308/619] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:52:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:10.371027 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d17e01f-af93-40b9-8bfa-a3898e7413ef None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:10.403131 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155b0023-0da0-477b-b0e0-de7f39a79f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:10.403826 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155b0023-0da0-477b-b0e0-de7f39a79f6f None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:10.404120 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155b0023-0da0-477b-b0e0-de7f39a79f6f None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:10.404400 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155b0023-0da0-477b-b0e0-de7f39a79f6f None admin] PATH_INFO: `/v3/users` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:10.405250 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155b0023-0da0-477b-b0e0-de7f39a79f6f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:10.405570 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155b0023-0da0-477b-b0e0-de7f39a79f6f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:10.407048 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155b0023-0da0-477b-b0e0-de7f39a79f6f None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:10.445595 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 312/620] 10.222.0.57 () {56 vars in 1135 bytes} [Thu May 19 14:52:10 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:10.460748 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155b0023-0da0-477b-b0e0-de7f39a79f6f None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:10.486740 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d17e01f-af93-40b9-8bfa-a3898e7413ef None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:10.490267 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:10.522449 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481bec2e-c005-469e-9818-28474c22305f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:10.523127 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481bec2e-c005-469e-9818-28474c22305f None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:10.523417 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481bec2e-c005-469e-9818-28474c22305f None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:10.523694 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481bec2e-c005-469e-9818-28474c22305f None admin] PATH_INFO: `/v3/domains` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:10.524427 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481bec2e-c005-469e-9818-28474c22305f None admin] RBAC: Adding query filter params (name=Default) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:10.524739 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481bec2e-c005-469e-9818-28474c22305f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:10.526157 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481bec2e-c005-469e-9818-28474c22305f None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:10.544000 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 309/621] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:52:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:10.545392 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481bec2e-c005-469e-9818-28474c22305f None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:10.652361 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481bec2e-c005-469e-9818-28474c22305f None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:10.693295 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a27937-a5d4-4996-b7ab-059441d4d0f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:10.694144 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a27937-a5d4-4996-b7ab-059441d4d0f8 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:10.694433 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a27937-a5d4-4996-b7ab-059441d4d0f8 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:10.694871 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a27937-a5d4-4996-b7ab-059441d4d0f8 None admin] PATH_INFO: `/v3/users/glance` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:10.702793 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a27937-a5d4-4996-b7ab-059441d4d0f8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:10.705117 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a27937-a5d4-4996-b7ab-059441d4d0f8 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:10.721492 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-a3a27937-a5d4-4996-b7ab-059441d4d0f8 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 19 14:52:10.721492 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:10.721492 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:10.721492 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:10.721492 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:10.721492 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:10.721492 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:10.721492 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:10.721492 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:10.721492 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:10.721492 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:10.721492 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:10.721492 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:52:10.721492 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:52:10.721492 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:52:10.721492 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:52:10.722635 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:10.722635 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:10.722635 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:52:10.722635 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:52:10.722635 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:52:10.722635 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:52:10.722635 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:10.722635 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:10.722635 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:10.722635 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:10.722635 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:10.722635 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:52:10.722635 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:10.722635 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:10.722635 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:10.722635 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:10.722635 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:10.722635 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:10.723687 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:10.723687 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:10.723687 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:52:10.723687 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:52:10.723687 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:52:10.723687 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:52:10.723687 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:52:10.723687 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:52:10.723687 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 19 14:52:10.723687 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:52:10.728161 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 310/622] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:52:10 2022] GET /identity/v3/users/glance => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:10.729418 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a27937-a5d4-4996-b7ab-059441d4d0f8 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:10.782294 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:10.841342 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a27937-a5d4-4996-b7ab-059441d4d0f8 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:10.851782 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155b0023-0da0-477b-b0e0-de7f39a79f6f None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:10.880286 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303a1db4-44ce-4621-aa14-cdf47c7be9c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:10.880972 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303a1db4-44ce-4621-aa14-cdf47c7be9c1 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:10.881258 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303a1db4-44ce-4621-aa14-cdf47c7be9c1 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:10.881538 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303a1db4-44ce-4621-aa14-cdf47c7be9c1 None admin] PATH_INFO: `/v3/users` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:10.882434 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303a1db4-44ce-4621-aa14-cdf47c7be9c1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:10.882749 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303a1db4-44ce-4621-aa14-cdf47c7be9c1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:10.884670 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303a1db4-44ce-4621-aa14-cdf47c7be9c1 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:10.908661 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 311/623] 10.222.0.57 () {56 vars in 1137 bytes} [Thu May 19 14:52:10 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:52:10.910042 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303a1db4-44ce-4621-aa14-cdf47c7be9c1 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:11.010393 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:11.014568 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c285001a-60cb-439d-b786-ca198bac53cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:11.015413 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c285001a-60cb-439d-b786-ca198bac53cb None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:11.015702 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c285001a-60cb-439d-b786-ca198bac53cb None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:11.015986 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c285001a-60cb-439d-b786-ca198bac53cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:11.027832 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303a1db4-44ce-4621-aa14-cdf47c7be9c1 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:11.050779 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c285001a-60cb-439d-b786-ca198bac53cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:11.052283 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c285001a-60cb-439d-b786-ca198bac53cb None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:11.065353 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1e68fd-d752-4d57-8ed7-569f3a3931e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:11.066244 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1e68fd-d752-4d57-8ed7-569f3a3931e5 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:11.066537 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1e68fd-d752-4d57-8ed7-569f3a3931e5 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:11.067276 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:11.068173 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1e68fd-d752-4d57-8ed7-569f3a3931e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:11.097126 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-c285001a-60cb-439d-b786-ca198bac53cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:52:11.097126 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:11.097126 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:11.097126 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:11.097126 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:11.097126 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:11.097126 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:11.097126 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:11.097126 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:11.097126 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:11.097126 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:11.097126 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:11.097126 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:52:11.097126 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:52:11.097126 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:52:11.099552 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:52:11.099552 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:11.099552 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:11.099552 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:11.099552 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:11.099552 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:11.099552 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:11.099552 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:11.099552 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:52:11.099552 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:11.099552 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:11.099552 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:11.099552 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:11.099552 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:11.099552 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:11.099552 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:11.099552 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:11.101982 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:52:11.101982 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:52:11.101982 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:52:11.101982 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:52:11.101982 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 313/624] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:52:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 638 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:52:11.105129 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c285001a-60cb-439d-b786-ca198bac53cb None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:11.208083 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1e68fd-d752-4d57-8ed7-569f3a3931e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:11.209483 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1e68fd-d752-4d57-8ed7-569f3a3931e5 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:11.265224 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c285001a-60cb-439d-b786-ca198bac53cb None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:11.305117 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99209d3c-524a-427d-bc10-0ccba9c13f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:11.305796 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99209d3c-524a-427d-bc10-0ccba9c13f8f None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:11.306116 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99209d3c-524a-427d-bc10-0ccba9c13f8f None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:11.306402 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99209d3c-524a-427d-bc10-0ccba9c13f8f None admin] PATH_INFO: `/v3/domains` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:11.307137 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99209d3c-524a-427d-bc10-0ccba9c13f8f None admin] RBAC: Adding query filter params (name=Default) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:11.307452 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99209d3c-524a-427d-bc10-0ccba9c13f8f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:11.308854 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99209d3c-524a-427d-bc10-0ccba9c13f8f None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:11.319564 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:11.326316 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 314/625] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:52:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:52:11.327847 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10c2dfb-3e0d-42b7-b344-2930ec51cc6d None None] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:11.328058 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10c2dfb-3e0d-42b7-b344-2930ec51cc6d None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:11.328346 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10c2dfb-3e0d-42b7-b344-2930ec51cc6d None None] PATH_INFO: `/` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:11.335197 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 315/626] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:52:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:52:11.336080 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10c2dfb-3e0d-42b7-b344-2930ec51cc6d None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:11.344535 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 312/627] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:52:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 14:52:11.346586 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854c5549-c707-4452-846a-5425182dab55 None None] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:11.347106 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854c5549-c707-4452-846a-5425182dab55 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:11.347622 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854c5549-c707-4452-846a-5425182dab55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:11.365880 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.auth.core [None req-854c5549-c707-4452-846a-5425182dab55 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:52:11.367531 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-854c5549-c707-4452-846a-5425182dab55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:52:11.402924 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:11.449005 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 313/628] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:52:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:52:11.450284 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854c5549-c707-4452-846a-5425182dab55 None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:11.462587 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10c2dfb-3e0d-42b7-b344-2930ec51cc6d None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:11.518444 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b9bc7a-ae07-4edd-ae05-7638d47d2d30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:11.518873 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b9bc7a-ae07-4edd-ae05-7638d47d2d30 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:11.519164 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b9bc7a-ae07-4edd-ae05-7638d47d2d30 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:11.519448 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b9bc7a-ae07-4edd-ae05-7638d47d2d30 None admin] PATH_INFO: `/v3/projects/service` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:11.527636 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b9bc7a-ae07-4edd-ae05-7638d47d2d30 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:11.529137 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b9bc7a-ae07-4edd-ae05-7638d47d2d30 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:11.546660 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-f3b9bc7a-ae07-4edd-ae05-7638d47d2d30 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:52:11.546660 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:11.546660 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:11.546660 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:11.546660 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:11.546660 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:11.546660 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:11.546660 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:11.546660 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:11.546660 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:11.546660 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:11.546660 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:11.546660 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:52:11.546660 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:52:11.546660 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:52:11.547728 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:52:11.547728 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:11.547728 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:11.547728 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:11.547728 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:11.547728 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:11.547728 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:11.547728 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:11.547728 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:52:11.547728 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:11.547728 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:11.547728 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:11.547728 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:11.547728 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:11.547728 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:11.547728 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:11.547728 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:11.548874 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:52:11.548874 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:52:11.548874 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:52:11.548874 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:52:11.548874 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:52:11.548874 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:52:11.548874 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:52:11.548874 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:52:11.553140 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 316/629] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:52:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:11.554138 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b9bc7a-ae07-4edd-ae05-7638d47d2d30 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:11.578386 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854c5549-c707-4452-846a-5425182dab55 None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:11.594603 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:11.620975 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b18583-e450-4e85-8173-ae7f9dd4c0e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:11.622344 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b18583-e450-4e85-8173-ae7f9dd4c0e2 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:11.622901 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b18583-e450-4e85-8173-ae7f9dd4c0e2 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:11.623410 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b18583-e450-4e85-8173-ae7f9dd4c0e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:11.636726 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b18583-e450-4e85-8173-ae7f9dd4c0e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:11.638593 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b18583-e450-4e85-8173-ae7f9dd4c0e2 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:11.660250 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-91b18583-e450-4e85-8173-ae7f9dd4c0e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:52:11.660250 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:11.660250 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:11.660250 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:11.660250 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:11.660250 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:11.660250 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:11.660250 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:11.660250 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:11.660250 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:11.660250 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:11.660250 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:11.660250 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:52:11.660250 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:52:11.660250 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:52:11.661287 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:52:11.661287 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:11.661287 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:11.661287 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:11.661287 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:11.661287 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:11.661287 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:11.661287 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:11.661287 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:52:11.661287 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:11.661287 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:11.661287 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:11.661287 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:11.661287 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:11.661287 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:11.661287 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:11.661287 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:11.662359 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:52:11.662359 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:52:11.662359 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:52:11.662359 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:52:11.664200 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b9bc7a-ae07-4edd-ae05-7638d47d2d30 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:11.670488 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 314/630] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:52:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:11.671169 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b18583-e450-4e85-8173-ae7f9dd4c0e2 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:11.694426 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:11.721696 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528864c4-38f2-4aaa-8d3c-e80a73777537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:11.722928 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528864c4-38f2-4aaa-8d3c-e80a73777537 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:11.723221 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528864c4-38f2-4aaa-8d3c-e80a73777537 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:11.723573 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528864c4-38f2-4aaa-8d3c-e80a73777537 None admin] PATH_INFO: `/v3/roles/service` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:11.738964 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528864c4-38f2-4aaa-8d3c-e80a73777537 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:52:11.750581 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-528864c4-38f2-4aaa-8d3c-e80a73777537 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:52:11.750581 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:11.750581 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:11.750581 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:11.750581 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:11.750581 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:11.750581 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:11.750581 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:11.750581 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:11.750581 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:11.750581 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:11.750581 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:11.750581 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:52:11.750581 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:52:11.750581 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:52:11.750581 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:52:11.752326 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:52:11.752326 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:52:11.752326 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:52:11.752326 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:52:11.752326 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:11.752326 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:11.752326 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:11.752326 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:11.752326 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:11.752326 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:11.752326 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:11.752326 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:52:11.752326 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:11.752326 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:11.752326 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:11.752326 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:11.753253 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:11.753253 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:11.753253 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:11.753253 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:11.753253 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:52:11.753253 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:52:11.753253 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:52:11.753253 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:52:11.753253 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:52:11.753253 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:52:11.753253 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:52:11.753253 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:52:11.754029 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 317/631] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 14:52:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:11.755293 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528864c4-38f2-4aaa-8d3c-e80a73777537 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:11.791984 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b18583-e450-4e85-8173-ae7f9dd4c0e2 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:11.830310 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451932a0-d622-4773-9987-9666be9b7993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:11.830984 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451932a0-d622-4773-9987-9666be9b7993 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:11.831279 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451932a0-d622-4773-9987-9666be9b7993 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:11.831944 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451932a0-d622-4773-9987-9666be9b7993 None admin] PATH_INFO: `/v3/projects` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:11.832370 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451932a0-d622-4773-9987-9666be9b7993 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:11.832692 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451932a0-d622-4773-9987-9666be9b7993 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:11.834283 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451932a0-d622-4773-9987-9666be9b7993 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:11.846517 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 315/632] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:52:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:11.847867 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a251beb1-e933-4fa8-9927-08ef37a1b616 None None] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:11.848274 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a251beb1-e933-4fa8-9927-08ef37a1b616 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:11.848513 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a251beb1-e933-4fa8-9927-08ef37a1b616 None None] PATH_INFO: `/` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:11.853869 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 316/633] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:52:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:52:11.853869 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a251beb1-e933-4fa8-9927-08ef37a1b616 None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:11.878322 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528864c4-38f2-4aaa-8d3c-e80a73777537 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:11.882633 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:11.916403 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5067f6-be28-450d-ae70-b6b9268c9480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:11.917171 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5067f6-be28-450d-ae70-b6b9268c9480 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:11.917367 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5067f6-be28-450d-ae70-b6b9268c9480 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:11.917652 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5067f6-be28-450d-ae70-b6b9268c9480 None admin] PATH_INFO: `/v3/domains` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:11.918427 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5067f6-be28-450d-ae70-b6b9268c9480 None admin] RBAC: Adding query filter params (name=Default) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:11.918742 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5067f6-be28-450d-ae70-b6b9268c9480 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:11.920125 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5067f6-be28-450d-ae70-b6b9268c9480 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:11.936737 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 318/634] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:52:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:11.936737 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5067f6-be28-450d-ae70-b6b9268c9480 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:11.976242 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a251beb1-e933-4fa8-9927-08ef37a1b616 None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:11.977803 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:12.012475 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5258b6-67bc-4114-9649-7834f50b7079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:12.013460 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5258b6-67bc-4114-9649-7834f50b7079 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:12.013752 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5258b6-67bc-4114-9649-7834f50b7079 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:12.014111 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5258b6-67bc-4114-9649-7834f50b7079 None admin] PATH_INFO: `/v3/roles` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:12.014848 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5258b6-67bc-4114-9649-7834f50b7079 None admin] RBAC: Adding query filter params (name=service) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:12.015163 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5258b6-67bc-4114-9649-7834f50b7079 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:12.016535 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5258b6-67bc-4114-9649-7834f50b7079 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:12.025659 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 317/635] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 19 14:52:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:12.026773 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f933949-f784-457f-9956-3503dcbd09ef None None] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:12.027073 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f933949-f784-457f-9956-3503dcbd09ef None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:12.027362 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f933949-f784-457f-9956-3503dcbd09ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:12.045190 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.auth.core [None req-1f933949-f784-457f-9956-3503dcbd09ef None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:52:12.047049 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-1f933949-f784-457f-9956-3503dcbd09ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:52:12.054435 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5067f6-be28-450d-ae70-b6b9268c9480 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:12.092374 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2393213c-a867-4361-af1d-12b1b9a557c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:12.093276 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2393213c-a867-4361-af1d-12b1b9a557c8 None admin] REQUEST_METHOD: `PUT` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:12.093566 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2393213c-a867-4361-af1d-12b1b9a557c8 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:12.093863 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2393213c-a867-4361-af1d-12b1b9a557c8 None admin] PATH_INFO: `/v3/projects/fc99d742ae004954b6d841293b5fa79c/users/17afb27b2bbe481e959cf31db4470593/roles/08f16502894349708527e30032247468` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:12.109060 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2393213c-a867-4361-af1d-12b1b9a557c8 None admin] RBAC: Authorizing `identity:create_grant(project_id=fc99d742ae004954b6d841293b5fa79c, user_id=17afb27b2bbe481e959cf31db4470593, role_id=08f16502894349708527e30032247468)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:12.110604 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2393213c-a867-4361-af1d-12b1b9a557c8 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:12.129892 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 319/636] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 14:52:11 2022] PUT /identity/v3/projects/fc99d742ae004954b6d841293b5fa79c/users/17afb27b2bbe481e959cf31db4470593/roles/08f16502894349708527e30032247468 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:52:12.129892 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2393213c-a867-4361-af1d-12b1b9a557c8 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:12.130617 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 318/637] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:52:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:52:12.132915 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f933949-f784-457f-9956-3503dcbd09ef None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:12.160932 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:12.238956 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:12.261014 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2393213c-a867-4361-af1d-12b1b9a557c8 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:12.262843 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f933949-f784-457f-9956-3503dcbd09ef None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:12.301758 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e595d8-8025-43ba-aacf-02dfcc8cf3f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:12.302224 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c869e82-eb3b-4a8f-82b8-1a9ffd869025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:12.302619 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e595d8-8025-43ba-aacf-02dfcc8cf3f3 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:12.302913 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e595d8-8025-43ba-aacf-02dfcc8cf3f3 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:12.303026 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c869e82-eb3b-4a8f-82b8-1a9ffd869025 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:12.303201 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e595d8-8025-43ba-aacf-02dfcc8cf3f3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:12.303297 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c869e82-eb3b-4a8f-82b8-1a9ffd869025 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:12.303575 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c869e82-eb3b-4a8f-82b8-1a9ffd869025 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:12.312077 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e595d8-8025-43ba-aacf-02dfcc8cf3f3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:12.314614 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e595d8-8025-43ba-aacf-02dfcc8cf3f3 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:12.315869 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c869e82-eb3b-4a8f-82b8-1a9ffd869025 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:12.317347 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c869e82-eb3b-4a8f-82b8-1a9ffd869025 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:12.334136 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-76e595d8-8025-43ba-aacf-02dfcc8cf3f3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:52:12.334136 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:12.334136 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:12.334136 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:12.334136 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:12.334136 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:12.334136 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:12.334136 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:12.334136 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:12.334136 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:12.334136 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:12.334136 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:12.334136 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:52:12.334136 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:52:12.334136 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:52:12.335338 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:52:12.335338 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:12.335338 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:12.335338 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:12.335338 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:12.335338 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:12.335338 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:12.335338 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:12.335338 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:52:12.335338 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:12.335338 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:12.335338 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:12.335338 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:12.335338 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:12.335338 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:12.335338 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:12.335338 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:12.336329 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:52:12.336329 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:52:12.336329 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:52:12.336329 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:52:12.336329 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:52:12.336329 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:52:12.336329 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:52:12.336329 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:52:12.336329 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 320/638] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:52:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:12.336865 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177e9b27-5d36-481d-9878-eec38edc2950 None None] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:12.337061 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177e9b27-5d36-481d-9878-eec38edc2950 None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:12.337350 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177e9b27-5d36-481d-9878-eec38edc2950 None None] PATH_INFO: `/` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:12.338965 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 321/639] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:52:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:52:12.339692 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177e9b27-5d36-481d-9878-eec38edc2950 None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:12.343567 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-1c869e82-eb3b-4a8f-82b8-1a9ffd869025 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:52:12.343567 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:12.343567 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:12.343567 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:12.343567 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:12.343567 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:12.343567 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:12.343567 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:12.343567 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:12.343567 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:12.343567 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:12.343567 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:12.343567 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:52:12.343567 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:52:12.343567 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:52:12.344652 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:52:12.344652 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:12.344652 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:12.344652 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:12.344652 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:12.344652 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:12.344652 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:12.344652 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:12.344652 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:52:12.344652 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:12.344652 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:12.344652 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:12.344652 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:12.344652 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:12.344652 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:12.344652 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:12.344652 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:12.345745 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:52:12.345745 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:52:12.345745 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:52:12.345745 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:52:12.346315 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 319/640] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:52:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:12.347464 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c869e82-eb3b-4a8f-82b8-1a9ffd869025 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:12.421590 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:12.461833 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c869e82-eb3b-4a8f-82b8-1a9ffd869025 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:12.463687 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177e9b27-5d36-481d-9878-eec38edc2950 None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:12.498276 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8f0731-82ca-4e2a-b9b9-43f6780aa4a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:12.498966 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8f0731-82ca-4e2a-b9b9-43f6780aa4a8 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:12.499312 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8f0731-82ca-4e2a-b9b9-43f6780aa4a8 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:12.499584 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee537aa-9a3b-4528-afb9-8c50ab713d56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:12.499714 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8f0731-82ca-4e2a-b9b9-43f6780aa4a8 None admin] PATH_INFO: `/v3/projects` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:12.500517 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee537aa-9a3b-4528-afb9-8c50ab713d56 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:12.500616 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8f0731-82ca-4e2a-b9b9-43f6780aa4a8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:12.500936 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee537aa-9a3b-4528-afb9-8c50ab713d56 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:12.501039 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8f0731-82ca-4e2a-b9b9-43f6780aa4a8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:12.501547 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee537aa-9a3b-4528-afb9-8c50ab713d56 None admin] PATH_INFO: `/v3/roles/service` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:12.502753 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8f0731-82ca-4e2a-b9b9-43f6780aa4a8 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:12.512743 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee537aa-9a3b-4528-afb9-8c50ab713d56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:52:12.520299 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 320/641] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:52:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:12.520299 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6577223f-41cf-497d-8c98-ccf936aa122e None None] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:12.520299 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6577223f-41cf-497d-8c98-ccf936aa122e None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:12.520299 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6577223f-41cf-497d-8c98-ccf936aa122e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:12.522061 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-eee537aa-9a3b-4528-afb9-8c50ab713d56 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:52:12.522061 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:12.522061 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:12.522061 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:12.522061 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:12.522061 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:12.522061 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:12.522061 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:12.522061 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:12.522061 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:12.522061 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:12.522061 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:12.522061 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:52:12.522061 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:52:12.522061 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:52:12.522061 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:52:12.523278 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:52:12.523278 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:52:12.523278 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:52:12.523278 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:52:12.523278 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:12.523278 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:12.523278 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:12.523278 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:12.523278 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:12.523278 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:12.523278 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:12.523278 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:52:12.523278 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:12.523278 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:12.523278 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:12.523278 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:12.524211 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:12.524211 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:12.524211 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:12.524211 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:12.524211 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:52:12.524211 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:52:12.524211 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:52:12.524211 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:52:12.524211 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:52:12.524211 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:52:12.524211 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:52:12.524211 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:52:12.524211 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 322/642] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 14:52:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:12.525001 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee537aa-9a3b-4528-afb9-8c50ab713d56 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:12.535417 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:12.543589 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.auth.core [None req-6577223f-41cf-497d-8c98-ccf936aa122e None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:52:12.544783 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-6577223f-41cf-497d-8c98-ccf936aa122e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:52:12.617133 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 321/643] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:52:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:52:12.618207 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6577223f-41cf-497d-8c98-ccf936aa122e None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:12.649276 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee537aa-9a3b-4528-afb9-8c50ab713d56 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:12.692627 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db85bb1-0857-409a-8c79-c4a51b3f3500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:12.693306 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db85bb1-0857-409a-8c79-c4a51b3f3500 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:12.693593 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db85bb1-0857-409a-8c79-c4a51b3f3500 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:12.693889 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db85bb1-0857-409a-8c79-c4a51b3f3500 None admin] PATH_INFO: `/v3/domains` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:12.694623 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db85bb1-0857-409a-8c79-c4a51b3f3500 None admin] RBAC: Adding query filter params (name=Default) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:12.694931 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db85bb1-0857-409a-8c79-c4a51b3f3500 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:12.696564 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db85bb1-0857-409a-8c79-c4a51b3f3500 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:12.704366 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:12.711977 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 323/644] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:52:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:12.713022 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db85bb1-0857-409a-8c79-c4a51b3f3500 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:12.753673 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6577223f-41cf-497d-8c98-ccf936aa122e None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:12.786034 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:12.791387 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca5c9cb-78cd-4e4b-b92c-52a4ac883231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:12.792054 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca5c9cb-78cd-4e4b-b92c-52a4ac883231 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:12.792627 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca5c9cb-78cd-4e4b-b92c-52a4ac883231 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:12.792627 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca5c9cb-78cd-4e4b-b92c-52a4ac883231 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:12.793443 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca5c9cb-78cd-4e4b-b92c-52a4ac883231 None admin] RBAC: Adding query filter params (role.id=08f16502894349708527e30032247468, scope.project.id=fc99d742ae004954b6d841293b5fa79c, user.id=27d9bc13809245299fa5d401f0696215) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:12.793761 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca5c9cb-78cd-4e4b-b92c-52a4ac883231 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:12.795230 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca5c9cb-78cd-4e4b-b92c-52a4ac883231 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:12.809886 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 322/645] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:52:12 2022] GET /identity/v3/role_assignments?user.id=27d9bc13809245299fa5d401f0696215&scope.project.id=fc99d742ae004954b6d841293b5fa79c&role.id=08f16502894349708527e30032247468 => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:12.809886 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca5c9cb-78cd-4e4b-b92c-52a4ac883231 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:12.837441 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db85bb1-0857-409a-8c79-c4a51b3f3500 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:12.885963 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5b1bcb-266b-482e-b5a3-7cd00d91d296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:12.886935 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5b1bcb-266b-482e-b5a3-7cd00d91d296 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:12.887231 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5b1bcb-266b-482e-b5a3-7cd00d91d296 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:12.887517 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5b1bcb-266b-482e-b5a3-7cd00d91d296 None admin] PATH_INFO: `/v3/roles` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:12.888471 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5b1bcb-266b-482e-b5a3-7cd00d91d296 None admin] RBAC: Adding query filter params (name=service) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:12.888792 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5b1bcb-266b-482e-b5a3-7cd00d91d296 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:12.891034 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5b1bcb-266b-482e-b5a3-7cd00d91d296 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:12.903819 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 324/646] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 19 14:52:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:12.904771 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5b1bcb-266b-482e-b5a3-7cd00d91d296 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:12.944706 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca5c9cb-78cd-4e4b-b92c-52a4ac883231 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:12.982006 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:12.998072 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a36fcc-5527-4d1c-a1d8-160c311744da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:12.998920 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a36fcc-5527-4d1c-a1d8-160c311744da None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:12.999216 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a36fcc-5527-4d1c-a1d8-160c311744da None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:12.999505 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a36fcc-5527-4d1c-a1d8-160c311744da None admin] PATH_INFO: `/v3/roles/service` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:13.014647 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a36fcc-5527-4d1c-a1d8-160c311744da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:52:13.024271 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-e3a36fcc-5527-4d1c-a1d8-160c311744da None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:52:13.024271 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:13.024271 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:13.024271 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:13.024271 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:13.024271 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:13.024271 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:13.024271 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:13.024271 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:13.024271 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:13.024271 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:13.024271 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:13.024271 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:52:13.024271 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:52:13.024271 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:52:13.024271 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:52:13.025362 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:52:13.025362 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:52:13.025362 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:52:13.025362 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:52:13.025362 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:13.025362 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:13.025362 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:13.025362 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:13.025362 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:13.025362 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:13.025362 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:13.025362 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:52:13.025362 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:13.025362 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:13.025362 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:13.025362 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:13.027347 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:13.027347 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:13.027347 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:13.027347 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:13.027347 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:52:13.027347 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:52:13.027347 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:52:13.027347 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:52:13.027347 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:52:13.027347 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:52:13.027347 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:52:13.027347 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:52:13.027347 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 323/647] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 14:52:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:13.027347 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a36fcc-5527-4d1c-a1d8-160c311744da None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:13.054557 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5b1bcb-266b-482e-b5a3-7cd00d91d296 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:13.085990 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:13.098606 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afefc6d-9701-4102-9adc-9381da14d1a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:13.099900 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afefc6d-9701-4102-9adc-9381da14d1a1 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:13.100192 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afefc6d-9701-4102-9adc-9381da14d1a1 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:13.101155 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afefc6d-9701-4102-9adc-9381da14d1a1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:13.112108 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afefc6d-9701-4102-9adc-9381da14d1a1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:13.114388 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afefc6d-9701-4102-9adc-9381da14d1a1 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:13.131114 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-5afefc6d-9701-4102-9adc-9381da14d1a1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 14:52:13.131114 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:13.131114 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:13.131114 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:13.131114 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:13.131114 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:13.131114 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:13.131114 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:13.131114 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:13.131114 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:13.131114 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:13.131114 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:13.131114 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:52:13.131114 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:52:13.131114 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:52:13.131114 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:52:13.132411 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:13.132411 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:13.132411 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:52:13.132411 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:52:13.132411 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:52:13.132411 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:52:13.132411 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:13.132411 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:13.132411 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:13.132411 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:13.132411 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:13.132411 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:52:13.132411 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:13.132411 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:13.132411 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:13.132411 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:13.132411 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:13.132411 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:13.133685 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:13.133685 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:13.133685 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:52:13.133685 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:52:13.133685 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:52:13.133685 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:52:13.133685 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:52:13.133685 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:52:13.133685 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 14:52:13.133685 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:52:13.134689 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 325/648] 10.222.0.57 () {56 vars in 1093 bytes} [Thu May 19 14:52:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:13.135525 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afefc6d-9701-4102-9adc-9381da14d1a1 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:13.166148 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a36fcc-5527-4d1c-a1d8-160c311744da None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:13.203373 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eca7628-eee2-40ab-88ed-3b661a8ec5a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:13.204746 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eca7628-eee2-40ab-88ed-3b661a8ec5a4 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:13.205649 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eca7628-eee2-40ab-88ed-3b661a8ec5a4 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:13.205951 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eca7628-eee2-40ab-88ed-3b661a8ec5a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:13.217270 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eca7628-eee2-40ab-88ed-3b661a8ec5a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:13.218798 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eca7628-eee2-40ab-88ed-3b661a8ec5a4 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:13.238483 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-2eca7628-eee2-40ab-88ed-3b661a8ec5a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:52:13.238483 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:13.238483 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:13.238483 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:13.238483 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:13.238483 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:13.238483 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:13.238483 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:13.238483 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:13.238483 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:13.238483 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:13.238483 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:13.238483 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:52:13.238483 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:52:13.238483 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:52:13.239514 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:52:13.239514 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:13.239514 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:13.239514 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:13.239514 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:13.239514 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:13.239514 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:13.239514 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:13.239514 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:52:13.239514 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:13.239514 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:13.239514 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:13.239514 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:13.239514 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:13.239514 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:13.239514 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:13.239514 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:13.240495 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:52:13.240495 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:52:13.240495 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:52:13.240495 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:52:13.240495 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 324/649] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:52:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:13.240818 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eca7628-eee2-40ab-88ed-3b661a8ec5a4 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:13.268540 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afefc6d-9701-4102-9adc-9381da14d1a1 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:13.280755 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:13.318810 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5609c0c8-ae5e-4c1e-a862-bd0d3c56b6cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:13.319657 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5609c0c8-ae5e-4c1e-a862-bd0d3c56b6cd None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:13.320055 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5609c0c8-ae5e-4c1e-a862-bd0d3c56b6cd None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:13.320683 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5609c0c8-ae5e-4c1e-a862-bd0d3c56b6cd None admin] PATH_INFO: `/v3/roles` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:13.321535 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5609c0c8-ae5e-4c1e-a862-bd0d3c56b6cd None admin] RBAC: Adding query filter params (name=service) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:13.321972 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5609c0c8-ae5e-4c1e-a862-bd0d3c56b6cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:13.323566 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5609c0c8-ae5e-4c1e-a862-bd0d3c56b6cd None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:13.333187 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 326/650] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 19 14:52:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:13.334274 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5609c0c8-ae5e-4c1e-a862-bd0d3c56b6cd None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:13.363083 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eca7628-eee2-40ab-88ed-3b661a8ec5a4 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:13.390882 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:13.401995 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602f7178-69af-48a3-96ba-2c2e9460d116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:13.402689 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602f7178-69af-48a3-96ba-2c2e9460d116 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:13.402986 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602f7178-69af-48a3-96ba-2c2e9460d116 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:13.403272 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602f7178-69af-48a3-96ba-2c2e9460d116 None admin] PATH_INFO: `/v3/users` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:13.404300 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602f7178-69af-48a3-96ba-2c2e9460d116 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:13.404623 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602f7178-69af-48a3-96ba-2c2e9460d116 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:13.406735 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602f7178-69af-48a3-96ba-2c2e9460d116 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:13.429957 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 325/651] 10.222.0.57 () {56 vars in 1133 bytes} [Thu May 19 14:52:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:13.431184 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602f7178-69af-48a3-96ba-2c2e9460d116 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:13.467940 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5609c0c8-ae5e-4c1e-a862-bd0d3c56b6cd None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:13.513758 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8eece50-cc9d-4c15-8a8a-e009fef555ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:13.514929 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eece50-cc9d-4c15-8a8a-e009fef555ea None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:13.515479 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eece50-cc9d-4c15-8a8a-e009fef555ea None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:13.515959 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eece50-cc9d-4c15-8a8a-e009fef555ea None admin] PATH_INFO: `/v3/domains` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:13.518258 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8eece50-cc9d-4c15-8a8a-e009fef555ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:13.520700 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8eece50-cc9d-4c15-8a8a-e009fef555ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:13.522445 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8eece50-cc9d-4c15-8a8a-e009fef555ea None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:13.538926 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:13.544313 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 327/652] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:52:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:13.545427 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8eece50-cc9d-4c15-8a8a-e009fef555ea None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:13.557489 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602f7178-69af-48a3-96ba-2c2e9460d116 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:13.603746 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed133a6d-5b52-4291-a903-6e082bb8f2c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:13.604977 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed133a6d-5b52-4291-a903-6e082bb8f2c2 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:13.605583 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed133a6d-5b52-4291-a903-6e082bb8f2c2 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:13.606133 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed133a6d-5b52-4291-a903-6e082bb8f2c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:13.620284 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed133a6d-5b52-4291-a903-6e082bb8f2c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:13.622545 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed133a6d-5b52-4291-a903-6e082bb8f2c2 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:13.642539 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-ed133a6d-5b52-4291-a903-6e082bb8f2c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:52:13.642539 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:13.642539 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:13.642539 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:13.642539 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:13.642539 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:13.642539 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:13.642539 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:13.642539 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:13.642539 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:13.642539 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:13.642539 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:13.642539 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:52:13.642539 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:52:13.642539 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:52:13.643919 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:52:13.643919 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:13.643919 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:13.643919 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:13.643919 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:13.643919 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:13.643919 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:13.643919 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:13.643919 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:52:13.643919 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:13.643919 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:13.643919 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:13.643919 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:13.643919 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:13.643919 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:13.643919 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:13.643919 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:13.645140 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:52:13.645140 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:52:13.645140 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:52:13.645140 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:52:13.646487 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 326/653] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:52:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:13.647669 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed133a6d-5b52-4291-a903-6e082bb8f2c2 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:13.683089 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8eece50-cc9d-4c15-8a8a-e009fef555ea None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:13.699468 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:13.736187 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ff5940-6475-4fab-94a2-e017e9643f85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:13.737158 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ff5940-6475-4fab-94a2-e017e9643f85 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:13.737626 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ff5940-6475-4fab-94a2-e017e9643f85 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:13.738106 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ff5940-6475-4fab-94a2-e017e9643f85 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:13.771334 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed133a6d-5b52-4291-a903-6e082bb8f2c2 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:13.820866 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ff5940-6475-4fab-94a2-e017e9643f85 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:13.822600 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d07c9c-467b-4dde-91b7-8a5e584341c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:13.823097 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ff5940-6475-4fab-94a2-e017e9643f85 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:13.824051 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d07c9c-467b-4dde-91b7-8a5e584341c9 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:13.824532 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d07c9c-467b-4dde-91b7-8a5e584341c9 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:13.825204 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d07c9c-467b-4dde-91b7-8a5e584341c9 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:13.836414 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d07c9c-467b-4dde-91b7-8a5e584341c9 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:13.838491 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d07c9c-467b-4dde-91b7-8a5e584341c9 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:13.857424 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-c2d07c9c-467b-4dde-91b7-8a5e584341c9 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 19 14:52:13.857424 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:13.857424 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:13.857424 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:13.857424 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:13.857424 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:13.857424 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:13.857424 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:13.857424 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:13.857424 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:13.857424 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:13.857424 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:13.857424 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:52:13.857424 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:52:13.857424 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:52:13.857424 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:52:13.858757 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:13.858757 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:13.858757 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:52:13.858757 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:52:13.858757 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:52:13.858757 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:52:13.858757 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:13.858757 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:13.858757 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:13.858757 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:13.858757 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:13.858757 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:52:13.858757 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:13.858757 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:13.858757 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:13.858757 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:13.858757 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:13.858757 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:13.860048 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:13.860048 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:13.860048 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:52:13.860048 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:52:13.860048 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:52:13.860048 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:52:13.860048 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:52:13.860048 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:52:13.860048 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 19 14:52:13.860048 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:52:13.866261 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 327/654] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 14:52:13 2022] GET /identity/v3/users/neutron => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:13.867291 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d07c9c-467b-4dde-91b7-8a5e584341c9 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:13.872050 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:13.959730 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 328/655] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:52:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:52:13.959730 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ff5940-6475-4fab-94a2-e017e9643f85 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:13.973688 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:13.995239 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d07c9c-467b-4dde-91b7-8a5e584341c9 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:14.044664 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eabf4d0-6954-4326-aaa6-d109655d1417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:14.045255 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eabf4d0-6954-4326-aaa6-d109655d1417 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:14.045544 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eabf4d0-6954-4326-aaa6-d109655d1417 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:14.045824 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eabf4d0-6954-4326-aaa6-d109655d1417 None admin] PATH_INFO: `/v3/domains` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:14.046585 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eabf4d0-6954-4326-aaa6-d109655d1417 None admin] RBAC: Adding query filter params (name=Default) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:14.046900 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eabf4d0-6954-4326-aaa6-d109655d1417 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:14.048816 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eabf4d0-6954-4326-aaa6-d109655d1417 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:14.061834 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 328/656] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:52:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:14.062665 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eabf4d0-6954-4326-aaa6-d109655d1417 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:14.096911 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ff5940-6475-4fab-94a2-e017e9643f85 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:14.131176 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:14.147392 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffae102-c4d3-413d-84b3-2e3443d24309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:14.148475 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffae102-c4d3-413d-84b3-2e3443d24309 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:14.149420 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffae102-c4d3-413d-84b3-2e3443d24309 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:14.149949 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffae102-c4d3-413d-84b3-2e3443d24309 None admin] PATH_INFO: `/v3/users` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:14.151736 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffae102-c4d3-413d-84b3-2e3443d24309 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:14.152347 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffae102-c4d3-413d-84b3-2e3443d24309 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:14.155039 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffae102-c4d3-413d-84b3-2e3443d24309 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:14.180105 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eabf4d0-6954-4326-aaa6-d109655d1417 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:14.180457 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 329/657] 10.222.0.57 () {56 vars in 1139 bytes} [Thu May 19 14:52:13 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:14.181252 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffae102-c4d3-413d-84b3-2e3443d24309 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:14.229384 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07369484-ce1e-49fe-8152-c8dd7c983d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:14.231210 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07369484-ce1e-49fe-8152-c8dd7c983d3d None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:14.231722 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07369484-ce1e-49fe-8152-c8dd7c983d3d None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:14.232640 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07369484-ce1e-49fe-8152-c8dd7c983d3d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:14.247886 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07369484-ce1e-49fe-8152-c8dd7c983d3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:14.250908 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07369484-ce1e-49fe-8152-c8dd7c983d3d None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:14.260479 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:14.280745 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-07369484-ce1e-49fe-8152-c8dd7c983d3d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:52:14.280745 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:14.280745 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:14.280745 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:14.280745 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:14.280745 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:14.280745 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:14.280745 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:14.280745 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:14.280745 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:14.280745 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:14.280745 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:14.280745 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:52:14.280745 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:52:14.280745 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:52:14.281869 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:52:14.281869 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:14.281869 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:14.281869 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:14.281869 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:14.281869 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:14.281869 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:14.281869 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:14.281869 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:52:14.281869 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:14.281869 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:14.281869 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:14.281869 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:14.281869 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:14.281869 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:14.281869 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:14.281869 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:14.283420 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:52:14.283420 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:52:14.283420 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:52:14.283420 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:52:14.286217 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 329/658] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:52:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:14.287410 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07369484-ce1e-49fe-8152-c8dd7c983d3d None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:14.309746 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffae102-c4d3-413d-84b3-2e3443d24309 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:14.353084 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f84784-5081-48cd-8d0c-88cdc1b71939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:14.354093 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f84784-5081-48cd-8d0c-88cdc1b71939 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:14.354450 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f84784-5081-48cd-8d0c-88cdc1b71939 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:14.354829 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f84784-5081-48cd-8d0c-88cdc1b71939 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:14.363922 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f84784-5081-48cd-8d0c-88cdc1b71939 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:14.365320 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f84784-5081-48cd-8d0c-88cdc1b71939 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:14.381568 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-45f84784-5081-48cd-8d0c-88cdc1b71939 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 14:52:14.381568 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:14.381568 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:14.381568 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:14.381568 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:14.381568 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:14.381568 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:14.381568 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:14.381568 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:14.381568 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:14.381568 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:14.381568 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:14.381568 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:52:14.381568 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:52:14.381568 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:52:14.381568 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:52:14.382875 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:14.382875 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:14.382875 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:52:14.382875 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:52:14.382875 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:52:14.382875 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:52:14.382875 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:14.382875 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:14.382875 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:14.382875 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:14.382875 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:14.382875 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:52:14.382875 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:14.382875 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:14.382875 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:14.382875 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:14.382875 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:14.382875 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:14.384148 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:14.384148 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:14.384148 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:52:14.384148 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:52:14.384148 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:52:14.384148 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:52:14.384148 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:52:14.384148 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:52:14.384148 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 14:52:14.384148 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:52:14.385632 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 330/659] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:52:14 2022] GET /identity/v3/users/cinder => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:14.390864 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f84784-5081-48cd-8d0c-88cdc1b71939 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:14.417551 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07369484-ce1e-49fe-8152-c8dd7c983d3d None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:14.440103 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:14.467043 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741f491e-13ca-45da-9906-efa4a8863c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:14.467852 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741f491e-13ca-45da-9906-efa4a8863c8f None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:14.468245 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741f491e-13ca-45da-9906-efa4a8863c8f None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:14.468633 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741f491e-13ca-45da-9906-efa4a8863c8f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:14.504307 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f84784-5081-48cd-8d0c-88cdc1b71939 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:14.532388 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:14.541397 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42575227-c78d-4bde-a716-f4b5f40584c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:14.542242 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42575227-c78d-4bde-a716-f4b5f40584c6 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:14.542634 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42575227-c78d-4bde-a716-f4b5f40584c6 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:14.543021 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42575227-c78d-4bde-a716-f4b5f40584c6 None admin] PATH_INFO: `/v3/domains` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:14.543850 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42575227-c78d-4bde-a716-f4b5f40584c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:14.544315 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42575227-c78d-4bde-a716-f4b5f40584c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:14.545778 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42575227-c78d-4bde-a716-f4b5f40584c6 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:14.549259 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741f491e-13ca-45da-9906-efa4a8863c8f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:14.550827 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741f491e-13ca-45da-9906-efa4a8863c8f None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:14.562746 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 331/660] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:52:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:14.562746 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42575227-c78d-4bde-a716-f4b5f40584c6 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:14.667716 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42575227-c78d-4bde-a716-f4b5f40584c6 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:14.678357 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 330/661] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:52:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:52:14.678357 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c5604f-afb7-4070-a674-77ef74b305c8 None None] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:14.678357 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c5604f-afb7-4070-a674-77ef74b305c8 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:14.678357 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c5604f-afb7-4070-a674-77ef74b305c8 None None] PATH_INFO: `/` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:14.678357 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 331/662] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:52:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:52:14.678357 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c5604f-afb7-4070-a674-77ef74b305c8 None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:14.695692 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:14.711510 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f861303-badf-4375-9a93-eed1f18ce0fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:14.712521 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f861303-badf-4375-9a93-eed1f18ce0fc None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:14.712889 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f861303-badf-4375-9a93-eed1f18ce0fc None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:14.713288 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f861303-badf-4375-9a93-eed1f18ce0fc None admin] PATH_INFO: `/v3/users` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:14.714202 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f861303-badf-4375-9a93-eed1f18ce0fc None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:14.714597 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f861303-badf-4375-9a93-eed1f18ce0fc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:14.716706 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f861303-badf-4375-9a93-eed1f18ce0fc None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:14.735984 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 332/663] 10.222.0.57 () {56 vars in 1137 bytes} [Thu May 19 14:52:14 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:14.736755 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f861303-badf-4375-9a93-eed1f18ce0fc None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:14.790563 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:14.791114 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c5604f-afb7-4070-a674-77ef74b305c8 None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:14.830761 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a716fa6e-26d0-412b-b43b-c4270f2253a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:14.831622 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a716fa6e-26d0-412b-b43b-c4270f2253a1 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:14.832008 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a716fa6e-26d0-412b-b43b-c4270f2253a1 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:14.832384 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a716fa6e-26d0-412b-b43b-c4270f2253a1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:14.842202 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a716fa6e-26d0-412b-b43b-c4270f2253a1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:14.843655 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a716fa6e-26d0-412b-b43b-c4270f2253a1 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:14.852211 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f861303-badf-4375-9a93-eed1f18ce0fc None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:14.858370 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-a716fa6e-26d0-412b-b43b-c4270f2253a1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:52:14.858370 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:14.858370 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:14.858370 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:14.858370 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:14.858370 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:14.858370 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:14.858370 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:14.858370 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:14.858370 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:14.858370 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:14.858370 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:14.858370 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:52:14.858370 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:52:14.858370 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:52:14.859540 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:52:14.859540 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:14.859540 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:14.859540 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:14.859540 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:14.859540 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:14.859540 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:14.859540 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:14.859540 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:52:14.859540 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:14.859540 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:14.859540 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:14.859540 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:14.859540 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:14.859540 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:14.859540 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:14.859540 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:14.866070 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:52:14.866070 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:52:14.866070 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:52:14.866070 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:52:14.866070 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:52:14.866070 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:52:14.866070 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:52:14.866070 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:52:14.866070 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 332/664] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:52:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:14.866070 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da67e6ea-4a8b-417a-88a6-fde1bfb8423c None None] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:14.866070 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da67e6ea-4a8b-417a-88a6-fde1bfb8423c None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:14.866878 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da67e6ea-4a8b-417a-88a6-fde1bfb8423c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:14.879455 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.auth.core [None req-da67e6ea-4a8b-417a-88a6-fde1bfb8423c None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:52:14.880661 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-da67e6ea-4a8b-417a-88a6-fde1bfb8423c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:52:14.896232 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a87861d-9f61-4758-9a69-62ea68bad5ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:14.897132 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a87861d-9f61-4758-9a69-62ea68bad5ca None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:14.897483 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a87861d-9f61-4758-9a69-62ea68bad5ca None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:14.897835 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a87861d-9f61-4758-9a69-62ea68bad5ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:14.908788 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a87861d-9f61-4758-9a69-62ea68bad5ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:14.910485 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a87861d-9f61-4758-9a69-62ea68bad5ca None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:14.929225 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-4a87861d-9f61-4758-9a69-62ea68bad5ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:52:14.929225 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:14.929225 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:14.929225 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:14.929225 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:14.929225 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:14.929225 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:14.929225 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:14.929225 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:14.929225 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:14.929225 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:14.929225 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:14.929225 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:52:14.929225 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:52:14.929225 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:52:14.930520 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:52:14.930520 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:14.930520 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:14.930520 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:14.930520 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:14.930520 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:14.930520 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:14.930520 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:14.930520 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:52:14.930520 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:14.930520 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:14.930520 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:14.930520 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:14.930520 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:14.930520 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:14.930520 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:14.930520 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:14.931743 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:52:14.931743 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:52:14.931743 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:52:14.931743 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:52:14.936399 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 333/665] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:52:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:14.936399 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a87861d-9f61-4758-9a69-62ea68bad5ca None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:14.962011 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 333/666] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:52:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:52:14.962011 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da67e6ea-4a8b-417a-88a6-fde1bfb8423c None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:14.978069 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:15.037356 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:15.041668 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a87861d-9f61-4758-9a69-62ea68bad5ca None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:15.079715 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86e731b-3f98-4587-8a53-d27b4a7ae1a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:15.080397 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86e731b-3f98-4587-8a53-d27b4a7ae1a8 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:15.080673 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86e731b-3f98-4587-8a53-d27b4a7ae1a8 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:15.081022 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86e731b-3f98-4587-8a53-d27b4a7ae1a8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:15.082887 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da67e6ea-4a8b-417a-88a6-fde1bfb8423c None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:15.123570 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca13383-c814-491a-a653-749864d134c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:15.124260 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca13383-c814-491a-a653-749864d134c1 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:15.124559 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca13383-c814-491a-a653-749864d134c1 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:15.124823 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca13383-c814-491a-a653-749864d134c1 None admin] PATH_INFO: `/v3/projects` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:15.125634 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca13383-c814-491a-a653-749864d134c1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:15.126066 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca13383-c814-491a-a653-749864d134c1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:15.127762 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca13383-c814-491a-a653-749864d134c1 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:15.145873 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 334/667] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:52:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:15.145873 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca13383-c814-491a-a653-749864d134c1 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:15.165331 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86e731b-3f98-4587-8a53-d27b4a7ae1a8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:15.166947 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86e731b-3f98-4587-8a53-d27b4a7ae1a8 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:15.215824 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:15.257292 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca13383-c814-491a-a653-749864d134c1 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:15.275209 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:15.285677 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 334/668] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:52:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:52:15.287036 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86e731b-3f98-4587-8a53-d27b4a7ae1a8 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:15.299721 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbfca3c-525e-409e-9d16-f0672e750419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:15.300174 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbfca3c-525e-409e-9d16-f0672e750419 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:15.300642 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbfca3c-525e-409e-9d16-f0672e750419 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:15.300795 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbfca3c-525e-409e-9d16-f0672e750419 None admin] PATH_INFO: `/v3/domains` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:15.301471 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbfca3c-525e-409e-9d16-f0672e750419 None admin] RBAC: Adding query filter params (name=Default) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:15.301790 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbfca3c-525e-409e-9d16-f0672e750419 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:15.303224 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbfca3c-525e-409e-9d16-f0672e750419 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:15.316098 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 335/669] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:52:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:15.316796 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbfca3c-525e-409e-9d16-f0672e750419 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:15.390507 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86e731b-3f98-4587-8a53-d27b4a7ae1a8 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:15.418695 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbfca3c-525e-409e-9d16-f0672e750419 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:15.425509 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475a793b-3bbd-4f28-bc55-6a348359b1f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:15.426348 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475a793b-3bbd-4f28-bc55-6a348359b1f8 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:15.426642 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475a793b-3bbd-4f28-bc55-6a348359b1f8 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:15.426920 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475a793b-3bbd-4f28-bc55-6a348359b1f8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:15.437390 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475a793b-3bbd-4f28-bc55-6a348359b1f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:52:15.445766 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:15.447103 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-475a793b-3bbd-4f28-bc55-6a348359b1f8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:52:15.447103 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:15.447103 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:15.447103 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:15.447103 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:15.447103 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:15.447103 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:15.447103 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:15.447103 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:15.447103 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:15.447103 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:15.447103 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:15.447103 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:52:15.447103 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:52:15.447103 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:52:15.447103 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:52:15.448387 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:52:15.448387 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:52:15.448387 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:52:15.448387 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:52:15.448387 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:15.448387 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:15.448387 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:15.448387 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:15.448387 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:15.448387 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:15.448387 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:15.448387 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:52:15.448387 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:15.448387 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:15.448387 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:15.448387 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:15.454298 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:15.454298 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:15.454298 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:15.454298 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:15.454298 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:52:15.454298 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:52:15.454298 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:52:15.454298 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:52:15.454298 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:52:15.454298 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:52:15.454298 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:52:15.454298 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:52:15.454298 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 335/670] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 14:52:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:15.454298 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475a793b-3bbd-4f28-bc55-6a348359b1f8 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:15.458616 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c298e2c-7f69-4729-9d7f-b1aab9da9e0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:15.459316 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c298e2c-7f69-4729-9d7f-b1aab9da9e0d None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:15.459605 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c298e2c-7f69-4729-9d7f-b1aab9da9e0d None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:15.459908 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c298e2c-7f69-4729-9d7f-b1aab9da9e0d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:15.460750 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c298e2c-7f69-4729-9d7f-b1aab9da9e0d None admin] RBAC: Adding query filter params (role.id=08f16502894349708527e30032247468, scope.project.id=fc99d742ae004954b6d841293b5fa79c, user.id=5d2aa8c8870a49d98f23f26e036695ef) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:15.461069 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c298e2c-7f69-4729-9d7f-b1aab9da9e0d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:15.462535 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c298e2c-7f69-4729-9d7f-b1aab9da9e0d None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:15.468690 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 336/671] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:52:15 2022] GET /identity/v3/role_assignments?user.id=5d2aa8c8870a49d98f23f26e036695ef&scope.project.id=fc99d742ae004954b6d841293b5fa79c&role.id=08f16502894349708527e30032247468 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:15.470091 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebfd3a3-fcc3-4edb-8d67-92e3aa63825f None None] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:15.470391 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebfd3a3-fcc3-4edb-8d67-92e3aa63825f None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:15.470662 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebfd3a3-fcc3-4edb-8d67-92e3aa63825f None None] PATH_INFO: `/` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:15.471986 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 337/672] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:52:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:52:15.472708 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebfd3a3-fcc3-4edb-8d67-92e3aa63825f None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:15.535567 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:15.585439 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475a793b-3bbd-4f28-bc55-6a348359b1f8 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:15.612818 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebfd3a3-fcc3-4edb-8d67-92e3aa63825f None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:15.625000 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be6c8ef-35ad-4c98-8f32-35a7c1cb8ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:15.626720 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be6c8ef-35ad-4c98-8f32-35a7c1cb8ba1 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:15.627008 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be6c8ef-35ad-4c98-8f32-35a7c1cb8ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:15.627467 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be6c8ef-35ad-4c98-8f32-35a7c1cb8ba1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:15.639138 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be6c8ef-35ad-4c98-8f32-35a7c1cb8ba1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:15.640702 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be6c8ef-35ad-4c98-8f32-35a7c1cb8ba1 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:15.657091 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e707b0b0-ca9f-4cb8-ba9e-5de97a0157f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:15.658406 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e707b0b0-ca9f-4cb8-ba9e-5de97a0157f3 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:15.658892 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e707b0b0-ca9f-4cb8-ba9e-5de97a0157f3 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:15.659364 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e707b0b0-ca9f-4cb8-ba9e-5de97a0157f3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:15.659783 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-9be6c8ef-35ad-4c98-8f32-35a7c1cb8ba1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:52:15.659783 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:15.659783 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:15.659783 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:15.659783 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:15.659783 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:15.659783 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:15.659783 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:15.659783 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:15.659783 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:15.659783 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:15.659783 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:15.659783 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:52:15.659783 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:52:15.659783 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:52:15.660752 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:52:15.660752 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:15.660752 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:15.660752 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:15.660752 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:15.660752 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:15.660752 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:15.660752 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:15.660752 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:52:15.660752 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:15.660752 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:15.660752 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:15.660752 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:15.660752 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:15.660752 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:15.660752 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:15.660752 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:15.661774 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:52:15.661774 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:52:15.661774 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:52:15.661774 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:52:15.661774 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 336/673] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:52:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:15.662403 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be6c8ef-35ad-4c98-8f32-35a7c1cb8ba1 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:15.676253 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e707b0b0-ca9f-4cb8-ba9e-5de97a0157f3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:15.677731 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e707b0b0-ca9f-4cb8-ba9e-5de97a0157f3 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:15.693324 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-e707b0b0-ca9f-4cb8-ba9e-5de97a0157f3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:52:15.693324 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:15.693324 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:15.693324 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:15.693324 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:15.693324 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:15.693324 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:15.693324 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:15.693324 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:15.693324 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:15.693324 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:15.693324 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:15.693324 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:52:15.693324 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:52:15.693324 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:52:15.694414 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:52:15.694414 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:15.694414 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:15.694414 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:15.694414 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:15.694414 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:15.694414 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:15.694414 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:15.694414 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:52:15.694414 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:15.694414 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:15.694414 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:15.694414 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:15.694414 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:15.694414 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:15.694414 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:15.694414 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:15.695739 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:52:15.695739 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:52:15.695739 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:52:15.695739 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:52:15.695739 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:52:15.695739 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:52:15.695739 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:52:15.695739 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:52:15.695739 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 338/674] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:52:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:15.696925 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a29052-2c9b-4652-a7cb-ede63b31062c None None] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:15.697226 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a29052-2c9b-4652-a7cb-ede63b31062c None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:15.698218 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a29052-2c9b-4652-a7cb-ede63b31062c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:15.717813 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.auth.core [None req-39a29052-2c9b-4652-a7cb-ede63b31062c None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:52:15.719066 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-39a29052-2c9b-4652-a7cb-ede63b31062c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:52:15.773430 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:15.792845 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be6c8ef-35ad-4c98-8f32-35a7c1cb8ba1 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:15.795883 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 339/675] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:52:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:52:15.796630 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a29052-2c9b-4652-a7cb-ede63b31062c None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:15.821474 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:15.843899 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2436d92-5525-46ed-b70e-7ed5ebe9b238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:15.844577 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2436d92-5525-46ed-b70e-7ed5ebe9b238 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:15.845406 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2436d92-5525-46ed-b70e-7ed5ebe9b238 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:15.845688 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2436d92-5525-46ed-b70e-7ed5ebe9b238 None admin] PATH_INFO: `/v3/roles` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:15.846580 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2436d92-5525-46ed-b70e-7ed5ebe9b238 None admin] RBAC: Adding query filter params (name=service) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:15.846891 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2436d92-5525-46ed-b70e-7ed5ebe9b238 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:15.848280 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2436d92-5525-46ed-b70e-7ed5ebe9b238 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:15.856869 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 337/676] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 19 14:52:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:15.857674 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2436d92-5525-46ed-b70e-7ed5ebe9b238 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:15.917241 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a29052-2c9b-4652-a7cb-ede63b31062c None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:15.954995 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cc4c0c-cb9e-43d8-b32e-2e622a18f13a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:15.955778 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cc4c0c-cb9e-43d8-b32e-2e622a18f13a None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:15.956076 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cc4c0c-cb9e-43d8-b32e-2e622a18f13a None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:15.956435 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cc4c0c-cb9e-43d8-b32e-2e622a18f13a None admin] PATH_INFO: `/v3/domains` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:15.957183 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cc4c0c-cb9e-43d8-b32e-2e622a18f13a None admin] RBAC: Adding query filter params (name=Default) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:15.957501 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cc4c0c-cb9e-43d8-b32e-2e622a18f13a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:15.958918 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cc4c0c-cb9e-43d8-b32e-2e622a18f13a None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:15.970453 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2436d92-5525-46ed-b70e-7ed5ebe9b238 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:15.971820 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 340/677] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:52:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:15.972649 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cc4c0c-cb9e-43d8-b32e-2e622a18f13a None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:16.011739 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315c3244-eadb-4f80-9731-0b01ca504b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:16.012580 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315c3244-eadb-4f80-9731-0b01ca504b66 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:16.012868 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315c3244-eadb-4f80-9731-0b01ca504b66 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:16.013146 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315c3244-eadb-4f80-9731-0b01ca504b66 None admin] PATH_INFO: `/v3/projects` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:16.013967 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315c3244-eadb-4f80-9731-0b01ca504b66 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:16.014288 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315c3244-eadb-4f80-9731-0b01ca504b66 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:16.015725 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315c3244-eadb-4f80-9731-0b01ca504b66 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:16.018925 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:16.034183 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 338/678] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:52:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:16.035823 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315c3244-eadb-4f80-9731-0b01ca504b66 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:16.071297 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:16.086074 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cc4c0c-cb9e-43d8-b32e-2e622a18f13a None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:16.125205 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5906098-9806-4e4f-b9be-9c75afd117e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:16.127235 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5906098-9806-4e4f-b9be-9c75afd117e1 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:16.127676 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5906098-9806-4e4f-b9be-9c75afd117e1 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:16.127968 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5906098-9806-4e4f-b9be-9c75afd117e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:16.143146 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5906098-9806-4e4f-b9be-9c75afd117e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:16.145059 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5906098-9806-4e4f-b9be-9c75afd117e1 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:16.148467 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315c3244-eadb-4f80-9731-0b01ca504b66 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:16.165512 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-c5906098-9806-4e4f-b9be-9c75afd117e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:52:16.165512 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:16.165512 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:16.165512 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:16.165512 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:16.165512 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:16.165512 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:16.165512 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:16.165512 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:16.165512 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:16.165512 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:16.165512 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:16.165512 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:52:16.165512 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:52:16.165512 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:52:16.166573 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:52:16.166573 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:16.166573 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:16.166573 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:16.166573 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:16.166573 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:16.166573 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:16.166573 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:16.166573 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:52:16.166573 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:16.166573 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:16.166573 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:16.166573 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:16.166573 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:16.166573 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:16.166573 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:16.166573 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:16.167653 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:52:16.167653 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:52:16.167653 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:52:16.167653 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:52:16.167653 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 341/679] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:52:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:16.167653 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5906098-9806-4e4f-b9be-9c75afd117e1 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:16.189944 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ff6fdc-8634-4463-8914-a6f6de81b1df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:16.191095 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ff6fdc-8634-4463-8914-a6f6de81b1df None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:16.191575 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ff6fdc-8634-4463-8914-a6f6de81b1df None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:16.192133 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ff6fdc-8634-4463-8914-a6f6de81b1df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:16.204330 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ff6fdc-8634-4463-8914-a6f6de81b1df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:16.206194 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ff6fdc-8634-4463-8914-a6f6de81b1df None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:16.226386 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-a4ff6fdc-8634-4463-8914-a6f6de81b1df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:52:16.226386 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:16.226386 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:16.226386 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:16.226386 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:16.226386 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:16.226386 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:16.226386 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:16.226386 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:16.226386 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:16.226386 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:16.226386 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:16.226386 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:52:16.226386 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:52:16.226386 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:52:16.227438 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:52:16.227438 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:16.227438 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:16.227438 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:16.227438 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:16.227438 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:16.227438 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:16.227438 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:16.227438 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:52:16.227438 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:16.227438 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:16.227438 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:16.227438 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:16.227438 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:16.227438 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:16.227438 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:16.227438 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:16.228533 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:52:16.228533 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:52:16.228533 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:52:16.228533 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:52:16.229105 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 339/680] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:52:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:16.230008 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ff6fdc-8634-4463-8914-a6f6de81b1df None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:16.275188 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:16.285130 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5906098-9806-4e4f-b9be-9c75afd117e1 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:16.320985 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:16.324530 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcedfd7c-97eb-44df-ad7e-01a3add98a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:16.325362 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcedfd7c-97eb-44df-ad7e-01a3add98a94 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:16.325657 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcedfd7c-97eb-44df-ad7e-01a3add98a94 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:16.325978 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcedfd7c-97eb-44df-ad7e-01a3add98a94 None admin] PATH_INFO: `/v3/projects/service` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:16.333913 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcedfd7c-97eb-44df-ad7e-01a3add98a94 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:16.335145 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcedfd7c-97eb-44df-ad7e-01a3add98a94 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:16.340268 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ff6fdc-8634-4463-8914-a6f6de81b1df None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:16.349671 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-bcedfd7c-97eb-44df-ad7e-01a3add98a94 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:52:16.349671 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:16.349671 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:16.349671 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:16.349671 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:16.349671 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:16.349671 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:16.349671 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:16.349671 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:16.349671 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:16.349671 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:16.349671 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:16.349671 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:52:16.349671 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:52:16.349671 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:52:16.350693 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:52:16.350693 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:16.350693 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:16.350693 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:16.350693 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:16.350693 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:16.350693 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:16.350693 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:16.350693 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:52:16.350693 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:16.350693 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:16.350693 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:16.350693 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:16.350693 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:16.350693 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:16.350693 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:16.350693 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:16.351705 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:52:16.351705 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:52:16.351705 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:52:16.351705 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:52:16.351705 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:52:16.351705 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:52:16.351705 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:52:16.351705 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:52:16.351705 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 342/681] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:52:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:16.351705 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcedfd7c-97eb-44df-ad7e-01a3add98a94 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:16.383130 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7936737-2a30-4f78-ac04-062750e0025d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:16.383912 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7936737-2a30-4f78-ac04-062750e0025d None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:16.384205 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7936737-2a30-4f78-ac04-062750e0025d None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:16.384490 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7936737-2a30-4f78-ac04-062750e0025d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:16.385314 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7936737-2a30-4f78-ac04-062750e0025d None admin] RBAC: Adding query filter params (role.id=08f16502894349708527e30032247468, scope.project.id=fc99d742ae004954b6d841293b5fa79c, user.id=8de553016ea641cdbf9c4a8e08c96a67) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:16.385630 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7936737-2a30-4f78-ac04-062750e0025d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:16.387101 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7936737-2a30-4f78-ac04-062750e0025d None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:16.396724 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 340/682] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:52:16 2022] GET /identity/v3/role_assignments?user.id=8de553016ea641cdbf9c4a8e08c96a67&scope.project.id=fc99d742ae004954b6d841293b5fa79c&role.id=08f16502894349708527e30032247468 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:16.396724 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7936737-2a30-4f78-ac04-062750e0025d None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:16.489649 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcedfd7c-97eb-44df-ad7e-01a3add98a94 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:16.532535 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1db439-dc61-4cec-986b-eb1de5ca0bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:16.532880 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7936737-2a30-4f78-ac04-062750e0025d None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:16.533652 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1db439-dc61-4cec-986b-eb1de5ca0bcc None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:16.534067 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1db439-dc61-4cec-986b-eb1de5ca0bcc None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:16.534423 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1db439-dc61-4cec-986b-eb1de5ca0bcc None admin] PATH_INFO: `/v3/domains` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:16.535384 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1db439-dc61-4cec-986b-eb1de5ca0bcc None admin] RBAC: Adding query filter params (name=Default) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:16.535695 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1db439-dc61-4cec-986b-eb1de5ca0bcc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:16.537357 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1db439-dc61-4cec-986b-eb1de5ca0bcc None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:16.555643 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 343/683] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:52:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:16.556774 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1db439-dc61-4cec-986b-eb1de5ca0bcc None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:16.574459 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:16.583769 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1dabed-4aeb-4efe-89c9-a59616a54dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:16.584465 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1dabed-4aeb-4efe-89c9-a59616a54dd3 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:16.584760 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1dabed-4aeb-4efe-89c9-a59616a54dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:16.585048 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1dabed-4aeb-4efe-89c9-a59616a54dd3 None admin] PATH_INFO: `/v3/domains` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:16.585832 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1dabed-4aeb-4efe-89c9-a59616a54dd3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:16.586784 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1dabed-4aeb-4efe-89c9-a59616a54dd3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:16.588525 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1dabed-4aeb-4efe-89c9-a59616a54dd3 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:16.606622 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 341/684] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:52:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:16.607484 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1dabed-4aeb-4efe-89c9-a59616a54dd3 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:16.620643 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:16.692089 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1db439-dc61-4cec-986b-eb1de5ca0bcc None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:16.737770 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464320ac-6738-42af-b010-dfb2fa24a3e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:16.739395 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464320ac-6738-42af-b010-dfb2fa24a3e3 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:16.739906 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464320ac-6738-42af-b010-dfb2fa24a3e3 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:16.740616 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464320ac-6738-42af-b010-dfb2fa24a3e3 None admin] PATH_INFO: `/v3/projects` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:16.741856 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464320ac-6738-42af-b010-dfb2fa24a3e3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:16.742332 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464320ac-6738-42af-b010-dfb2fa24a3e3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:16.744591 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464320ac-6738-42af-b010-dfb2fa24a3e3 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:16.753012 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1dabed-4aeb-4efe-89c9-a59616a54dd3 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:16.765096 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 344/685] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:52:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:16.766570 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464320ac-6738-42af-b010-dfb2fa24a3e3 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:16.791006 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c44af8-8431-499d-9d60-174b0a384197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:16.791693 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c44af8-8431-499d-9d60-174b0a384197 None admin] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:16.791993 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c44af8-8431-499d-9d60-174b0a384197 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:16.792273 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c44af8-8431-499d-9d60-174b0a384197 None admin] PATH_INFO: `/v3/users` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:16.792997 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c44af8-8431-499d-9d60-174b0a384197 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:16.794493 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c44af8-8431-499d-9d60-174b0a384197 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:16.820445 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 342/686] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 19 14:52:16 2022] POST /identity/v3/users => generated 254 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:52:16.821646 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c44af8-8431-499d-9d60-174b0a384197 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:16.871608 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:16.903837 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:16.922555 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464320ac-6738-42af-b010-dfb2fa24a3e3 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:16.958885 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c44af8-8431-499d-9d60-174b0a384197 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:16.969574 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d3414f-05f8-4fbe-9f31-dac2db128e83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:16.971033 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d3414f-05f8-4fbe-9f31-dac2db128e83 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:16.971481 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d3414f-05f8-4fbe-9f31-dac2db128e83 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:16.972003 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d3414f-05f8-4fbe-9f31-dac2db128e83 None admin] PATH_INFO: `/v3/users/swift` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:16.980943 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d3414f-05f8-4fbe-9f31-dac2db128e83 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:16.982650 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d3414f-05f8-4fbe-9f31-dac2db128e83 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:17.001836 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-21d3414f-05f8-4fbe-9f31-dac2db128e83 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 14:52:17.001836 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:17.001836 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:17.001836 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:17.001836 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:17.001836 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:17.001836 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:17.001836 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:17.001836 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:17.001836 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:17.001836 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:17.001836 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:17.001836 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:52:17.001836 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:52:17.001836 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:52:17.001836 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:52:17.003201 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:17.003201 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:17.003201 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:52:17.003201 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:52:17.003201 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:52:17.003201 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:52:17.003201 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:17.003201 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:17.003201 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:17.003201 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:17.003201 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:17.003201 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:52:17.003201 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:17.003201 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:17.003201 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:17.003201 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:17.003201 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:17.003201 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:17.004539 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:17.004539 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:17.004539 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:52:17.004539 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:52:17.004539 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:52:17.004539 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:52:17.004539 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:52:17.004539 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:52:17.004539 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 14:52:17.004539 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:52:17.011759 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 345/687] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:52:16 2022] GET /identity/v3/users/swift => generated 83 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:17.013344 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d3414f-05f8-4fbe-9f31-dac2db128e83 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:17.026633 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1debd5-97e1-4c27-9275-fb50b6e2bc37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:17.027331 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1debd5-97e1-4c27-9275-fb50b6e2bc37 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:17.027623 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1debd5-97e1-4c27-9275-fb50b6e2bc37 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:17.027908 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1debd5-97e1-4c27-9275-fb50b6e2bc37 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:17.028764 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1debd5-97e1-4c27-9275-fb50b6e2bc37 None admin] RBAC: Adding query filter params (role.id=08f16502894349708527e30032247468, scope.project.id=fc99d742ae004954b6d841293b5fa79c, user.id=7f75f8f59dcd42c480aac3eca18fc8db) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:17.029090 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1debd5-97e1-4c27-9275-fb50b6e2bc37 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:17.030627 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1debd5-97e1-4c27-9275-fb50b6e2bc37 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:17.049896 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 343/688] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:52:16 2022] GET /identity/v3/role_assignments?user.id=7f75f8f59dcd42c480aac3eca18fc8db&scope.project.id=fc99d742ae004954b6d841293b5fa79c&role.id=08f16502894349708527e30032247468 => generated 608 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:17.170100 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d3414f-05f8-4fbe-9f31-dac2db128e83 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:17.204939 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:17.211258 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c745a78-3d4c-4d8f-9b51-ee08a6aba150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:17.212208 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c745a78-3d4c-4d8f-9b51-ee08a6aba150 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:17.212679 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c745a78-3d4c-4d8f-9b51-ee08a6aba150 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:17.213136 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c745a78-3d4c-4d8f-9b51-ee08a6aba150 None admin] PATH_INFO: `/v3/users` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:17.214192 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c745a78-3d4c-4d8f-9b51-ee08a6aba150 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:17.214698 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c745a78-3d4c-4d8f-9b51-ee08a6aba150 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:17.216440 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c745a78-3d4c-4d8f-9b51-ee08a6aba150 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:17.241982 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 346/689] 10.222.0.57 () {56 vars in 1135 bytes} [Thu May 19 14:52:17 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:52:17.244415 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1debd5-97e1-4c27-9275-fb50b6e2bc37 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:17.370503 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1debd5-97e1-4c27-9275-fb50b6e2bc37 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:17.402083 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:17.415000 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d64cee9-ac34-4395-b7b1-d1efe8178a83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:17.415000 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d64cee9-ac34-4395-b7b1-d1efe8178a83 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:17.415000 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d64cee9-ac34-4395-b7b1-d1efe8178a83 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:17.415000 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d64cee9-ac34-4395-b7b1-d1efe8178a83 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:17.492258 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d64cee9-ac34-4395-b7b1-d1efe8178a83 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:17.494085 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d64cee9-ac34-4395-b7b1-d1efe8178a83 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:17.607747 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 344/690] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:52:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:52:17.609506 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c745a78-3d4c-4d8f-9b51-ee08a6aba150 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:17.766408 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c745a78-3d4c-4d8f-9b51-ee08a6aba150 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:17.811302 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1937a1-7714-4b30-bb0a-cfde006b179d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:17.812121 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1937a1-7714-4b30-bb0a-cfde006b179d None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:17.812418 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1937a1-7714-4b30-bb0a-cfde006b179d None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:17.812706 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1937a1-7714-4b30-bb0a-cfde006b179d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:17.824016 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1937a1-7714-4b30-bb0a-cfde006b179d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:17.825495 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1937a1-7714-4b30-bb0a-cfde006b179d None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:17.846079 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-de1937a1-7714-4b30-bb0a-cfde006b179d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:52:17.846079 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:17.846079 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:17.846079 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:17.846079 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:17.846079 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:17.846079 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:17.846079 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:17.846079 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:17.846079 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:17.846079 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:17.846079 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:17.846079 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:52:17.846079 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:52:17.846079 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:52:17.847251 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:52:17.847251 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:17.847251 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:17.847251 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:17.847251 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:17.847251 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:17.847251 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:17.847251 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:17.847251 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:52:17.847251 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:17.847251 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:17.847251 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:17.847251 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:17.847251 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:17.847251 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:17.847251 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:17.847251 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:17.848459 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:52:17.848459 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:52:17.848459 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:52:17.848459 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:52:17.848459 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 347/691] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:52:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:17.854643 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d64cee9-ac34-4395-b7b1-d1efe8178a83 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:17.906823 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:18.012506 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d64cee9-ac34-4395-b7b1-d1efe8178a83 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:18.067506 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac31255-c405-45a6-9955-c58b60a6a15d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:18.068197 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac31255-c405-45a6-9955-c58b60a6a15d None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:18.068493 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac31255-c405-45a6-9955-c58b60a6a15d None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:18.068778 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac31255-c405-45a6-9955-c58b60a6a15d None admin] PATH_INFO: `/v3/domains` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:18.069516 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac31255-c405-45a6-9955-c58b60a6a15d None admin] RBAC: Adding query filter params (name=Default) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:18.069832 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac31255-c405-45a6-9955-c58b60a6a15d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:18.071513 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac31255-c405-45a6-9955-c58b60a6a15d None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:18.077037 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5454ceb-9aa4-4edb-b505-bf28e01998b9 None None] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:18.077547 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5454ceb-9aa4-4edb-b505-bf28e01998b9 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:18.077635 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5454ceb-9aa4-4edb-b505-bf28e01998b9 None None] PATH_INFO: `/` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:18.079248 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 348/692] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:52:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:52:18.084920 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dbc1cb-f307-4508-9884-4811d3df9d7d None None] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:18.085495 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dbc1cb-f307-4508-9884-4811d3df9d7d None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:18.086086 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dbc1cb-f307-4508-9884-4811d3df9d7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:18.093894 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 345/693] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:52:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:18.099123 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac31255-c405-45a6-9955-c58b60a6a15d None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:18.110436 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.auth.core [None req-91dbc1cb-f307-4508-9884-4811d3df9d7d None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:52:18.111992 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-91dbc1cb-f307-4508-9884-4811d3df9d7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:52:18.170372 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:18.196862 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 349/694] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:52:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:52:18.219370 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dbc1cb-f307-4508-9884-4811d3df9d7d None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:18.254495 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:18.292410 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac31255-c405-45a6-9955-c58b60a6a15d None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:18.359760 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d40161f-2607-4c34-a88d-486ce8c8c266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:18.361341 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d40161f-2607-4c34-a88d-486ce8c8c266 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:18.361818 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d40161f-2607-4c34-a88d-486ce8c8c266 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:18.362134 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d40161f-2607-4c34-a88d-486ce8c8c266 None admin] PATH_INFO: `/v3/projects/service` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:18.374856 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d40161f-2607-4c34-a88d-486ce8c8c266 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:18.376680 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d40161f-2607-4c34-a88d-486ce8c8c266 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:18.379028 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dbc1cb-f307-4508-9884-4811d3df9d7d None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:18.404099 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-7d40161f-2607-4c34-a88d-486ce8c8c266 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:52:18.404099 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:18.404099 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:18.404099 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:18.404099 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:18.404099 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:18.404099 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:18.404099 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:18.404099 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:18.404099 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:18.404099 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:18.404099 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:18.404099 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:52:18.404099 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:52:18.404099 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:52:18.405164 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:52:18.405164 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:18.405164 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:18.405164 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:18.405164 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:18.405164 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:18.405164 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:18.405164 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:18.405164 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:52:18.405164 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:18.405164 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:18.405164 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:18.405164 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:18.405164 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:18.405164 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:18.405164 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:18.405164 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:18.406181 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:52:18.406181 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:52:18.406181 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:52:18.406181 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:52:18.406181 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:52:18.406181 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:52:18.406181 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:52:18.406181 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:52:18.407911 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 346/695] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:52:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:52:18.419114 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d40161f-2607-4c34-a88d-486ce8c8c266 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:18.436875 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aead17d-6866-4940-92e6-5019bddd393c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:18.438103 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aead17d-6866-4940-92e6-5019bddd393c None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:18.439235 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aead17d-6866-4940-92e6-5019bddd393c None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:18.439742 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aead17d-6866-4940-92e6-5019bddd393c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:18.451362 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aead17d-6866-4940-92e6-5019bddd393c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:52:18.459843 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-4aead17d-6866-4940-92e6-5019bddd393c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:52:18.459843 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:18.459843 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:18.459843 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:18.459843 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:18.459843 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:18.459843 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:18.459843 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:18.459843 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:18.459843 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:18.459843 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:18.459843 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:18.459843 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:52:18.459843 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:52:18.459843 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:52:18.459843 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:52:18.460942 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:52:18.460942 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:52:18.460942 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:52:18.460942 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:52:18.460942 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:18.460942 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:18.460942 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:18.460942 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:18.460942 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:18.460942 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:18.460942 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:18.460942 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:52:18.460942 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:18.460942 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:18.460942 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:18.460942 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:18.461930 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:18.461930 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:18.461930 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:18.461930 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:18.461930 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:52:18.461930 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:52:18.461930 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:52:18.461930 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:52:18.461930 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:52:18.461930 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:52:18.461930 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:52:18.461930 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:52:18.469580 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 350/696] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:52:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:52:18.471975 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aead17d-6866-4940-92e6-5019bddd393c None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:18.534241 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:18.625369 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d40161f-2607-4c34-a88d-486ce8c8c266 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:18.667907 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aead17d-6866-4940-92e6-5019bddd393c None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:18.696374 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1983f00-85c2-4304-8d42-42d14c9ed076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:18.697604 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1983f00-85c2-4304-8d42-42d14c9ed076 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:18.698159 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1983f00-85c2-4304-8d42-42d14c9ed076 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:18.698689 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1983f00-85c2-4304-8d42-42d14c9ed076 None admin] PATH_INFO: `/v3/projects` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:18.705748 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1983f00-85c2-4304-8d42-42d14c9ed076 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:18.705748 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1983f00-85c2-4304-8d42-42d14c9ed076 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:18.705748 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1983f00-85c2-4304-8d42-42d14c9ed076 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:18.709853 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9448484f-797d-4480-9bd7-bba3d4a0469d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:18.710550 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9448484f-797d-4480-9bd7-bba3d4a0469d None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:18.710843 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9448484f-797d-4480-9bd7-bba3d4a0469d None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:18.711130 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9448484f-797d-4480-9bd7-bba3d4a0469d None admin] PATH_INFO: `/v3/roles` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:18.711893 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9448484f-797d-4480-9bd7-bba3d4a0469d None admin] RBAC: Adding query filter params (name=admin) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:18.712210 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9448484f-797d-4480-9bd7-bba3d4a0469d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:18.713611 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9448484f-797d-4480-9bd7-bba3d4a0469d None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:18.729965 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 347/697] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:52:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:18.732137 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 351/698] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:52:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:52:18.737001 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1983f00-85c2-4304-8d42-42d14c9ed076 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:18.738880 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9448484f-797d-4480-9bd7-bba3d4a0469d None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:18.761031 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:18.868735 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:18.879985 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9448484f-797d-4480-9bd7-bba3d4a0469d None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:18.920768 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb02b5ba-2015-48a1-8cc6-07b0346f2e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:18.922098 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb02b5ba-2015-48a1-8cc6-07b0346f2e38 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:18.922610 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb02b5ba-2015-48a1-8cc6-07b0346f2e38 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:18.923762 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb02b5ba-2015-48a1-8cc6-07b0346f2e38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:18.939444 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb02b5ba-2015-48a1-8cc6-07b0346f2e38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:18.941396 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb02b5ba-2015-48a1-8cc6-07b0346f2e38 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:18.964458 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-fb02b5ba-2015-48a1-8cc6-07b0346f2e38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:52:18.964458 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:18.964458 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:18.964458 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:18.964458 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:18.964458 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:18.964458 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:18.964458 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:18.964458 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:18.964458 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:18.964458 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:18.964458 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:18.964458 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:52:18.964458 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:52:18.964458 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:52:18.965524 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:52:18.965524 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:18.965524 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:18.965524 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:18.965524 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:18.965524 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:18.965524 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:18.965524 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:18.965524 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:52:18.965524 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:18.965524 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:18.965524 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:18.965524 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:18.965524 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:18.965524 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:18.965524 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:18.965524 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:18.966556 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:52:18.966556 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:52:18.966556 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:52:18.966556 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:52:18.973718 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 352/699] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:52:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:18.976321 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb02b5ba-2015-48a1-8cc6-07b0346f2e38 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:19.015778 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1983f00-85c2-4304-8d42-42d14c9ed076 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:19.091970 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da4b1a6-d91c-4ef4-b0b5-6704699bcc92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:19.092647 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da4b1a6-d91c-4ef4-b0b5-6704699bcc92 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:19.092939 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da4b1a6-d91c-4ef4-b0b5-6704699bcc92 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:19.093225 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da4b1a6-d91c-4ef4-b0b5-6704699bcc92 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:19.094110 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da4b1a6-d91c-4ef4-b0b5-6704699bcc92 None admin] RBAC: Adding query filter params (role.id=08f16502894349708527e30032247468, scope.project.id=fc99d742ae004954b6d841293b5fa79c, user.id=17afb27b2bbe481e959cf31db4470593) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:19.094435 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da4b1a6-d91c-4ef4-b0b5-6704699bcc92 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:19.095970 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da4b1a6-d91c-4ef4-b0b5-6704699bcc92 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:19.097499 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb02b5ba-2015-48a1-8cc6-07b0346f2e38 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:19.109861 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 348/700] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:52:18 2022] GET /identity/v3/role_assignments?user.id=17afb27b2bbe481e959cf31db4470593&scope.project.id=fc99d742ae004954b6d841293b5fa79c&role.id=08f16502894349708527e30032247468 => generated 608 bytes in 374 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:19.138967 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58685a41-0067-4cfe-a2dc-0b7c4691bd66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:19.140065 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58685a41-0067-4cfe-a2dc-0b7c4691bd66 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:19.140608 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58685a41-0067-4cfe-a2dc-0b7c4691bd66 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:19.141125 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58685a41-0067-4cfe-a2dc-0b7c4691bd66 None admin] PATH_INFO: `/v3/domains` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:19.142883 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58685a41-0067-4cfe-a2dc-0b7c4691bd66 None admin] RBAC: Adding query filter params (name=Default) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:19.143461 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58685a41-0067-4cfe-a2dc-0b7c4691bd66 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:19.145261 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58685a41-0067-4cfe-a2dc-0b7c4691bd66 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:19.164948 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 353/701] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:52:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:19.166803 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da4b1a6-d91c-4ef4-b0b5-6704699bcc92 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:19.198189 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d24c88-6cda-473a-8369-c38bee2512ce None None] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:19.198921 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d24c88-6cda-473a-8369-c38bee2512ce None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:19.199415 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d24c88-6cda-473a-8369-c38bee2512ce None None] PATH_INFO: `/` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:19.201118 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 354/702] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:52:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:52:19.208475 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d69acd9-86af-4eb1-9919-155d5d038aa9 None None] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:19.209074 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d69acd9-86af-4eb1-9919-155d5d038aa9 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:19.209566 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d69acd9-86af-4eb1-9919-155d5d038aa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:19.224307 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:19.241278 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.auth.core [None req-0d69acd9-86af-4eb1-9919-155d5d038aa9 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:52:19.243420 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-0d69acd9-86af-4eb1-9919-155d5d038aa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:52:19.330928 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:19.333259 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 355/703] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:52:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:52:19.355919 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d69acd9-86af-4eb1-9919-155d5d038aa9 None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:19.393659 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da4b1a6-d91c-4ef4-b0b5-6704699bcc92 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:19.433475 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea60607-3b42-46d6-bacd-afeb38197783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:19.434501 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea60607-3b42-46d6-bacd-afeb38197783 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:19.434795 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea60607-3b42-46d6-bacd-afeb38197783 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:19.435076 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea60607-3b42-46d6-bacd-afeb38197783 None admin] PATH_INFO: `/v3/users/nova` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:19.448989 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea60607-3b42-46d6-bacd-afeb38197783 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:19.450667 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea60607-3b42-46d6-bacd-afeb38197783 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:19.468685 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-aea60607-3b42-46d6-bacd-afeb38197783 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 14:52:19.468685 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:19.468685 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:19.468685 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:19.468685 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:19.468685 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:19.468685 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:19.468685 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:19.468685 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:19.468685 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:19.468685 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:19.468685 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:19.468685 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:52:19.468685 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:52:19.468685 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:52:19.468685 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:52:19.470365 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:19.470365 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:19.470365 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:52:19.470365 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:52:19.470365 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:52:19.470365 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:52:19.470365 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:19.470365 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:19.470365 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:19.470365 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:19.470365 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:19.470365 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:52:19.470365 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:19.470365 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:19.470365 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:19.470365 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:19.470365 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:19.470365 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:19.471461 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:19.471461 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:19.471461 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:52:19.471461 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:52:19.471461 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:52:19.471461 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:52:19.471461 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:52:19.471461 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:52:19.471461 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 14:52:19.471461 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:52:19.479293 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 349/704] 10.222.0.57 () {56 vars in 1093 bytes} [Thu May 19 14:52:19 2022] GET /identity/v3/users/nova => generated 82 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:19.480731 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea60607-3b42-46d6-bacd-afeb38197783 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:19.489142 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d69acd9-86af-4eb1-9919-155d5d038aa9 None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:19.527715 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507f35bc-f6d3-4099-b60d-a832b4edfb04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:19.528719 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507f35bc-f6d3-4099-b60d-a832b4edfb04 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:19.529140 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507f35bc-f6d3-4099-b60d-a832b4edfb04 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:19.529786 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507f35bc-f6d3-4099-b60d-a832b4edfb04 None admin] PATH_INFO: `/v3/services/network` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:19.530334 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507f35bc-f6d3-4099-b60d-a832b4edfb04 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:19.531815 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507f35bc-f6d3-4099-b60d-a832b4edfb04 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:19.536812 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:19.551041 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-507f35bc-f6d3-4099-b60d-a832b4edfb04 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 19 14:52:19.551041 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:19.551041 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:52:19.551041 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:52:19.551041 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:52:19.551041 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:52:19.551041 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:52:19.551041 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:52:19.551041 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 14:52:19.551041 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:52:19.551041 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:52:19.551041 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:52:19.551041 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:19.551041 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:19.551041 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:19.551041 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:19.551041 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:19.552491 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:19.552491 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:19.552491 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:19.552491 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:19.552491 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:19.552491 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:19.552491 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:52:19.552491 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:52:19.552491 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:52:19.552491 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:52:19.552491 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:19.552491 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:19.552491 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:19.552491 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:19.552491 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:19.552491 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:19.553452 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:19.553452 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:52:19.553452 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:19.553452 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:19.553452 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:19.553452 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:19.553452 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:19.553452 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:19.553452 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:19.553452 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:19.553452 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:52:19.553452 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:52:19.553452 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 14:52:19.553452 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:52:19.554611 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 356/705] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:52:19 2022] GET /identity/v3/services/network => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:52:19.559261 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507f35bc-f6d3-4099-b60d-a832b4edfb04 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:19.602962 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea60607-3b42-46d6-bacd-afeb38197783 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:19.638407 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:19.653180 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebabc73-e20e-40fd-9aad-bea06f158fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:19.653180 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebabc73-e20e-40fd-9aad-bea06f158fd7 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:19.653180 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebabc73-e20e-40fd-9aad-bea06f158fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:19.653180 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebabc73-e20e-40fd-9aad-bea06f158fd7 None admin] PATH_INFO: `/v3/users` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:19.653657 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebabc73-e20e-40fd-9aad-bea06f158fd7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:19.653955 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebabc73-e20e-40fd-9aad-bea06f158fd7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:19.655431 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebabc73-e20e-40fd-9aad-bea06f158fd7 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:19.678589 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 350/706] 10.222.0.57 () {56 vars in 1133 bytes} [Thu May 19 14:52:19 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:52:19.684184 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebabc73-e20e-40fd-9aad-bea06f158fd7 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:19.686585 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507f35bc-f6d3-4099-b60d-a832b4edfb04 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:19.726398 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558e153c-44f8-4e88-882c-fd30d583360e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:19.727081 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558e153c-44f8-4e88-882c-fd30d583360e None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:19.727432 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558e153c-44f8-4e88-882c-fd30d583360e None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:19.727716 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558e153c-44f8-4e88-882c-fd30d583360e None admin] PATH_INFO: `/v3/services` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:19.728499 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558e153c-44f8-4e88-882c-fd30d583360e None admin] RBAC: Adding query filter params (name=network) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:19.728817 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558e153c-44f8-4e88-882c-fd30d583360e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:19.731098 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558e153c-44f8-4e88-882c-fd30d583360e None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:19.737091 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 357/707] 10.222.0.57 () {56 vars in 1112 bytes} [Thu May 19 14:52:19 2022] GET /identity/v3/services?name=network => generated 124 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:19.742315 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558e153c-44f8-4e88-882c-fd30d583360e None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:19.799451 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebabc73-e20e-40fd-9aad-bea06f158fd7 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:19.815665 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:19.836699 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627ad30a-bd24-4931-9f5c-2bd27e19c652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:19.837369 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627ad30a-bd24-4931-9f5c-2bd27e19c652 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:19.837663 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627ad30a-bd24-4931-9f5c-2bd27e19c652 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:19.837973 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627ad30a-bd24-4931-9f5c-2bd27e19c652 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:19.860301 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558e153c-44f8-4e88-882c-fd30d583360e None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:19.903291 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1aeb92-0b49-4eb8-bb8c-04f165d34b0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:19.903897 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1aeb92-0b49-4eb8-bb8c-04f165d34b0f None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:19.904243 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1aeb92-0b49-4eb8-bb8c-04f165d34b0f None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:19.904613 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:19.905248 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1aeb92-0b49-4eb8-bb8c-04f165d34b0f None admin] PATH_INFO: `/v3/services` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:19.906174 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1aeb92-0b49-4eb8-bb8c-04f165d34b0f None admin] RBAC: Adding query filter params (type=network) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:19.906494 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1aeb92-0b49-4eb8-bb8c-04f165d34b0f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:19.908736 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1aeb92-0b49-4eb8-bb8c-04f165d34b0f None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:19.915816 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 358/708] 10.222.0.57 () {56 vars in 1112 bytes} [Thu May 19 14:52:19 2022] GET /identity/v3/services?type=network => generated 124 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:52:19.917221 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627ad30a-bd24-4931-9f5c-2bd27e19c652 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:19.917487 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeed863-398a-4f53-9814-1c4a916d0e7b None None] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:19.917789 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeed863-398a-4f53-9814-1c4a916d0e7b None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:19.918144 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeed863-398a-4f53-9814-1c4a916d0e7b None None] PATH_INFO: `/` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:19.918746 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627ad30a-bd24-4931-9f5c-2bd27e19c652 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:19.919893 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 359/709] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:52:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:52:19.926393 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e254054-8e2c-47d4-bcc7-1c8c040e3b58 None None] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:19.926839 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e254054-8e2c-47d4-bcc7-1c8c040e3b58 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:19.927129 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e254054-8e2c-47d4-bcc7-1c8c040e3b58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:19.946951 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.auth.core [None req-8e254054-8e2c-47d4-bcc7-1c8c040e3b58 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:52:19.948476 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-8e254054-8e2c-47d4-bcc7-1c8c040e3b58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:52:20.027368 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 360/710] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:52:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:52:20.037400 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 351/711] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:52:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:52:20.043335 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e254054-8e2c-47d4-bcc7-1c8c040e3b58 None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:20.058101 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627ad30a-bd24-4931-9f5c-2bd27e19c652 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:20.130244 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:20.176020 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e254054-8e2c-47d4-bcc7-1c8c040e3b58 None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:20.190015 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:20.203296 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627ad30a-bd24-4931-9f5c-2bd27e19c652 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:20.220610 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b4aef2-279a-411e-ac16-e116ef109197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:20.221919 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b4aef2-279a-411e-ac16-e116ef109197 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:20.222395 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b4aef2-279a-411e-ac16-e116ef109197 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:20.223020 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b4aef2-279a-411e-ac16-e116ef109197 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:20.234818 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b4aef2-279a-411e-ac16-e116ef109197 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:20.236661 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b4aef2-279a-411e-ac16-e116ef109197 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:20.245690 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c863d8-03da-423f-9569-7d07960ff832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:20.246681 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c863d8-03da-423f-9569-7d07960ff832 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:20.246964 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c863d8-03da-423f-9569-7d07960ff832 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:20.247245 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c863d8-03da-423f-9569-7d07960ff832 None admin] PATH_INFO: `/v3/roles/service` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:20.257663 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-f3b4aef2-279a-411e-ac16-e116ef109197 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:52:20.257663 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:20.257663 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:20.257663 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:20.257663 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:20.257663 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:20.257663 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:20.257663 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:20.257663 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:20.257663 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:20.257663 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:20.257663 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:20.257663 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:52:20.257663 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:52:20.257663 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:52:20.258762 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:52:20.258762 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:20.258762 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:20.258762 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:20.258762 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:20.258762 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:20.258762 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:20.258762 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:20.258762 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:52:20.258762 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:20.258762 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:20.258762 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:20.258762 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:20.258762 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:20.258762 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:20.258762 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:20.258762 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:20.259757 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:52:20.259757 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:52:20.259757 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:52:20.259757 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:52:20.259757 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 361/712] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:52:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:20.261806 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35c62df-2bba-489f-8ed6-88f959199876 None None] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:20.261806 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35c62df-2bba-489f-8ed6-88f959199876 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:20.261806 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35c62df-2bba-489f-8ed6-88f959199876 None None] PATH_INFO: `/` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:20.262849 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 362/713] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:52:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:52:20.262984 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c863d8-03da-423f-9569-7d07960ff832 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:52:20.266538 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35c62df-2bba-489f-8ed6-88f959199876 None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:20.276050 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-90c863d8-03da-423f-9569-7d07960ff832 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:52:20.276050 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:20.276050 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:20.276050 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:20.276050 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:20.276050 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:20.276050 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:20.276050 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:20.276050 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:20.276050 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:20.276050 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:20.276050 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:20.276050 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:52:20.276050 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:52:20.276050 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:52:20.276050 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:52:20.278500 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:52:20.278500 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:52:20.278500 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:52:20.278500 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:52:20.278500 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:20.278500 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:20.278500 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:20.278500 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:20.278500 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:20.278500 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:20.278500 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:20.278500 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:52:20.278500 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:20.278500 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:20.278500 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:20.278500 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:20.280667 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:20.280667 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:20.280667 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:20.280667 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:20.280667 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:52:20.280667 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:52:20.280667 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:52:20.280667 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:52:20.280667 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:52:20.280667 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:52:20.280667 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:52:20.280667 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:52:20.286604 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 352/714] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 14:52:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:20.286604 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7b6d70-df11-4d45-9251-56574fa3ad2f None None] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:20.286604 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7b6d70-df11-4d45-9251-56574fa3ad2f None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:20.286604 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7b6d70-df11-4d45-9251-56574fa3ad2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:20.308948 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.auth.core [None req-3d7b6d70-df11-4d45-9251-56574fa3ad2f None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:52:20.310175 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.fernet_utils [None req-3d7b6d70-df11-4d45-9251-56574fa3ad2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:52:20.378718 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35c62df-2bba-489f-8ed6-88f959199876 None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:20.392119 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:20.399551 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 353/715] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:52:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:52:20.401126 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7b6d70-df11-4d45-9251-56574fa3ad2f None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:20.419256 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4de8c71-89fa-4352-a960-9806196e4291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:20.420276 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4de8c71-89fa-4352-a960-9806196e4291 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:20.420802 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4de8c71-89fa-4352-a960-9806196e4291 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:20.421463 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4de8c71-89fa-4352-a960-9806196e4291 None admin] PATH_INFO: `/v3/domains` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:20.422548 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4de8c71-89fa-4352-a960-9806196e4291 None admin] RBAC: Adding query filter params (name=Default) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:20.423133 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4de8c71-89fa-4352-a960-9806196e4291 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:20.424731 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4de8c71-89fa-4352-a960-9806196e4291 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:20.443461 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 363/716] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:52:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:20.444660 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4de8c71-89fa-4352-a960-9806196e4291 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:20.503001 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:20.535680 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7b6d70-df11-4d45-9251-56574fa3ad2f None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:20.573252 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50345411-2970-44b4-bd9b-d6de8a664d4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:20.574351 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50345411-2970-44b4-bd9b-d6de8a664d4e None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:20.574821 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50345411-2970-44b4-bd9b-d6de8a664d4e None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:20.575002 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4de8c71-89fa-4352-a960-9806196e4291 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:20.575459 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50345411-2970-44b4-bd9b-d6de8a664d4e None admin] PATH_INFO: `/v3/roles` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:20.576412 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50345411-2970-44b4-bd9b-d6de8a664d4e None admin] RBAC: Adding query filter params (name=service) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:20.576913 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50345411-2970-44b4-bd9b-d6de8a664d4e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:20.578518 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50345411-2970-44b4-bd9b-d6de8a664d4e None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:20.594284 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 354/717] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 19 14:52:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:20.594284 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50345411-2970-44b4-bd9b-d6de8a664d4e None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:20.618894 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee269097-6780-463a-bee2-e6b57c9d9156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:20.619761 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee269097-6780-463a-bee2-e6b57c9d9156 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:20.620067 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee269097-6780-463a-bee2-e6b57c9d9156 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:20.620380 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee269097-6780-463a-bee2-e6b57c9d9156 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:20.621047 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee269097-6780-463a-bee2-e6b57c9d9156 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:20.622605 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee269097-6780-463a-bee2-e6b57c9d9156 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:20.635646 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-ee269097-6780-463a-bee2-e6b57c9d9156 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 14:52:20.635646 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:20.635646 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:52:20.635646 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:52:20.635646 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:52:20.635646 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:52:20.635646 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:52:20.635646 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:52:20.635646 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 14:52:20.635646 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:52:20.635646 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:52:20.635646 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:52:20.635646 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:20.635646 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:20.635646 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:20.635646 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:20.637503 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:20.637503 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:20.637503 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:20.637503 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:20.637503 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:20.637503 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:20.637503 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:20.637503 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:52:20.637503 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:52:20.637503 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:52:20.637503 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:52:20.637503 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:20.637503 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:20.637503 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:20.637503 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:20.637503 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:20.643907 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:20.643907 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:20.643907 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:52:20.643907 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:20.643907 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:20.643907 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:20.643907 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:20.643907 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:20.643907 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:20.643907 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:20.643907 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:20.643907 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:52:20.643907 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:52:20.643907 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 14:52:20.643907 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:52:20.643907 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 364/718] 10.222.0.57 () {56 vars in 1129 bytes} [Thu May 19 14:52:20 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:20.644869 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee269097-6780-463a-bee2-e6b57c9d9156 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:20.686669 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:20.753061 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50345411-2970-44b4-bd9b-d6de8a664d4e None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:20.760600 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee269097-6780-463a-bee2-e6b57c9d9156 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:20.796923 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fd9107-d66c-4e5c-96fd-83e9d4e31d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:20.798551 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fd9107-d66c-4e5c-96fd-83e9d4e31d7f None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:20.798905 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fd9107-d66c-4e5c-96fd-83e9d4e31d7f None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:20.799251 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fd9107-d66c-4e5c-96fd-83e9d4e31d7f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:20.800920 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90dc180-69fe-4a52-a1df-b8dc306410d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:20.802290 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90dc180-69fe-4a52-a1df-b8dc306410d0 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:20.803388 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90dc180-69fe-4a52-a1df-b8dc306410d0 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:20.803884 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90dc180-69fe-4a52-a1df-b8dc306410d0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:20.811790 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fd9107-d66c-4e5c-96fd-83e9d4e31d7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:20.813032 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90dc180-69fe-4a52-a1df-b8dc306410d0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:20.813424 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fd9107-d66c-4e5c-96fd-83e9d4e31d7f None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:20.815152 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90dc180-69fe-4a52-a1df-b8dc306410d0 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:20.833114 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-f0fd9107-d66c-4e5c-96fd-83e9d4e31d7f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:52:20.833114 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:20.833114 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:20.833114 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:20.833114 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:20.833114 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:20.833114 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:20.833114 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:20.833114 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:20.833114 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:20.833114 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:20.833114 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:20.833114 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:52:20.833114 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:52:20.833114 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:52:20.834192 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:52:20.834192 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:20.834192 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:20.834192 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:20.834192 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:20.834192 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:20.834192 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:20.834192 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:20.834192 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:52:20.834192 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:20.834192 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:20.834192 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:20.834192 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:20.834192 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:20.834192 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:20.834192 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:20.834192 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:20.835207 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:52:20.835207 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:52:20.835207 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:52:20.835207 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:52:20.835852 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-f90dc180-69fe-4a52-a1df-b8dc306410d0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:52:20.835852 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:20.835852 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:20.835852 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:20.835852 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:20.835852 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:20.835852 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:20.835852 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:20.835852 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:20.835852 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:20.835852 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:20.835852 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:20.835852 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:52:20.835852 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:52:20.835852 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:52:20.836757 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:52:20.836757 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:20.836757 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:20.836757 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:20.836757 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:20.836757 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:20.836757 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:20.836757 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:20.836757 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:52:20.836757 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:20.836757 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:20.836757 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:20.836757 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:20.836757 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:20.836757 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:20.836757 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:20.836757 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:20.838213 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:52:20.838213 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:52:20.838213 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:52:20.838213 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:52:20.838213 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:52:20.838213 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:52:20.838213 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:52:20.838213 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:52:20.839248 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 365/719] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:52:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:20.840173 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fd9107-d66c-4e5c-96fd-83e9d4e31d7f None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:20.841002 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 355/720] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:52:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:20.848463 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90dc180-69fe-4a52-a1df-b8dc306410d0 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:20.852762 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:20.966386 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90dc180-69fe-4a52-a1df-b8dc306410d0 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:20.972952 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:20.991753 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fd9107-d66c-4e5c-96fd-83e9d4e31d7f None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:21.011845 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9e3188-85aa-4243-b39e-7a6933097d6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:21.012444 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9e3188-85aa-4243-b39e-7a6933097d6b None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:21.012736 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9e3188-85aa-4243-b39e-7a6933097d6b None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:21.013021 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9e3188-85aa-4243-b39e-7a6933097d6b None admin] PATH_INFO: `/v3/domains` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:21.013760 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9e3188-85aa-4243-b39e-7a6933097d6b None admin] RBAC: Adding query filter params (name=Default) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:21.014121 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9e3188-85aa-4243-b39e-7a6933097d6b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:21.016248 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9e3188-85aa-4243-b39e-7a6933097d6b None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:21.029512 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5994d9e2-7f9e-4ee2-bd46-b49e12896d5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:21.030455 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5994d9e2-7f9e-4ee2-bd46-b49e12896d5d None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:21.030894 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5994d9e2-7f9e-4ee2-bd46-b49e12896d5d None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:21.031090 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 356/721] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:52:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:21.031587 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5994d9e2-7f9e-4ee2-bd46-b49e12896d5d None admin] PATH_INFO: `/v3/services` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:21.031916 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9e3188-85aa-4243-b39e-7a6933097d6b None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:21.033783 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5994d9e2-7f9e-4ee2-bd46-b49e12896d5d None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:21.034348 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5994d9e2-7f9e-4ee2-bd46-b49e12896d5d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:21.035941 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5994d9e2-7f9e-4ee2-bd46-b49e12896d5d None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:21.041289 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 366/722] 10.222.0.57 () {56 vars in 1124 bytes} [Thu May 19 14:52:20 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:21.042366 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5994d9e2-7f9e-4ee2-bd46-b49e12896d5d None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:21.119297 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:21.155193 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5994d9e2-7f9e-4ee2-bd46-b49e12896d5d None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:21.173096 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9e3188-85aa-4243-b39e-7a6933097d6b None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:21.193813 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad6a7a2-b821-45b2-bad5-fd540717ac31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:21.194978 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad6a7a2-b821-45b2-bad5-fd540717ac31 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:21.195751 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad6a7a2-b821-45b2-bad5-fd540717ac31 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:21.196244 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad6a7a2-b821-45b2-bad5-fd540717ac31 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:21.205212 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad6a7a2-b821-45b2-bad5-fd540717ac31 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:21.207039 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad6a7a2-b821-45b2-bad5-fd540717ac31 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:21.215054 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ad7551-18c4-4a03-950f-22216641b399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:21.215725 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ad7551-18c4-4a03-950f-22216641b399 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:21.216016 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ad7551-18c4-4a03-950f-22216641b399 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:21.216297 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ad7551-18c4-4a03-950f-22216641b399 None admin] PATH_INFO: `/v3/projects` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:21.217091 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ad7551-18c4-4a03-950f-22216641b399 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:21.217407 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ad7551-18c4-4a03-950f-22216641b399 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:21.218880 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ad7551-18c4-4a03-950f-22216641b399 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:21.224482 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-2ad6a7a2-b821-45b2-bad5-fd540717ac31 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 14:52:21.224482 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:21.224482 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:21.224482 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:21.224482 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:21.224482 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:21.224482 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:21.224482 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:21.224482 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:21.224482 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:21.224482 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:21.224482 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:21.224482 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:52:21.224482 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:52:21.224482 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:52:21.224482 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:52:21.225515 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:21.225515 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:21.225515 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:52:21.225515 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:52:21.225515 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:52:21.225515 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:52:21.225515 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:21.225515 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:21.225515 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:21.225515 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:21.225515 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:21.225515 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:52:21.225515 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:21.225515 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:21.225515 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:21.225515 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:21.225515 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:21.225515 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:21.226930 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:21.226930 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:21.226930 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:52:21.226930 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:52:21.226930 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:52:21.226930 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:52:21.226930 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:52:21.226930 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:52:21.226930 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 14:52:21.226930 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:52:21.229398 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 367/723] 10.222.0.57 () {56 vars in 1117 bytes} [Thu May 19 14:52:21 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:21.230505 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad6a7a2-b821-45b2-bad5-fd540717ac31 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:21.232947 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 357/724] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:52:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:21.240710 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ad7551-18c4-4a03-950f-22216641b399 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:21.252695 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:21.359854 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ad7551-18c4-4a03-950f-22216641b399 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:21.370933 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad6a7a2-b821-45b2-bad5-fd540717ac31 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:21.389187 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:21.404820 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dade2d8a-cce0-4cfb-a9de-9241a804d552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:21.406367 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dade2d8a-cce0-4cfb-a9de-9241a804d552 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:21.407025 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dade2d8a-cce0-4cfb-a9de-9241a804d552 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:21.408245 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dade2d8a-cce0-4cfb-a9de-9241a804d552 None admin] PATH_INFO: `/v3/users` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:21.409316 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dade2d8a-cce0-4cfb-a9de-9241a804d552 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:21.410006 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dade2d8a-cce0-4cfb-a9de-9241a804d552 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:21.411824 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dade2d8a-cce0-4cfb-a9de-9241a804d552 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:21.412852 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f012292-2f7d-436f-9825-9759fa7871a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:21.413527 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f012292-2f7d-436f-9825-9759fa7871a7 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:21.413823 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f012292-2f7d-436f-9825-9759fa7871a7 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:21.414142 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f012292-2f7d-436f-9825-9759fa7871a7 None admin] PATH_INFO: `/v3/services` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:21.415103 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f012292-2f7d-436f-9825-9759fa7871a7 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:21.415424 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f012292-2f7d-436f-9825-9759fa7871a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:21.417618 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f012292-2f7d-436f-9825-9759fa7871a7 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:21.425931 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 368/725] 10.222.0.57 () {56 vars in 1124 bytes} [Thu May 19 14:52:21 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:21.427188 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f012292-2f7d-436f-9825-9759fa7871a7 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:21.451728 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 358/726] 10.222.0.57 () {56 vars in 1149 bytes} [Thu May 19 14:52:21 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:21.453091 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dade2d8a-cce0-4cfb-a9de-9241a804d552 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:21.621814 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:21.665371 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dade2d8a-cce0-4cfb-a9de-9241a804d552 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:21.683532 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f012292-2f7d-436f-9825-9759fa7871a7 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:21.723833 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c648cfe-a953-4316-adb3-8185a1c8aa28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:21.724506 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c648cfe-a953-4316-adb3-8185a1c8aa28 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:21.724795 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c648cfe-a953-4316-adb3-8185a1c8aa28 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:21.725106 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c648cfe-a953-4316-adb3-8185a1c8aa28 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:21.737485 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b4a2b7-7a17-440c-a79c-43921993fafa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:21.738606 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b4a2b7-7a17-440c-a79c-43921993fafa None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:21.739236 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b4a2b7-7a17-440c-a79c-43921993fafa None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:21.739729 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b4a2b7-7a17-440c-a79c-43921993fafa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:21.740959 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b4a2b7-7a17-440c-a79c-43921993fafa None admin] RBAC: Adding query filter params (role.id=028379e01f9245e9b8fc56e8b9e8a6ad, scope.project.id=fc99d742ae004954b6d841293b5fa79c, user.id=5d2aa8c8870a49d98f23f26e036695ef) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:21.741507 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b4a2b7-7a17-440c-a79c-43921993fafa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:21.743321 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b4a2b7-7a17-440c-a79c-43921993fafa None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:21.757102 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 369/727] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:52:21 2022] GET /identity/v3/role_assignments?user.id=5d2aa8c8870a49d98f23f26e036695ef&scope.project.id=fc99d742ae004954b6d841293b5fa79c&role.id=028379e01f9245e9b8fc56e8b9e8a6ad => generated 259 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:21.794897 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:21.820689 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c648cfe-a953-4316-adb3-8185a1c8aa28 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:21.822238 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c648cfe-a953-4316-adb3-8185a1c8aa28 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:21.956438 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 359/728] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:52:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 504 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:52:21.962296 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b4a2b7-7a17-440c-a79c-43921993fafa None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:22.178154 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad24b0eb-f563-4165-8c30-3eaed7ad59ec None None] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:22.178581 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad24b0eb-f563-4165-8c30-3eaed7ad59ec None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:22.178782 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad24b0eb-f563-4165-8c30-3eaed7ad59ec None None] PATH_INFO: `/` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:22.180364 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 360/729] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:52:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:52:22.185833 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357b593d-8b6d-4d29-8355-08d0a041db0b None None] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:22.186211 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357b593d-8b6d-4d29-8355-08d0a041db0b None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:22.186501 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357b593d-8b6d-4d29-8355-08d0a041db0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:22.233570 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.auth.core [None req-357b593d-8b6d-4d29-8355-08d0a041db0b None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:52:22.237232 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.fernet_utils [None req-357b593d-8b6d-4d29-8355-08d0a041db0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:52:22.254685 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b4a2b7-7a17-440c-a79c-43921993fafa None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:22.323651 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cd0957-3400-4321-8e76-e12a2dac192b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:22.324460 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cd0957-3400-4321-8e76-e12a2dac192b None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:22.324754 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cd0957-3400-4321-8e76-e12a2dac192b None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:22.325020 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cd0957-3400-4321-8e76-e12a2dac192b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:22.337912 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:22.340469 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 361/730] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:52:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:52:22.343809 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cd0957-3400-4321-8e76-e12a2dac192b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:22.347395 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cd0957-3400-4321-8e76-e12a2dac192b None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:22.366133 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-87cd0957-3400-4321-8e76-e12a2dac192b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:52:22.366133 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:22.366133 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:22.366133 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:22.366133 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:22.366133 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:22.366133 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:22.366133 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:22.366133 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:22.366133 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:22.366133 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:22.366133 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:22.366133 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:52:22.366133 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:52:22.366133 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:52:22.367171 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:52:22.367171 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:22.367171 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:22.367171 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:22.367171 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:22.367171 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:22.367171 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:22.367171 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:22.367171 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:52:22.367171 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:22.367171 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:22.367171 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:22.367171 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:22.367171 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:22.367171 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:22.367171 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:22.367171 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:22.368166 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:52:22.368166 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:52:22.368166 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:52:22.368166 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:52:22.370511 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 370/731] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:52:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 409 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:22.376824 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357b593d-8b6d-4d29-8355-08d0a041db0b None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:22.388934 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cd0957-3400-4321-8e76-e12a2dac192b None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:22.397877 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:22.500986 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cd0957-3400-4321-8e76-e12a2dac192b None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:22.513972 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357b593d-8b6d-4d29-8355-08d0a041db0b None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:22.540705 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39aba4e0-be3e-461f-a1fa-6d89788dc35d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:22.541960 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aba4e0-be3e-461f-a1fa-6d89788dc35d None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:22.542469 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aba4e0-be3e-461f-a1fa-6d89788dc35d None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:22.542961 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aba4e0-be3e-461f-a1fa-6d89788dc35d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:22.553695 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a6560d-f91f-4c8c-b660-67216d186414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:22.554416 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a6560d-f91f-4c8c-b660-67216d186414 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:22.554706 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a6560d-f91f-4c8c-b660-67216d186414 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:22.554988 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a6560d-f91f-4c8c-b660-67216d186414 None admin] PATH_INFO: `/v3/domains` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:22.555731 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a6560d-f91f-4c8c-b660-67216d186414 None admin] RBAC: Adding query filter params (name=Default) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:22.556044 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a6560d-f91f-4c8c-b660-67216d186414 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:22.557551 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a6560d-f91f-4c8c-b660-67216d186414 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:22.558369 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39aba4e0-be3e-461f-a1fa-6d89788dc35d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:52:22.566471 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-39aba4e0-be3e-461f-a1fa-6d89788dc35d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:52:22.566471 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:22.566471 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:22.566471 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:22.566471 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:22.566471 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:22.566471 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:22.566471 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:22.566471 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:22.566471 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:22.566471 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:22.566471 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:22.566471 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:52:22.566471 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:52:22.566471 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:52:22.566471 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:52:22.567564 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:52:22.567564 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:52:22.567564 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:52:22.567564 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:52:22.567564 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:22.567564 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:22.567564 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:22.567564 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:22.567564 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:22.567564 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:22.567564 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:22.567564 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:52:22.567564 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:22.567564 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:22.567564 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:22.567564 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:22.568516 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:22.568516 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:22.568516 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:22.568516 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:22.568516 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:52:22.568516 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:52:22.568516 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:52:22.568516 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:52:22.568516 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:52:22.568516 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:52:22.568516 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:52:22.568516 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:52:22.575245 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 362/732] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:52:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:22.575245 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f56320-c760-45a4-b1f7-bfb3be18d1b3 None None] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:22.575245 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f56320-c760-45a4-b1f7-bfb3be18d1b3 None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:22.575245 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f56320-c760-45a4-b1f7-bfb3be18d1b3 None None] PATH_INFO: `/` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:22.576534 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 371/733] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:52:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:22.578814 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 363/734] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:52:22 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:52:22.580078 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39aba4e0-be3e-461f-a1fa-6d89788dc35d None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:22.580831 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddbd112-bdf0-4329-83f5-754f4e337bfc None None] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:22.581231 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddbd112-bdf0-4329-83f5-754f4e337bfc None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:22.581618 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddbd112-bdf0-4329-83f5-754f4e337bfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:22.601121 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.auth.core [None req-0ddbd112-bdf0-4329-83f5-754f4e337bfc None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:52:22.602676 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.fernet_utils [None req-0ddbd112-bdf0-4329-83f5-754f4e337bfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:52:22.639620 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:22.682237 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:22.687538 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 364/735] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:52:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:52:22.688416 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddbd112-bdf0-4329-83f5-754f4e337bfc None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:22.700681 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39aba4e0-be3e-461f-a1fa-6d89788dc35d None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:22.739395 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8e4249-89b2-4231-a411-2dae517ac74d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:22.740613 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8e4249-89b2-4231-a411-2dae517ac74d None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:22.741115 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8e4249-89b2-4231-a411-2dae517ac74d None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:22.741604 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8e4249-89b2-4231-a411-2dae517ac74d None admin] PATH_INFO: `/v3/projects/service` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:22.749372 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8e4249-89b2-4231-a411-2dae517ac74d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:22.751946 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8e4249-89b2-4231-a411-2dae517ac74d None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:22.767476 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-fc8e4249-89b2-4231-a411-2dae517ac74d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:52:22.767476 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:22.767476 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:22.767476 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:22.767476 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:22.767476 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:22.767476 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:22.767476 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:22.767476 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:22.767476 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:22.767476 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:22.767476 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:22.767476 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:52:22.767476 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:52:22.767476 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:52:22.768646 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:52:22.768646 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:22.768646 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:22.768646 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:22.768646 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:22.768646 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:22.768646 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:22.768646 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:22.768646 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:52:22.768646 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:22.768646 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:22.768646 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:22.768646 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:22.768646 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:22.768646 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:22.768646 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:22.768646 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:22.769678 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:52:22.769678 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:52:22.769678 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:52:22.769678 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:52:22.769678 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:52:22.769678 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:52:22.769678 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:52:22.769678 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:52:22.772382 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 372/736] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:52:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:22.777873 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8e4249-89b2-4231-a411-2dae517ac74d None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:22.819782 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddbd112-bdf0-4329-83f5-754f4e337bfc None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:22.859729 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f976a9-41d0-4a2e-bc54-f41b5bc389d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:22.861045 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f976a9-41d0-4a2e-bc54-f41b5bc389d2 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:22.861259 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f976a9-41d0-4a2e-bc54-f41b5bc389d2 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:22.861549 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f976a9-41d0-4a2e-bc54-f41b5bc389d2 None admin] PATH_INFO: `/v3/roles` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:22.862794 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f976a9-41d0-4a2e-bc54-f41b5bc389d2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:22.862794 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f976a9-41d0-4a2e-bc54-f41b5bc389d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:22.864281 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f976a9-41d0-4a2e-bc54-f41b5bc389d2 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:22.874977 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 365/737] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:52:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:22.876811 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f976a9-41d0-4a2e-bc54-f41b5bc389d2 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:22.894113 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8e4249-89b2-4231-a411-2dae517ac74d None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:22.907067 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:22.933547 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb1c047-9423-466d-83e4-6de9fce9a183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:22.934671 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb1c047-9423-466d-83e4-6de9fce9a183 None admin] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:22.935242 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb1c047-9423-466d-83e4-6de9fce9a183 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:22.935751 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb1c047-9423-466d-83e4-6de9fce9a183 None admin] PATH_INFO: `/v3/services` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:22.936709 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb1c047-9423-466d-83e4-6de9fce9a183 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:22.939690 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb1c047-9423-466d-83e4-6de9fce9a183 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:22.951487 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:22.964195 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 373/738] 10.222.0.57 () {60 vars in 1141 bytes} [Thu May 19 14:52:22 2022] POST /identity/v3/services => generated 240 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:52:22.965735 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb1c047-9423-466d-83e4-6de9fce9a183 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:23.044440 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f976a9-41d0-4a2e-bc54-f41b5bc389d2 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:23.146684 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbab9b1-9d80-4b1a-8679-afcd0824380e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:23.147839 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbab9b1-9d80-4b1a-8679-afcd0824380e None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:23.148344 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbab9b1-9d80-4b1a-8679-afcd0824380e None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:23.154453 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbab9b1-9d80-4b1a-8679-afcd0824380e None admin] PATH_INFO: `/v3/projects` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:23.155669 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbab9b1-9d80-4b1a-8679-afcd0824380e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:23.156214 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbab9b1-9d80-4b1a-8679-afcd0824380e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:23.158049 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb1c047-9423-466d-83e4-6de9fce9a183 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:23.158807 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbab9b1-9d80-4b1a-8679-afcd0824380e None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:23.185372 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 366/739] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:52:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:23.191691 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbab9b1-9d80-4b1a-8679-afcd0824380e None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:23.201264 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fc04ed-99a8-42bf-bcaa-afa7a5857f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:23.202341 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fc04ed-99a8-42bf-bcaa-afa7a5857f7b None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:23.202761 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fc04ed-99a8-42bf-bcaa-afa7a5857f7b None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:23.203146 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fc04ed-99a8-42bf-bcaa-afa7a5857f7b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:23.217188 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fc04ed-99a8-42bf-bcaa-afa7a5857f7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:23.219823 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fc04ed-99a8-42bf-bcaa-afa7a5857f7b None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:23.242044 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-28fc04ed-99a8-42bf-bcaa-afa7a5857f7b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:52:23.242044 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:23.242044 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:23.242044 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:23.242044 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:23.242044 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:23.242044 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:23.242044 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:23.242044 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:23.242044 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:23.242044 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:23.242044 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:23.242044 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:52:23.242044 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:52:23.242044 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:52:23.243301 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:52:23.243301 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:23.243301 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:23.243301 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:23.243301 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:23.243301 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:23.243301 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:23.243301 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:23.243301 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:52:23.243301 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:23.243301 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:23.243301 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:23.243301 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:23.243301 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:23.243301 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:23.243301 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:23.243301 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:23.244592 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:52:23.244592 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:52:23.244592 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:52:23.244592 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:52:23.249884 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 374/740] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:52:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:23.252401 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fc04ed-99a8-42bf-bcaa-afa7a5857f7b None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:23.287239 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:23.370048 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbab9b1-9d80-4b1a-8679-afcd0824380e None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:23.379414 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fc04ed-99a8-42bf-bcaa-afa7a5857f7b None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:23.393014 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:23.422946 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1627da8-130d-4fb0-9fb0-e407cc20469b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:23.423627 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1627da8-130d-4fb0-9fb0-e407cc20469b None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:23.423918 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1627da8-130d-4fb0-9fb0-e407cc20469b None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:23.424203 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1627da8-130d-4fb0-9fb0-e407cc20469b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:23.425051 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1627da8-130d-4fb0-9fb0-e407cc20469b None admin] RBAC: Adding query filter params (role.id=08f16502894349708527e30032247468, scope.project.id=fc99d742ae004954b6d841293b5fa79c, user.id=cd0da70fc3c1477d8202726187ab5fde) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:23.425371 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1627da8-130d-4fb0-9fb0-e407cc20469b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:23.426840 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1627da8-130d-4fb0-9fb0-e407cc20469b None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:23.429588 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd09f852-7e38-40d8-8bbb-eb1dc4770748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:23.430617 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd09f852-7e38-40d8-8bbb-eb1dc4770748 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:23.431126 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd09f852-7e38-40d8-8bbb-eb1dc4770748 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:23.431610 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd09f852-7e38-40d8-8bbb-eb1dc4770748 None admin] PATH_INFO: `/v3/domains` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:23.432567 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd09f852-7e38-40d8-8bbb-eb1dc4770748 None admin] RBAC: Adding query filter params (name=Default) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:23.433097 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd09f852-7e38-40d8-8bbb-eb1dc4770748 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:23.433334 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 367/741] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:52:23 2022] GET /identity/v3/role_assignments?user.id=cd0da70fc3c1477d8202726187ab5fde&scope.project.id=fc99d742ae004954b6d841293b5fa79c&role.id=08f16502894349708527e30032247468 => generated 259 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:23.437056 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd09f852-7e38-40d8-8bbb-eb1dc4770748 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:23.458036 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 375/742] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:52:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:23.459816 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1627da8-130d-4fb0-9fb0-e407cc20469b None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:23.612035 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1627da8-130d-4fb0-9fb0-e407cc20469b None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:23.653475 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32cc984-2527-43d1-9323-ac0442aa62fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:23.654322 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32cc984-2527-43d1-9323-ac0442aa62fd None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:23.654612 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32cc984-2527-43d1-9323-ac0442aa62fd None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:23.654892 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32cc984-2527-43d1-9323-ac0442aa62fd None admin] PATH_INFO: `/v3/users/swift` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:23.663741 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32cc984-2527-43d1-9323-ac0442aa62fd None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:23.671335 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32cc984-2527-43d1-9323-ac0442aa62fd None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:23.696062 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-b32cc984-2527-43d1-9323-ac0442aa62fd None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 14:52:23.696062 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:23.696062 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:23.696062 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:23.696062 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:23.696062 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:23.696062 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:23.696062 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:23.696062 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:23.696062 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:23.696062 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:23.696062 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:23.696062 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:52:23.696062 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:52:23.696062 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:52:23.696062 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:52:23.697173 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:23.697173 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:23.697173 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:52:23.697173 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:52:23.697173 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:52:23.697173 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:52:23.697173 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:23.697173 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:23.697173 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:23.697173 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:23.697173 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:23.697173 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:52:23.697173 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:23.697173 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:23.697173 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:23.697173 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:23.697173 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:23.697173 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:23.698264 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:23.698264 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:23.698264 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:52:23.698264 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:52:23.698264 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:52:23.698264 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:52:23.698264 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:52:23.698264 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:52:23.698264 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 14:52:23.698264 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:52:23.698264 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 368/743] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:52:23 2022] GET /identity/v3/users/swift => generated 83 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:23.707502 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd09f852-7e38-40d8-8bbb-eb1dc4770748 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:23.821680 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:23.841423 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd09f852-7e38-40d8-8bbb-eb1dc4770748 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:23.896811 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3244fab-29a0-4cdf-b639-46b801a71248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:23.897534 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3244fab-29a0-4cdf-b639-46b801a71248 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:23.897831 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3244fab-29a0-4cdf-b639-46b801a71248 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:23.898150 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3244fab-29a0-4cdf-b639-46b801a71248 None admin] PATH_INFO: `/v3/users` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:23.899037 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3244fab-29a0-4cdf-b639-46b801a71248 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:23.899361 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3244fab-29a0-4cdf-b639-46b801a71248 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:23.900866 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3244fab-29a0-4cdf-b639-46b801a71248 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:23.923032 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 376/744] 10.222.0.57 () {56 vars in 1135 bytes} [Thu May 19 14:52:23 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:23.928402 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32cc984-2527-43d1-9323-ac0442aa62fd None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:24.009813 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:24.095132 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32cc984-2527-43d1-9323-ac0442aa62fd None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:24.143585 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f673de71-ad39-43f5-8810-8ef5b32941da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:24.144254 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f673de71-ad39-43f5-8810-8ef5b32941da None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:24.144541 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f673de71-ad39-43f5-8810-8ef5b32941da None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:24.144819 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f673de71-ad39-43f5-8810-8ef5b32941da None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:24.242192 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683f7af7-7ec7-4792-b26b-66647cbe4f17 None None] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:24.242501 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683f7af7-7ec7-4792-b26b-66647cbe4f17 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:24.242697 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683f7af7-7ec7-4792-b26b-66647cbe4f17 None None] PATH_INFO: `/` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:24.246464 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f673de71-ad39-43f5-8810-8ef5b32941da None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:24.247055 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 377/745] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:52:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:52:24.247947 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f673de71-ad39-43f5-8810-8ef5b32941da None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:24.251224 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36283a40-0f57-4528-93b1-1615c054cb98 None None] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:24.251857 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36283a40-0f57-4528-93b1-1615c054cb98 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:24.252362 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36283a40-0f57-4528-93b1-1615c054cb98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:24.285992 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.auth.core [None req-36283a40-0f57-4528-93b1-1615c054cb98 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:52:24.287545 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-36283a40-0f57-4528-93b1-1615c054cb98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:52:24.294255 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:24.363616 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 369/746] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:52:23 2022] GET /identity/v3/auth/tokens => generated 898 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:52:24.369926 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f673de71-ad39-43f5-8810-8ef5b32941da None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:24.481101 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f673de71-ad39-43f5-8810-8ef5b32941da None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:24.511650 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 378/747] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:52:24 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:52:24.519315 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7438ae1e-795a-4366-809e-94e3ec5da865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:24.520337 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7438ae1e-795a-4366-809e-94e3ec5da865 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:24.520823 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7438ae1e-795a-4366-809e-94e3ec5da865 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:24.521263 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7438ae1e-795a-4366-809e-94e3ec5da865 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:24.532785 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7438ae1e-795a-4366-809e-94e3ec5da865 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:24.534551 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7438ae1e-795a-4366-809e-94e3ec5da865 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:24.542157 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:24.543399 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929862dc-a71b-4bfe-9825-a1b8faecd3d5 None None] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:24.543865 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929862dc-a71b-4bfe-9825-a1b8faecd3d5 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:24.544348 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929862dc-a71b-4bfe-9825-a1b8faecd3d5 None None] PATH_INFO: `/` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:24.550715 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 379/748] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:52:24 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:52:24.552596 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929862dc-a71b-4bfe-9825-a1b8faecd3d5 None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:24.557069 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-7438ae1e-795a-4366-809e-94e3ec5da865 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:52:24.557069 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:24.557069 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:24.557069 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:24.557069 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:24.557069 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:24.557069 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:24.557069 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:24.557069 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:24.557069 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:24.557069 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:24.557069 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:24.557069 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:52:24.557069 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:52:24.557069 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:52:24.558107 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:52:24.558107 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:24.558107 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:24.558107 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:24.558107 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:24.558107 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:24.558107 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:24.558107 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:24.558107 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:52:24.558107 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:24.558107 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:24.558107 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:24.558107 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:24.558107 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:24.558107 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:24.558107 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:24.558107 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:24.559123 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:52:24.559123 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:52:24.559123 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:52:24.559123 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:52:24.559123 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 370/749] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:52:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:24.559957 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0852ee77-743e-4af1-a0eb-d52eb3045e49 None None] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:24.560250 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0852ee77-743e-4af1-a0eb-d52eb3045e49 None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:24.560583 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0852ee77-743e-4af1-a0eb-d52eb3045e49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:24.581599 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.auth.core [None req-0852ee77-743e-4af1-a0eb-d52eb3045e49 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:52:24.583321 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.fernet_utils [None req-0852ee77-743e-4af1-a0eb-d52eb3045e49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:52:24.645071 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:24.665697 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 371/750] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:52:24 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:52:24.667179 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0852ee77-743e-4af1-a0eb-d52eb3045e49 None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:24.712103 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929862dc-a71b-4bfe-9825-a1b8faecd3d5 None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:24.757860 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfdae21-39c1-4f13-b0db-3e939413b2f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:24.759668 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfdae21-39c1-4f13-b0db-3e939413b2f4 None admin] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:24.760206 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfdae21-39c1-4f13-b0db-3e939413b2f4 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:24.760715 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfdae21-39c1-4f13-b0db-3e939413b2f4 None admin] PATH_INFO: `/v3/services` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:24.761733 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfdae21-39c1-4f13-b0db-3e939413b2f4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:24.763780 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfdae21-39c1-4f13-b0db-3e939413b2f4 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:24.783964 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 380/751] 10.222.0.57 () {60 vars in 1141 bytes} [Thu May 19 14:52:24 2022] POST /identity/v3/services => generated 251 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:52:24.785333 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfdae21-39c1-4f13-b0db-3e939413b2f4 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:24.807325 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0852ee77-743e-4af1-a0eb-d52eb3045e49 None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:24.858157 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:24.914611 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9db0c06-dd27-431d-a25f-4f6e3868b6e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:24.915304 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9db0c06-dd27-431d-a25f-4f6e3868b6e4 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:24.915594 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9db0c06-dd27-431d-a25f-4f6e3868b6e4 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:24.915873 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9db0c06-dd27-431d-a25f-4f6e3868b6e4 None admin] PATH_INFO: `/v3/domains` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:24.916412 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfdae21-39c1-4f13-b0db-3e939413b2f4 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:24.916611 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9db0c06-dd27-431d-a25f-4f6e3868b6e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:24.916917 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9db0c06-dd27-431d-a25f-4f6e3868b6e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:24.918352 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9db0c06-dd27-431d-a25f-4f6e3868b6e4 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:24.948460 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 372/752] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:52:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:24.953954 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9db0c06-dd27-431d-a25f-4f6e3868b6e4 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:24.955460 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:24.965459 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80427dc6-5047-4806-8d26-b2ef927927a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:24.966742 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80427dc6-5047-4806-8d26-b2ef927927a9 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:24.967392 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80427dc6-5047-4806-8d26-b2ef927927a9 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:24.967885 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80427dc6-5047-4806-8d26-b2ef927927a9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:24.983833 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80427dc6-5047-4806-8d26-b2ef927927a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:52:24.994163 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-80427dc6-5047-4806-8d26-b2ef927927a9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:52:24.994163 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:24.994163 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:24.994163 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:24.994163 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:24.994163 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:24.994163 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:24.994163 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:24.994163 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:24.994163 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:24.994163 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:24.994163 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:24.994163 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:52:24.994163 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:52:24.994163 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:52:24.994163 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:52:24.995400 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:52:24.995400 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:52:24.995400 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:52:24.995400 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:52:24.995400 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:24.995400 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:24.995400 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:24.995400 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:24.995400 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:24.995400 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:24.995400 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:24.995400 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:52:24.995400 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:24.995400 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:24.995400 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:24.995400 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:24.996342 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:24.996342 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:24.996342 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:24.996342 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:24.996342 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:52:24.996342 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:52:24.996342 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:52:24.996342 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:52:24.996342 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:52:24.996342 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:52:24.996342 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:52:24.996342 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:52:24.999528 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 381/753] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:52:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:25.007883 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80427dc6-5047-4806-8d26-b2ef927927a9 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:25.116893 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9db0c06-dd27-431d-a25f-4f6e3868b6e4 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:25.165033 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4426ce82-1301-45d5-9113-ec492ba2c503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:25.165876 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4426ce82-1301-45d5-9113-ec492ba2c503 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:25.166180 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4426ce82-1301-45d5-9113-ec492ba2c503 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:25.166453 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4426ce82-1301-45d5-9113-ec492ba2c503 None admin] PATH_INFO: `/v3/projects/service` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:25.179465 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4426ce82-1301-45d5-9113-ec492ba2c503 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:25.180972 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4426ce82-1301-45d5-9113-ec492ba2c503 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:25.191102 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80427dc6-5047-4806-8d26-b2ef927927a9 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:25.196332 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-4426ce82-1301-45d5-9113-ec492ba2c503 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:52:25.196332 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:25.196332 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:25.196332 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:25.196332 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:25.196332 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:25.196332 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:25.196332 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:25.196332 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:25.196332 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:25.196332 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:25.196332 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:25.196332 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:52:25.196332 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:52:25.196332 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:52:25.197456 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:52:25.197456 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:25.197456 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:25.197456 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:25.197456 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:25.197456 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:25.197456 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:25.197456 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:25.197456 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:52:25.197456 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:25.197456 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:25.197456 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:25.197456 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:25.197456 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:25.197456 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:25.197456 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:25.197456 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:25.198505 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:52:25.198505 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:52:25.198505 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:52:25.198505 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:52:25.198505 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:52:25.198505 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:52:25.198505 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:52:25.198505 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:52:25.198985 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 373/754] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:52:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:52:25.205247 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4426ce82-1301-45d5-9113-ec492ba2c503 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:25.235981 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09028bad-10e4-4cc1-a225-52470c98eef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:25.237010 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09028bad-10e4-4cc1-a225-52470c98eef3 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:25.237516 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09028bad-10e4-4cc1-a225-52470c98eef3 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:25.238044 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09028bad-10e4-4cc1-a225-52470c98eef3 None admin] PATH_INFO: `/v3/roles` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:25.239139 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09028bad-10e4-4cc1-a225-52470c98eef3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:25.239653 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09028bad-10e4-4cc1-a225-52470c98eef3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:25.241304 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09028bad-10e4-4cc1-a225-52470c98eef3 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:25.251357 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 382/755] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:52:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:25.255752 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:25.259616 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09028bad-10e4-4cc1-a225-52470c98eef3 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:25.323977 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4426ce82-1301-45d5-9113-ec492ba2c503 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:25.324526 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:25.361647 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74028a84-b072-4881-a08e-e3b0f889133b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:25.362463 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74028a84-b072-4881-a08e-e3b0f889133b None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:25.362849 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74028a84-b072-4881-a08e-e3b0f889133b None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:25.363212 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74028a84-b072-4881-a08e-e3b0f889133b None admin] PATH_INFO: `/v3/projects` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:25.364086 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74028a84-b072-4881-a08e-e3b0f889133b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:25.364556 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74028a84-b072-4881-a08e-e3b0f889133b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:25.366066 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74028a84-b072-4881-a08e-e3b0f889133b None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:25.379986 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 374/756] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:52:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:52:25.384157 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09028bad-10e4-4cc1-a225-52470c98eef3 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:25.385763 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74028a84-b072-4881-a08e-e3b0f889133b None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:25.424794 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f34a5a-ee1e-4685-9da6-abba70decf9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:25.426083 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f34a5a-ee1e-4685-9da6-abba70decf9f None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:25.426623 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f34a5a-ee1e-4685-9da6-abba70decf9f None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:25.427107 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f34a5a-ee1e-4685-9da6-abba70decf9f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:25.440299 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f34a5a-ee1e-4685-9da6-abba70decf9f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:25.442065 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f34a5a-ee1e-4685-9da6-abba70decf9f None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:25.462494 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-81f34a5a-ee1e-4685-9da6-abba70decf9f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:52:25.462494 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:25.462494 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:25.462494 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:25.462494 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:25.462494 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:25.462494 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:25.462494 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:25.462494 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:25.462494 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:25.462494 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:25.462494 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:25.462494 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:52:25.462494 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:52:25.462494 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:52:25.463568 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:52:25.463568 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:25.463568 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:25.463568 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:25.463568 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:25.463568 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:25.463568 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:25.463568 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:25.463568 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:52:25.463568 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:25.463568 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:25.463568 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:25.463568 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:25.463568 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:25.463568 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:25.463568 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:25.463568 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:25.464574 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:52:25.464574 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:52:25.464574 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:52:25.464574 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:52:25.466582 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 383/757] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:52:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:52:25.472977 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f34a5a-ee1e-4685-9da6-abba70decf9f None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:25.493106 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:25.496293 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74028a84-b072-4881-a08e-e3b0f889133b None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:25.534124 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ca685f-598b-49aa-8c29-e5f91c1c58cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:25.535281 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ca685f-598b-49aa-8c29-e5f91c1c58cd None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:25.536160 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ca685f-598b-49aa-8c29-e5f91c1c58cd None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:25.536289 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ca685f-598b-49aa-8c29-e5f91c1c58cd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:25.537703 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ca685f-598b-49aa-8c29-e5f91c1c58cd None admin] RBAC: Adding query filter params (role.id=028379e01f9245e9b8fc56e8b9e8a6ad, scope.project.id=fc99d742ae004954b6d841293b5fa79c, user.id=17afb27b2bbe481e959cf31db4470593) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:25.538299 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ca685f-598b-49aa-8c29-e5f91c1c58cd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:25.539832 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ca685f-598b-49aa-8c29-e5f91c1c58cd None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:25.546315 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 375/758] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:52:25 2022] GET /identity/v3/role_assignments?user.id=17afb27b2bbe481e959cf31db4470593&scope.project.id=fc99d742ae004954b6d841293b5fa79c&role.id=028379e01f9245e9b8fc56e8b9e8a6ad => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:52:25.596723 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f34a5a-ee1e-4685-9da6-abba70decf9f None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:25.599835 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:25.640259 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dcc4d7-295b-4edd-a773-eb7e96872516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:25.641303 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dcc4d7-295b-4edd-a773-eb7e96872516 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:25.641988 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dcc4d7-295b-4edd-a773-eb7e96872516 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:25.642464 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dcc4d7-295b-4edd-a773-eb7e96872516 None admin] PATH_INFO: `/v3/domains` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:25.643427 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dcc4d7-295b-4edd-a773-eb7e96872516 None admin] RBAC: Adding query filter params (name=Default) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:25.643936 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dcc4d7-295b-4edd-a773-eb7e96872516 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:25.645544 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dcc4d7-295b-4edd-a773-eb7e96872516 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:25.661485 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 384/759] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:52:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:52:25.668124 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ca685f-598b-49aa-8c29-e5f91c1c58cd None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:25.831590 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ca685f-598b-49aa-8c29-e5f91c1c58cd None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:25.878082 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b328d9e-9446-4189-886e-e9421e68317c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:25.878906 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b328d9e-9446-4189-886e-e9421e68317c None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:25.879196 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b328d9e-9446-4189-886e-e9421e68317c None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:25.879476 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b328d9e-9446-4189-886e-e9421e68317c None admin] PATH_INFO: `/v3/users/nova` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:25.897313 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f79f62-3823-471f-9158-e1432a3b7d61 None None] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:25.897505 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f79f62-3823-471f-9158-e1432a3b7d61 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:25.897779 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f79f62-3823-471f-9158-e1432a3b7d61 None None] PATH_INFO: `/` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:25.899025 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b328d9e-9446-4189-886e-e9421e68317c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:25.900553 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b328d9e-9446-4189-886e-e9421e68317c None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:25.900672 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 385/760] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:52:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:52:25.904569 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39b892b-f54a-4683-aee7-ee8bad6a3c02 None None] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:25.904863 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39b892b-f54a-4683-aee7-ee8bad6a3c02 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:25.905150 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39b892b-f54a-4683-aee7-ee8bad6a3c02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:25.918115 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-6b328d9e-9446-4189-886e-e9421e68317c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 14:52:25.918115 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:25.918115 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:25.918115 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:25.918115 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:25.918115 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:25.918115 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:25.918115 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:25.918115 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:25.918115 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:25.918115 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:25.918115 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:25.918115 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:52:25.918115 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:52:25.918115 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:52:25.918115 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:52:25.919211 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:25.919211 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:25.919211 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:52:25.919211 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:52:25.919211 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:52:25.919211 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:52:25.919211 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:25.919211 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:25.919211 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:25.919211 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:25.919211 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:25.919211 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:52:25.919211 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:25.919211 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:25.919211 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:25.919211 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:25.919211 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:25.919211 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:25.921475 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:25.921475 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:25.921475 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:52:25.921475 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:52:25.921475 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:52:25.921475 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:52:25.921475 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:52:25.921475 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:52:25.921475 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 14:52:25.921475 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:52:25.921475 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 376/761] 10.222.0.57 () {56 vars in 1093 bytes} [Thu May 19 14:52:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:25.925740 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b328d9e-9446-4189-886e-e9421e68317c None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:25.932158 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.auth.core [None req-e39b892b-f54a-4683-aee7-ee8bad6a3c02 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:52:25.933322 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-e39b892b-f54a-4683-aee7-ee8bad6a3c02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:52:25.978225 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:26.062608 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 386/762] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:52:25 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:52:26.102693 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39b892b-f54a-4683-aee7-ee8bad6a3c02 None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:26.121698 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b328d9e-9446-4189-886e-e9421e68317c None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:26.191003 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb415ca9-bcc9-4c4a-af33-a206e9b0b0f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:26.191594 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb415ca9-bcc9-4c4a-af33-a206e9b0b0f9 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:26.191877 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb415ca9-bcc9-4c4a-af33-a206e9b0b0f9 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:26.192151 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb415ca9-bcc9-4c4a-af33-a206e9b0b0f9 None admin] PATH_INFO: `/v3/users` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:26.193001 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb415ca9-bcc9-4c4a-af33-a206e9b0b0f9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:26.193313 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb415ca9-bcc9-4c4a-af33-a206e9b0b0f9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:26.194784 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb415ca9-bcc9-4c4a-af33-a206e9b0b0f9 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:26.221968 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 377/763] 10.222.0.57 () {56 vars in 1133 bytes} [Thu May 19 14:52:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:52:26.224378 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb415ca9-bcc9-4c4a-af33-a206e9b0b0f9 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:26.284408 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:26.293197 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39b892b-f54a-4683-aee7-ee8bad6a3c02 None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:26.312672 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:26.332289 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb415ca9-bcc9-4c4a-af33-a206e9b0b0f9 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:26.333260 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fd0bec-9074-4287-a275-d79907f1669d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:26.334276 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fd0bec-9074-4287-a275-d79907f1669d None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:26.334720 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fd0bec-9074-4287-a275-d79907f1669d None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:26.335076 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fd0bec-9074-4287-a275-d79907f1669d None admin] PATH_INFO: `/v3/services/network` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:26.335818 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fd0bec-9074-4287-a275-d79907f1669d None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:26.337261 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fd0bec-9074-4287-a275-d79907f1669d None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:26.351117 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-d7fd0bec-9074-4287-a275-d79907f1669d None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 19 14:52:26.351117 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:26.351117 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:52:26.351117 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:52:26.351117 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:52:26.351117 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:52:26.351117 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:52:26.351117 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:52:26.351117 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 14:52:26.351117 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:52:26.351117 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:52:26.351117 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:52:26.351117 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:26.351117 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:26.351117 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:26.351117 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:26.351117 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:26.352467 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:26.352467 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:26.352467 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:26.352467 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:26.352467 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:26.352467 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:26.352467 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:52:26.352467 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:52:26.352467 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:52:26.352467 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:52:26.352467 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:26.352467 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:26.352467 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:26.352467 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:26.352467 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:26.352467 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:26.353598 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:26.353598 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:52:26.353598 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:26.353598 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:26.353598 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:26.353598 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:26.353598 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:26.353598 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:26.353598 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:26.353598 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:26.353598 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:52:26.353598 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:52:26.353598 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 14:52:26.353598 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:52:26.355524 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 387/764] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:52:26 2022] GET /identity/v3/services/network => generated 88 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:26.355673 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea91f4c0-97b0-4436-b329-8ed70b50c9a0 None None] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:26.361831 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea91f4c0-97b0-4436-b329-8ed70b50c9a0 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:26.361831 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea91f4c0-97b0-4436-b329-8ed70b50c9a0 None None] PATH_INFO: `/` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:26.362323 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 388/765] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:52:26 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:52:26.363151 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea91f4c0-97b0-4436-b329-8ed70b50c9a0 None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:26.372575 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb1848b-e1eb-4ddf-8255-983598088c37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:26.373923 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb1848b-e1eb-4ddf-8255-983598088c37 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:26.374218 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb1848b-e1eb-4ddf-8255-983598088c37 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:26.374498 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb1848b-e1eb-4ddf-8255-983598088c37 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:26.389381 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb1848b-e1eb-4ddf-8255-983598088c37 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:26.391254 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb1848b-e1eb-4ddf-8255-983598088c37 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:26.410944 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-ebb1848b-e1eb-4ddf-8255-983598088c37 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:52:26.410944 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:26.410944 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:26.410944 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:26.410944 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:26.410944 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:26.410944 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:26.410944 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:26.410944 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:26.410944 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:26.410944 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:26.410944 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:26.410944 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:52:26.410944 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:52:26.410944 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:52:26.411959 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:52:26.411959 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:26.411959 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:26.411959 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:26.411959 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:26.411959 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:26.411959 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:26.411959 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:26.411959 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:52:26.411959 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:26.411959 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:26.411959 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:26.411959 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:26.411959 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:26.411959 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:26.411959 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:26.411959 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:26.412930 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:52:26.412930 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:52:26.412930 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:52:26.412930 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:52:26.419553 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 378/766] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:52:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:52:26.419553 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252b3224-649e-436e-a834-7a7be00f9242 None None] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:26.419553 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252b3224-649e-436e-a834-7a7be00f9242 None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:26.419553 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252b3224-649e-436e-a834-7a7be00f9242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:26.435713 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.auth.core [None req-252b3224-649e-436e-a834-7a7be00f9242 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:52:26.437349 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.fernet_utils [None req-252b3224-649e-436e-a834-7a7be00f9242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:52:26.504844 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea91f4c0-97b0-4436-b329-8ed70b50c9a0 None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:26.529010 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 379/767] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:52:26 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:52:26.530266 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252b3224-649e-436e-a834-7a7be00f9242 None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:26.564469 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7d352f-4bf6-4f58-be5f-4ae2c9446cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:26.565071 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7d352f-4bf6-4f58-be5f-4ae2c9446cac None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:26.565375 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7d352f-4bf6-4f58-be5f-4ae2c9446cac None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:26.565665 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7d352f-4bf6-4f58-be5f-4ae2c9446cac None admin] PATH_INFO: `/v3/services` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:26.566451 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7d352f-4bf6-4f58-be5f-4ae2c9446cac None admin] RBAC: Adding query filter params (name=network) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:26.566858 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7d352f-4bf6-4f58-be5f-4ae2c9446cac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:26.568257 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7d352f-4bf6-4f58-be5f-4ae2c9446cac None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:26.576792 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 389/768] 10.222.0.57 () {56 vars in 1112 bytes} [Thu May 19 14:52:26 2022] GET /identity/v3/services?name=network => generated 124 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:26.577290 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7d352f-4bf6-4f58-be5f-4ae2c9446cac None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:26.606148 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:26.618254 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:26.675041 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252b3224-649e-436e-a834-7a7be00f9242 None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:26.712196 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7d352f-4bf6-4f58-be5f-4ae2c9446cac None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:26.721960 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676a8904-7efc-4407-85d2-88f406e37e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:26.723397 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676a8904-7efc-4407-85d2-88f406e37e6b None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:26.723898 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676a8904-7efc-4407-85d2-88f406e37e6b None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:26.724801 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676a8904-7efc-4407-85d2-88f406e37e6b None admin] PATH_INFO: `/v3/domains` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:26.726078 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676a8904-7efc-4407-85d2-88f406e37e6b None admin] RBAC: Adding query filter params (name=Default) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:26.726626 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676a8904-7efc-4407-85d2-88f406e37e6b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:26.728988 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676a8904-7efc-4407-85d2-88f406e37e6b None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:26.749514 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 380/769] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:52:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:26.752055 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676a8904-7efc-4407-85d2-88f406e37e6b None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:26.753905 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60375ff0-82d0-48c4-9341-1b7b5a0e653a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:26.754783 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60375ff0-82d0-48c4-9341-1b7b5a0e653a None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:26.755132 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60375ff0-82d0-48c4-9341-1b7b5a0e653a None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:26.755418 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60375ff0-82d0-48c4-9341-1b7b5a0e653a None admin] PATH_INFO: `/v3/roles/service` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:26.767619 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60375ff0-82d0-48c4-9341-1b7b5a0e653a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:52:26.775999 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-60375ff0-82d0-48c4-9341-1b7b5a0e653a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:52:26.775999 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:26.775999 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:26.775999 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:26.775999 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:26.775999 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:26.775999 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:26.775999 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:26.775999 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:26.775999 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:26.775999 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:26.775999 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:26.775999 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:52:26.775999 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:52:26.775999 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:52:26.775999 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:52:26.777009 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:52:26.777009 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:52:26.777009 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:52:26.777009 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:52:26.777009 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:26.777009 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:26.777009 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:26.777009 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:26.777009 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:26.777009 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:26.777009 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:26.777009 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:52:26.777009 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:26.777009 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:26.777009 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:26.777009 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:26.777989 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:26.777989 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:26.777989 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:26.777989 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:26.777989 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:52:26.777989 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:52:26.777989 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:52:26.777989 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:52:26.777989 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:52:26.777989 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:52:26.777989 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:52:26.777989 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:52:26.782580 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 390/770] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 14:52:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:26.783346 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60375ff0-82d0-48c4-9341-1b7b5a0e653a None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:26.865217 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676a8904-7efc-4407-85d2-88f406e37e6b None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:26.873534 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:26.892811 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:26.897773 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60375ff0-82d0-48c4-9341-1b7b5a0e653a None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:26.904223 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79b3930-d4ea-424c-a337-bacb235e1eb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:26.904896 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79b3930-d4ea-424c-a337-bacb235e1eb2 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:26.905323 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79b3930-d4ea-424c-a337-bacb235e1eb2 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:26.905518 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79b3930-d4ea-424c-a337-bacb235e1eb2 None admin] PATH_INFO: `/v3/services` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:26.906178 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79b3930-d4ea-424c-a337-bacb235e1eb2 None admin] RBAC: Adding query filter params (type=network) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:26.906500 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79b3930-d4ea-424c-a337-bacb235e1eb2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:26.909223 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79b3930-d4ea-424c-a337-bacb235e1eb2 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:26.916377 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 381/771] 10.222.0.57 () {56 vars in 1112 bytes} [Thu May 19 14:52:26 2022] GET /identity/v3/services?type=network => generated 350 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:26.917021 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79b3930-d4ea-424c-a337-bacb235e1eb2 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:26.939744 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e19f08-9e0c-4496-b810-800683a415cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:26.941133 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e19f08-9e0c-4496-b810-800683a415cc None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:26.941663 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e19f08-9e0c-4496-b810-800683a415cc None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:26.942206 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e19f08-9e0c-4496-b810-800683a415cc None admin] PATH_INFO: `/v3/projects/service` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:26.950419 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e19f08-9e0c-4496-b810-800683a415cc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:26.952150 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e19f08-9e0c-4496-b810-800683a415cc None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:26.967910 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-76e19f08-9e0c-4496-b810-800683a415cc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:52:26.967910 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:26.967910 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:26.967910 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:26.967910 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:26.967910 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:26.967910 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:26.967910 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:26.967910 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:26.967910 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:26.967910 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:26.967910 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:26.967910 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:52:26.967910 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:52:26.967910 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:52:26.968961 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:52:26.968961 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:26.968961 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:26.968961 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:26.968961 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:26.968961 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:26.968961 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:26.968961 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:26.968961 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:52:26.968961 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:26.968961 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:26.968961 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:26.968961 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:26.968961 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:26.968961 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:26.968961 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:26.968961 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:26.970644 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:52:26.970644 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:52:26.970644 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:52:26.970644 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:52:26.970644 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:52:26.970644 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:52:26.970644 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:52:26.970644 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:52:26.977756 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 391/772] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:52:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:26.978938 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e19f08-9e0c-4496-b810-800683a415cc None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:27.027974 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79b3930-d4ea-424c-a337-bacb235e1eb2 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:27.064583 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c5e0c8-5254-4a33-80d2-04258a55398f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:27.065457 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c5e0c8-5254-4a33-80d2-04258a55398f None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:27.065743 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c5e0c8-5254-4a33-80d2-04258a55398f None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:27.066062 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c5e0c8-5254-4a33-80d2-04258a55398f None admin] PATH_INFO: `/v3/roles` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:27.067034 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c5e0c8-5254-4a33-80d2-04258a55398f None admin] RBAC: Adding query filter params (name=service) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:27.067153 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c5e0c8-5254-4a33-80d2-04258a55398f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:27.068495 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c5e0c8-5254-4a33-80d2-04258a55398f None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:27.082215 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 382/773] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 19 14:52:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:27.082215 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c5e0c8-5254-4a33-80d2-04258a55398f None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:27.096833 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e19f08-9e0c-4496-b810-800683a415cc None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:27.131712 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:27.136868 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd54f51-52d1-4320-b4b5-e1493add62c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:27.137739 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd54f51-52d1-4320-b4b5-e1493add62c5 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:27.138336 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd54f51-52d1-4320-b4b5-e1493add62c5 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:27.138725 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd54f51-52d1-4320-b4b5-e1493add62c5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:27.140276 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd54f51-52d1-4320-b4b5-e1493add62c5 None admin] RBAC: Adding query filter params (interface=public, service_id=8a3648abfff84564a463c4643fd0c786, region_id=RegionOne) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:27.140971 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd54f51-52d1-4320-b4b5-e1493add62c5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:27.142605 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd54f51-52d1-4320-b4b5-e1493add62c5 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:27.153218 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 392/774] 10.222.0.57 () {56 vars in 1251 bytes} [Thu May 19 14:52:26 2022] GET /identity/v3/endpoints?service_id=8a3648abfff84564a463c4643fd0c786&interface=public®ion_id=RegionOne => generated 194 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:27.154231 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd54f51-52d1-4320-b4b5-e1493add62c5 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:27.164980 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:27.188822 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c5e0c8-5254-4a33-80d2-04258a55398f None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:27.225497 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b955482-4e9f-4450-be18-9783806855a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:27.226445 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b955482-4e9f-4450-be18-9783806855a2 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:27.226883 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b955482-4e9f-4450-be18-9783806855a2 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:27.227358 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b955482-4e9f-4450-be18-9783806855a2 None admin] PATH_INFO: `/v3/projects` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:27.228294 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b955482-4e9f-4450-be18-9783806855a2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:27.228758 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b955482-4e9f-4450-be18-9783806855a2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:27.230436 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b955482-4e9f-4450-be18-9783806855a2 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:27.244148 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 383/775] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:52:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:27.245401 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b955482-4e9f-4450-be18-9783806855a2 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:27.280403 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd54f51-52d1-4320-b4b5-e1493add62c5 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:27.321376 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2ae501-cc84-4fa5-8206-94d98ca2d08c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:27.322845 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2ae501-cc84-4fa5-8206-94d98ca2d08c None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:27.322845 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2ae501-cc84-4fa5-8206-94d98ca2d08c None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:27.322845 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2ae501-cc84-4fa5-8206-94d98ca2d08c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:27.335345 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2ae501-cc84-4fa5-8206-94d98ca2d08c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:27.336961 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2ae501-cc84-4fa5-8206-94d98ca2d08c None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:27.354750 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b955482-4e9f-4450-be18-9783806855a2 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:27.356537 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-ac2ae501-cc84-4fa5-8206-94d98ca2d08c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:52:27.356537 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:27.356537 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:27.356537 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:27.356537 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:27.356537 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:27.356537 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:27.356537 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:27.356537 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:27.356537 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:27.356537 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:27.356537 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:27.356537 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:52:27.356537 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:52:27.356537 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:52:27.357483 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:52:27.357483 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:27.357483 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:27.357483 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:27.357483 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:27.357483 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:27.357483 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:27.357483 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:27.357483 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:52:27.357483 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:27.357483 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:27.357483 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:27.357483 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:27.357483 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:27.357483 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:27.357483 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:27.357483 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:27.358532 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:52:27.358532 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:52:27.358532 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:52:27.358532 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:52:27.363194 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 393/776] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:52:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:27.364074 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2ae501-cc84-4fa5-8206-94d98ca2d08c None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:27.371180 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:27.398031 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a48d7a-b0c2-4495-a974-e5edc1709760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:27.398850 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a48d7a-b0c2-4495-a974-e5edc1709760 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:27.399238 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a48d7a-b0c2-4495-a974-e5edc1709760 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:27.399629 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a48d7a-b0c2-4495-a974-e5edc1709760 None admin] PATH_INFO: `/v3/services` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:27.400422 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a48d7a-b0c2-4495-a974-e5edc1709760 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:27.402010 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a48d7a-b0c2-4495-a974-e5edc1709760 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:27.409674 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 384/777] 10.222.0.57 () {56 vars in 1087 bytes} [Thu May 19 14:52:27 2022] GET /identity/v3/services => generated 772 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:27.410723 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a48d7a-b0c2-4495-a974-e5edc1709760 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:27.437320 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:27.491302 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2ae501-cc84-4fa5-8206-94d98ca2d08c None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:27.523974 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a48d7a-b0c2-4495-a974-e5edc1709760 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:27.532692 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a617480-12f8-435e-a9f7-82486a273189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:27.534206 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a617480-12f8-435e-a9f7-82486a273189 None admin] REQUEST_METHOD: `PUT` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:27.534744 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a617480-12f8-435e-a9f7-82486a273189 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:27.535272 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a617480-12f8-435e-a9f7-82486a273189 None admin] PATH_INFO: `/v3/projects/fc99d742ae004954b6d841293b5fa79c/users/5d2aa8c8870a49d98f23f26e036695ef/roles/028379e01f9245e9b8fc56e8b9e8a6ad` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:27.538044 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a617480-12f8-435e-a9f7-82486a273189 None admin] RBAC: Authorizing `identity:create_grant(project_id=fc99d742ae004954b6d841293b5fa79c, user_id=5d2aa8c8870a49d98f23f26e036695ef, role_id=028379e01f9245e9b8fc56e8b9e8a6ad)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:27.539849 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a617480-12f8-435e-a9f7-82486a273189 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:27.558328 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 394/778] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 14:52:27 2022] PUT /identity/v3/projects/fc99d742ae004954b6d841293b5fa79c/users/5d2aa8c8870a49d98f23f26e036695ef/roles/028379e01f9245e9b8fc56e8b9e8a6ad => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:52:27.568386 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8cdd53-e6d9-4eef-afca-ac6089647a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:27.569092 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8cdd53-e6d9-4eef-afca-ac6089647a6a None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:27.569437 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8cdd53-e6d9-4eef-afca-ac6089647a6a None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:27.569774 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8cdd53-e6d9-4eef-afca-ac6089647a6a None admin] PATH_INFO: `/v3/domains` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:27.570955 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8cdd53-e6d9-4eef-afca-ac6089647a6a None admin] RBAC: Adding query filter params (name=Default) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:27.571298 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8cdd53-e6d9-4eef-afca-ac6089647a6a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:27.572839 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8cdd53-e6d9-4eef-afca-ac6089647a6a None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:27.589170 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 385/779] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:52:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:27.595444 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a617480-12f8-435e-a9f7-82486a273189 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:27.754895 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a617480-12f8-435e-a9f7-82486a273189 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:27.756023 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7237427a-00a4-4338-8e38-b561dcdbec14 None None] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:27.756499 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7237427a-00a4-4338-8e38-b561dcdbec14 None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:27.756593 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7237427a-00a4-4338-8e38-b561dcdbec14 None None] PATH_INFO: `/` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:27.759253 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 386/780] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:52:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:52:27.765043 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cff6d6-2f30-4d8a-935a-4cf0e3bc1e34 None None] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:27.765348 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cff6d6-2f30-4d8a-935a-4cf0e3bc1e34 None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:27.765628 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cff6d6-2f30-4d8a-935a-4cf0e3bc1e34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:27.779369 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:27.789503 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.auth.core [None req-f6cff6d6-2f30-4d8a-935a-4cf0e3bc1e34 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:52:27.792654 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.fernet_utils [None req-f6cff6d6-2f30-4d8a-935a-4cf0e3bc1e34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:52:27.800992 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93f9fa2-0cff-4acb-a323-d74babdf5279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:27.801832 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93f9fa2-0cff-4acb-a323-d74babdf5279 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:27.802173 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93f9fa2-0cff-4acb-a323-d74babdf5279 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:27.802459 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93f9fa2-0cff-4acb-a323-d74babdf5279 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:27.811612 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93f9fa2-0cff-4acb-a323-d74babdf5279 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:27.813261 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93f9fa2-0cff-4acb-a323-d74babdf5279 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:27.827534 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:27.831831 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-f93f9fa2-0cff-4acb-a323-d74babdf5279 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 14:52:27.831831 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:27.831831 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:27.831831 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:27.831831 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:27.831831 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:27.831831 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:27.831831 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:27.831831 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:27.831831 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:27.831831 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:27.831831 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:27.831831 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:52:27.831831 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:52:27.831831 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:52:27.831831 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:52:27.832908 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:27.832908 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:27.832908 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:52:27.832908 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:52:27.832908 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:52:27.832908 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:52:27.832908 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:27.832908 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:27.832908 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:27.832908 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:27.832908 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:27.832908 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:52:27.832908 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:27.832908 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:27.832908 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:27.832908 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:27.832908 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:27.832908 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:27.834005 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:27.834005 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:27.834005 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:52:27.834005 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:52:27.834005 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:52:27.834005 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:52:27.834005 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:52:27.834005 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:52:27.834005 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 14:52:27.834005 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:52:27.834005 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 395/781] 10.222.0.57 () {56 vars in 1117 bytes} [Thu May 19 14:52:27 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:52:27.838929 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93f9fa2-0cff-4acb-a323-d74babdf5279 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:27.890528 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 387/782] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:52:27 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:52:27.920593 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cff6d6-2f30-4d8a-935a-4cf0e3bc1e34 None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:27.968766 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93f9fa2-0cff-4acb-a323-d74babdf5279 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:28.024853 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabe89fd-e45e-40c6-bf58-fe27f1572671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:28.026090 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabe89fd-e45e-40c6-bf58-fe27f1572671 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:28.026629 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabe89fd-e45e-40c6-bf58-fe27f1572671 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:28.027194 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabe89fd-e45e-40c6-bf58-fe27f1572671 None admin] PATH_INFO: `/v3/users` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:28.028266 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabe89fd-e45e-40c6-bf58-fe27f1572671 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:28.028793 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabe89fd-e45e-40c6-bf58-fe27f1572671 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:28.030624 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabe89fd-e45e-40c6-bf58-fe27f1572671 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:28.042513 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cff6d6-2f30-4d8a-935a-4cf0e3bc1e34 None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:28.065711 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 396/783] 10.222.0.57 () {56 vars in 1149 bytes} [Thu May 19 14:52:27 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:52:28.071054 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabe89fd-e45e-40c6-bf58-fe27f1572671 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:28.093075 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c9fc52-411a-4b0e-9aa7-89d63e17c6ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:28.094397 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c9fc52-411a-4b0e-9aa7-89d63e17c6ed None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:28.096440 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c9fc52-411a-4b0e-9aa7-89d63e17c6ed None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:28.097869 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c9fc52-411a-4b0e-9aa7-89d63e17c6ed None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:28.100173 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c9fc52-411a-4b0e-9aa7-89d63e17c6ed None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:28.101828 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c9fc52-411a-4b0e-9aa7-89d63e17c6ed None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:28.139941 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-e6c9fc52-411a-4b0e-9aa7-89d63e17c6ed None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 14:52:28.139941 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:28.139941 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:52:28.139941 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:52:28.139941 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:52:28.139941 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:52:28.139941 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:52:28.139941 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:52:28.139941 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 14:52:28.139941 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:52:28.139941 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:52:28.139941 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:52:28.139941 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:28.139941 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:28.139941 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:28.139941 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:28.141713 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:28.141713 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:28.141713 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:28.141713 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:28.141713 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:28.141713 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:28.141713 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:28.141713 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:52:28.141713 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:52:28.141713 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:52:28.141713 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:52:28.141713 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:28.141713 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:28.141713 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:28.141713 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:28.141713 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:28.142871 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:28.142871 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:28.142871 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:52:28.142871 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:28.142871 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:28.142871 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:28.142871 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:28.142871 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:28.142871 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:28.142871 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:28.142871 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:28.142871 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:52:28.142871 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:52:28.142871 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 14:52:28.142871 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:52:28.146031 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 388/784] 10.222.0.57 () {56 vars in 1129 bytes} [Thu May 19 14:52:27 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:52:28.148000 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:28.152263 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c9fc52-411a-4b0e-9aa7-89d63e17c6ed None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:28.215865 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:28.258333 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabe89fd-e45e-40c6-bf58-fe27f1572671 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:28.296066 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd449aef-8730-460d-aab4-23125843913a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:28.297216 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd449aef-8730-460d-aab4-23125843913a None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:28.298298 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd449aef-8730-460d-aab4-23125843913a None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:28.298954 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd449aef-8730-460d-aab4-23125843913a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:28.311711 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd449aef-8730-460d-aab4-23125843913a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:28.314278 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd449aef-8730-460d-aab4-23125843913a None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:28.333359 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-dd449aef-8730-460d-aab4-23125843913a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:52:28.333359 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:28.333359 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:28.333359 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:28.333359 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:28.333359 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:28.333359 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:28.333359 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:28.333359 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:28.333359 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:28.333359 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:28.333359 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:28.333359 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:52:28.333359 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:52:28.333359 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:52:28.334464 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:52:28.334464 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:28.334464 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:28.334464 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:28.334464 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:28.334464 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:28.334464 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:28.334464 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:28.334464 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:52:28.334464 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:28.334464 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:28.334464 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:28.334464 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:28.334464 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:28.334464 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:28.334464 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:28.334464 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:28.335895 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:52:28.335895 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:52:28.335895 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:52:28.335895 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:52:28.338217 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 397/785] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:52:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:28.339819 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0324854b-2f57-4aaf-a92b-4872147e8daa None None] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:28.340349 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0324854b-2f57-4aaf-a92b-4872147e8daa None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:28.340842 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0324854b-2f57-4aaf-a92b-4872147e8daa None None] PATH_INFO: `/` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:28.342626 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 398/786] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:52:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:52:28.343549 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0324854b-2f57-4aaf-a92b-4872147e8daa None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:28.357749 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c9fc52-411a-4b0e-9aa7-89d63e17c6ed None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:28.407351 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26bb33d-3f21-4da9-9d92-e8c03c31ea73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:28.408036 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26bb33d-3f21-4da9-9d92-e8c03c31ea73 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:28.408325 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26bb33d-3f21-4da9-9d92-e8c03c31ea73 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:28.408603 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26bb33d-3f21-4da9-9d92-e8c03c31ea73 None admin] PATH_INFO: `/v3/services` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:28.409334 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26bb33d-3f21-4da9-9d92-e8c03c31ea73 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:28.409888 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26bb33d-3f21-4da9-9d92-e8c03c31ea73 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:28.411707 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26bb33d-3f21-4da9-9d92-e8c03c31ea73 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:28.417501 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 389/787] 10.222.0.57 () {56 vars in 1124 bytes} [Thu May 19 14:52:28 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:28.419540 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f4512c-c294-4bcb-8af9-dd753a2816e1 None None] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:28.419830 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f4512c-c294-4bcb-8af9-dd753a2816e1 None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:28.420119 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f4512c-c294-4bcb-8af9-dd753a2816e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:28.442421 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.auth.core [None req-77f4512c-c294-4bcb-8af9-dd753a2816e1 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:52:28.443635 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.fernet_utils [None req-77f4512c-c294-4bcb-8af9-dd753a2816e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:52:28.459908 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:28.478231 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0324854b-2f57-4aaf-a92b-4872147e8daa None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:28.519964 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7327107-65df-4ddb-b068-3b63cf61c367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:28.521102 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7327107-65df-4ddb-b068-3b63cf61c367 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:28.521629 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7327107-65df-4ddb-b068-3b63cf61c367 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:28.522226 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7327107-65df-4ddb-b068-3b63cf61c367 None admin] PATH_INFO: `/v3/domains` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:28.523194 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7327107-65df-4ddb-b068-3b63cf61c367 None admin] RBAC: Adding query filter params (name=Default) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:28.523755 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7327107-65df-4ddb-b068-3b63cf61c367 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:28.525420 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7327107-65df-4ddb-b068-3b63cf61c367 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:28.538946 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 399/788] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:52:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:28.540162 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7327107-65df-4ddb-b068-3b63cf61c367 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:28.543896 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 390/789] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:52:28 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:52:28.544811 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f4512c-c294-4bcb-8af9-dd753a2816e1 None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:28.582641 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:28.686879 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f4512c-c294-4bcb-8af9-dd753a2816e1 None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:28.699641 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7327107-65df-4ddb-b068-3b63cf61c367 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:28.736393 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7dc510-d71b-45b3-8a84-d7390feefea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:28.737193 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7dc510-d71b-45b3-8a84-d7390feefea3 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:28.737485 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7dc510-d71b-45b3-8a84-d7390feefea3 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:28.737769 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7dc510-d71b-45b3-8a84-d7390feefea3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:28.738525 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95579ec9-f417-484b-a3d7-4db4dc5dc1f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:28.739575 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95579ec9-f417-484b-a3d7-4db4dc5dc1f2 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:28.740081 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95579ec9-f417-484b-a3d7-4db4dc5dc1f2 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:28.740571 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95579ec9-f417-484b-a3d7-4db4dc5dc1f2 None admin] PATH_INFO: `/v3/services` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:28.741492 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95579ec9-f417-484b-a3d7-4db4dc5dc1f2 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:28.742042 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95579ec9-f417-484b-a3d7-4db4dc5dc1f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:28.743648 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95579ec9-f417-484b-a3d7-4db4dc5dc1f2 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:28.745161 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7dc510-d71b-45b3-8a84-d7390feefea3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:28.747075 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7dc510-d71b-45b3-8a84-d7390feefea3 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:28.749875 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 400/790] 10.222.0.57 () {56 vars in 1124 bytes} [Thu May 19 14:52:28 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:28.751123 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95579ec9-f417-484b-a3d7-4db4dc5dc1f2 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:28.764816 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-7f7dc510-d71b-45b3-8a84-d7390feefea3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:52:28.764816 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:28.764816 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:28.764816 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:28.764816 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:28.764816 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:28.764816 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:28.764816 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:28.764816 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:28.764816 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:28.764816 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:28.764816 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:28.764816 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:52:28.764816 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:52:28.764816 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:52:28.767033 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:52:28.767033 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:28.767033 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:28.767033 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:28.767033 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:28.767033 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:28.767033 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:28.767033 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:28.767033 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:52:28.767033 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:28.767033 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:28.767033 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:28.767033 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:28.767033 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:28.767033 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:28.767033 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:28.767033 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:28.768045 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:52:28.768045 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:52:28.768045 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:52:28.768045 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:52:28.768045 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:52:28.768045 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:52:28.768045 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:52:28.768045 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:52:28.769914 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 391/791] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:52:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:28.771135 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7dc510-d71b-45b3-8a84-d7390feefea3 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:28.774129 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:28.876793 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:28.889633 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95579ec9-f417-484b-a3d7-4db4dc5dc1f2 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:28.898273 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7dc510-d71b-45b3-8a84-d7390feefea3 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:28.931746 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fac791-ee91-4387-9586-c8a9674a5bc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:28.933156 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fac791-ee91-4387-9586-c8a9674a5bc9 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:28.933671 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fac791-ee91-4387-9586-c8a9674a5bc9 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:28.934251 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fac791-ee91-4387-9586-c8a9674a5bc9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:28.944113 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fac63e-5f96-4f43-9577-52bfaae28860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:28.946291 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fac63e-5f96-4f43-9577-52bfaae28860 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:28.946795 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fac63e-5f96-4f43-9577-52bfaae28860 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:28.947265 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fac63e-5f96-4f43-9577-52bfaae28860 None admin] PATH_INFO: `/v3/endpoints` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:28.948292 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fac63e-5f96-4f43-9577-52bfaae28860 None admin] RBAC: Adding query filter params (interface=public, service_id=778f840a164840a985adb27a8224b247, region_id=RegionOne) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:28.949084 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fac63e-5f96-4f43-9577-52bfaae28860 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:28.949492 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fac791-ee91-4387-9586-c8a9674a5bc9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:52:28.950937 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fac63e-5f96-4f43-9577-52bfaae28860 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:28.959024 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-d9fac791-ee91-4387-9586-c8a9674a5bc9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:52:28.959024 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:28.959024 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:28.959024 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:28.959024 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:28.959024 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:28.959024 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:28.959024 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:28.959024 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:28.959024 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:28.959024 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:28.959024 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:28.959024 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:52:28.959024 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:52:28.959024 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:52:28.959024 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:52:28.960098 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:52:28.960098 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:52:28.960098 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:52:28.960098 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:52:28.960098 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:28.960098 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:28.960098 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:28.960098 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:28.960098 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:28.960098 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:28.960098 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:28.960098 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:52:28.960098 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:28.960098 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:28.960098 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:28.960098 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:28.961020 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:28.961020 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:28.961020 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:28.961020 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:28.961020 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:52:28.961020 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:52:28.961020 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:52:28.961020 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:52:28.961020 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:52:28.961020 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:52:28.961020 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:52:28.961020 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:52:28.961020 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 401/792] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:52:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:28.961949 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fac791-ee91-4387-9586-c8a9674a5bc9 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:28.963764 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 392/793] 10.222.0.57 () {56 vars in 1251 bytes} [Thu May 19 14:52:28 2022] GET /identity/v3/endpoints?service_id=778f840a164840a985adb27a8224b247&interface=public®ion_id=RegionOne => generated 194 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:28.972237 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fac63e-5f96-4f43-9577-52bfaae28860 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:29.055249 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:29.089166 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fac63e-5f96-4f43-9577-52bfaae28860 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:29.097428 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fac791-ee91-4387-9586-c8a9674a5bc9 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:29.134120 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91ff409-c0a7-49fd-88a4-2caa95d02d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:29.135026 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91ff409-c0a7-49fd-88a4-2caa95d02d53 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:29.135734 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91ff409-c0a7-49fd-88a4-2caa95d02d53 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:29.136394 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91ff409-c0a7-49fd-88a4-2caa95d02d53 None admin] PATH_INFO: `/v3/services` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:29.137303 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91ff409-c0a7-49fd-88a4-2caa95d02d53 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:29.138929 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91ff409-c0a7-49fd-88a4-2caa95d02d53 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:29.140606 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1285c7b-ab42-40f9-8f4d-45905b4f7b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:29.142260 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1285c7b-ab42-40f9-8f4d-45905b4f7b0a None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:29.142694 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1285c7b-ab42-40f9-8f4d-45905b4f7b0a None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:29.143071 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1285c7b-ab42-40f9-8f4d-45905b4f7b0a None admin] PATH_INFO: `/v3/projects` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:29.143892 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1285c7b-ab42-40f9-8f4d-45905b4f7b0a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:29.144215 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1285c7b-ab42-40f9-8f4d-45905b4f7b0a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:29.145647 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1285c7b-ab42-40f9-8f4d-45905b4f7b0a None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:29.151518 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 393/794] 10.222.0.57 () {56 vars in 1087 bytes} [Thu May 19 14:52:28 2022] GET /identity/v3/services => generated 772 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:29.152349 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91ff409-c0a7-49fd-88a4-2caa95d02d53 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:29.170337 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:29.176337 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 402/795] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:52:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:29.182590 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1285c7b-ab42-40f9-8f4d-45905b4f7b0a None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:29.287657 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91ff409-c0a7-49fd-88a4-2caa95d02d53 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:29.329262 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12d2205-9e27-47ff-b6fb-aadc3ef4c526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:29.330439 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12d2205-9e27-47ff-b6fb-aadc3ef4c526 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:29.331022 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12d2205-9e27-47ff-b6fb-aadc3ef4c526 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:29.331461 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12d2205-9e27-47ff-b6fb-aadc3ef4c526 None admin] PATH_INFO: `/v3/roles` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:29.332380 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12d2205-9e27-47ff-b6fb-aadc3ef4c526 None admin] RBAC: Adding query filter params (name=admin) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:29.333143 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12d2205-9e27-47ff-b6fb-aadc3ef4c526 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:29.335289 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12d2205-9e27-47ff-b6fb-aadc3ef4c526 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:29.346269 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 394/796] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:52:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:29.358935 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12d2205-9e27-47ff-b6fb-aadc3ef4c526 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:29.416598 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1285c7b-ab42-40f9-8f4d-45905b4f7b0a None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:29.467791 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:29.488092 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12d2205-9e27-47ff-b6fb-aadc3ef4c526 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:29.494230 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:29.497926 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49581d1c-e0f8-448c-a3ee-76041665c745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:29.498838 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49581d1c-e0f8-448c-a3ee-76041665c745 None admin] REQUEST_METHOD: `PUT` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:29.499132 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49581d1c-e0f8-448c-a3ee-76041665c745 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:29.499420 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49581d1c-e0f8-448c-a3ee-76041665c745 None admin] PATH_INFO: `/v3/projects/fc99d742ae004954b6d841293b5fa79c/users/cd0da70fc3c1477d8202726187ab5fde/roles/08f16502894349708527e30032247468` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:29.521356 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49581d1c-e0f8-448c-a3ee-76041665c745 None admin] RBAC: Authorizing `identity:create_grant(project_id=fc99d742ae004954b6d841293b5fa79c, user_id=cd0da70fc3c1477d8202726187ab5fde, role_id=08f16502894349708527e30032247468)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:29.523340 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49581d1c-e0f8-448c-a3ee-76041665c745 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:29.533449 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff0a639-0acc-434c-a714-2201f6ccc568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:29.534521 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff0a639-0acc-434c-a714-2201f6ccc568 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:29.534875 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff0a639-0acc-434c-a714-2201f6ccc568 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:29.535103 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff0a639-0acc-434c-a714-2201f6ccc568 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:29.545201 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 403/797] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 14:52:29 2022] PUT /identity/v3/projects/fc99d742ae004954b6d841293b5fa79c/users/cd0da70fc3c1477d8202726187ab5fde/roles/08f16502894349708527e30032247468 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:52:29.553504 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff0a639-0acc-434c-a714-2201f6ccc568 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:29.555066 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff0a639-0acc-434c-a714-2201f6ccc568 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:29.575134 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-3ff0a639-0acc-434c-a714-2201f6ccc568 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:52:29.575134 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:29.575134 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:29.575134 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:29.575134 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:29.575134 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:29.575134 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:29.575134 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:29.575134 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:29.575134 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:29.575134 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:29.575134 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:29.575134 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:52:29.575134 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:52:29.575134 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:52:29.576204 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:52:29.576204 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:29.576204 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:29.576204 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:29.576204 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:29.576204 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:29.576204 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:29.576204 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:29.576204 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:52:29.576204 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:29.576204 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:29.576204 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:29.576204 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:29.576204 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:29.576204 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:29.576204 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:29.576204 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:29.577224 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:52:29.577224 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:52:29.577224 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:52:29.577224 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:52:29.577224 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 395/798] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:52:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:29.583223 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49581d1c-e0f8-448c-a3ee-76041665c745 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:29.739544 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49581d1c-e0f8-448c-a3ee-76041665c745 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:29.777438 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d208d7-d073-41b9-a758-c245227c3693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:29.778160 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d208d7-d073-41b9-a758-c245227c3693 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:29.778457 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d208d7-d073-41b9-a758-c245227c3693 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:29.778746 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d208d7-d073-41b9-a758-c245227c3693 None admin] PATH_INFO: `/v3/domains` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:29.779554 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d208d7-d073-41b9-a758-c245227c3693 None admin] RBAC: Adding query filter params (name=Default) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:29.779876 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d208d7-d073-41b9-a758-c245227c3693 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:29.781272 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d208d7-d073-41b9-a758-c245227c3693 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:29.796116 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 404/799] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:52:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:29.800426 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff0a639-0acc-434c-a714-2201f6ccc568 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:29.920395 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff0a639-0acc-434c-a714-2201f6ccc568 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:29.947213 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:29.965151 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f1ca8f-8396-4579-9642-a594c0077128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:29.966464 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f1ca8f-8396-4579-9642-a594c0077128 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:29.967020 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f1ca8f-8396-4579-9642-a594c0077128 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:29.967543 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f1ca8f-8396-4579-9642-a594c0077128 None admin] PATH_INFO: `/v3/users/swift` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:29.979989 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f1ca8f-8396-4579-9642-a594c0077128 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:29.981997 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f1ca8f-8396-4579-9642-a594c0077128 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:30.007241 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-28f1ca8f-8396-4579-9642-a594c0077128 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 14:52:30.007241 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:30.007241 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:30.007241 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:30.007241 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:30.007241 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:30.007241 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:30.007241 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:30.007241 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:30.007241 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:30.007241 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:30.007241 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:30.007241 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:52:30.007241 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:52:30.007241 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:52:30.007241 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:52:30.008329 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:30.008329 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:30.008329 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:52:30.008329 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:52:30.008329 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:52:30.008329 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:52:30.008329 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:30.008329 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:30.008329 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:30.008329 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:30.008329 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:30.008329 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:52:30.008329 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:30.008329 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:30.008329 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:30.008329 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:30.008329 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:30.008329 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:30.009399 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:30.009399 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:30.009399 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:52:30.009399 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:52:30.009399 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:52:30.009399 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:52:30.009399 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:52:30.009399 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:52:30.009399 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 14:52:30.009399 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:52:30.012276 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 396/800] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:52:29 2022] GET /identity/v3/users/swift => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:30.017955 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d208d7-d073-41b9-a758-c245227c3693 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:30.095178 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:30.181883 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d208d7-d073-41b9-a758-c245227c3693 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:30.211336 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afa5a7f-b836-4475-b268-35c947cbe669 None None] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:30.211684 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afa5a7f-b836-4475-b268-35c947cbe669 None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:30.211912 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afa5a7f-b836-4475-b268-35c947cbe669 None None] PATH_INFO: `/` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:30.213591 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 397/801] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:52:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:52:30.220333 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe154891-368f-4a28-8bc5-b8869ab074a3 None None] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:30.220625 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe154891-368f-4a28-8bc5-b8869ab074a3 None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:30.220910 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe154891-368f-4a28-8bc5-b8869ab074a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:30.230452 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7531f92-1eb7-4e62-90b9-10d255192de9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:30.231394 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7531f92-1eb7-4e62-90b9-10d255192de9 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:30.231873 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7531f92-1eb7-4e62-90b9-10d255192de9 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:30.232342 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7531f92-1eb7-4e62-90b9-10d255192de9 None admin] PATH_INFO: `/v3/users` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:30.233407 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7531f92-1eb7-4e62-90b9-10d255192de9 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:30.233950 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7531f92-1eb7-4e62-90b9-10d255192de9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:30.236113 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7531f92-1eb7-4e62-90b9-10d255192de9 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:30.241890 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.auth.core [None req-fe154891-368f-4a28-8bc5-b8869ab074a3 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:52:30.243133 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.fernet_utils [None req-fe154891-368f-4a28-8bc5-b8869ab074a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:52:30.255608 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 405/802] 10.222.0.57 () {56 vars in 1135 bytes} [Thu May 19 14:52:30 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:30.257227 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcccac9-9715-4fd8-bd92-863b84b51b31 None None] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:30.262101 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcccac9-9715-4fd8-bd92-863b84b51b31 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:30.262426 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcccac9-9715-4fd8-bd92-863b84b51b31 None None] PATH_INFO: `/` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:30.264305 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 406/803] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:52:30 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:52:30.265697 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcccac9-9715-4fd8-bd92-863b84b51b31 None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:30.341874 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 398/804] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:52:30 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:52:30.341874 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35faa495-9b20-403c-962a-7a1593c9d3c5 None None] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:30.341874 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35faa495-9b20-403c-962a-7a1593c9d3c5 None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:30.341874 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35faa495-9b20-403c-962a-7a1593c9d3c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:30.361503 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.auth.core [None req-35faa495-9b20-403c-962a-7a1593c9d3c5 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:52:30.364149 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.fernet_utils [None req-35faa495-9b20-403c-962a-7a1593c9d3c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:52:30.397236 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:30.404083 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcccac9-9715-4fd8-bd92-863b84b51b31 None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:30.449224 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 399/805] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:52:30 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:52:30.450396 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2d28ce-a00b-43c8-9955-713b287630b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:30.451207 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2d28ce-a00b-43c8-9955-713b287630b1 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:30.451503 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2d28ce-a00b-43c8-9955-713b287630b1 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:30.451773 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35faa495-9b20-403c-962a-7a1593c9d3c5 None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:30.451882 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2d28ce-a00b-43c8-9955-713b287630b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:30.472931 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2d28ce-a00b-43c8-9955-713b287630b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:30.474451 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2d28ce-a00b-43c8-9955-713b287630b1 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:30.499035 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-fd2d28ce-a00b-43c8-9955-713b287630b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:52:30.499035 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:30.499035 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:30.499035 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:30.499035 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:30.499035 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:30.499035 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:30.499035 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:30.499035 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:30.499035 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:30.499035 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:30.499035 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:30.499035 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:52:30.499035 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:52:30.499035 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:52:30.500110 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:52:30.500110 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:30.500110 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:30.500110 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:30.500110 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:30.500110 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:30.500110 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:30.500110 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:30.500110 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:52:30.500110 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:30.500110 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:30.500110 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:30.500110 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:30.500110 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:30.500110 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:30.500110 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:30.500110 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:30.501254 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:52:30.501254 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:52:30.501254 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:52:30.501254 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:52:30.501254 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 407/806] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:52:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:52:30.502109 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2d28ce-a00b-43c8-9955-713b287630b1 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:30.517810 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:30.580150 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35faa495-9b20-403c-962a-7a1593c9d3c5 None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:30.618770 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63750f55-44d8-45ce-9270-54e6d66a95e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:30.619914 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63750f55-44d8-45ce-9270-54e6d66a95e8 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:30.620295 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63750f55-44d8-45ce-9270-54e6d66a95e8 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:30.620749 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63750f55-44d8-45ce-9270-54e6d66a95e8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:30.632465 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63750f55-44d8-45ce-9270-54e6d66a95e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:52:30.642036 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-63750f55-44d8-45ce-9270-54e6d66a95e8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:52:30.642036 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:30.642036 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:30.642036 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:30.642036 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:30.642036 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:30.642036 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:30.642036 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:30.642036 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:30.642036 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:30.642036 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:30.642036 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:30.642036 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:52:30.642036 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:52:30.642036 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:52:30.642036 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:52:30.643093 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:52:30.643093 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:52:30.643093 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:52:30.643093 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:52:30.643093 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:30.643093 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:30.643093 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:30.643093 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:30.643093 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:30.643093 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:30.643093 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:30.643093 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:52:30.643093 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:30.643093 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:30.643093 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:30.643093 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:30.644039 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:30.644039 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:30.644039 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:30.644039 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:30.644039 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:52:30.644039 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:52:30.644039 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:52:30.644039 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:52:30.644039 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:52:30.644039 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:52:30.644039 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:52:30.644039 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:52:30.644039 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 400/807] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:52:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:30.644817 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2d28ce-a00b-43c8-9955-713b287630b1 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:30.644898 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63750f55-44d8-45ce-9270-54e6d66a95e8 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:30.688686 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f09573-b682-46e1-a01e-9cb75af0985c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:30.689537 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f09573-b682-46e1-a01e-9cb75af0985c None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:30.689824 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f09573-b682-46e1-a01e-9cb75af0985c None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:30.690135 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f09573-b682-46e1-a01e-9cb75af0985c None admin] PATH_INFO: `/v3/services/network` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:30.690801 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f09573-b682-46e1-a01e-9cb75af0985c None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:30.692176 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f09573-b682-46e1-a01e-9cb75af0985c None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:30.705145 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-24f09573-b682-46e1-a01e-9cb75af0985c None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 19 14:52:30.705145 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:30.705145 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:52:30.705145 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:52:30.705145 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:52:30.705145 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:52:30.705145 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:52:30.705145 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:52:30.705145 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 14:52:30.705145 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:52:30.705145 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:52:30.705145 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:52:30.705145 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:30.705145 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:30.705145 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:30.705145 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:30.705145 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:30.706551 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:30.706551 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:30.706551 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:30.706551 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:30.706551 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:30.706551 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:30.706551 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:52:30.706551 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:52:30.706551 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:52:30.706551 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:52:30.706551 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:30.706551 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:30.706551 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:30.706551 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:30.706551 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:30.706551 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:30.707813 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:30.707813 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:52:30.707813 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:30.707813 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:30.707813 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:30.707813 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:30.707813 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:30.707813 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:30.707813 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:30.707813 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:30.707813 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:52:30.707813 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:52:30.707813 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 14:52:30.707813 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:52:30.707813 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 408/808] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:52:30 2022] GET /identity/v3/services/network => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:30.708705 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f09573-b682-46e1-a01e-9cb75af0985c None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:30.728111 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:30.768135 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63750f55-44d8-45ce-9270-54e6d66a95e8 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:30.771220 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:30.810632 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338116ee-b56d-41c9-8059-0795e07ab909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:30.812152 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338116ee-b56d-41c9-8059-0795e07ab909 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:30.812638 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338116ee-b56d-41c9-8059-0795e07ab909 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:30.813674 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338116ee-b56d-41c9-8059-0795e07ab909 None admin] PATH_INFO: `/v3/domains` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:30.814726 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338116ee-b56d-41c9-8059-0795e07ab909 None admin] RBAC: Adding query filter params (name=Default) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:30.815251 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338116ee-b56d-41c9-8059-0795e07ab909 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:30.816868 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338116ee-b56d-41c9-8059-0795e07ab909 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:30.818372 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f09573-b682-46e1-a01e-9cb75af0985c None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:30.831030 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 401/809] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:52:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:30.832238 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338116ee-b56d-41c9-8059-0795e07ab909 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:30.862025 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f889230-bcbf-483d-9d7e-992220843238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:30.862025 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f889230-bcbf-483d-9d7e-992220843238 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:30.862025 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f889230-bcbf-483d-9d7e-992220843238 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:30.862025 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f889230-bcbf-483d-9d7e-992220843238 None admin] PATH_INFO: `/v3/roles` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:30.862478 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f889230-bcbf-483d-9d7e-992220843238 None admin] RBAC: Adding query filter params (name=admin) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:30.862478 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f889230-bcbf-483d-9d7e-992220843238 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:30.863746 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f889230-bcbf-483d-9d7e-992220843238 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:30.877646 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 409/810] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:52:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:30.878755 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f889230-bcbf-483d-9d7e-992220843238 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:30.946570 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338116ee-b56d-41c9-8059-0795e07ab909 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:30.982429 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:30.987684 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bbe094-a87d-4e71-8b95-f169bd1e2fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:30.988605 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bbe094-a87d-4e71-8b95-f169bd1e2fa2 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:30.989072 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bbe094-a87d-4e71-8b95-f169bd1e2fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:30.989196 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f889230-bcbf-483d-9d7e-992220843238 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:30.989613 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bbe094-a87d-4e71-8b95-f169bd1e2fa2 None admin] PATH_INFO: `/v3/services` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:30.990641 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bbe094-a87d-4e71-8b95-f169bd1e2fa2 None admin] RBAC: Adding query filter params (name=network) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:30.991110 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bbe094-a87d-4e71-8b95-f169bd1e2fa2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:30.992743 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bbe094-a87d-4e71-8b95-f169bd1e2fa2 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:31.002865 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 402/811] 10.222.0.57 () {56 vars in 1112 bytes} [Thu May 19 14:52:30 2022] GET /identity/v3/services?name=network => generated 124 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:31.004074 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bbe094-a87d-4e71-8b95-f169bd1e2fa2 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:31.025879 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c398195-b5f4-45d4-beb8-86c4147648e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:31.026631 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c398195-b5f4-45d4-beb8-86c4147648e0 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:31.026919 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c398195-b5f4-45d4-beb8-86c4147648e0 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:31.027202 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c398195-b5f4-45d4-beb8-86c4147648e0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:31.035417 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c398195-b5f4-45d4-beb8-86c4147648e0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:31.037140 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:31.038209 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c398195-b5f4-45d4-beb8-86c4147648e0 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:31.055076 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-4c398195-b5f4-45d4-beb8-86c4147648e0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:52:31.055076 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:31.055076 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:31.055076 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:31.055076 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:31.055076 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:31.055076 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:31.055076 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:31.055076 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:31.055076 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:31.055076 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:31.055076 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:31.055076 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:52:31.055076 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:52:31.055076 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:52:31.056293 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:52:31.056293 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:31.056293 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:31.056293 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:31.056293 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:31.056293 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:31.056293 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:31.056293 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:31.056293 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:52:31.056293 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:31.056293 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:31.056293 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:31.056293 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:31.056293 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:31.056293 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:31.056293 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:31.056293 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:31.057448 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:52:31.057448 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:52:31.057448 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:52:31.057448 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:52:31.057448 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:52:31.057448 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:52:31.057448 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:52:31.057448 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:52:31.064303 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 410/812] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:52:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:31.065699 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c398195-b5f4-45d4-beb8-86c4147648e0 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:31.118602 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bbe094-a87d-4e71-8b95-f169bd1e2fa2 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:31.157736 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a81cf23-a313-45e6-b2c5-8a2e23c3dba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:31.158592 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a81cf23-a313-45e6-b2c5-8a2e23c3dba8 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:31.158861 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a81cf23-a313-45e6-b2c5-8a2e23c3dba8 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:31.159183 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a81cf23-a313-45e6-b2c5-8a2e23c3dba8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:31.170577 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a81cf23-a313-45e6-b2c5-8a2e23c3dba8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:31.172263 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a81cf23-a313-45e6-b2c5-8a2e23c3dba8 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:31.178112 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c398195-b5f4-45d4-beb8-86c4147648e0 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:31.196716 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-4a81cf23-a313-45e6-b2c5-8a2e23c3dba8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:52:31.196716 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:31.196716 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:31.196716 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:31.196716 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:31.196716 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:31.196716 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:31.196716 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:31.196716 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:31.196716 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:31.196716 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:31.196716 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:31.196716 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:52:31.196716 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:52:31.196716 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:52:31.197948 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:52:31.197948 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:31.197948 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:31.197948 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:31.197948 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:31.197948 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:31.197948 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:31.197948 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:31.197948 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:52:31.197948 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:31.197948 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:31.197948 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:31.197948 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:31.197948 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:31.197948 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:31.197948 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:31.197948 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:31.199062 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:52:31.199062 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:52:31.199062 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:52:31.199062 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:52:31.199062 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 403/813] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:52:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:31.199062 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a81cf23-a313-45e6-b2c5-8a2e23c3dba8 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:31.225035 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25700daf-7e6a-40de-baf2-52db8dd4bc4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:31.226053 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25700daf-7e6a-40de-baf2-52db8dd4bc4c None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:31.226520 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25700daf-7e6a-40de-baf2-52db8dd4bc4c None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:31.227014 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25700daf-7e6a-40de-baf2-52db8dd4bc4c None admin] PATH_INFO: `/v3/services` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:31.227919 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25700daf-7e6a-40de-baf2-52db8dd4bc4c None admin] RBAC: Adding query filter params (type=network) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:31.228391 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25700daf-7e6a-40de-baf2-52db8dd4bc4c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:31.230071 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25700daf-7e6a-40de-baf2-52db8dd4bc4c None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:31.236086 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 411/814] 10.222.0.57 () {56 vars in 1112 bytes} [Thu May 19 14:52:31 2022] GET /identity/v3/services?type=network => generated 350 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:31.237280 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25700daf-7e6a-40de-baf2-52db8dd4bc4c None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:31.241632 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:31.297743 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:31.319704 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a81cf23-a313-45e6-b2c5-8a2e23c3dba8 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:31.353254 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25700daf-7e6a-40de-baf2-52db8dd4bc4c None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:31.359351 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e204b7-666d-41e5-be91-72243a617593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:31.360264 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e204b7-666d-41e5-be91-72243a617593 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:31.360595 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e204b7-666d-41e5-be91-72243a617593 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:31.360826 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e204b7-666d-41e5-be91-72243a617593 None admin] PATH_INFO: `/v3/projects` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:31.361621 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e204b7-666d-41e5-be91-72243a617593 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:31.362513 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e204b7-666d-41e5-be91-72243a617593 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:31.363976 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e204b7-666d-41e5-be91-72243a617593 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:31.381986 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 404/815] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:52:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:31.383172 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e204b7-666d-41e5-be91-72243a617593 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:31.403409 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fabfe9-4dd0-4b44-b0e5-e1348ef52a74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:31.404566 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fabfe9-4dd0-4b44-b0e5-e1348ef52a74 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:31.404863 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fabfe9-4dd0-4b44-b0e5-e1348ef52a74 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:31.405203 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fabfe9-4dd0-4b44-b0e5-e1348ef52a74 None admin] PATH_INFO: `/v3/domains` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:31.406251 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fabfe9-4dd0-4b44-b0e5-e1348ef52a74 None admin] RBAC: Adding query filter params (name=Default) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:31.407206 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fabfe9-4dd0-4b44-b0e5-e1348ef52a74 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:31.409651 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fabfe9-4dd0-4b44-b0e5-e1348ef52a74 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:31.430006 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 412/816] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:52:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:31.431275 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fabfe9-4dd0-4b44-b0e5-e1348ef52a74 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:31.509049 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e204b7-666d-41e5-be91-72243a617593 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:31.520629 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:31.559983 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956862c8-0256-4049-b7b6-bf31c613a056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:31.560575 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956862c8-0256-4049-b7b6-bf31c613a056 None admin] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:31.560864 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956862c8-0256-4049-b7b6-bf31c613a056 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:31.561147 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956862c8-0256-4049-b7b6-bf31c613a056 None admin] PATH_INFO: `/v3/endpoints` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:31.561928 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956862c8-0256-4049-b7b6-bf31c613a056 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:31.564053 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956862c8-0256-4049-b7b6-bf31c613a056 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:31.571545 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fabfe9-4dd0-4b44-b0e5-e1348ef52a74 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:31.576388 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:31.595596 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 405/817] 10.222.0.57 () {60 vars in 1144 bytes} [Thu May 19 14:52:31 2022] POST /identity/v3/endpoints => generated 327 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:52:31.598600 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956862c8-0256-4049-b7b6-bf31c613a056 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:31.633157 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f590dcc-e396-4a78-b0f6-0fb8f0675ae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:31.634515 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f590dcc-e396-4a78-b0f6-0fb8f0675ae2 None admin] REQUEST_METHOD: `PUT` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:31.635066 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f590dcc-e396-4a78-b0f6-0fb8f0675ae2 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:31.635561 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f590dcc-e396-4a78-b0f6-0fb8f0675ae2 None admin] PATH_INFO: `/v3/projects/fc99d742ae004954b6d841293b5fa79c/users/17afb27b2bbe481e959cf31db4470593/roles/028379e01f9245e9b8fc56e8b9e8a6ad` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:31.638116 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f590dcc-e396-4a78-b0f6-0fb8f0675ae2 None admin] RBAC: Authorizing `identity:create_grant(project_id=fc99d742ae004954b6d841293b5fa79c, user_id=17afb27b2bbe481e959cf31db4470593, role_id=028379e01f9245e9b8fc56e8b9e8a6ad)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:31.639870 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f590dcc-e396-4a78-b0f6-0fb8f0675ae2 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:31.671627 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 413/818] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 14:52:31 2022] PUT /identity/v3/projects/fc99d742ae004954b6d841293b5fa79c/users/17afb27b2bbe481e959cf31db4470593/roles/028379e01f9245e9b8fc56e8b9e8a6ad => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:52:31.925283 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956862c8-0256-4049-b7b6-bf31c613a056 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:31.964884 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20505cbf-c007-4872-9325-cfc5a94341cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:31.965726 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20505cbf-c007-4872-9325-cfc5a94341cb None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:31.966052 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20505cbf-c007-4872-9325-cfc5a94341cb None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:31.966331 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20505cbf-c007-4872-9325-cfc5a94341cb None admin] PATH_INFO: `/v3/users/nova` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:31.977073 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20505cbf-c007-4872-9325-cfc5a94341cb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:31.978668 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20505cbf-c007-4872-9325-cfc5a94341cb None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:31.996276 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-20505cbf-c007-4872-9325-cfc5a94341cb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 14:52:31.996276 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:31.996276 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:31.996276 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:31.996276 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:31.996276 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:31.996276 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:31.996276 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:31.996276 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:31.996276 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:31.996276 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:31.996276 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:31.996276 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:52:31.996276 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:52:31.996276 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:52:31.996276 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:52:31.997494 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:31.997494 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:31.997494 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:52:31.997494 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:52:31.997494 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:52:31.997494 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:52:31.997494 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:31.997494 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:31.997494 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:31.997494 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:31.997494 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:31.997494 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:52:31.997494 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:31.997494 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:31.997494 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:31.997494 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:31.997494 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:31.997494 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:31.998739 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:31.998739 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:31.998739 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:52:31.998739 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:52:31.998739 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:52:31.998739 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:52:31.998739 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:52:31.998739 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:52:31.998739 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 14:52:31.998739 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:52:31.999403 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83271def-85f2-4aef-9f59-88150e3a472c None None] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:31.999403 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83271def-85f2-4aef-9f59-88150e3a472c None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:31.999403 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83271def-85f2-4aef-9f59-88150e3a472c None None] PATH_INFO: `/` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:31.999646 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 406/819] 10.222.0.57 () {56 vars in 1093 bytes} [Thu May 19 14:52:31 2022] GET /identity/v3/users/nova => generated 82 bytes in 402 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:31.999728 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 414/820] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:52:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:52:32.006291 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04625f35-c001-4587-962b-b8d451049986 None None] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:32.006585 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04625f35-c001-4587-962b-b8d451049986 None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:32.006706 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83271def-85f2-4aef-9f59-88150e3a472c None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:32.006871 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04625f35-c001-4587-962b-b8d451049986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:32.027571 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.auth.core [None req-04625f35-c001-4587-962b-b8d451049986 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:52:32.028801 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.fernet_utils [None req-04625f35-c001-4587-962b-b8d451049986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:52:32.105248 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:32.124189 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 407/821] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:52:32 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:52:32.140642 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83271def-85f2-4aef-9f59-88150e3a472c None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:32.148387 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04625f35-c001-4587-962b-b8d451049986 None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:32.170266 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:32.182837 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7751f3-bba1-4419-afde-0f1ac05d2064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:32.183517 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7751f3-bba1-4419-afde-0f1ac05d2064 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:32.184834 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7751f3-bba1-4419-afde-0f1ac05d2064 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:32.185132 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7751f3-bba1-4419-afde-0f1ac05d2064 None admin] PATH_INFO: `/v3/users` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:32.186163 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7751f3-bba1-4419-afde-0f1ac05d2064 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:32.186490 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7751f3-bba1-4419-afde-0f1ac05d2064 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:32.188701 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7751f3-bba1-4419-afde-0f1ac05d2064 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:32.216304 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 415/822] 10.222.0.57 () {56 vars in 1133 bytes} [Thu May 19 14:52:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:32.217673 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d881ab60-fd1e-4a0a-b90f-92a9cd65f96b None None] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:32.218026 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d881ab60-fd1e-4a0a-b90f-92a9cd65f96b None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:32.218583 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d881ab60-fd1e-4a0a-b90f-92a9cd65f96b None None] PATH_INFO: `/` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:32.221483 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 416/823] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:52:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:52:32.222596 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d881ab60-fd1e-4a0a-b90f-92a9cd65f96b None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:32.259495 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04625f35-c001-4587-962b-b8d451049986 None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:32.299762 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79794474-3801-41b0-b8bf-738b0cf0babb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:32.300585 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79794474-3801-41b0-b8bf-738b0cf0babb None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:32.300873 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79794474-3801-41b0-b8bf-738b0cf0babb None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:32.301180 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79794474-3801-41b0-b8bf-738b0cf0babb None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:32.301824 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79794474-3801-41b0-b8bf-738b0cf0babb None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:32.303241 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79794474-3801-41b0-b8bf-738b0cf0babb None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:32.315731 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-79794474-3801-41b0-b8bf-738b0cf0babb None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 14:52:32.315731 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:32.315731 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:52:32.315731 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:52:32.315731 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:52:32.315731 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:52:32.315731 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:52:32.315731 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:52:32.315731 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 14:52:32.315731 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:52:32.315731 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:52:32.315731 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:52:32.315731 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:32.315731 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:32.315731 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:32.315731 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:32.316861 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:32.316861 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:32.316861 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:32.316861 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:32.316861 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:32.316861 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:32.316861 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:32.316861 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:52:32.316861 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:52:32.316861 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:52:32.316861 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:52:32.316861 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:32.316861 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:32.316861 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:32.316861 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:32.316861 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:32.318000 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:32.318000 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:32.318000 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:52:32.318000 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:32.318000 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:32.318000 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:32.318000 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:32.318000 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:32.318000 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:32.318000 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:32.318000 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:32.318000 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:52:32.318000 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:52:32.318000 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 14:52:32.318000 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:52:32.318000 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 408/824] 10.222.0.57 () {56 vars in 1129 bytes} [Thu May 19 14:52:32 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:52:32.318990 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0abdc52-49b5-454a-8d44-6d1ce727a222 None None] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:32.318990 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0abdc52-49b5-454a-8d44-6d1ce727a222 None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:32.319141 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0abdc52-49b5-454a-8d44-6d1ce727a222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:32.337505 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.auth.core [None req-d0abdc52-49b5-454a-8d44-6d1ce727a222 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:52:32.338733 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.fernet_utils [None req-d0abdc52-49b5-454a-8d44-6d1ce727a222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:52:32.341368 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d881ab60-fd1e-4a0a-b90f-92a9cd65f96b None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:32.379028 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb467d49-4ec2-4134-84ce-186fd22e6d24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:32.379690 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb467d49-4ec2-4134-84ce-186fd22e6d24 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:32.379984 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb467d49-4ec2-4134-84ce-186fd22e6d24 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:32.380266 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb467d49-4ec2-4134-84ce-186fd22e6d24 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:32.389804 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:32.413753 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 409/825] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:52:32 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:52:32.414963 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0abdc52-49b5-454a-8d44-6d1ce727a222 None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:32.437436 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:32.467050 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb467d49-4ec2-4134-84ce-186fd22e6d24 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:32.468540 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb467d49-4ec2-4134-84ce-186fd22e6d24 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:32.529086 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0abdc52-49b5-454a-8d44-6d1ce727a222 None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:32.567496 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3678f15-8a35-4d8c-a64d-21c09f2a7d4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:32.568490 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3678f15-8a35-4d8c-a64d-21c09f2a7d4e None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:32.568949 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3678f15-8a35-4d8c-a64d-21c09f2a7d4e None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:32.569386 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3678f15-8a35-4d8c-a64d-21c09f2a7d4e None admin] PATH_INFO: `/v3/services` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:32.570543 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3678f15-8a35-4d8c-a64d-21c09f2a7d4e None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:32.571022 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3678f15-8a35-4d8c-a64d-21c09f2a7d4e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:32.571495 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 417/826] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:52:32 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:52:32.572346 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb467d49-4ec2-4134-84ce-186fd22e6d24 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:32.573234 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3678f15-8a35-4d8c-a64d-21c09f2a7d4e None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:32.579935 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 410/827] 10.222.0.57 () {56 vars in 1124 bytes} [Thu May 19 14:52:32 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:32.581029 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3678f15-8a35-4d8c-a64d-21c09f2a7d4e None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:32.636783 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:32.660775 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:32.687748 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb467d49-4ec2-4134-84ce-186fd22e6d24 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:32.708760 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3678f15-8a35-4d8c-a64d-21c09f2a7d4e None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:32.727623 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9881ff-87be-4282-9036-35bef91c3c69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:32.728436 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9881ff-87be-4282-9036-35bef91c3c69 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:32.728731 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9881ff-87be-4282-9036-35bef91c3c69 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:32.729021 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9881ff-87be-4282-9036-35bef91c3c69 None admin] PATH_INFO: `/v3/roles/service` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:32.740239 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9881ff-87be-4282-9036-35bef91c3c69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:52:32.745331 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eb786b-1aba-4635-a621-07baef417ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:32.746151 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eb786b-1aba-4635-a621-07baef417ecb None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:32.746435 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eb786b-1aba-4635-a621-07baef417ecb None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:32.746711 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eb786b-1aba-4635-a621-07baef417ecb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:32.748058 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-6e9881ff-87be-4282-9036-35bef91c3c69 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:52:32.748058 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:32.748058 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:32.748058 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:32.748058 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:32.748058 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:32.748058 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:32.748058 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:32.748058 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:32.748058 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:32.748058 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:32.748058 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:32.748058 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:52:32.748058 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:52:32.748058 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:52:32.748058 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:52:32.749095 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:52:32.749095 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:52:32.749095 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:52:32.749095 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:52:32.749095 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:32.749095 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:32.749095 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:32.749095 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:32.749095 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:32.749095 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:32.749095 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:32.749095 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:52:32.749095 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:32.749095 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:32.749095 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:32.749095 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:32.751055 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:32.751055 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:32.751055 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:32.751055 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:32.751055 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:52:32.751055 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:52:32.751055 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:52:32.751055 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:52:32.751055 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:52:32.751055 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:52:32.751055 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:52:32.751055 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:52:32.751055 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 418/828] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 14:52:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:32.751055 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9881ff-87be-4282-9036-35bef91c3c69 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:32.766078 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3eb786b-1aba-4635-a621-07baef417ecb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:32.768047 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3eb786b-1aba-4635-a621-07baef417ecb None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:32.788249 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-c3eb786b-1aba-4635-a621-07baef417ecb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:52:32.788249 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:32.788249 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:32.788249 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:32.788249 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:32.788249 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:32.788249 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:32.788249 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:32.788249 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:32.788249 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:32.788249 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:32.788249 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:32.788249 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:52:32.788249 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:52:32.788249 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:52:32.789417 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:52:32.789417 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:32.789417 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:32.789417 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:32.789417 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:32.789417 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:32.789417 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:32.789417 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:32.789417 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:52:32.789417 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:32.789417 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:32.789417 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:32.789417 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:32.789417 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:32.789417 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:32.789417 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:32.789417 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:32.790465 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:52:32.790465 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:52:32.790465 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:52:32.790465 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:52:32.790465 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 411/829] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:52:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:32.790465 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eb786b-1aba-4635-a621-07baef417ecb None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:32.869498 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9881ff-87be-4282-9036-35bef91c3c69 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:32.895333 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:32.902416 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eb786b-1aba-4635-a621-07baef417ecb None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:32.906173 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5a9dc7-a0a6-40a6-953e-48ca68a98646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:32.907419 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5a9dc7-a0a6-40a6-953e-48ca68a98646 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:32.907712 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5a9dc7-a0a6-40a6-953e-48ca68a98646 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:32.907997 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5a9dc7-a0a6-40a6-953e-48ca68a98646 None admin] PATH_INFO: `/v3/services` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:32.908724 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5a9dc7-a0a6-40a6-953e-48ca68a98646 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:32.909041 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5a9dc7-a0a6-40a6-953e-48ca68a98646 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:32.910517 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5a9dc7-a0a6-40a6-953e-48ca68a98646 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:32.912636 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:32.920641 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 419/830] 10.222.0.57 () {56 vars in 1124 bytes} [Thu May 19 14:52:32 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:32.920641 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5a9dc7-a0a6-40a6-953e-48ca68a98646 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:32.949162 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c9ba7b-76d2-444e-b749-5f935332add3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:32.949767 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c9ba7b-76d2-444e-b749-5f935332add3 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:32.950085 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c9ba7b-76d2-444e-b749-5f935332add3 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:32.950434 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c9ba7b-76d2-444e-b749-5f935332add3 None admin] PATH_INFO: `/v3/roles` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:32.951447 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c9ba7b-76d2-444e-b749-5f935332add3 None admin] RBAC: Adding query filter params (name=service) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:32.951770 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c9ba7b-76d2-444e-b749-5f935332add3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:32.953666 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c9ba7b-76d2-444e-b749-5f935332add3 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:32.964407 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 412/831] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 19 14:52:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:32.965252 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c9ba7b-76d2-444e-b749-5f935332add3 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:33.033836 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5a9dc7-a0a6-40a6-953e-48ca68a98646 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:33.071282 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf9579c-589d-4d6b-89bc-68fbeaae1f98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:33.071961 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf9579c-589d-4d6b-89bc-68fbeaae1f98 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:33.072259 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf9579c-589d-4d6b-89bc-68fbeaae1f98 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:33.072547 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf9579c-589d-4d6b-89bc-68fbeaae1f98 None admin] PATH_INFO: `/v3/domains` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:33.073286 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf9579c-589d-4d6b-89bc-68fbeaae1f98 None admin] RBAC: Adding query filter params (name=Default) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:33.073603 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf9579c-589d-4d6b-89bc-68fbeaae1f98 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:33.075029 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf9579c-589d-4d6b-89bc-68fbeaae1f98 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:33.088969 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 420/832] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:52:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:33.089901 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf9579c-589d-4d6b-89bc-68fbeaae1f98 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:33.090630 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c9ba7b-76d2-444e-b749-5f935332add3 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:33.132906 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d545dfeb-f634-476c-b9ea-676e761e6f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:33.133914 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d545dfeb-f634-476c-b9ea-676e761e6f8f None admin] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:33.134381 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d545dfeb-f634-476c-b9ea-676e761e6f8f None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:33.134826 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d545dfeb-f634-476c-b9ea-676e761e6f8f None admin] PATH_INFO: `/v3/endpoints` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:33.135755 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d545dfeb-f634-476c-b9ea-676e761e6f8f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:33.137320 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d545dfeb-f634-476c-b9ea-676e761e6f8f None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:33.165891 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 413/833] 10.222.0.57 () {60 vars in 1144 bytes} [Thu May 19 14:52:32 2022] POST /identity/v3/endpoints => generated 346 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:52:33.165891 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d545dfeb-f634-476c-b9ea-676e761e6f8f None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:33.173253 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:33.196336 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:33.257619 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf9579c-589d-4d6b-89bc-68fbeaae1f98 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:33.308841 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d545dfeb-f634-476c-b9ea-676e761e6f8f None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:33.321084 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea5ca98-b50b-4a11-a6d8-7179c11f6c64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:33.322570 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea5ca98-b50b-4a11-a6d8-7179c11f6c64 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:33.323105 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea5ca98-b50b-4a11-a6d8-7179c11f6c64 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:33.323623 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea5ca98-b50b-4a11-a6d8-7179c11f6c64 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:33.340483 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea5ca98-b50b-4a11-a6d8-7179c11f6c64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:33.344924 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea5ca98-b50b-4a11-a6d8-7179c11f6c64 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:33.349770 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616b481d-abed-496d-b907-a67ce5a0df76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:33.350758 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616b481d-abed-496d-b907-a67ce5a0df76 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:33.351241 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616b481d-abed-496d-b907-a67ce5a0df76 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:33.351241 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616b481d-abed-496d-b907-a67ce5a0df76 None admin] PATH_INFO: `/v3/projects/service` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:33.360361 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616b481d-abed-496d-b907-a67ce5a0df76 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:33.360976 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616b481d-abed-496d-b907-a67ce5a0df76 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:33.375715 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-2ea5ca98-b50b-4a11-a6d8-7179c11f6c64 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:52:33.375715 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:33.375715 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:33.375715 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:33.375715 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:33.375715 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:33.375715 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:33.375715 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:33.375715 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:33.375715 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:33.375715 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:33.375715 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:33.375715 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:52:33.375715 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:52:33.375715 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:52:33.376952 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:52:33.376952 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:33.376952 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:33.376952 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:33.376952 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:33.376952 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:33.376952 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:33.376952 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:33.376952 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:52:33.376952 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:33.376952 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:33.376952 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:33.376952 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:33.376952 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:33.376952 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:33.376952 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:33.376952 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:33.379565 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:52:33.379565 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:52:33.379565 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:52:33.379565 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:52:33.379836 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-616b481d-abed-496d-b907-a67ce5a0df76 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:52:33.379836 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:33.379836 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:33.379836 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:33.379836 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:33.379836 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:33.379836 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:33.379836 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:33.379836 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:33.379836 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:33.379836 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:33.379836 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:33.379836 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:52:33.379836 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:52:33.379836 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:52:33.381146 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:52:33.381146 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:33.381146 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:33.381146 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:33.381146 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:33.381146 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:33.381146 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:33.381146 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:33.381146 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:52:33.381146 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:33.381146 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:33.381146 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:33.381146 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:33.381146 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:33.381146 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:33.381146 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:33.381146 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:33.385498 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:52:33.385498 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:52:33.385498 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:52:33.385498 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:52:33.385498 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:52:33.385498 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:52:33.385498 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:52:33.385498 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:52:33.385498 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 414/834] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:52:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:33.389581 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 421/835] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:52:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:33.389745 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616b481d-abed-496d-b907-a67ce5a0df76 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:33.389967 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea5ca98-b50b-4a11-a6d8-7179c11f6c64 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:33.496738 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:33.525459 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea5ca98-b50b-4a11-a6d8-7179c11f6c64 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:33.537657 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616b481d-abed-496d-b907-a67ce5a0df76 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:33.540423 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:33.566177 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b97b8f-5e6d-4fd5-9985-cfdb82489d5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:33.566861 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b97b8f-5e6d-4fd5-9985-cfdb82489d5d None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:33.567156 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b97b8f-5e6d-4fd5-9985-cfdb82489d5d None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:33.567445 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b97b8f-5e6d-4fd5-9985-cfdb82489d5d None admin] PATH_INFO: `/v3/domains` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:33.570103 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b97b8f-5e6d-4fd5-9985-cfdb82489d5d None admin] RBAC: Adding query filter params (name=Default) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:33.570103 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b97b8f-5e6d-4fd5-9985-cfdb82489d5d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:33.570308 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b97b8f-5e6d-4fd5-9985-cfdb82489d5d None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:33.585010 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 422/836] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:52:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:33.591366 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b97b8f-5e6d-4fd5-9985-cfdb82489d5d None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:33.591648 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49d870d-a44c-423e-9fb8-732140b88037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:33.592498 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49d870d-a44c-423e-9fb8-732140b88037 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:33.592940 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49d870d-a44c-423e-9fb8-732140b88037 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:33.594157 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49d870d-a44c-423e-9fb8-732140b88037 None admin] PATH_INFO: `/v3/projects` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:33.595444 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49d870d-a44c-423e-9fb8-732140b88037 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:33.595942 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49d870d-a44c-423e-9fb8-732140b88037 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:33.598247 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49d870d-a44c-423e-9fb8-732140b88037 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:33.614215 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 415/837] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:52:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:33.619705 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49d870d-a44c-423e-9fb8-732140b88037 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:33.739767 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b97b8f-5e6d-4fd5-9985-cfdb82489d5d None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:33.756871 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49d870d-a44c-423e-9fb8-732140b88037 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:33.792690 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543d078a-20a4-4a79-b566-e08922056802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:33.793696 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543d078a-20a4-4a79-b566-e08922056802 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:33.794092 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543d078a-20a4-4a79-b566-e08922056802 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:33.794480 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543d078a-20a4-4a79-b566-e08922056802 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:33.796903 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:33.803788 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543d078a-20a4-4a79-b566-e08922056802 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:33.805279 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543d078a-20a4-4a79-b566-e08922056802 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:33.815973 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30298cb-92a1-4837-bd56-55e3234b486e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:33.817180 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30298cb-92a1-4837-bd56-55e3234b486e None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:33.817762 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30298cb-92a1-4837-bd56-55e3234b486e None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:33.818315 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30298cb-92a1-4837-bd56-55e3234b486e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:33.819382 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30298cb-92a1-4837-bd56-55e3234b486e None admin] RBAC: Adding query filter params (role.id=028379e01f9245e9b8fc56e8b9e8a6ad, scope.project.id=fc99d742ae004954b6d841293b5fa79c, user.id=5d2aa8c8870a49d98f23f26e036695ef) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:33.820193 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30298cb-92a1-4837-bd56-55e3234b486e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:33.821913 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30298cb-92a1-4837-bd56-55e3234b486e None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:33.825183 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-543d078a-20a4-4a79-b566-e08922056802 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 14:52:33.825183 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:33.825183 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:33.825183 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:33.825183 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:33.825183 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:33.825183 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:33.825183 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:33.825183 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:33.825183 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:33.825183 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:33.825183 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:33.825183 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:52:33.825183 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:52:33.825183 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:52:33.825183 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:52:33.826320 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:33.826320 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:33.826320 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:52:33.826320 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:52:33.826320 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:52:33.826320 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:52:33.826320 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:33.826320 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:33.826320 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:33.826320 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:33.826320 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:33.826320 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:52:33.826320 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:33.826320 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:33.826320 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:33.826320 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:33.826320 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:33.826320 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:33.827478 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:33.827478 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:33.827478 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:52:33.827478 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:52:33.827478 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:52:33.827478 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:52:33.827478 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:52:33.827478 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:52:33.827478 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 14:52:33.827478 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:52:33.832091 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 423/838] 10.222.0.57 () {56 vars in 1117 bytes} [Thu May 19 14:52:33 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:33.833030 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543d078a-20a4-4a79-b566-e08922056802 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:33.842243 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 416/839] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:52:33 2022] GET /identity/v3/role_assignments?user.id=5d2aa8c8870a49d98f23f26e036695ef&scope.project.id=fc99d742ae004954b6d841293b5fa79c&role.id=028379e01f9245e9b8fc56e8b9e8a6ad => generated 608 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:52:33.876765 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:33.958437 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543d078a-20a4-4a79-b566-e08922056802 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:33.997864 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fee3315-6782-4a37-9c4a-bd6a4e5ab314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:33.998588 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fee3315-6782-4a37-9c4a-bd6a4e5ab314 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:33.998872 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fee3315-6782-4a37-9c4a-bd6a4e5ab314 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:33.999149 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fee3315-6782-4a37-9c4a-bd6a4e5ab314 None admin] PATH_INFO: `/v3/users` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:34.000007 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fee3315-6782-4a37-9c4a-bd6a4e5ab314 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:34.000352 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fee3315-6782-4a37-9c4a-bd6a4e5ab314 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:34.001818 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fee3315-6782-4a37-9c4a-bd6a4e5ab314 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:34.022653 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 424/840] 10.222.0.57 () {56 vars in 1149 bytes} [Thu May 19 14:52:33 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:34.028278 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30298cb-92a1-4837-bd56-55e3234b486e None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:34.144542 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30298cb-92a1-4837-bd56-55e3234b486e None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:34.184262 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fce1c7-7dcf-4769-8da2-7ced329af70d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:34.185179 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fce1c7-7dcf-4769-8da2-7ced329af70d None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:34.185649 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fce1c7-7dcf-4769-8da2-7ced329af70d None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:34.186146 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fce1c7-7dcf-4769-8da2-7ced329af70d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:34.252178 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:34.261358 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fce1c7-7dcf-4769-8da2-7ced329af70d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:34.262967 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fce1c7-7dcf-4769-8da2-7ced329af70d None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:34.341131 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86438cf6-9540-4f67-97e6-fc17b958045a None None] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:34.341422 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86438cf6-9540-4f67-97e6-fc17b958045a None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:34.341702 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86438cf6-9540-4f67-97e6-fc17b958045a None None] PATH_INFO: `/` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:34.343426 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 425/841] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:52:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:52:34.349157 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083cacd3-0fd9-4685-aba6-0e213e1cbeb7 None None] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:34.349685 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083cacd3-0fd9-4685-aba6-0e213e1cbeb7 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:34.350283 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083cacd3-0fd9-4685-aba6-0e213e1cbeb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:34.367994 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.auth.core [None req-083cacd3-0fd9-4685-aba6-0e213e1cbeb7 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:52:34.369487 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-083cacd3-0fd9-4685-aba6-0e213e1cbeb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:52:34.375076 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 417/842] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:52:34 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:52:34.381159 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fce1c7-7dcf-4769-8da2-7ced329af70d None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:34.456696 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 426/843] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:52:34 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:52:34.482344 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083cacd3-0fd9-4685-aba6-0e213e1cbeb7 None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:34.511102 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:34.516332 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:34.518535 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fce1c7-7dcf-4769-8da2-7ced329af70d None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:34.558841 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcbaadb-6c4f-4624-aec1-4ce63d8a6bd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:34.559670 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcbaadb-6c4f-4624-aec1-4ce63d8a6bd5 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:34.559956 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcbaadb-6c4f-4624-aec1-4ce63d8a6bd5 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:34.560226 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcbaadb-6c4f-4624-aec1-4ce63d8a6bd5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:34.572800 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcbaadb-6c4f-4624-aec1-4ce63d8a6bd5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:34.574433 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcbaadb-6c4f-4624-aec1-4ce63d8a6bd5 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:34.594420 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-dfcbaadb-6c4f-4624-aec1-4ce63d8a6bd5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:52:34.594420 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:34.594420 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:34.594420 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:34.594420 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:34.594420 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:34.594420 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:34.594420 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:34.594420 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:34.594420 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:34.594420 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:34.594420 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:34.594420 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:52:34.594420 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:52:34.594420 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:52:34.595742 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:52:34.595742 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:34.595742 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:34.595742 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:34.595742 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:34.595742 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:34.595742 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:34.595742 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:34.595742 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:52:34.595742 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:34.595742 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:34.595742 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:34.595742 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:34.595742 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:34.595742 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:34.595742 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:34.595742 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:34.596989 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:52:34.596989 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:52:34.596989 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:52:34.596989 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:52:34.600532 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 418/844] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:52:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:52:34.601401 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcbaadb-6c4f-4624-aec1-4ce63d8a6bd5 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:34.619489 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083cacd3-0fd9-4685-aba6-0e213e1cbeb7 None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:34.667600 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e032167c-095c-4347-afb8-ac8e4e6211f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:34.668474 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e032167c-095c-4347-afb8-ac8e4e6211f9 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:34.668827 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e032167c-095c-4347-afb8-ac8e4e6211f9 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:34.669204 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e032167c-095c-4347-afb8-ac8e4e6211f9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:34.680920 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e032167c-095c-4347-afb8-ac8e4e6211f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:52:34.689046 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-e032167c-095c-4347-afb8-ac8e4e6211f9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:52:34.689046 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:34.689046 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:34.689046 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:34.689046 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:34.689046 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:34.689046 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:34.689046 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:34.689046 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:34.689046 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:34.689046 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:34.689046 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:34.689046 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:52:34.689046 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:52:34.689046 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:52:34.689046 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:52:34.690387 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:52:34.690387 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:52:34.690387 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:52:34.690387 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:52:34.690387 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:34.690387 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:34.690387 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:34.690387 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:34.690387 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:34.690387 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:34.690387 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:34.690387 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:52:34.690387 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:34.690387 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:34.690387 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:34.690387 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:34.691594 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:34.691594 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:34.691594 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:34.691594 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:34.691594 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:52:34.691594 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:52:34.691594 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:52:34.691594 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:52:34.691594 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:52:34.691594 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:52:34.691594 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:52:34.691594 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:52:34.692517 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 427/845] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:52:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:34.698804 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e032167c-095c-4347-afb8-ac8e4e6211f9 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:34.743589 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcbaadb-6c4f-4624-aec1-4ce63d8a6bd5 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:34.784652 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183fb270-3fa9-467d-9532-556e4763c469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:34.785876 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:34.786783 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183fb270-3fa9-467d-9532-556e4763c469 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:34.787302 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183fb270-3fa9-467d-9532-556e4763c469 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:34.787753 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183fb270-3fa9-467d-9532-556e4763c469 None admin] PATH_INFO: `/v3/domains` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:34.788648 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183fb270-3fa9-467d-9532-556e4763c469 None admin] RBAC: Adding query filter params (name=Default) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:34.789142 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183fb270-3fa9-467d-9532-556e4763c469 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:34.790754 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183fb270-3fa9-467d-9532-556e4763c469 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:34.805606 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 419/846] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:52:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:34.813469 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e032167c-095c-4347-afb8-ac8e4e6211f9 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:34.813661 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183fb270-3fa9-467d-9532-556e4763c469 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:34.821465 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:34.844077 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee3c460-d87e-4fef-b409-417235c4d94a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:34.844761 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee3c460-d87e-4fef-b409-417235c4d94a None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:34.845055 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee3c460-d87e-4fef-b409-417235c4d94a None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:34.845341 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee3c460-d87e-4fef-b409-417235c4d94a None admin] PATH_INFO: `/v3/roles` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:34.846148 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee3c460-d87e-4fef-b409-417235c4d94a None admin] RBAC: Adding query filter params (name=admin) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:34.846468 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee3c460-d87e-4fef-b409-417235c4d94a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:34.847857 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee3c460-d87e-4fef-b409-417235c4d94a None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:34.861806 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 428/847] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:52:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:34.863118 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee3c460-d87e-4fef-b409-417235c4d94a None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:34.935474 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183fb270-3fa9-467d-9532-556e4763c469 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:34.971202 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee3c460-d87e-4fef-b409-417235c4d94a None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:34.979496 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf863ec-ab8c-44b9-ad05-c94c2dd48152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:34.980405 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf863ec-ab8c-44b9-ad05-c94c2dd48152 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:34.980703 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf863ec-ab8c-44b9-ad05-c94c2dd48152 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:34.980988 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf863ec-ab8c-44b9-ad05-c94c2dd48152 None admin] PATH_INFO: `/v3/projects/service` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:34.989524 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf863ec-ab8c-44b9-ad05-c94c2dd48152 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:34.991204 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf863ec-ab8c-44b9-ad05-c94c2dd48152 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:35.006954 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-ebf863ec-ab8c-44b9-ad05-c94c2dd48152 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:52:35.006954 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:35.006954 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:35.006954 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:35.006954 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:35.006954 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:35.006954 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:35.006954 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:35.006954 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:35.006954 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:35.006954 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:35.006954 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:35.006954 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:52:35.006954 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:52:35.006954 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:52:35.008198 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:52:35.008198 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:35.008198 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:35.008198 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:35.008198 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:35.008198 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:35.008198 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:35.008198 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:35.008198 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:52:35.008198 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:35.008198 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:35.008198 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:35.008198 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:35.008198 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:35.008198 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:35.008198 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:35.008198 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:35.009419 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:52:35.009419 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:52:35.009419 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:52:35.009419 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:52:35.009419 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:52:35.009419 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:52:35.009419 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:52:35.009419 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:52:35.010041 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b18ccf-c6c9-439d-bc6f-23ad4baf43cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:35.010167 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 420/848] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:52:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:35.013186 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b18ccf-c6c9-439d-bc6f-23ad4baf43cc None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:35.013490 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b18ccf-c6c9-439d-bc6f-23ad4baf43cc None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:35.013780 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b18ccf-c6c9-439d-bc6f-23ad4baf43cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:35.013936 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf863ec-ab8c-44b9-ad05-c94c2dd48152 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:35.027466 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b18ccf-c6c9-439d-bc6f-23ad4baf43cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:35.028920 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b18ccf-c6c9-439d-bc6f-23ad4baf43cc None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:35.030930 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:35.050648 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-24b18ccf-c6c9-439d-bc6f-23ad4baf43cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:52:35.050648 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:35.050648 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:35.050648 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:35.050648 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:35.050648 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:35.050648 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:35.050648 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:35.050648 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:35.050648 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:35.050648 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:35.050648 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:35.050648 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:52:35.050648 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:52:35.050648 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:52:35.051929 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:52:35.051929 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:35.051929 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:35.051929 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:35.051929 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:35.051929 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:35.051929 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:35.051929 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:35.051929 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:52:35.051929 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:35.051929 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:35.051929 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:35.051929 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:35.051929 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:35.051929 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:35.051929 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:35.051929 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:35.053030 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:52:35.053030 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:52:35.053030 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:52:35.053030 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:52:35.053030 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 429/849] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:52:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:52:35.059287 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b18ccf-c6c9-439d-bc6f-23ad4baf43cc None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:35.094566 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:35.141639 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf863ec-ab8c-44b9-ad05-c94c2dd48152 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:35.186869 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b18ccf-c6c9-439d-bc6f-23ad4baf43cc None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:35.190841 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadb1fac-93b4-46a7-83f2-be8013119d3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:35.191992 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadb1fac-93b4-46a7-83f2-be8013119d3f None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:35.192514 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadb1fac-93b4-46a7-83f2-be8013119d3f None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:35.193004 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadb1fac-93b4-46a7-83f2-be8013119d3f None admin] PATH_INFO: `/v3/projects` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:35.194108 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadb1fac-93b4-46a7-83f2-be8013119d3f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:35.194906 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadb1fac-93b4-46a7-83f2-be8013119d3f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:35.197350 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadb1fac-93b4-46a7-83f2-be8013119d3f None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:35.218762 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 421/850] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:52:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:35.220451 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadb1fac-93b4-46a7-83f2-be8013119d3f None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:35.237654 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adcc6b1-515c-4ad4-a0c5-77339cb4115e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:35.238729 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adcc6b1-515c-4ad4-a0c5-77339cb4115e None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:35.239202 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adcc6b1-515c-4ad4-a0c5-77339cb4115e None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:35.239677 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adcc6b1-515c-4ad4-a0c5-77339cb4115e None admin] PATH_INFO: `/v3/domains` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:35.240603 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adcc6b1-515c-4ad4-a0c5-77339cb4115e None admin] RBAC: Adding query filter params (name=Default) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:35.241115 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adcc6b1-515c-4ad4-a0c5-77339cb4115e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:35.242713 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adcc6b1-515c-4ad4-a0c5-77339cb4115e None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:35.256112 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 430/851] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:52:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:35.261958 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adcc6b1-515c-4ad4-a0c5-77339cb4115e None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:35.333758 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadb1fac-93b4-46a7-83f2-be8013119d3f None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:35.334250 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:35.362767 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:35.371583 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adcc6b1-515c-4ad4-a0c5-77339cb4115e None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:35.374233 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6246975b-67f5-40fd-927d-0282a27da92a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:35.375279 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6246975b-67f5-40fd-927d-0282a27da92a None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:35.375946 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6246975b-67f5-40fd-927d-0282a27da92a None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:35.376453 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6246975b-67f5-40fd-927d-0282a27da92a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:35.377511 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6246975b-67f5-40fd-927d-0282a27da92a None admin] RBAC: Adding query filter params (role.id=08f16502894349708527e30032247468, scope.project.id=fc99d742ae004954b6d841293b5fa79c, user.id=cd0da70fc3c1477d8202726187ab5fde) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:35.378085 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6246975b-67f5-40fd-927d-0282a27da92a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:35.379765 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6246975b-67f5-40fd-927d-0282a27da92a None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:35.386837 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 422/852] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:52:35 2022] GET /identity/v3/role_assignments?user.id=cd0da70fc3c1477d8202726187ab5fde&scope.project.id=fc99d742ae004954b6d841293b5fa79c&role.id=08f16502894349708527e30032247468 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:52:35.411210 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549bd05a-3385-4e1f-a1cd-2a2deb9ab0ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:35.412120 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549bd05a-3385-4e1f-a1cd-2a2deb9ab0ba None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:35.412421 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549bd05a-3385-4e1f-a1cd-2a2deb9ab0ba None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:35.412711 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549bd05a-3385-4e1f-a1cd-2a2deb9ab0ba None admin] PATH_INFO: `/v3/users/swift` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:35.423573 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549bd05a-3385-4e1f-a1cd-2a2deb9ab0ba None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:35.425198 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549bd05a-3385-4e1f-a1cd-2a2deb9ab0ba None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:35.442770 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-549bd05a-3385-4e1f-a1cd-2a2deb9ab0ba None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 14:52:35.442770 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:35.442770 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:35.442770 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:35.442770 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:35.442770 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:35.442770 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:35.442770 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:35.442770 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:35.442770 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:35.442770 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:35.442770 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:35.442770 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:52:35.442770 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:52:35.442770 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:52:35.442770 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:52:35.443952 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:35.443952 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:35.443952 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:52:35.443952 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:52:35.443952 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:52:35.443952 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:52:35.443952 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:35.443952 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:35.443952 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:35.443952 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:35.443952 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:35.443952 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:52:35.443952 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:35.443952 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:35.443952 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:35.443952 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:35.443952 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:35.443952 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:35.445331 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:35.445331 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:35.445331 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:52:35.445331 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:52:35.445331 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:52:35.445331 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:52:35.445331 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:52:35.445331 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:52:35.445331 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 14:52:35.445331 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:52:35.449443 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 431/853] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:52:35 2022] GET /identity/v3/users/swift => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:35.451063 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6246975b-67f5-40fd-927d-0282a27da92a None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:35.572439 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6246975b-67f5-40fd-927d-0282a27da92a None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:35.615415 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb6d7e3-3aaf-490a-a622-8a57c270bbbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:35.616111 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb6d7e3-3aaf-490a-a622-8a57c270bbbc None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:35.616403 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb6d7e3-3aaf-490a-a622-8a57c270bbbc None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:35.616686 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb6d7e3-3aaf-490a-a622-8a57c270bbbc None admin] PATH_INFO: `/v3/users` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:35.617536 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb6d7e3-3aaf-490a-a622-8a57c270bbbc None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:35.617879 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb6d7e3-3aaf-490a-a622-8a57c270bbbc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:35.619328 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb6d7e3-3aaf-490a-a622-8a57c270bbbc None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:35.646210 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 423/854] 10.222.0.57 () {56 vars in 1135 bytes} [Thu May 19 14:52:35 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:35.649834 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549bd05a-3385-4e1f-a1cd-2a2deb9ab0ba None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:35.709130 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8109cb09-f0f8-4bbc-8f1d-f77a6557a654 None None] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:35.709483 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8109cb09-f0f8-4bbc-8f1d-f77a6557a654 None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:35.709711 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8109cb09-f0f8-4bbc-8f1d-f77a6557a654 None None] PATH_INFO: `/` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:35.711306 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 424/855] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:52:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:52:35.716993 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d53eae-8ad9-4d0e-ae91-c4d142158f00 None None] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:35.717280 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d53eae-8ad9-4d0e-ae91-c4d142158f00 None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:35.717895 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d53eae-8ad9-4d0e-ae91-c4d142158f00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:35.738514 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.auth.core [None req-e5d53eae-8ad9-4d0e-ae91-c4d142158f00 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:52:35.739872 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.fernet_utils [None req-e5d53eae-8ad9-4d0e-ae91-c4d142158f00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:52:35.773232 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549bd05a-3385-4e1f-a1cd-2a2deb9ab0ba None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:35.789252 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:35.790972 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:35.812049 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55e790a-2641-470d-990c-fa65a89646d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:35.812951 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55e790a-2641-470d-990c-fa65a89646d9 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:35.813316 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55e790a-2641-470d-990c-fa65a89646d9 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:35.813714 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55e790a-2641-470d-990c-fa65a89646d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:35.818704 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 425/856] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:52:35 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:52:35.845795 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d53eae-8ad9-4d0e-ae91-c4d142158f00 None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:35.890064 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55e790a-2641-470d-990c-fa65a89646d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:35.891827 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55e790a-2641-470d-990c-fa65a89646d9 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:35.955693 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d53eae-8ad9-4d0e-ae91-c4d142158f00 None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:35.996091 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc1e627-a762-4de3-a558-7249004e63ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:35.996895 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc1e627-a762-4de3-a558-7249004e63ff None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:35.997176 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc1e627-a762-4de3-a558-7249004e63ff None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:35.997448 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc1e627-a762-4de3-a558-7249004e63ff None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:35.998144 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc1e627-a762-4de3-a558-7249004e63ff None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:35.998287 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 432/857] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:52:35 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:52:36.004288 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55e790a-2641-470d-990c-fa65a89646d9 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:36.004475 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc1e627-a762-4de3-a558-7249004e63ff None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:36.017572 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-cdc1e627-a762-4de3-a558-7249004e63ff None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 14:52:36.017572 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:36.017572 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:52:36.017572 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:52:36.017572 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:52:36.017572 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:52:36.017572 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:52:36.017572 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:52:36.017572 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 14:52:36.017572 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:52:36.017572 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:52:36.017572 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:52:36.017572 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:36.017572 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:36.017572 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:36.017572 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:36.017572 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:36.018915 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:36.018915 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:36.018915 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:36.018915 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:36.018915 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:36.018915 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:36.018915 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:52:36.018915 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:52:36.018915 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:52:36.018915 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:52:36.018915 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:36.018915 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:36.018915 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:36.018915 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:36.018915 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:36.018915 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:36.019914 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:36.019914 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:52:36.019914 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:36.019914 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:36.019914 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:36.019914 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:36.019914 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:36.019914 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:36.019914 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:36.019914 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:36.019914 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:52:36.019914 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:52:36.019914 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 14:52:36.019914 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:52:36.019914 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 426/858] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 19 14:52:35 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:36.024449 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:36.027058 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc1e627-a762-4de3-a558-7249004e63ff None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:36.085519 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:36.144263 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55e790a-2641-470d-990c-fa65a89646d9 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:36.152498 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc1e627-a762-4de3-a558-7249004e63ff None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:36.195056 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc5ca8b-1403-49d0-a31e-f3f9589ab90e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:36.195924 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc5ca8b-1403-49d0-a31e-f3f9589ab90e None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:36.196319 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc5ca8b-1403-49d0-a31e-f3f9589ab90e None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:36.196704 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc5ca8b-1403-49d0-a31e-f3f9589ab90e None admin] PATH_INFO: `/v3/services` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:36.197534 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc5ca8b-1403-49d0-a31e-f3f9589ab90e None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:36.197994 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc5ca8b-1403-49d0-a31e-f3f9589ab90e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:36.198498 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37edb99-20d3-4eac-a87d-527e4498f2f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:36.199479 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37edb99-20d3-4eac-a87d-527e4498f2f7 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:36.199696 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc5ca8b-1403-49d0-a31e-f3f9589ab90e None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:36.200099 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37edb99-20d3-4eac-a87d-527e4498f2f7 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:36.200538 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37edb99-20d3-4eac-a87d-527e4498f2f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:36.206707 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 427/859] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 19 14:52:36 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:36.212195 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc5ca8b-1403-49d0-a31e-f3f9589ab90e None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:36.214435 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37edb99-20d3-4eac-a87d-527e4498f2f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:36.217361 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37edb99-20d3-4eac-a87d-527e4498f2f7 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:36.238645 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-e37edb99-20d3-4eac-a87d-527e4498f2f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:52:36.238645 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:36.238645 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:36.238645 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:36.238645 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:36.238645 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:36.238645 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:36.238645 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:36.238645 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:36.238645 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:36.238645 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:36.238645 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:36.238645 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:52:36.238645 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:52:36.238645 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:52:36.239753 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:52:36.239753 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:36.239753 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:36.239753 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:36.239753 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:36.239753 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:36.239753 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:36.239753 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:36.239753 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:52:36.239753 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:36.239753 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:36.239753 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:36.239753 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:36.239753 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:36.239753 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:36.239753 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:36.239753 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:36.240803 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:52:36.240803 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:52:36.240803 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:52:36.240803 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:52:36.247599 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 433/860] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:52:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:36.249210 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37edb99-20d3-4eac-a87d-527e4498f2f7 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:36.319257 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc5ca8b-1403-49d0-a31e-f3f9589ab90e None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:36.325024 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:36.327013 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:36.357774 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b642805b-8476-41c1-a7d5-df18410a5e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:36.358477 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b642805b-8476-41c1-a7d5-df18410a5e72 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:36.358767 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b642805b-8476-41c1-a7d5-df18410a5e72 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:36.359052 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b642805b-8476-41c1-a7d5-df18410a5e72 None admin] PATH_INFO: `/v3/services` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:36.359915 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b642805b-8476-41c1-a7d5-df18410a5e72 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:36.360284 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b642805b-8476-41c1-a7d5-df18410a5e72 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:36.361822 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b642805b-8476-41c1-a7d5-df18410a5e72 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:36.367139 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 428/861] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 19 14:52:36 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:36.367792 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37edb99-20d3-4eac-a87d-527e4498f2f7 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:36.368429 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c262408b-2f3a-420f-b321-b008eb844bb3 None None] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:36.368721 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c262408b-2f3a-420f-b321-b008eb844bb3 None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:36.369004 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c262408b-2f3a-420f-b321-b008eb844bb3 None None] PATH_INFO: `/` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:36.370643 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 429/862] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:52:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:52:36.380599 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ee98f7-8670-4e06-9ffb-b6250c419422 None None] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:36.380944 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ee98f7-8670-4e06-9ffb-b6250c419422 None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:36.381170 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ee98f7-8670-4e06-9ffb-b6250c419422 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:36.401352 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.auth.core [None req-52ee98f7-8670-4e06-9ffb-b6250c419422 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:52:36.402583 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.fernet_utils [None req-52ee98f7-8670-4e06-9ffb-b6250c419422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:52:36.414447 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cae6bc-f346-4f46-9454-cb1547c6d6b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:36.415642 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cae6bc-f346-4f46-9454-cb1547c6d6b0 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:36.416169 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cae6bc-f346-4f46-9454-cb1547c6d6b0 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:36.416668 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cae6bc-f346-4f46-9454-cb1547c6d6b0 None admin] PATH_INFO: `/v3/domains` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:36.417697 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cae6bc-f346-4f46-9454-cb1547c6d6b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:36.418261 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cae6bc-f346-4f46-9454-cb1547c6d6b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:36.419997 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cae6bc-f346-4f46-9454-cb1547c6d6b0 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:36.441073 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 434/863] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:52:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:52:36.442888 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cae6bc-f346-4f46-9454-cb1547c6d6b0 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:36.492494 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 430/864] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:52:36 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:52:36.510048 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ee98f7-8670-4e06-9ffb-b6250c419422 None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:36.560893 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cae6bc-f346-4f46-9454-cb1547c6d6b0 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:36.591649 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:36.610799 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf05c5fa-cc3c-4bd1-9941-aa19f840f0c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:36.611616 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf05c5fa-cc3c-4bd1-9941-aa19f840f0c5 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:36.611909 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf05c5fa-cc3c-4bd1-9941-aa19f840f0c5 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:36.612194 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf05c5fa-cc3c-4bd1-9941-aa19f840f0c5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:36.623091 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:36.623739 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf05c5fa-cc3c-4bd1-9941-aa19f840f0c5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:36.625969 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf05c5fa-cc3c-4bd1-9941-aa19f840f0c5 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:36.627767 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ee98f7-8670-4e06-9ffb-b6250c419422 None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:36.646751 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-cf05c5fa-cc3c-4bd1-9941-aa19f840f0c5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:52:36.646751 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:36.646751 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:36.646751 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:36.646751 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:36.646751 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:36.646751 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:36.646751 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:36.646751 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:36.646751 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:36.646751 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:36.646751 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:36.646751 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:52:36.646751 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:52:36.646751 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:52:36.648103 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:52:36.648103 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:36.648103 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:36.648103 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:36.648103 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:36.648103 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:36.648103 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:36.648103 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:36.648103 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:52:36.648103 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:36.648103 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:36.648103 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:36.648103 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:36.648103 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:36.648103 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:36.648103 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:36.648103 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:36.649449 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:52:36.649449 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:52:36.649449 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:52:36.649449 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:52:36.649449 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:52:36.649449 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:52:36.649449 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:52:36.649449 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:52:36.649449 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 435/865] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:52:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:52:36.669470 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955bc7db-e98d-45e3-8755-c7c6c1690b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:36.670336 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955bc7db-e98d-45e3-8755-c7c6c1690b00 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:36.670634 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955bc7db-e98d-45e3-8755-c7c6c1690b00 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:36.670897 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf05c5fa-cc3c-4bd1-9941-aa19f840f0c5 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:36.671045 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955bc7db-e98d-45e3-8755-c7c6c1690b00 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:36.671630 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955bc7db-e98d-45e3-8755-c7c6c1690b00 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:36.674239 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955bc7db-e98d-45e3-8755-c7c6c1690b00 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:36.688691 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-955bc7db-e98d-45e3-8755-c7c6c1690b00 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 14:52:36.688691 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:36.688691 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:52:36.688691 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:52:36.688691 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:52:36.688691 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:52:36.688691 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:52:36.688691 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:52:36.688691 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 14:52:36.688691 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:52:36.688691 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:52:36.688691 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:52:36.688691 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:36.688691 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:36.688691 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:36.688691 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:36.690053 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:36.690053 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:36.690053 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:36.690053 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:36.690053 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:36.690053 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:36.690053 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:36.690053 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:52:36.690053 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:52:36.690053 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:52:36.690053 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:52:36.690053 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:36.690053 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:36.690053 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:36.690053 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:36.690053 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:36.691631 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:36.691631 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:36.691631 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:52:36.691631 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:36.691631 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:36.691631 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:36.691631 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:36.691631 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:36.691631 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:36.691631 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:36.691631 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:36.691631 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:52:36.691631 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:52:36.691631 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 14:52:36.691631 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:52:36.691631 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 431/866] 10.222.0.57 () {56 vars in 1132 bytes} [Thu May 19 14:52:36 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:52:36.697374 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955bc7db-e98d-45e3-8755-c7c6c1690b00 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:36.800306 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf05c5fa-cc3c-4bd1-9941-aa19f840f0c5 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:36.815791 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955bc7db-e98d-45e3-8755-c7c6c1690b00 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:36.845147 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9198edf4-f503-425b-ab3d-1de7fbd71a34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:36.846733 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9198edf4-f503-425b-ab3d-1de7fbd71a34 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:36.847467 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9198edf4-f503-425b-ab3d-1de7fbd71a34 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:36.847969 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9198edf4-f503-425b-ab3d-1de7fbd71a34 None admin] PATH_INFO: `/v3/projects` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:36.849139 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9198edf4-f503-425b-ab3d-1de7fbd71a34 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:36.849919 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9198edf4-f503-425b-ab3d-1de7fbd71a34 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:36.852051 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9198edf4-f503-425b-ab3d-1de7fbd71a34 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:36.854395 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a97c46e-6423-4a1d-95bf-22b43c873ace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:36.855067 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a97c46e-6423-4a1d-95bf-22b43c873ace None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:36.855377 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a97c46e-6423-4a1d-95bf-22b43c873ace None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:36.855753 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a97c46e-6423-4a1d-95bf-22b43c873ace None admin] PATH_INFO: `/v3/services` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:36.856472 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a97c46e-6423-4a1d-95bf-22b43c873ace None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:36.856784 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a97c46e-6423-4a1d-95bf-22b43c873ace None admin] RBAC: Authorizing `identity:list_services()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:36.858168 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a97c46e-6423-4a1d-95bf-22b43c873ace None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:36.871937 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 432/868] 10.222.0.57 () {56 vars in 1126 bytes} [Thu May 19 14:52:36 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:52:36.871937 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a97c46e-6423-4a1d-95bf-22b43c873ace None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:36.872222 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 436/868] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:52:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:36.875405 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9198edf4-f503-425b-ab3d-1de7fbd71a34 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:36.903590 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:36.940651 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:36.993543 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a97c46e-6423-4a1d-95bf-22b43c873ace None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:37.008417 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9198edf4-f503-425b-ab3d-1de7fbd71a34 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:37.044625 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0be522-4981-4f59-9777-e4750b666c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:37.045925 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0be522-4981-4f59-9777-e4750b666c13 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:37.046506 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0be522-4981-4f59-9777-e4750b666c13 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:37.047556 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0be522-4981-4f59-9777-e4750b666c13 None admin] PATH_INFO: `/v3/services` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:37.047726 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b132a88-6b69-4404-9021-537a14317caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:37.048455 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b132a88-6b69-4404-9021-537a14317caa None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:37.048809 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b132a88-6b69-4404-9021-537a14317caa None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:37.049074 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0be522-4981-4f59-9777-e4750b666c13 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:37.049202 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b132a88-6b69-4404-9021-537a14317caa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:37.049694 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0be522-4981-4f59-9777-e4750b666c13 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:37.050155 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b132a88-6b69-4404-9021-537a14317caa None admin] RBAC: Adding query filter params (role.id=028379e01f9245e9b8fc56e8b9e8a6ad, scope.project.id=fc99d742ae004954b6d841293b5fa79c, user.id=17afb27b2bbe481e959cf31db4470593) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:37.050554 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b132a88-6b69-4404-9021-537a14317caa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:37.051843 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0be522-4981-4f59-9777-e4750b666c13 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:37.052171 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b132a88-6b69-4404-9021-537a14317caa None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:37.058663 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 433/869] 10.222.0.57 () {56 vars in 1126 bytes} [Thu May 19 14:52:36 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:37.065078 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 437/870] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:52:36 2022] GET /identity/v3/role_assignments?user.id=17afb27b2bbe481e959cf31db4470593&scope.project.id=fc99d742ae004954b6d841293b5fa79c&role.id=028379e01f9245e9b8fc56e8b9e8a6ad => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:37.905191 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b967648-6624-48fe-981a-435a0c63dfd0 None None] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:37.905771 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b967648-6624-48fe-981a-435a0c63dfd0 None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:37.905771 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b967648-6624-48fe-981a-435a0c63dfd0 None None] PATH_INFO: `/` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:37.907457 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 434/871] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:52:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:52:37.913205 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901d09d7-9c79-41b7-9a53-fff7b692e995 None None] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:37.913728 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901d09d7-9c79-41b7-9a53-fff7b692e995 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:37.913728 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901d09d7-9c79-41b7-9a53-fff7b692e995 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:37.931971 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.auth.core [None req-901d09d7-9c79-41b7-9a53-fff7b692e995 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:52:37.933189 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-901d09d7-9c79-41b7-9a53-fff7b692e995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:52:38.007366 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 438/872] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:52:37 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:52:38.031528 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b967648-6624-48fe-981a-435a0c63dfd0 None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:38.132298 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:38.143871 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b967648-6624-48fe-981a-435a0c63dfd0 None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:38.179445 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54042d65-b98b-4e81-be9a-07ee5bd73954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:38.180188 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54042d65-b98b-4e81-be9a-07ee5bd73954 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:38.180481 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54042d65-b98b-4e81-be9a-07ee5bd73954 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:38.180761 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54042d65-b98b-4e81-be9a-07ee5bd73954 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:38.193563 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54042d65-b98b-4e81-be9a-07ee5bd73954 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:52:38.202018 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-54042d65-b98b-4e81-be9a-07ee5bd73954 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 14:52:38.202018 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:38.202018 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:38.202018 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:38.202018 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:38.202018 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:38.202018 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:38.202018 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:38.202018 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:38.202018 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:38.202018 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:38.202018 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:38.202018 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:52:38.202018 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:52:38.202018 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:52:38.202018 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:52:38.203141 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:52:38.203141 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:52:38.203141 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:52:38.203141 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:52:38.203141 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:38.203141 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:38.203141 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:38.203141 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:38.203141 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:38.203141 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:38.203141 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:38.203141 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:52:38.203141 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:38.203141 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:38.203141 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:38.203141 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:38.204951 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:38.204951 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:38.204951 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:38.204951 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:38.204951 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:52:38.204951 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:52:38.204951 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:52:38.204951 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:52:38.204951 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:52:38.204951 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:52:38.204951 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 14:52:38.204951 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:52:38.204951 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 435/873] 10.222.0.57 () {56 vars in 1120 bytes} [Thu May 19 14:52:38 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:38.209801 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901d09d7-9c79-41b7-9a53-fff7b692e995 None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:38.256077 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:38.324094 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901d09d7-9c79-41b7-9a53-fff7b692e995 None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:38.361753 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f279273a-0cfb-49ac-a764-cea5451402fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:38.362480 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f279273a-0cfb-49ac-a764-cea5451402fb None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:38.362771 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f279273a-0cfb-49ac-a764-cea5451402fb None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:38.363052 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f279273a-0cfb-49ac-a764-cea5451402fb None admin] PATH_INFO: `/v3/roles` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:38.363822 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f279273a-0cfb-49ac-a764-cea5451402fb None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:38.364134 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f279273a-0cfb-49ac-a764-cea5451402fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:38.365516 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f279273a-0cfb-49ac-a764-cea5451402fb None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:38.375517 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 439/874] 10.222.0.57 () {56 vars in 1115 bytes} [Thu May 19 14:52:38 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:38.381009 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54042d65-b98b-4e81-be9a-07ee5bd73954 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:38.493057 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54042d65-b98b-4e81-be9a-07ee5bd73954 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:38.529454 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c521961d-c125-4971-a1ab-0c2817537de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:38.530292 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c521961d-c125-4971-a1ab-0c2817537de8 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:38.530581 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c521961d-c125-4971-a1ab-0c2817537de8 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:38.530862 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c521961d-c125-4971-a1ab-0c2817537de8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:38.541665 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c521961d-c125-4971-a1ab-0c2817537de8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:38.543161 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c521961d-c125-4971-a1ab-0c2817537de8 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:38.563322 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-c521961d-c125-4971-a1ab-0c2817537de8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:52:38.563322 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:38.563322 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:38.563322 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:38.563322 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:38.563322 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:38.563322 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:38.563322 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:38.563322 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:38.563322 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:38.563322 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:38.563322 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:38.563322 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:52:38.563322 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:52:38.563322 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:52:38.564470 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:52:38.564470 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:38.564470 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:38.564470 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:38.564470 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:38.564470 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:38.564470 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:38.564470 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:38.564470 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:52:38.564470 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:38.564470 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:38.564470 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:38.564470 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:38.564470 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:38.564470 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:38.564470 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:38.564470 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:38.565555 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:52:38.565555 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:52:38.565555 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:52:38.565555 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:52:38.565555 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 436/875] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:52:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:38.572578 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f279273a-0cfb-49ac-a764-cea5451402fb None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:38.689468 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f279273a-0cfb-49ac-a764-cea5451402fb None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:38.697388 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:38.731658 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337f1716-667e-4324-9b1d-4a03f5141b39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:38.732332 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337f1716-667e-4324-9b1d-4a03f5141b39 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:38.732620 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337f1716-667e-4324-9b1d-4a03f5141b39 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:38.732902 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337f1716-667e-4324-9b1d-4a03f5141b39 None admin] PATH_INFO: `/v3/domains` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:38.733696 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337f1716-667e-4324-9b1d-4a03f5141b39 None admin] RBAC: Adding query filter params (name=Default) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:38.734040 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337f1716-667e-4324-9b1d-4a03f5141b39 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:38.736632 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337f1716-667e-4324-9b1d-4a03f5141b39 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:38.749078 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 440/876] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:52:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:38.754899 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337f1716-667e-4324-9b1d-4a03f5141b39 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:38.755087 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610ab457-e9a3-4e07-96e4-67d66a392f51 None None] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:38.755292 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610ab457-e9a3-4e07-96e4-67d66a392f51 None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:38.755578 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610ab457-e9a3-4e07-96e4-67d66a392f51 None None] PATH_INFO: `/` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:38.757196 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 437/877] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:52:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:52:38.762460 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068ba709-9243-4b15-bc8d-0ce948e84f23 None None] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:38.762755 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068ba709-9243-4b15-bc8d-0ce948e84f23 None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:38.763039 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068ba709-9243-4b15-bc8d-0ce948e84f23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:38.769789 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:38.784586 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.auth.core [None req-068ba709-9243-4b15-bc8d-0ce948e84f23 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:52:38.786029 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.fernet_utils [None req-068ba709-9243-4b15-bc8d-0ce948e84f23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:52:38.879051 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337f1716-667e-4324-9b1d-4a03f5141b39 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:38.879290 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 438/878] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:52:38 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:52:38.901715 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068ba709-9243-4b15-bc8d-0ce948e84f23 None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:38.925320 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8654911f-4e47-4b8c-a146-61ad5e71a62a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:38.927361 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8654911f-4e47-4b8c-a146-61ad5e71a62a None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:38.928167 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8654911f-4e47-4b8c-a146-61ad5e71a62a None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:38.928663 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8654911f-4e47-4b8c-a146-61ad5e71a62a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:38.940626 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8654911f-4e47-4b8c-a146-61ad5e71a62a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:38.942412 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8654911f-4e47-4b8c-a146-61ad5e71a62a None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:38.963937 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-8654911f-4e47-4b8c-a146-61ad5e71a62a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 14:52:38.963937 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:38.963937 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:38.963937 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:38.963937 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:38.963937 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:38.963937 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:38.963937 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:38.963937 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:38.963937 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:38.963937 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:38.963937 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:38.963937 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:52:38.963937 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:52:38.963937 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:52:38.963937 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:52:38.965122 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:38.965122 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:38.965122 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:52:38.965122 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:52:38.965122 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:52:38.965122 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:52:38.965122 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:38.965122 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:38.965122 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:38.965122 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:38.965122 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:38.965122 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:52:38.965122 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:38.965122 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:38.965122 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:38.965122 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:38.965122 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:38.965122 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:38.966350 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:38.966350 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:38.966350 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:52:38.966350 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:52:38.966350 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:52:38.966350 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:52:38.966350 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:52:38.966350 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:52:38.966350 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 14:52:38.966350 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:52:38.972105 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 441/879] 10.222.0.57 () {56 vars in 1117 bytes} [Thu May 19 14:52:38 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:38.973158 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8654911f-4e47-4b8c-a146-61ad5e71a62a None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:38.999019 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:39.027370 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068ba709-9243-4b15-bc8d-0ce948e84f23 None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:39.072336 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e80639b-4d08-4571-bba8-065472d43859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:39.073320 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e80639b-4d08-4571-bba8-065472d43859 None admin] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:39.073786 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e80639b-4d08-4571-bba8-065472d43859 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:39.074250 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e80639b-4d08-4571-bba8-065472d43859 None admin] PATH_INFO: `/v3/services` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:39.075625 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e80639b-4d08-4571-bba8-065472d43859 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:39.077478 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e80639b-4d08-4571-bba8-065472d43859 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:39.081134 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:39.094944 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8654911f-4e47-4b8c-a146-61ad5e71a62a None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:39.105833 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 439/880] 10.222.0.57 () {60 vars in 1141 bytes} [Thu May 19 14:52:38 2022] POST /identity/v3/services => generated 251 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:52:39.144816 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87df82eb-3b8a-4f8a-9955-37873037e235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:39.145807 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87df82eb-3b8a-4f8a-9955-37873037e235 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:39.146380 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87df82eb-3b8a-4f8a-9955-37873037e235 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:39.146825 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87df82eb-3b8a-4f8a-9955-37873037e235 None admin] PATH_INFO: `/v3/users` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:39.147861 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87df82eb-3b8a-4f8a-9955-37873037e235 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:39.148347 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87df82eb-3b8a-4f8a-9955-37873037e235 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:39.150165 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87df82eb-3b8a-4f8a-9955-37873037e235 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:39.176265 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 442/881] 10.222.0.57 () {56 vars in 1149 bytes} [Thu May 19 14:52:38 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:39.181771 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e80639b-4d08-4571-bba8-065472d43859 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:39.310387 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e80639b-4d08-4571-bba8-065472d43859 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:39.353733 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192945e6-bfdd-475f-9e20-e732b58ecb03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:39.354440 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192945e6-bfdd-475f-9e20-e732b58ecb03 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:39.354732 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192945e6-bfdd-475f-9e20-e732b58ecb03 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:39.355016 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192945e6-bfdd-475f-9e20-e732b58ecb03 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:39.424639 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:39.433569 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192945e6-bfdd-475f-9e20-e732b58ecb03 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:39.435500 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192945e6-bfdd-475f-9e20-e732b58ecb03 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:39.476845 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446ec9d2-6e66-47d7-8f93-cf72eab8f909 None None] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:39.477173 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446ec9d2-6e66-47d7-8f93-cf72eab8f909 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:39.477425 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446ec9d2-6e66-47d7-8f93-cf72eab8f909 None None] PATH_INFO: `/` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:39.479096 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 443/882] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:52:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:52:39.484226 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c79334b-ee26-4f5a-8842-0c5aff524cf5 None None] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:39.485085 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c79334b-ee26-4f5a-8842-0c5aff524cf5 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:39.485329 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c79334b-ee26-4f5a-8842-0c5aff524cf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:39.507158 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.auth.core [None req-3c79334b-ee26-4f5a-8842-0c5aff524cf5 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:52:39.508385 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-3c79334b-ee26-4f5a-8842-0c5aff524cf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:52:39.554410 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 440/883] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:52:39 2022] GET /identity/v3/auth/tokens => generated 1252 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 19 14:52:39.554410 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1836a8b-1d08-45dd-84b1-c257869fcd1c None None] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:39.554410 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1836a8b-1d08-45dd-84b1-c257869fcd1c None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:39.554410 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1836a8b-1d08-45dd-84b1-c257869fcd1c None None] PATH_INFO: `/` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:39.554410 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 441/884] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:52:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:52:39.554410 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1836a8b-1d08-45dd-84b1-c257869fcd1c None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:39.584634 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 444/885] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:52:39 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:52:39.587190 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc498c16-89a4-4993-9c23-fbdd4bbc296a None None] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:39.587485 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc498c16-89a4-4993-9c23-fbdd4bbc296a None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:39.587769 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc498c16-89a4-4993-9c23-fbdd4bbc296a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:39.603810 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:39.610844 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.auth.core [None req-dc498c16-89a4-4993-9c23-fbdd4bbc296a None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:52:39.612520 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-dc498c16-89a4-4993-9c23-fbdd4bbc296a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:52:39.680738 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:39.690895 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1836a8b-1d08-45dd-84b1-c257869fcd1c None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:39.701912 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 445/886] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:52:39 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:52:39.701912 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc498c16-89a4-4993-9c23-fbdd4bbc296a None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:39.740803 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ff29b9-269c-46bf-9f61-2dc01fc383d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:39.742098 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ff29b9-269c-46bf-9f61-2dc01fc383d8 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:39.742582 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ff29b9-269c-46bf-9f61-2dc01fc383d8 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:39.743063 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ff29b9-269c-46bf-9f61-2dc01fc383d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:39.759750 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ff29b9-269c-46bf-9f61-2dc01fc383d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:39.762521 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ff29b9-269c-46bf-9f61-2dc01fc383d8 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:39.794331 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-43ff29b9-269c-46bf-9f61-2dc01fc383d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:52:39.794331 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:39.794331 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:39.794331 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:39.794331 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:39.794331 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:39.794331 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:39.794331 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:39.794331 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:39.794331 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:39.794331 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:39.794331 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:39.794331 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:52:39.794331 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:52:39.794331 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:52:39.795430 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:52:39.795430 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:39.795430 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:39.795430 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:39.795430 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:39.795430 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:39.795430 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:39.795430 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:39.795430 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:52:39.795430 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:39.795430 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:39.795430 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:39.795430 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:39.795430 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:39.795430 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:39.795430 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:39.795430 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:39.796517 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:52:39.796517 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:52:39.796517 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:52:39.796517 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:52:39.796517 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 442/887] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:52:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:52:39.798259 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ff29b9-269c-46bf-9f61-2dc01fc383d8 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:39.851566 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc498c16-89a4-4993-9c23-fbdd4bbc296a None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:39.889364 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2887d839-9964-4216-9bed-ed2c439a70fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:39.890610 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2887d839-9964-4216-9bed-ed2c439a70fb None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:39.891180 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2887d839-9964-4216-9bed-ed2c439a70fb None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:39.891756 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2887d839-9964-4216-9bed-ed2c439a70fb None admin] PATH_INFO: `/v3/services/object-store` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:39.892711 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2887d839-9964-4216-9bed-ed2c439a70fb None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:39.894283 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2887d839-9964-4216-9bed-ed2c439a70fb None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:39.897205 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:39.908427 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ff29b9-269c-46bf-9f61-2dc01fc383d8 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:39.909596 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-2887d839-9964-4216-9bed-ed2c439a70fb None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 14:52:39.909596 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:39.909596 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:52:39.909596 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:52:39.909596 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:52:39.909596 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:52:39.909596 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:52:39.909596 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:52:39.909596 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 14:52:39.909596 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:52:39.909596 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:52:39.909596 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:52:39.909596 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:39.909596 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:39.909596 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:39.909596 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:39.909596 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:39.910955 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:39.910955 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:39.910955 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:39.910955 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:39.910955 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:39.910955 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:39.910955 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:52:39.910955 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:52:39.910955 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:52:39.910955 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:52:39.910955 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:39.910955 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:39.910955 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:39.910955 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:39.910955 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:39.910955 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:39.912148 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:39.912148 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:52:39.912148 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:39.912148 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:39.912148 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:39.912148 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:39.912148 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:39.912148 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:39.912148 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:39.912148 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:39.912148 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:52:39.912148 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:52:39.912148 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 14:52:39.912148 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:52:39.914977 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 446/888] 10.222.0.57 () {56 vars in 1126 bytes} [Thu May 19 14:52:39 2022] GET /identity/v3/services/object-store => generated 93 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:39.915933 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2887d839-9964-4216-9bed-ed2c439a70fb None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:39.949026 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e4afab-2680-4b65-9ecd-4b935d6c1872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:39.949697 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e4afab-2680-4b65-9ecd-4b935d6c1872 None admin] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:39.950013 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e4afab-2680-4b65-9ecd-4b935d6c1872 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:39.950293 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e4afab-2680-4b65-9ecd-4b935d6c1872 None admin] PATH_INFO: `/v3/services` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:39.951019 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e4afab-2680-4b65-9ecd-4b935d6c1872 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:39.952386 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e4afab-2680-4b65-9ecd-4b935d6c1872 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:39.969987 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 443/889] 10.222.0.57 () {60 vars in 1141 bytes} [Thu May 19 14:52:39 2022] POST /identity/v3/services => generated 269 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:52:39.969987 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e4afab-2680-4b65-9ecd-4b935d6c1872 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:39.979889 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:40.081884 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2887d839-9964-4216-9bed-ed2c439a70fb None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:40.112434 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e4afab-2680-4b65-9ecd-4b935d6c1872 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:40.133653 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5e6fb5-1faf-4f58-adf7-5981407f34fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:40.134758 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5e6fb5-1faf-4f58-adf7-5981407f34fe None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:40.135282 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5e6fb5-1faf-4f58-adf7-5981407f34fe None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:40.136447 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5e6fb5-1faf-4f58-adf7-5981407f34fe None admin] PATH_INFO: `/v3/domains` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:40.137731 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5e6fb5-1faf-4f58-adf7-5981407f34fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:40.138293 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5e6fb5-1faf-4f58-adf7-5981407f34fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:40.141522 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5e6fb5-1faf-4f58-adf7-5981407f34fe None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:40.152232 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee8857b-08fc-4c26-997e-e525b7d455ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:40.153391 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee8857b-08fc-4c26-997e-e525b7d455ae None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:40.153391 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee8857b-08fc-4c26-997e-e525b7d455ae None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:40.153619 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee8857b-08fc-4c26-997e-e525b7d455ae None admin] PATH_INFO: `/v3/services` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:40.154283 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee8857b-08fc-4c26-997e-e525b7d455ae None admin] RBAC: Adding query filter params (name=object-store) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:40.154599 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee8857b-08fc-4c26-997e-e525b7d455ae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:40.156909 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee8857b-08fc-4c26-997e-e525b7d455ae None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:40.162511 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 447/890] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:52:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:40.168725 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 444/891] 10.222.0.57 () {56 vars in 1122 bytes} [Thu May 19 14:52:39 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:40.169110 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5e6fb5-1faf-4f58-adf7-5981407f34fe None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:40.174655 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee8857b-08fc-4c26-997e-e525b7d455ae None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:40.245020 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:40.264327 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:40.305019 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5e6fb5-1faf-4f58-adf7-5981407f34fe None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:40.306049 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee8857b-08fc-4c26-997e-e525b7d455ae None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:40.346083 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac2703d-73b8-48e0-8de2-357969f87e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:40.346949 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac2703d-73b8-48e0-8de2-357969f87e5d None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:40.347243 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac2703d-73b8-48e0-8de2-357969f87e5d None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:40.347528 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac2703d-73b8-48e0-8de2-357969f87e5d None admin] PATH_INFO: `/v3/projects/service` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:40.351985 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5986f0c9-0b8e-42a0-bbcb-f9ddd06f4f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:40.352666 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5986f0c9-0b8e-42a0-bbcb-f9ddd06f4f6a None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:40.352951 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5986f0c9-0b8e-42a0-bbcb-f9ddd06f4f6a None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:40.353239 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5986f0c9-0b8e-42a0-bbcb-f9ddd06f4f6a None admin] PATH_INFO: `/v3/services` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:40.354282 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5986f0c9-0b8e-42a0-bbcb-f9ddd06f4f6a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:40.354604 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5986f0c9-0b8e-42a0-bbcb-f9ddd06f4f6a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:40.354853 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac2703d-73b8-48e0-8de2-357969f87e5d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:40.355990 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5986f0c9-0b8e-42a0-bbcb-f9ddd06f4f6a None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:40.356283 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac2703d-73b8-48e0-8de2-357969f87e5d None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:40.366052 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 445/892] 10.222.0.57 () {56 vars in 1122 bytes} [Thu May 19 14:52:40 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:40.379567 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-4ac2703d-73b8-48e0-8de2-357969f87e5d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:52:40.379567 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:40.379567 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:40.379567 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:40.379567 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:40.379567 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:40.379567 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:40.379567 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:40.379567 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:40.379567 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:40.379567 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:40.379567 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:40.379567 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:52:40.379567 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:52:40.379567 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:52:40.380706 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:52:40.380706 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:40.380706 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:40.380706 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:40.380706 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:40.380706 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:40.380706 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:40.380706 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:40.380706 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:52:40.380706 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:40.380706 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:40.380706 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:40.380706 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:40.380706 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:40.380706 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:40.380706 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:40.380706 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:40.382832 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:52:40.382832 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:52:40.382832 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:52:40.382832 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:52:40.382832 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:52:40.382832 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:52:40.382832 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:52:40.382832 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:52:40.382832 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 448/893] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:52:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:40.391152 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5986f0c9-0b8e-42a0-bbcb-f9ddd06f4f6a None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:40.512612 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5986f0c9-0b8e-42a0-bbcb-f9ddd06f4f6a None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:40.546249 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:40.553968 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05082abd-c661-44b5-bc96-5aaad4ab2e4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:40.554665 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05082abd-c661-44b5-bc96-5aaad4ab2e4a None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:40.554956 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05082abd-c661-44b5-bc96-5aaad4ab2e4a None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:40.555238 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05082abd-c661-44b5-bc96-5aaad4ab2e4a None admin] PATH_INFO: `/v3/projects` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:40.556049 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05082abd-c661-44b5-bc96-5aaad4ab2e4a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:40.556365 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05082abd-c661-44b5-bc96-5aaad4ab2e4a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:40.557854 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05082abd-c661-44b5-bc96-5aaad4ab2e4a None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:40.572326 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 446/894] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:52:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:40.578476 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac2703d-73b8-48e0-8de2-357969f87e5d None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:40.695179 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac2703d-73b8-48e0-8de2-357969f87e5d None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:40.717277 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:40.732768 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc24845-14c2-47fc-85ae-60ff91e67518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:40.733435 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc24845-14c2-47fc-85ae-60ff91e67518 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:40.733721 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc24845-14c2-47fc-85ae-60ff91e67518 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:40.734031 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc24845-14c2-47fc-85ae-60ff91e67518 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:40.735612 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc24845-14c2-47fc-85ae-60ff91e67518 None admin] RBAC: Adding query filter params (role.id=d07cc726aa754554a8a566bac4b7d7fe, scope.project.id=fc99d742ae004954b6d841293b5fa79c, user.id=cd0da70fc3c1477d8202726187ab5fde) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:40.735933 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc24845-14c2-47fc-85ae-60ff91e67518 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:40.737362 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc24845-14c2-47fc-85ae-60ff91e67518 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:40.743564 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 449/895] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:52:40 2022] GET /identity/v3/role_assignments?user.id=cd0da70fc3c1477d8202726187ab5fde&scope.project.id=fc99d742ae004954b6d841293b5fa79c&role.id=d07cc726aa754554a8a566bac4b7d7fe => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:41.540105 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4036e1-ae98-4fba-8d9f-7e5d1397476b None None] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:41.540683 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4036e1-ae98-4fba-8d9f-7e5d1397476b None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:41.540683 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4036e1-ae98-4fba-8d9f-7e5d1397476b None None] PATH_INFO: `/` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:41.542341 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 447/896] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:52:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:52:41.548653 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29244f8-4095-4ba9-b3db-fe614f5a4ca4 None None] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:41.549235 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29244f8-4095-4ba9-b3db-fe614f5a4ca4 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:41.549235 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29244f8-4095-4ba9-b3db-fe614f5a4ca4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:41.567615 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.auth.core [None req-f29244f8-4095-4ba9-b3db-fe614f5a4ca4 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:52:41.568822 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-f29244f8-4095-4ba9-b3db-fe614f5a4ca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:52:41.645924 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 450/897] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:52:41 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:52:41.668783 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4036e1-ae98-4fba-8d9f-7e5d1397476b None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:41.781750 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4036e1-ae98-4fba-8d9f-7e5d1397476b None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:41.818635 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97089cd-37c2-479c-b7e2-9de71bd195bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:41.819952 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97089cd-37c2-479c-b7e2-9de71bd195bc None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:41.820413 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97089cd-37c2-479c-b7e2-9de71bd195bc None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:41.820872 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97089cd-37c2-479c-b7e2-9de71bd195bc None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:41.821698 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97089cd-37c2-479c-b7e2-9de71bd195bc None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:41.823260 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97089cd-37c2-479c-b7e2-9de71bd195bc None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:41.836094 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-b97089cd-37c2-479c-b7e2-9de71bd195bc None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 14:52:41.836094 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:41.836094 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:52:41.836094 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:52:41.836094 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:52:41.836094 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:52:41.836094 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:52:41.836094 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:52:41.836094 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 14:52:41.836094 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:52:41.836094 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:52:41.836094 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:52:41.836094 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:41.836094 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:41.836094 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:41.836094 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:41.836094 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:41.837495 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:41.837495 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:41.837495 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:41.837495 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:41.837495 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:41.837495 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:41.837495 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:52:41.837495 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:52:41.837495 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:52:41.837495 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:52:41.837495 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:41.837495 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:41.837495 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:41.837495 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:41.837495 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:41.837495 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:41.838716 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:41.838716 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:52:41.838716 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:41.838716 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:41.838716 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:41.838716 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:41.838716 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:41.838716 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:41.838716 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:41.838716 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:41.838716 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:52:41.838716 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:52:41.838716 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 14:52:41.838716 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:52:41.841358 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 448/898] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 19 14:52:41 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:41.847752 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29244f8-4095-4ba9-b3db-fe614f5a4ca4 None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:41.962895 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29244f8-4095-4ba9-b3db-fe614f5a4ca4 None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:41.978132 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:42.006572 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7e1c15-3dac-4d90-ba5e-2a98439b9e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:42.007649 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7e1c15-3dac-4d90-ba5e-2a98439b9e9e None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:42.008205 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7e1c15-3dac-4d90-ba5e-2a98439b9e9e None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:42.008713 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7e1c15-3dac-4d90-ba5e-2a98439b9e9e None admin] PATH_INFO: `/v3/services` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:42.009656 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7e1c15-3dac-4d90-ba5e-2a98439b9e9e None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:42.010229 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7e1c15-3dac-4d90-ba5e-2a98439b9e9e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:42.011829 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7e1c15-3dac-4d90-ba5e-2a98439b9e9e None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:42.017577 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 451/899] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 19 14:52:41 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:42.022955 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97089cd-37c2-479c-b7e2-9de71bd195bc None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:42.090713 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:42.138140 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97089cd-37c2-479c-b7e2-9de71bd195bc None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:42.173871 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b14655-70cd-4455-970c-82d633704f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:42.174564 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b14655-70cd-4455-970c-82d633704f0d None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:42.174853 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b14655-70cd-4455-970c-82d633704f0d None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:42.175134 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b14655-70cd-4455-970c-82d633704f0d None admin] PATH_INFO: `/v3/services` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:42.175849 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b14655-70cd-4455-970c-82d633704f0d None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:42.176161 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b14655-70cd-4455-970c-82d633704f0d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:42.177551 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b14655-70cd-4455-970c-82d633704f0d None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:42.182983 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 449/900] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 19 14:52:42 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:42.191150 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7e1c15-3dac-4d90-ba5e-2a98439b9e9e None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:42.308278 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7e1c15-3dac-4d90-ba5e-2a98439b9e9e None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:42.346353 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70138bb-62d8-4705-8096-7edd846de4af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:42.347413 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70138bb-62d8-4705-8096-7edd846de4af None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:42.347941 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70138bb-62d8-4705-8096-7edd846de4af None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:42.348440 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70138bb-62d8-4705-8096-7edd846de4af None admin] PATH_INFO: `/v3/endpoints` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:42.349483 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70138bb-62d8-4705-8096-7edd846de4af None admin] RBAC: Adding query filter params (interface=public, service_id=5f5904c22bb740c28da10b3961372b85, region_id=RegionOne) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:42.350057 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70138bb-62d8-4705-8096-7edd846de4af None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:42.351817 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70138bb-62d8-4705-8096-7edd846de4af None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:42.358009 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 452/901] 10.222.0.57 () {56 vars in 1251 bytes} [Thu May 19 14:52:42 2022] GET /identity/v3/endpoints?service_id=5f5904c22bb740c28da10b3961372b85&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:42.364431 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b14655-70cd-4455-970c-82d633704f0d None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:42.469714 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447ce70a-3538-4290-a1ff-9c76b71f2a32 None None] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:42.470422 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447ce70a-3538-4290-a1ff-9c76b71f2a32 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:42.471065 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447ce70a-3538-4290-a1ff-9c76b71f2a32 None None] PATH_INFO: `/` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:42.472961 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 453/902] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:52:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:52:42.477277 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b14655-70cd-4455-970c-82d633704f0d None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:42.478936 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9088f5-836d-4cbb-a08b-a4902c3f53c0 None None] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:42.479478 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9088f5-836d-4cbb-a08b-a4902c3f53c0 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:42.479982 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9088f5-836d-4cbb-a08b-a4902c3f53c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:42.498559 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.auth.core [None req-4b9088f5-836d-4cbb-a08b-a4902c3f53c0 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:52:42.500240 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-4b9088f5-836d-4cbb-a08b-a4902c3f53c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:52:42.510283 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:42.518902 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deda6d5-ea3c-429f-9e7c-a67cf6891af8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:42.519771 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deda6d5-ea3c-429f-9e7c-a67cf6891af8 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:42.520178 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deda6d5-ea3c-429f-9e7c-a67cf6891af8 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:42.520484 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deda6d5-ea3c-429f-9e7c-a67cf6891af8 None admin] PATH_INFO: `/v3/services` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:42.521212 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3deda6d5-ea3c-429f-9e7c-a67cf6891af8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:42.522996 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3deda6d5-ea3c-429f-9e7c-a67cf6891af8 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:42.533151 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 450/903] 10.222.0.57 () {56 vars in 1087 bytes} [Thu May 19 14:52:42 2022] GET /identity/v3/services => generated 1268 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 14:52:42.548034 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:42.588577 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 454/904] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:52:42 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:52:42.612924 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deda6d5-ea3c-429f-9e7c-a67cf6891af8 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:42.731224 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84104e4-8e59-47aa-860d-4204eef405fc None None] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:42.731574 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84104e4-8e59-47aa-860d-4204eef405fc None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:42.731819 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84104e4-8e59-47aa-860d-4204eef405fc None None] PATH_INFO: `/` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:42.733497 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 455/905] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:52:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:52:42.733884 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deda6d5-ea3c-429f-9e7c-a67cf6891af8 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:42.739140 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1008649f-f45b-48fc-8f35-a4e4b8f101c1 None None] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:42.739429 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1008649f-f45b-48fc-8f35-a4e4b8f101c1 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:42.739715 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1008649f-f45b-48fc-8f35-a4e4b8f101c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:42.759753 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.auth.core [None req-1008649f-f45b-48fc-8f35-a4e4b8f101c1 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:52:42.761344 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-1008649f-f45b-48fc-8f35-a4e4b8f101c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:52:42.778473 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb71c887-8e81-44d5-ad3c-4925bbf231b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:42.779303 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb71c887-8e81-44d5-ad3c-4925bbf231b0 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:42.779592 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb71c887-8e81-44d5-ad3c-4925bbf231b0 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:42.779874 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb71c887-8e81-44d5-ad3c-4925bbf231b0 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:42.780542 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb71c887-8e81-44d5-ad3c-4925bbf231b0 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:42.782620 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb71c887-8e81-44d5-ad3c-4925bbf231b0 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:42.801505 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-bb71c887-8e81-44d5-ad3c-4925bbf231b0 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 14:52:42.801505 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:42.801505 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:52:42.801505 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:52:42.801505 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:52:42.801505 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:52:42.801505 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:52:42.801505 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:52:42.801505 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 14:52:42.801505 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:52:42.801505 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:52:42.801505 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:52:42.801505 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:42.801505 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:42.801505 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:42.801505 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:42.802701 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:42.802701 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:42.802701 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:42.802701 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:42.802701 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:42.802701 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:42.802701 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:42.802701 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:52:42.802701 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:52:42.802701 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:52:42.802701 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:52:42.802701 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:42.802701 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:42.802701 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:42.802701 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:42.802701 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:42.803695 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:42.803695 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:42.803695 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:52:42.803695 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:42.803695 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:42.803695 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:42.803695 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:42.803695 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:42.803695 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:42.803695 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:42.803695 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:42.803695 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:52:42.803695 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:52:42.803695 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 14:52:42.803695 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:52:42.803695 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 451/906] 10.222.0.57 () {56 vars in 1132 bytes} [Thu May 19 14:52:42 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:42.810792 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb71c887-8e81-44d5-ad3c-4925bbf231b0 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:42.864245 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 456/907] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:52:42 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:52:42.871455 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:42.887540 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1008649f-f45b-48fc-8f35-a4e4b8f101c1 None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:42.937014 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb71c887-8e81-44d5-ad3c-4925bbf231b0 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:42.976981 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071c793a-8ab9-4e93-9bc8-5f58db7bba41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:42.978081 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071c793a-8ab9-4e93-9bc8-5f58db7bba41 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:42.978686 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071c793a-8ab9-4e93-9bc8-5f58db7bba41 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:42.979205 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071c793a-8ab9-4e93-9bc8-5f58db7bba41 None admin] PATH_INFO: `/v3/services` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:42.980166 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071c793a-8ab9-4e93-9bc8-5f58db7bba41 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:42.980782 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071c793a-8ab9-4e93-9bc8-5f58db7bba41 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:42.982525 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071c793a-8ab9-4e93-9bc8-5f58db7bba41 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:42.989046 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 452/908] 10.222.0.57 () {56 vars in 1126 bytes} [Thu May 19 14:52:42 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:42.997166 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071c793a-8ab9-4e93-9bc8-5f58db7bba41 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:43.005226 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1008649f-f45b-48fc-8f35-a4e4b8f101c1 None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:43.006831 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:43.042860 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bc1a42-f585-4531-81b5-a792f8d22d4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:43.043538 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bc1a42-f585-4531-81b5-a792f8d22d4b None admin] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:43.043829 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bc1a42-f585-4531-81b5-a792f8d22d4b None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:43.044111 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bc1a42-f585-4531-81b5-a792f8d22d4b None admin] PATH_INFO: `/v3/services` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:43.044847 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bc1a42-f585-4531-81b5-a792f8d22d4b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:43.046267 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bc1a42-f585-4531-81b5-a792f8d22d4b None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:43.063902 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 457/909] 10.222.0.57 () {60 vars in 1141 bytes} [Thu May 19 14:52:42 2022] POST /identity/v3/services => generated 241 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 14:52:43.065606 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2a4aa1-b5af-4ab6-a2c6-38d53ad02b38 None None] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:43.066137 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2a4aa1-b5af-4ab6-a2c6-38d53ad02b38 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:43.066428 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2a4aa1-b5af-4ab6-a2c6-38d53ad02b38 None None] PATH_INFO: `/` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:43.068002 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 458/910] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:52:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:52:43.080353 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48741dd-d407-4273-9049-44ef35c406b5 None None] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:43.080823 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48741dd-d407-4273-9049-44ef35c406b5 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:43.081218 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48741dd-d407-4273-9049-44ef35c406b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:43.099301 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.auth.core [None req-f48741dd-d407-4273-9049-44ef35c406b5 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:52:43.100688 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-f48741dd-d407-4273-9049-44ef35c406b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:52:43.136726 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071c793a-8ab9-4e93-9bc8-5f58db7bba41 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:43.152397 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:43.193210 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b76bb5b-64eb-40e4-bf67-e68752e38e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:43.193933 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b76bb5b-64eb-40e4-bf67-e68752e38e3e None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:43.194227 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b76bb5b-64eb-40e4-bf67-e68752e38e3e None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:43.194765 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b76bb5b-64eb-40e4-bf67-e68752e38e3e None admin] PATH_INFO: `/v3/services` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:43.195489 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b76bb5b-64eb-40e4-bf67-e68752e38e3e None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:43.195802 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b76bb5b-64eb-40e4-bf67-e68752e38e3e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:43.197204 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b76bb5b-64eb-40e4-bf67-e68752e38e3e None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:43.202934 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 459/911] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:52:43 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:52:43.208362 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 453/912] 10.222.0.57 () {56 vars in 1126 bytes} [Thu May 19 14:52:42 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:43.215299 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48741dd-d407-4273-9049-44ef35c406b5 None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:43.220212 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b76bb5b-64eb-40e4-bf67-e68752e38e3e None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:43.328588 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:43.341579 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b76bb5b-64eb-40e4-bf67-e68752e38e3e None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:43.345794 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48741dd-d407-4273-9049-44ef35c406b5 None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:43.379715 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebed1a5f-cd22-4098-87a9-871e26e3f552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:43.380540 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebed1a5f-cd22-4098-87a9-871e26e3f552 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:43.380884 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebed1a5f-cd22-4098-87a9-871e26e3f552 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:43.381171 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebed1a5f-cd22-4098-87a9-871e26e3f552 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:43.390973 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6691fd-75f2-4255-b2ce-b36dbf09c3aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:43.391587 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6691fd-75f2-4255-b2ce-b36dbf09c3aa None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:43.391878 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6691fd-75f2-4255-b2ce-b36dbf09c3aa None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:43.392457 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6691fd-75f2-4255-b2ce-b36dbf09c3aa None admin] PATH_INFO: `/v3/endpoints` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:43.392841 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebed1a5f-cd22-4098-87a9-871e26e3f552 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:52:43.393269 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6691fd-75f2-4255-b2ce-b36dbf09c3aa None admin] RBAC: Adding query filter params (interface=public, service_id=f4e2835615014c0ea93ced416645b45c, region_id=RegionOne) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:43.393591 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6691fd-75f2-4255-b2ce-b36dbf09c3aa None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:43.395004 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6691fd-75f2-4255-b2ce-b36dbf09c3aa None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:43.400945 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 460/913] 10.222.0.57 () {56 vars in 1251 bytes} [Thu May 19 14:52:43 2022] GET /identity/v3/endpoints?service_id=f4e2835615014c0ea93ced416645b45c&interface=public®ion_id=RegionOne => generated 194 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:43.401170 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-ebed1a5f-cd22-4098-87a9-871e26e3f552 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 14:52:43.401170 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:43.401170 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:43.401170 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:43.401170 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:43.401170 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:43.401170 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:43.401170 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:43.401170 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:43.401170 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:43.401170 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:43.401170 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:43.401170 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:52:43.401170 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:52:43.401170 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:52:43.401170 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:52:43.402237 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:52:43.402237 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:52:43.402237 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:52:43.402237 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:52:43.402237 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:43.402237 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:43.402237 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:43.402237 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:43.402237 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:43.402237 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:43.402237 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:43.402237 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:52:43.402237 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:43.402237 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:43.402237 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:43.402237 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:43.403296 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:43.403296 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:43.403296 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:43.403296 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:43.403296 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:52:43.403296 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:52:43.403296 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:52:43.403296 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:52:43.403296 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:52:43.403296 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:52:43.403296 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 14:52:43.403296 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:52:43.407485 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 454/914] 10.222.0.57 () {56 vars in 1120 bytes} [Thu May 19 14:52:43 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:43.408495 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6691fd-75f2-4255-b2ce-b36dbf09c3aa None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:43.414482 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebed1a5f-cd22-4098-87a9-871e26e3f552 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:43.450489 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:43.532646 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebed1a5f-cd22-4098-87a9-871e26e3f552 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:43.549645 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6691fd-75f2-4255-b2ce-b36dbf09c3aa None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:43.569914 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed94bff0-92ef-47c7-8377-88a18eb51979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:43.570925 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed94bff0-92ef-47c7-8377-88a18eb51979 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:43.571463 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed94bff0-92ef-47c7-8377-88a18eb51979 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:43.571970 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed94bff0-92ef-47c7-8377-88a18eb51979 None admin] PATH_INFO: `/v3/roles` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:43.572931 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed94bff0-92ef-47c7-8377-88a18eb51979 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:43.573483 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed94bff0-92ef-47c7-8377-88a18eb51979 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:43.575110 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed94bff0-92ef-47c7-8377-88a18eb51979 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:43.589324 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 455/915] 10.222.0.57 () {56 vars in 1115 bytes} [Thu May 19 14:52:43 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:43.590801 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed94bff0-92ef-47c7-8377-88a18eb51979 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:43.592823 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00dd7c1-6e63-4d9b-ab6e-8f6c976d40c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:43.593558 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00dd7c1-6e63-4d9b-ab6e-8f6c976d40c1 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:43.593910 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00dd7c1-6e63-4d9b-ab6e-8f6c976d40c1 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:43.594189 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00dd7c1-6e63-4d9b-ab6e-8f6c976d40c1 None admin] PATH_INFO: `/v3/services` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:43.594901 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00dd7c1-6e63-4d9b-ab6e-8f6c976d40c1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:43.596483 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00dd7c1-6e63-4d9b-ab6e-8f6c976d40c1 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:43.604960 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 461/916] 10.222.0.57 () {56 vars in 1087 bytes} [Thu May 19 14:52:43 2022] GET /identity/v3/services => generated 1497 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 14:52:43.720682 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:43.723932 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed94bff0-92ef-47c7-8377-88a18eb51979 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:43.763418 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c799131-8683-4aa8-971b-1ae3b4824a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:43.764236 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c799131-8683-4aa8-971b-1ae3b4824a96 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:43.764525 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c799131-8683-4aa8-971b-1ae3b4824a96 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:43.764806 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c799131-8683-4aa8-971b-1ae3b4824a96 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:43.778235 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c799131-8683-4aa8-971b-1ae3b4824a96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:43.779722 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c799131-8683-4aa8-971b-1ae3b4824a96 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:43.799570 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-6c799131-8683-4aa8-971b-1ae3b4824a96 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:52:43.799570 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:43.799570 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:43.799570 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:43.799570 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:43.799570 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:43.799570 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:43.799570 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:43.799570 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:43.799570 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:43.799570 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:43.799570 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:43.799570 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:52:43.799570 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:52:43.799570 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:52:43.800723 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:52:43.800723 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:43.800723 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:43.800723 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:43.800723 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:43.800723 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:43.800723 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:43.800723 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:43.800723 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:52:43.800723 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:43.800723 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:43.800723 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:43.800723 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:43.800723 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:43.800723 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:43.800723 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:43.800723 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:43.801809 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:52:43.801809 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:52:43.801809 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:52:43.801809 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:52:43.801809 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 456/917] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:52:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:43.806752 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00dd7c1-6e63-4d9b-ab6e-8f6c976d40c1 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:43.832357 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:43.918999 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00dd7c1-6e63-4d9b-ab6e-8f6c976d40c1 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:43.961320 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4f15c4-7b2b-4585-b8a0-a0473f48ec25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:43.961941 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4f15c4-7b2b-4585-b8a0-a0473f48ec25 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:43.962235 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4f15c4-7b2b-4585-b8a0-a0473f48ec25 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:43.962527 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4f15c4-7b2b-4585-b8a0-a0473f48ec25 None admin] PATH_INFO: `/v3/domains` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:43.963260 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4f15c4-7b2b-4585-b8a0-a0473f48ec25 None admin] RBAC: Adding query filter params (name=Default) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:43.963573 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4f15c4-7b2b-4585-b8a0-a0473f48ec25 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:43.964944 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4f15c4-7b2b-4585-b8a0-a0473f48ec25 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:43.977810 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 462/918] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:52:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:52:43.983027 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c799131-8683-4aa8-971b-1ae3b4824a96 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:44.094498 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c799131-8683-4aa8-971b-1ae3b4824a96 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:44.129970 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c627014e-b901-47dc-b0f5-d69173ebbd2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:44.130787 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c627014e-b901-47dc-b0f5-d69173ebbd2a None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:44.131069 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c627014e-b901-47dc-b0f5-d69173ebbd2a None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:44.131340 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c627014e-b901-47dc-b0f5-d69173ebbd2a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:44.139520 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c627014e-b901-47dc-b0f5-d69173ebbd2a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:44.141361 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c627014e-b901-47dc-b0f5-d69173ebbd2a None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:44.156686 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-c627014e-b901-47dc-b0f5-d69173ebbd2a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 14:52:44.156686 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:44.156686 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:44.156686 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:44.156686 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:44.156686 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:44.156686 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:44.156686 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:44.156686 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:44.156686 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:44.156686 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:44.156686 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:44.156686 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:52:44.156686 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:52:44.156686 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:52:44.156686 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:52:44.158129 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:44.158129 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:44.158129 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:52:44.158129 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:52:44.158129 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:52:44.158129 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:52:44.158129 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:44.158129 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:44.158129 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:44.158129 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:44.158129 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:44.158129 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:52:44.158129 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:44.158129 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:44.158129 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:44.158129 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:44.158129 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:44.158129 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:44.159277 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:44.159277 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:44.159277 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:52:44.159277 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:52:44.159277 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:52:44.159277 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:52:44.159277 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:52:44.159277 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:52:44.159277 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 14:52:44.159277 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:52:44.159277 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 457/919] 10.222.0.57 () {56 vars in 1117 bytes} [Thu May 19 14:52:43 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:44.163890 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4f15c4-7b2b-4585-b8a0-a0473f48ec25 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:44.253896 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:44.274374 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4f15c4-7b2b-4585-b8a0-a0473f48ec25 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:44.310604 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f009ad-408a-4c46-b875-bbcf76b1fe90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:44.311292 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f009ad-408a-4c46-b875-bbcf76b1fe90 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:44.311590 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f009ad-408a-4c46-b875-bbcf76b1fe90 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:44.311875 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f009ad-408a-4c46-b875-bbcf76b1fe90 None admin] PATH_INFO: `/v3/users` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:44.313261 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f009ad-408a-4c46-b875-bbcf76b1fe90 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:44.313586 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f009ad-408a-4c46-b875-bbcf76b1fe90 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:44.315087 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f009ad-408a-4c46-b875-bbcf76b1fe90 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:44.336111 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 463/920] 10.222.0.57 () {56 vars in 1149 bytes} [Thu May 19 14:52:44 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:44.341683 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c627014e-b901-47dc-b0f5-d69173ebbd2a None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:44.346688 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:44.458759 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c627014e-b901-47dc-b0f5-d69173ebbd2a None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:44.494735 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e32f84-1574-49d9-8784-b993e4dd0bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:44.495533 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e32f84-1574-49d9-8784-b993e4dd0bbf None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:44.495820 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e32f84-1574-49d9-8784-b993e4dd0bbf None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:44.496092 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e32f84-1574-49d9-8784-b993e4dd0bbf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:44.508077 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e32f84-1574-49d9-8784-b993e4dd0bbf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:44.509462 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e32f84-1574-49d9-8784-b993e4dd0bbf None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:44.527704 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-51e32f84-1574-49d9-8784-b993e4dd0bbf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:52:44.527704 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:44.527704 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:44.527704 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:44.527704 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:44.527704 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:44.527704 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:44.527704 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:44.527704 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:44.527704 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:44.527704 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:44.527704 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:44.527704 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:52:44.527704 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:52:44.527704 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:52:44.528809 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:52:44.528809 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:44.528809 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:44.528809 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:44.528809 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:44.528809 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:44.528809 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:44.528809 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:44.528809 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:52:44.528809 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:44.528809 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:44.528809 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:44.528809 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:44.528809 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:44.528809 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:44.528809 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:44.528809 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:44.530351 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:52:44.530351 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:52:44.530351 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:52:44.530351 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:52:44.530351 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 458/921] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:52:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:44.537342 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f009ad-408a-4c46-b875-bbcf76b1fe90 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:44.644041 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f009ad-408a-4c46-b875-bbcf76b1fe90 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:44.680337 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca16c64e-45e9-4900-850e-28a04f8f9b63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:44.681016 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca16c64e-45e9-4900-850e-28a04f8f9b63 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:44.681310 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca16c64e-45e9-4900-850e-28a04f8f9b63 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:44.681596 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca16c64e-45e9-4900-850e-28a04f8f9b63 None admin] PATH_INFO: `/v3/domains` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:44.682362 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca16c64e-45e9-4900-850e-28a04f8f9b63 None admin] RBAC: Adding query filter params (name=Default) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:44.682677 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca16c64e-45e9-4900-850e-28a04f8f9b63 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:44.684047 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca16c64e-45e9-4900-850e-28a04f8f9b63 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:44.690940 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:44.697336 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 464/922] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:52:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:44.703812 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e32f84-1574-49d9-8784-b993e4dd0bbf None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:44.763237 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:44.811478 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e32f84-1574-49d9-8784-b993e4dd0bbf None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:44.853289 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e606b4-a63b-4b05-bee6-84d6dccc2d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:44.854169 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e606b4-a63b-4b05-bee6-84d6dccc2d1b None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:44.854462 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e606b4-a63b-4b05-bee6-84d6dccc2d1b None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:44.854746 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e606b4-a63b-4b05-bee6-84d6dccc2d1b None admin] PATH_INFO: `/v3/projects/service` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:44.862979 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e606b4-a63b-4b05-bee6-84d6dccc2d1b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:44.864459 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e606b4-a63b-4b05-bee6-84d6dccc2d1b None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:44.880264 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-75e606b4-a63b-4b05-bee6-84d6dccc2d1b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:52:44.880264 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:44.880264 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:44.880264 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:44.880264 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:44.880264 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:44.880264 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:44.880264 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:44.880264 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:44.880264 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:44.880264 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:44.880264 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:44.880264 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:52:44.880264 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:52:44.880264 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:52:44.881344 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:52:44.881344 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:44.881344 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:44.881344 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:44.881344 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:44.881344 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:44.881344 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:44.881344 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:44.881344 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:52:44.881344 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:44.881344 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:44.881344 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:44.881344 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:44.881344 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:44.881344 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:44.881344 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:44.881344 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:44.882447 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:52:44.882447 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:52:44.882447 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:52:44.882447 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:52:44.882447 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:52:44.882447 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:52:44.882447 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:52:44.882447 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:52:44.882447 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 459/923] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:52:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:44.889321 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca16c64e-45e9-4900-850e-28a04f8f9b63 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:45.000491 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca16c64e-45e9-4900-850e-28a04f8f9b63 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:45.029439 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dfe189-62c2-4526-8612-3460e4a5967a None None] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:45.030093 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dfe189-62c2-4526-8612-3460e4a5967a None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:45.030608 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dfe189-62c2-4526-8612-3460e4a5967a None None] PATH_INFO: `/` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:45.033027 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 460/924] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:52:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:52:45.038064 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476ebeb2-9e3a-4e3e-9223-e7b15352077c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:45.039052 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476ebeb2-9e3a-4e3e-9223-e7b15352077c None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:45.039263 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cb7384-bca4-4c1e-93cb-7ea251de2239 None None] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:45.039559 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cb7384-bca4-4c1e-93cb-7ea251de2239 None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:45.039840 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cb7384-bca4-4c1e-93cb-7ea251de2239 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:45.040197 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476ebeb2-9e3a-4e3e-9223-e7b15352077c None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:45.040707 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476ebeb2-9e3a-4e3e-9223-e7b15352077c None admin] PATH_INFO: `/v3/projects` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:45.041961 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476ebeb2-9e3a-4e3e-9223-e7b15352077c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:45.042574 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476ebeb2-9e3a-4e3e-9223-e7b15352077c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:45.044321 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476ebeb2-9e3a-4e3e-9223-e7b15352077c None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:45.058772 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 465/925] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:52:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:45.064565 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.auth.core [None req-33cb7384-bca4-4c1e-93cb-7ea251de2239 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:52:45.064565 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.fernet_utils [None req-33cb7384-bca4-4c1e-93cb-7ea251de2239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:52:45.065874 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476ebeb2-9e3a-4e3e-9223-e7b15352077c None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:45.130967 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:45.153930 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 461/926] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:52:45 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:52:45.179455 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cb7384-bca4-4c1e-93cb-7ea251de2239 None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:45.186209 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476ebeb2-9e3a-4e3e-9223-e7b15352077c None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:45.224947 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb18767c-b34b-4ddc-a94d-cb4966da3238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:45.226285 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb18767c-b34b-4ddc-a94d-cb4966da3238 None admin] REQUEST_METHOD: `PUT` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:45.226865 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb18767c-b34b-4ddc-a94d-cb4966da3238 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:45.227381 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb18767c-b34b-4ddc-a94d-cb4966da3238 None admin] PATH_INFO: `/v3/projects/fc99d742ae004954b6d841293b5fa79c/users/cd0da70fc3c1477d8202726187ab5fde/roles/d07cc726aa754554a8a566bac4b7d7fe` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:45.229976 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb18767c-b34b-4ddc-a94d-cb4966da3238 None admin] RBAC: Authorizing `identity:create_grant(project_id=fc99d742ae004954b6d841293b5fa79c, user_id=cd0da70fc3c1477d8202726187ab5fde, role_id=d07cc726aa754554a8a566bac4b7d7fe)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:45.231777 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb18767c-b34b-4ddc-a94d-cb4966da3238 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:45.239961 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:45.251352 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 466/927] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 14:52:45 2022] PUT /identity/v3/projects/fc99d742ae004954b6d841293b5fa79c/users/cd0da70fc3c1477d8202726187ab5fde/roles/d07cc726aa754554a8a566bac4b7d7fe => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:52:45.305494 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cb7384-bca4-4c1e-93cb-7ea251de2239 None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:45.351603 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcced11-c8b8-4849-8f27-58ea858447c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:45.352438 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcced11-c8b8-4849-8f27-58ea858447c9 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:45.352726 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcced11-c8b8-4849-8f27-58ea858447c9 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:45.353005 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcced11-c8b8-4849-8f27-58ea858447c9 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:45.353689 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcced11-c8b8-4849-8f27-58ea858447c9 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:45.355481 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcced11-c8b8-4849-8f27-58ea858447c9 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:45.369514 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-2fcced11-c8b8-4849-8f27-58ea858447c9 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 14:52:45.369514 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:45.369514 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:52:45.369514 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:52:45.369514 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:52:45.369514 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:52:45.369514 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:52:45.369514 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:52:45.369514 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 14:52:45.369514 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:52:45.369514 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:52:45.369514 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:52:45.369514 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:45.369514 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:45.369514 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:45.369514 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:45.369514 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:45.370826 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:45.370826 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:45.370826 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:45.370826 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:45.370826 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:45.370826 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:45.370826 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:52:45.370826 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:52:45.370826 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:52:45.370826 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:52:45.370826 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:45.370826 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:45.370826 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:45.370826 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:45.370826 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:45.370826 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:45.371824 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:45.371824 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:52:45.371824 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:45.371824 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:45.371824 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:45.371824 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:45.371824 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:45.371824 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:45.371824 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:45.371824 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:45.371824 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:52:45.371824 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:52:45.371824 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 14:52:45.371824 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:52:45.376416 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 462/928] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 19 14:52:45 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:52:45.377858 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb18767c-b34b-4ddc-a94d-cb4966da3238 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:45.505945 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb18767c-b34b-4ddc-a94d-cb4966da3238 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:45.529124 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:45.535247 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e2c5b4-2002-42ba-8ec7-ee372be18446 None None] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:45.535682 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e2c5b4-2002-42ba-8ec7-ee372be18446 None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:45.536073 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e2c5b4-2002-42ba-8ec7-ee372be18446 None None] PATH_INFO: `/` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:45.537829 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 463/929] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:52:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:52:45.543908 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e989ced3-1209-4f52-b2bb-d226d3944864 None None] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:45.544390 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e989ced3-1209-4f52-b2bb-d226d3944864 None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:45.544989 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e989ced3-1209-4f52-b2bb-d226d3944864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:45.546338 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6872e723-5019-4224-b662-be85a8809f94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:45.547514 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6872e723-5019-4224-b662-be85a8809f94 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:45.548044 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6872e723-5019-4224-b662-be85a8809f94 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:45.548738 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6872e723-5019-4224-b662-be85a8809f94 None admin] PATH_INFO: `/v3/services` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:45.549540 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6872e723-5019-4224-b662-be85a8809f94 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:45.549956 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6872e723-5019-4224-b662-be85a8809f94 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:45.551529 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6872e723-5019-4224-b662-be85a8809f94 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:45.560296 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 467/930] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 19 14:52:45 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:45.563560 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6872e723-5019-4224-b662-be85a8809f94 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:45.563707 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.auth.core [None req-e989ced3-1209-4f52-b2bb-d226d3944864 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:52:45.565232 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.fernet_utils [None req-e989ced3-1209-4f52-b2bb-d226d3944864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:52:45.646521 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 464/931] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:52:45 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:52:45.671100 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e989ced3-1209-4f52-b2bb-d226d3944864 None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:45.712461 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6872e723-5019-4224-b662-be85a8809f94 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:45.721721 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:45.753540 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7a2fa1-f6c6-45cf-a68c-8c7618a934fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:45.754407 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7a2fa1-f6c6-45cf-a68c-8c7618a934fd None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:45.754838 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7a2fa1-f6c6-45cf-a68c-8c7618a934fd None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:45.755193 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7a2fa1-f6c6-45cf-a68c-8c7618a934fd None admin] PATH_INFO: `/v3/services` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:45.755985 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7a2fa1-f6c6-45cf-a68c-8c7618a934fd None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:45.756378 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7a2fa1-f6c6-45cf-a68c-8c7618a934fd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:45.758193 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7a2fa1-f6c6-45cf-a68c-8c7618a934fd None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:45.769038 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 468/932] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 19 14:52:45 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:45.771093 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7a2fa1-f6c6-45cf-a68c-8c7618a934fd None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:45.799029 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e989ced3-1209-4f52-b2bb-d226d3944864 None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:45.825988 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:45.839276 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007d6c50-20f8-4541-8a36-6485b03dfebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:45.839910 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007d6c50-20f8-4541-8a36-6485b03dfebf None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:45.840272 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007d6c50-20f8-4541-8a36-6485b03dfebf None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:45.840557 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007d6c50-20f8-4541-8a36-6485b03dfebf None admin] PATH_INFO: `/v3/services/object-store` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:45.841299 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007d6c50-20f8-4541-8a36-6485b03dfebf None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:45.842768 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007d6c50-20f8-4541-8a36-6485b03dfebf None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:45.857002 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-007d6c50-20f8-4541-8a36-6485b03dfebf None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 14:52:45.857002 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:45.857002 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:52:45.857002 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:52:45.857002 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:52:45.857002 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:52:45.857002 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:52:45.857002 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:52:45.857002 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 14:52:45.857002 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:52:45.857002 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:52:45.857002 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:52:45.857002 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:45.857002 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:45.857002 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:45.857002 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:45.857002 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:45.858286 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:45.858286 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:45.858286 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:45.858286 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:45.858286 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:45.858286 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:45.858286 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:52:45.858286 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:52:45.858286 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:52:45.858286 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:52:45.858286 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:45.858286 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:45.858286 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:45.858286 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:45.858286 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:45.858286 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:45.860112 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:45.860112 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:52:45.860112 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:45.860112 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:45.860112 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:45.860112 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:45.860112 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:45.860112 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:45.860112 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:45.860112 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:45.860112 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:52:45.860112 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:52:45.860112 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 14:52:45.860112 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:52:45.863894 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 465/933] 10.222.0.57 () {56 vars in 1126 bytes} [Thu May 19 14:52:45 2022] GET /identity/v3/services/object-store => generated 93 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:45.864905 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007d6c50-20f8-4541-8a36-6485b03dfebf None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:45.897248 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7a2fa1-f6c6-45cf-a68c-8c7618a934fd None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:45.936163 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df18fab-3d5d-448c-952d-d60d6b0b4c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:45.938219 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df18fab-3d5d-448c-952d-d60d6b0b4c98 None admin] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:45.938934 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df18fab-3d5d-448c-952d-d60d6b0b4c98 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:45.939447 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df18fab-3d5d-448c-952d-d60d6b0b4c98 None admin] PATH_INFO: `/v3/endpoints` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:45.940425 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df18fab-3d5d-448c-952d-d60d6b0b4c98 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:45.942047 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df18fab-3d5d-448c-952d-d60d6b0b4c98 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:45.974357 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 469/934] 10.222.0.57 () {60 vars in 1144 bytes} [Thu May 19 14:52:45 2022] POST /identity/v3/endpoints => generated 346 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 14:52:45.982629 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007d6c50-20f8-4541-8a36-6485b03dfebf None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:45.986439 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:46.034214 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc12870-cb6f-4664-bae2-c7b4fa27302b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:46.034908 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc12870-cb6f-4664-bae2-c7b4fa27302b None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:46.035197 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc12870-cb6f-4664-bae2-c7b4fa27302b None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:46.035480 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc12870-cb6f-4664-bae2-c7b4fa27302b None admin] PATH_INFO: `/v3/services` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:46.035846 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3c5efc-2556-45fe-a56d-0ee8e8678c83 None None] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:46.036286 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc12870-cb6f-4664-bae2-c7b4fa27302b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:46.036586 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3c5efc-2556-45fe-a56d-0ee8e8678c83 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:46.036700 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc12870-cb6f-4664-bae2-c7b4fa27302b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:46.037310 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3c5efc-2556-45fe-a56d-0ee8e8678c83 None None] PATH_INFO: `/` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:46.038089 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc12870-cb6f-4664-bae2-c7b4fa27302b None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:46.039723 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 470/935] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:52:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:52:46.045208 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 466/936] 10.222.0.57 () {56 vars in 1122 bytes} [Thu May 19 14:52:45 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:46.045375 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a6d8ca-3217-4f31-9d63-4d3650913f47 None None] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:46.045509 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a6d8ca-3217-4f31-9d63-4d3650913f47 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:46.045867 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a6d8ca-3217-4f31-9d63-4d3650913f47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:46.051273 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc12870-cb6f-4664-bae2-c7b4fa27302b None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:46.065822 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.auth.core [None req-f4a6d8ca-3217-4f31-9d63-4d3650913f47 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:52:46.067481 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-f4a6d8ca-3217-4f31-9d63-4d3650913f47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:52:46.147599 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 471/937] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:52:46 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:52:46.169636 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc12870-cb6f-4664-bae2-c7b4fa27302b None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:46.171036 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a6d8ca-3217-4f31-9d63-4d3650913f47 None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:46.211409 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:46.213259 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8761a20-4b54-4c34-8b5a-e1465cb70acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:46.213719 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8761a20-4b54-4c34-8b5a-e1465cb70acb None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:46.213719 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8761a20-4b54-4c34-8b5a-e1465cb70acb None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:46.213937 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8761a20-4b54-4c34-8b5a-e1465cb70acb None admin] PATH_INFO: `/v3/services` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:46.214649 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8761a20-4b54-4c34-8b5a-e1465cb70acb None admin] RBAC: Adding query filter params (type=object-store) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:46.214958 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8761a20-4b54-4c34-8b5a-e1465cb70acb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:46.217524 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8761a20-4b54-4c34-8b5a-e1465cb70acb None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:46.226551 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 467/938] 10.222.0.57 () {56 vars in 1122 bytes} [Thu May 19 14:52:46 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:46.229717 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8761a20-4b54-4c34-8b5a-e1465cb70acb None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:46.265108 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:46.301783 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a6d8ca-3217-4f31-9d63-4d3650913f47 None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:46.337745 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bccb1a-c922-42bd-bec1-63c440c99753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:46.338607 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bccb1a-c922-42bd-bec1-63c440c99753 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:46.338904 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bccb1a-c922-42bd-bec1-63c440c99753 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:46.339193 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bccb1a-c922-42bd-bec1-63c440c99753 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:46.339872 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bccb1a-c922-42bd-bec1-63c440c99753 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:46.341276 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bccb1a-c922-42bd-bec1-63c440c99753 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:46.354113 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-f0bccb1a-c922-42bd-bec1-63c440c99753 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 14:52:46.354113 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:46.354113 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:52:46.354113 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:52:46.354113 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:52:46.354113 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:52:46.354113 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:52:46.354113 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:52:46.354113 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 14:52:46.354113 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:52:46.354113 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:52:46.354113 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:52:46.354113 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:46.354113 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:46.354113 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:46.354113 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:46.355244 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:46.355244 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:46.355244 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:46.355244 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:46.355244 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:46.355244 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:46.355244 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:46.355244 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:52:46.355244 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:52:46.355244 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:52:46.355244 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:52:46.355244 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:46.355244 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:46.355244 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:46.355244 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:46.355244 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:46.356256 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:46.356256 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:46.356256 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:52:46.356256 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:46.356256 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:46.356256 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:46.356256 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:46.356256 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:46.356256 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:46.356256 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:46.356256 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:46.356256 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:52:46.356256 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:52:46.356256 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 14:52:46.356256 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:52:46.356256 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 472/939] 10.222.0.57 () {56 vars in 1132 bytes} [Thu May 19 14:52:46 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:46.361998 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bccb1a-c922-42bd-bec1-63c440c99753 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:46.362863 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8761a20-4b54-4c34-8b5a-e1465cb70acb None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:46.402207 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a6d909-84a4-43e1-96fe-258babbdeb07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:46.402800 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a6d909-84a4-43e1-96fe-258babbdeb07 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:46.403094 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a6d909-84a4-43e1-96fe-258babbdeb07 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:46.403382 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a6d909-84a4-43e1-96fe-258babbdeb07 None admin] PATH_INFO: `/v3/endpoints` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:46.404182 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a6d909-84a4-43e1-96fe-258babbdeb07 None admin] RBAC: Adding query filter params (interface=public, service_id=7165ab06af02447ebd3a0956ac881fd5, region_id=RegionOne) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:46.404503 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a6d909-84a4-43e1-96fe-258babbdeb07 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:46.406437 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a6d909-84a4-43e1-96fe-258babbdeb07 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:46.417521 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 468/940] 10.222.0.57 () {56 vars in 1251 bytes} [Thu May 19 14:52:46 2022] GET /identity/v3/endpoints?service_id=7165ab06af02447ebd3a0956ac881fd5&interface=public®ion_id=RegionOne => generated 194 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:52:46.418489 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a6d909-84a4-43e1-96fe-258babbdeb07 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:46.498255 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:46.506780 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bccb1a-c922-42bd-bec1-63c440c99753 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:46.526220 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:46.540405 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a6d909-84a4-43e1-96fe-258babbdeb07 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:46.552027 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98a0be4-fa91-4562-bc7c-5b9a2222a58b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:46.552796 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98a0be4-fa91-4562-bc7c-5b9a2222a58b None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:46.553002 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98a0be4-fa91-4562-bc7c-5b9a2222a58b None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:46.553286 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98a0be4-fa91-4562-bc7c-5b9a2222a58b None admin] PATH_INFO: `/v3/services` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:46.554044 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98a0be4-fa91-4562-bc7c-5b9a2222a58b None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:46.554361 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98a0be4-fa91-4562-bc7c-5b9a2222a58b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:46.555737 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98a0be4-fa91-4562-bc7c-5b9a2222a58b None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:46.566020 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 473/941] 10.222.0.57 () {56 vars in 1126 bytes} [Thu May 19 14:52:46 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:46.567060 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98a0be4-fa91-4562-bc7c-5b9a2222a58b None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:46.585657 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97170e45-398d-4990-89dd-f25b38abeea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:46.586754 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97170e45-398d-4990-89dd-f25b38abeea9 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:46.587227 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97170e45-398d-4990-89dd-f25b38abeea9 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:46.587921 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97170e45-398d-4990-89dd-f25b38abeea9 None admin] PATH_INFO: `/v3/services` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:46.589172 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97170e45-398d-4990-89dd-f25b38abeea9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:46.590740 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97170e45-398d-4990-89dd-f25b38abeea9 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:46.600547 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 469/942] 10.222.0.57 () {56 vars in 1087 bytes} [Thu May 19 14:52:46 2022] GET /identity/v3/services => generated 1497 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 14:52:46.697454 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98a0be4-fa91-4562-bc7c-5b9a2222a58b None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:46.738851 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7424b5f9-5f27-4374-ab17-774dbeb3b0cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:46.739567 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7424b5f9-5f27-4374-ab17-774dbeb3b0cd None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:46.739837 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7424b5f9-5f27-4374-ab17-774dbeb3b0cd None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:46.740128 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7424b5f9-5f27-4374-ab17-774dbeb3b0cd None admin] PATH_INFO: `/v3/services` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:46.740873 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7424b5f9-5f27-4374-ab17-774dbeb3b0cd None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:46.741194 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7424b5f9-5f27-4374-ab17-774dbeb3b0cd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:46.742713 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7424b5f9-5f27-4374-ab17-774dbeb3b0cd None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:46.749154 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 474/943] 10.222.0.57 () {56 vars in 1126 bytes} [Thu May 19 14:52:46 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:46.754757 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97170e45-398d-4990-89dd-f25b38abeea9 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:46.867925 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97170e45-398d-4990-89dd-f25b38abeea9 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:46.902775 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ab03cc-0255-42b2-8351-6d1954146b33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:46.903446 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ab03cc-0255-42b2-8351-6d1954146b33 None admin] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:46.903737 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ab03cc-0255-42b2-8351-6d1954146b33 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:46.904021 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ab03cc-0255-42b2-8351-6d1954146b33 None admin] PATH_INFO: `/v3/endpoints` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:46.904780 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ab03cc-0255-42b2-8351-6d1954146b33 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:46.906321 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ab03cc-0255-42b2-8351-6d1954146b33 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:46.930299 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 470/944] 10.222.0.57 () {60 vars in 1144 bytes} [Thu May 19 14:52:46 2022] POST /identity/v3/endpoints => generated 347 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:52:47.643294 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fafd4bf-4d8a-47a4-87f5-14c753b6ace9 None None] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:47.643711 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fafd4bf-4d8a-47a4-87f5-14c753b6ace9 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:47.643893 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fafd4bf-4d8a-47a4-87f5-14c753b6ace9 None None] PATH_INFO: `/` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:47.645523 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 475/945] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:52:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:52:47.652786 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff59e20f-183e-4d98-ab44-c0ad42deefb5 None None] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:47.652786 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff59e20f-183e-4d98-ab44-c0ad42deefb5 None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:47.652786 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff59e20f-183e-4d98-ab44-c0ad42deefb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:47.669536 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.auth.core [None req-ff59e20f-183e-4d98-ab44-c0ad42deefb5 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:52:47.670909 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.fernet_utils [None req-ff59e20f-183e-4d98-ab44-c0ad42deefb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:52:47.679762 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:47.753160 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 471/946] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:52:47 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:52:47.777538 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fafd4bf-4d8a-47a4-87f5-14c753b6ace9 None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:47.807612 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:47.891645 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fafd4bf-4d8a-47a4-87f5-14c753b6ace9 None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:47.929940 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ed794c-0df9-4a3b-b183-8c2a910a76eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:47.930704 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ed794c-0df9-4a3b-b183-8c2a910a76eb None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:47.931000 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ed794c-0df9-4a3b-b183-8c2a910a76eb None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:47.931295 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ed794c-0df9-4a3b-b183-8c2a910a76eb None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:47.943290 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ed794c-0df9-4a3b-b183-8c2a910a76eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:52:47.951845 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-00ed794c-0df9-4a3b-b183-8c2a910a76eb None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 14:52:47.951845 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:47.951845 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:47.951845 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:47.951845 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:47.951845 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:47.951845 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:47.951845 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:47.951845 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:47.951845 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:47.951845 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:47.951845 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:47.951845 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:52:47.951845 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:52:47.951845 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:52:47.951845 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:52:47.953554 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:52:47.953554 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:52:47.953554 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:52:47.953554 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:52:47.953554 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:47.953554 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:47.953554 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:47.953554 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:47.953554 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:47.953554 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:47.953554 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:47.953554 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:52:47.953554 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:47.953554 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:47.953554 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:47.953554 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:47.955564 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:47.955564 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:47.955564 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:47.955564 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:47.955564 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:52:47.955564 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:52:47.955564 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:52:47.955564 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:52:47.955564 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:52:47.955564 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:52:47.955564 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 14:52:47.955564 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:52:47.955564 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 476/947] 10.222.0.57 () {56 vars in 1120 bytes} [Thu May 19 14:52:47 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:47.959796 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff59e20f-183e-4d98-ab44-c0ad42deefb5 None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:48.064148 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff59e20f-183e-4d98-ab44-c0ad42deefb5 None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:48.101209 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a528ba-b11f-4ef9-b67b-59868e3ea3c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:48.102292 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a528ba-b11f-4ef9-b67b-59868e3ea3c9 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:48.102798 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a528ba-b11f-4ef9-b67b-59868e3ea3c9 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:48.103333 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a528ba-b11f-4ef9-b67b-59868e3ea3c9 None admin] PATH_INFO: `/v3/roles` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:48.104295 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a528ba-b11f-4ef9-b67b-59868e3ea3c9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:48.104916 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a528ba-b11f-4ef9-b67b-59868e3ea3c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:48.106557 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a528ba-b11f-4ef9-b67b-59868e3ea3c9 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:48.115815 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 472/948] 10.222.0.57 () {56 vars in 1115 bytes} [Thu May 19 14:52:47 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:48.116108 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:48.122026 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ed794c-0df9-4a3b-b183-8c2a910a76eb None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:48.216501 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:48.235514 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ed794c-0df9-4a3b-b183-8c2a910a76eb None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:48.276861 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f5b9a9-754f-4efc-b4ad-10bdeed8275a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:48.277675 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f5b9a9-754f-4efc-b4ad-10bdeed8275a None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:48.278074 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f5b9a9-754f-4efc-b4ad-10bdeed8275a None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:48.278361 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f5b9a9-754f-4efc-b4ad-10bdeed8275a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:48.290159 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f5b9a9-754f-4efc-b4ad-10bdeed8275a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:48.291739 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f5b9a9-754f-4efc-b4ad-10bdeed8275a None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:48.310249 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-70f5b9a9-754f-4efc-b4ad-10bdeed8275a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:52:48.310249 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:48.310249 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:48.310249 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:48.310249 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:48.310249 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:48.310249 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:48.310249 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:48.310249 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:48.310249 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:48.310249 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:48.310249 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:48.310249 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:52:48.310249 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:52:48.310249 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:52:48.311362 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:52:48.311362 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:48.311362 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:48.311362 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:48.311362 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:48.311362 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:48.311362 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:48.311362 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:48.311362 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:52:48.311362 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:48.311362 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:48.311362 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:48.311362 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:48.311362 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:48.311362 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:48.311362 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:48.311362 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:48.313090 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:52:48.313090 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:52:48.313090 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:52:48.313090 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:52:48.313090 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 477/949] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:52:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:48.317405 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a528ba-b11f-4ef9-b67b-59868e3ea3c9 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:48.427216 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a528ba-b11f-4ef9-b67b-59868e3ea3c9 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:48.438677 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ae7982-2acf-4ab4-915e-4c41df83c576 None None] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:48.439295 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ae7982-2acf-4ab4-915e-4c41df83c576 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:48.439827 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ae7982-2acf-4ab4-915e-4c41df83c576 None None] PATH_INFO: `/` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:48.441766 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 478/950] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:52:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:52:48.448827 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03adf66f-f79b-4f02-9958-bc05d0967734 None None] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:48.449307 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03adf66f-f79b-4f02-9958-bc05d0967734 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:48.449789 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03adf66f-f79b-4f02-9958-bc05d0967734 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:48.464180 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf93e07-fec2-4d30-8b48-19e0a2d4da00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:48.465133 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf93e07-fec2-4d30-8b48-19e0a2d4da00 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:48.465626 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf93e07-fec2-4d30-8b48-19e0a2d4da00 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:48.466123 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf93e07-fec2-4d30-8b48-19e0a2d4da00 None admin] PATH_INFO: `/v3/domains` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:48.467095 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf93e07-fec2-4d30-8b48-19e0a2d4da00 None admin] RBAC: Adding query filter params (name=Default) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:48.467622 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf93e07-fec2-4d30-8b48-19e0a2d4da00 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:48.469229 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf93e07-fec2-4d30-8b48-19e0a2d4da00 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:48.475972 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.auth.core [None req-03adf66f-f79b-4f02-9958-bc05d0967734 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:52:48.478102 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-03adf66f-f79b-4f02-9958-bc05d0967734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:52:48.485881 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 473/951] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:52:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:48.488749 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf93e07-fec2-4d30-8b48-19e0a2d4da00 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:48.557648 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:48.581334 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 479/952] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:52:48 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:52:48.601275 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03adf66f-f79b-4f02-9958-bc05d0967734 None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:48.613487 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf93e07-fec2-4d30-8b48-19e0a2d4da00 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:48.651336 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94594db-51fd-4ba5-96a8-91f0d92149bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:48.652592 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94594db-51fd-4ba5-96a8-91f0d92149bc None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:48.653108 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94594db-51fd-4ba5-96a8-91f0d92149bc None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:48.653609 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94594db-51fd-4ba5-96a8-91f0d92149bc None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:48.662823 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94594db-51fd-4ba5-96a8-91f0d92149bc None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:48.665224 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94594db-51fd-4ba5-96a8-91f0d92149bc None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:48.683066 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-d94594db-51fd-4ba5-96a8-91f0d92149bc None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 14:52:48.683066 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:48.683066 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:48.683066 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:48.683066 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:48.683066 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:48.683066 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:48.683066 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:48.683066 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:48.683066 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:48.683066 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:48.683066 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:48.683066 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:52:48.683066 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:52:48.683066 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:52:48.683066 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:52:48.684240 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:48.684240 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:48.684240 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:52:48.684240 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:52:48.684240 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:52:48.684240 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:52:48.684240 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:48.684240 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:48.684240 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:48.684240 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:48.684240 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:48.684240 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:52:48.684240 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:48.684240 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:48.684240 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:48.684240 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:48.684240 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:48.684240 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:48.685480 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:48.685480 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:48.685480 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:52:48.685480 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:52:48.685480 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:52:48.685480 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:52:48.685480 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:52:48.685480 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:52:48.685480 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 14:52:48.685480 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:52:48.686145 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:48.689273 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 474/953] 10.222.0.57 () {56 vars in 1117 bytes} [Thu May 19 14:52:48 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:52:48.695002 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94594db-51fd-4ba5-96a8-91f0d92149bc None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:48.734696 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03adf66f-f79b-4f02-9958-bc05d0967734 None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:48.783474 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a1ac06-2c2e-4a46-acde-d22dbe4a2289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:48.785546 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a1ac06-2c2e-4a46-acde-d22dbe4a2289 None admin] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:48.786069 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a1ac06-2c2e-4a46-acde-d22dbe4a2289 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:48.786627 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a1ac06-2c2e-4a46-acde-d22dbe4a2289 None admin] PATH_INFO: `/v3/projects` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:48.787730 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a1ac06-2c2e-4a46-acde-d22dbe4a2289 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:48.790134 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a1ac06-2c2e-4a46-acde-d22dbe4a2289 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:48.792301 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: WARNING keystone.server.flask.common [None req-e2a1ac06-2c2e-4a46-acde-d22dbe4a2289 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 19 14:52:48.811232 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94594db-51fd-4ba5-96a8-91f0d92149bc None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:48.819918 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.sql.core [None req-e2a1ac06-2c2e-4a46-acde-d22dbe4a2289 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 19 14:52:48.819918 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 19 14:52:48.819918 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [parameters: {'id': 'b1c7705330f14cf7a0100cdfb9a5b9fe', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 19 14:52:48.819918 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=281785) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 19 14:52:48.832152 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-e2a1ac06-2c2e-4a46-acde-d22dbe4a2289 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id b1c7705330f14cf7a0100cdfb9a5b9fe.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id b1c7705330f14cf7a0100cdfb9a5b9fe. May 19 14:52:48.832152 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:48.832152 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 14:52:48.832152 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 14:52:48.832152 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 14:52:48.832152 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 14:52:48.832152 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 14:52:48.832152 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application result = self._query(query) May 19 14:52:48.832152 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 14:52:48.832152 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application conn.query(q) May 19 14:52:48.832152 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 14:52:48.832152 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 14:52:48.832152 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 14:52:48.833200 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application result.read() May 19 14:52:48.833200 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 14:52:48.833200 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 14:52:48.833200 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 14:52:48.833200 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 14:52:48.833200 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 14:52:48.833200 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 14:52:48.833200 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 14:52:48.833200 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 14:52:48.833200 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 19 14:52:48.833200 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:52:48.833200 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 19 14:52:48.833200 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:52:48.833200 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:48.833200 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 19 14:52:48.833200 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 19 14:52:48.833200 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project May 19 14:52:48.833200 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return project_ref.to_dict() May 19 14:52:48.834396 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 14:52:48.834396 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application next(self.gen) May 19 14:52:48.834396 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 19 14:52:48.834396 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application yield resource May 19 14:52:48.834396 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 14:52:48.834396 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application next(self.gen) May 19 14:52:48.834396 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 19 14:52:48.834396 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application self.session.rollback() May 19 14:52:48.834396 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 19 14:52:48.834396 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application self.force_reraise() May 19 14:52:48.834396 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 19 14:52:48.834396 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise self.value May 19 14:52:48.834396 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 19 14:52:48.834396 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 19 14:52:48.834396 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 19 14:52:48.834396 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application session.commit() May 19 14:52:48.834396 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 19 14:52:48.836314 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 19 14:52:48.836314 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 19 14:52:48.836314 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application self._prepare_impl() May 19 14:52:48.836314 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 19 14:52:48.836314 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application self.session.flush() May 19 14:52:48.836314 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 19 14:52:48.836314 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application self._flush(objects) May 19 14:52:48.836314 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 19 14:52:48.836314 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 19 14:52:48.836314 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 19 14:52:48.836314 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application compat.raise_( May 19 14:52:48.836314 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 14:52:48.836314 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception May 19 14:52:48.836314 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 19 14:52:48.836314 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application flush_context.execute() May 19 14:52:48.836314 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 19 14:52:48.836314 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rec.execute(self) May 19 14:52:48.837399 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 19 14:52:48.837399 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 19 14:52:48.837399 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 19 14:52:48.837399 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application _emit_insert_statements( May 19 14:52:48.837399 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 19 14:52:48.837399 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application c = connection._execute_20( May 19 14:52:48.837399 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 19 14:52:48.837399 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 19 14:52:48.837399 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 19 14:52:48.837399 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 19 14:52:48.837399 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 19 14:52:48.837399 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application ret = self._execute_context( May 19 14:52:48.837399 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 19 14:52:48.837399 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 19 14:52:48.837399 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 19 14:52:48.838393 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 19 14:52:48.838393 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 14:52:48.838393 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception May 19 14:52:48.838393 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 14:52:48.838393 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 14:52:48.838393 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 14:52:48.838393 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 14:52:48.838393 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 14:52:48.838393 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application result = self._query(query) May 19 14:52:48.838393 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 14:52:48.838393 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application conn.query(q) May 19 14:52:48.838393 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 14:52:48.838393 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 14:52:48.838393 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 14:52:48.838393 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application result.read() May 19 14:52:48.838393 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 14:52:48.838393 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 14:52:48.839476 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 14:52:48.839476 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 14:52:48.839476 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 14:52:48.839476 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 14:52:48.839476 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 14:52:48.839476 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 14:52:48.839476 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 19 14:52:48.839476 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 19 14:52:48.839476 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [parameters: {'id': 'b1c7705330f14cf7a0100cdfb9a5b9fe', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 19 14:52:48.839476 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 19 14:52:48.839476 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:52:48.839476 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:52:48.839476 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:52:48.839476 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:48.839476 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project May 19 14:52:48.840538 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) May 19 14:52:48.840538 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 19 14:52:48.840538 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 19 14:52:48.840538 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. May 19 14:52:48.840538 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:52:48.840538 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:52:48.840538 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:52:48.840538 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:48.840538 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:48.840538 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:48.840538 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:48.840538 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:48.840538 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:48.840538 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:48.840538 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:48.840538 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:48.840538 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:48.843017 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:48.843017 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post May 19 14:52:48.843017 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( May 19 14:52:48.843017 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:48.843017 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:48.843017 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project May 19 14:52:48.843017 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.Conflict( May 19 14:52:48.843017 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id b1c7705330f14cf7a0100cdfb9a5b9fe. May 19 14:52:48.843017 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:52:48.843017 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 480/954] 10.222.0.57 () {60 vars in 1140 bytes} [Thu May 19 14:52:48 2022] POST /identity/v3/projects => generated 262 bytes in 241 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:48.846644 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a1ac06-2c2e-4a46-acde-d22dbe4a2289 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:48.848590 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:48.866408 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a9ebcb-2b9f-4f26-a4fa-9176a4754b5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:48.867356 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a9ebcb-2b9f-4f26-a4fa-9176a4754b5d None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:48.867356 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a9ebcb-2b9f-4f26-a4fa-9176a4754b5d None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:48.867662 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a9ebcb-2b9f-4f26-a4fa-9176a4754b5d None admin] PATH_INFO: `/v3/users` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:48.868552 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a9ebcb-2b9f-4f26-a4fa-9176a4754b5d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:48.868875 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a9ebcb-2b9f-4f26-a4fa-9176a4754b5d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:48.870410 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a9ebcb-2b9f-4f26-a4fa-9176a4754b5d None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:48.891787 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 475/955] 10.222.0.57 () {56 vars in 1149 bytes} [Thu May 19 14:52:48 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:52:48.897291 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a9ebcb-2b9f-4f26-a4fa-9176a4754b5d None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:48.985582 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a1ac06-2c2e-4a46-acde-d22dbe4a2289 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:49.010971 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:49.026064 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8d1964-7c82-4ba7-9df2-411a59a39c63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:49.026870 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8d1964-7c82-4ba7-9df2-411a59a39c63 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:49.027156 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8d1964-7c82-4ba7-9df2-411a59a39c63 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:49.027433 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8d1964-7c82-4ba7-9df2-411a59a39c63 None admin] PATH_INFO: `/v3/projects/service` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:49.032160 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a9ebcb-2b9f-4f26-a4fa-9176a4754b5d None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:49.035012 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8d1964-7c82-4ba7-9df2-411a59a39c63 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:49.036662 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8d1964-7c82-4ba7-9df2-411a59a39c63 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:49.051602 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-4b8d1964-7c82-4ba7-9df2-411a59a39c63 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:52:49.051602 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:49.051602 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:49.051602 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:49.051602 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:49.051602 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:49.051602 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:49.051602 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:49.051602 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:49.051602 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:49.051602 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:49.051602 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:49.051602 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:52:49.051602 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:52:49.051602 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:52:49.052851 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:52:49.052851 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:49.052851 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:49.052851 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:49.052851 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:49.052851 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:49.052851 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:49.052851 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:49.052851 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:52:49.052851 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:49.052851 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:49.052851 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:49.052851 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:49.052851 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:49.052851 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:49.052851 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:49.052851 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:49.053975 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:52:49.053975 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:52:49.053975 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:52:49.053975 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:52:49.053975 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:52:49.053975 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:52:49.053975 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:52:49.053975 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:52:49.053975 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 481/956] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:52:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:49.054565 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3664344b-69d7-40cb-b962-b9074175df14 None None] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:49.054651 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3664344b-69d7-40cb-b962-b9074175df14 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:49.054885 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3664344b-69d7-40cb-b962-b9074175df14 None None] PATH_INFO: `/` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:49.059167 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 482/957] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:52:49 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:52:49.059803 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3664344b-69d7-40cb-b962-b9074175df14 None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:49.082278 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad629d8-02be-4348-a6df-314ef02c6da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:49.083221 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad629d8-02be-4348-a6df-314ef02c6da3 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:49.084208 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad629d8-02be-4348-a6df-314ef02c6da3 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:49.084653 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad629d8-02be-4348-a6df-314ef02c6da3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:49.145077 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:49.168956 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad629d8-02be-4348-a6df-314ef02c6da3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:49.170674 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad629d8-02be-4348-a6df-314ef02c6da3 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:49.178708 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3664344b-69d7-40cb-b962-b9074175df14 None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:49.218313 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd5c9df-2e20-4feb-a051-21ffc99e3578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:49.219013 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd5c9df-2e20-4feb-a051-21ffc99e3578 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:49.219305 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd5c9df-2e20-4feb-a051-21ffc99e3578 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:49.219589 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd5c9df-2e20-4feb-a051-21ffc99e3578 None admin] PATH_INFO: `/v3/projects` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:49.220433 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd5c9df-2e20-4feb-a051-21ffc99e3578 None admin] RBAC: Adding query filter params (name=service) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:49.220675 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd5c9df-2e20-4feb-a051-21ffc99e3578 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:49.222132 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd5c9df-2e20-4feb-a051-21ffc99e3578 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:49.247429 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 483/958] 10.222.0.57 () {56 vars in 1112 bytes} [Thu May 19 14:52:49 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:49.247429 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74418567-c1e4-45e2-8283-ae6203ad9ab7 None None] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:49.247429 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74418567-c1e4-45e2-8283-ae6203ad9ab7 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:49.247429 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74418567-c1e4-45e2-8283-ae6203ad9ab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:49.268382 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.auth.core [None req-74418567-c1e4-45e2-8283-ae6203ad9ab7 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:52:49.270129 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-74418567-c1e4-45e2-8283-ae6203ad9ab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:52:49.294014 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 476/959] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:52:48 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:52:49.299669 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc98d2b-e508-48ac-be4a-0a3e8ea05d5b None None] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:49.299977 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc98d2b-e508-48ac-be4a-0a3e8ea05d5b None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:49.300267 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc98d2b-e508-48ac-be4a-0a3e8ea05d5b None None] PATH_INFO: `/` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:49.301657 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 477/960] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:52:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:52:49.302404 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc98d2b-e508-48ac-be4a-0a3e8ea05d5b None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:49.317192 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:49.355532 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 484/961] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:52:49 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:52:49.356708 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b358aef3-888a-40d7-a2e4-08ac32e2ea24 None None] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:49.357011 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b358aef3-888a-40d7-a2e4-08ac32e2ea24 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:49.357298 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b358aef3-888a-40d7-a2e4-08ac32e2ea24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:49.384346 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.auth.core [None req-b358aef3-888a-40d7-a2e4-08ac32e2ea24 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:52:49.385699 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-b358aef3-888a-40d7-a2e4-08ac32e2ea24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:52:49.409165 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:49.438692 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc98d2b-e508-48ac-be4a-0a3e8ea05d5b None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:49.466974 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 485/962] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:52:49 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:52:49.466974 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b358aef3-888a-40d7-a2e4-08ac32e2ea24 None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:49.483306 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33da6959-43c4-4f2c-b4f1-79dc3cc1127a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:49.484203 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33da6959-43c4-4f2c-b4f1-79dc3cc1127a None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:49.484636 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33da6959-43c4-4f2c-b4f1-79dc3cc1127a None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:49.484924 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33da6959-43c4-4f2c-b4f1-79dc3cc1127a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:49.497777 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33da6959-43c4-4f2c-b4f1-79dc3cc1127a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:49.499229 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33da6959-43c4-4f2c-b4f1-79dc3cc1127a None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:49.518168 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-33da6959-43c4-4f2c-b4f1-79dc3cc1127a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:52:49.518168 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:49.518168 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:49.518168 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:49.518168 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:49.518168 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:49.518168 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:49.518168 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:49.518168 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:49.518168 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:49.518168 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:49.518168 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:49.518168 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:52:49.518168 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:52:49.518168 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:52:49.519282 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:52:49.519282 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:49.519282 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:49.519282 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:49.519282 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:49.519282 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:49.519282 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:49.519282 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:49.519282 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:52:49.519282 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:49.519282 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:49.519282 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:49.519282 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:49.519282 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:49.519282 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:49.519282 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:49.519282 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:49.520349 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:52:49.520349 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:52:49.520349 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:52:49.520349 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:52:49.520349 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 478/963] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:52:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:49.520349 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33da6959-43c4-4f2c-b4f1-79dc3cc1127a None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:49.589677 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b358aef3-888a-40d7-a2e4-08ac32e2ea24 None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:49.591763 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:49.630662 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86cec1c-9d53-4903-9117-e70c91aa9670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:49.631474 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86cec1c-9d53-4903-9117-e70c91aa9670 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:49.631763 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86cec1c-9d53-4903-9117-e70c91aa9670 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:49.632096 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86cec1c-9d53-4903-9117-e70c91aa9670 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:49.632762 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86cec1c-9d53-4903-9117-e70c91aa9670 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:49.634155 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86cec1c-9d53-4903-9117-e70c91aa9670 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:49.636265 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33da6959-43c4-4f2c-b4f1-79dc3cc1127a None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:49.647280 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-e86cec1c-9d53-4903-9117-e70c91aa9670 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 14:52:49.647280 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:49.647280 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:52:49.647280 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:52:49.647280 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:52:49.647280 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:52:49.647280 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:52:49.647280 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:52:49.647280 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 14:52:49.647280 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:52:49.647280 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:52:49.647280 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:52:49.647280 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:49.647280 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:49.647280 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:49.647280 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:49.647280 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:49.648504 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:49.648504 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:49.648504 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:49.648504 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:49.648504 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:49.648504 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:49.648504 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:52:49.648504 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:52:49.648504 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:52:49.648504 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:52:49.648504 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:49.648504 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:49.648504 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:49.648504 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:49.648504 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:49.648504 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:49.650197 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:49.650197 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:52:49.650197 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:49.650197 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:49.650197 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:49.650197 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:49.650197 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:49.650197 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:49.650197 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:49.650197 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:49.650197 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:52:49.650197 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:52:49.650197 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 14:52:49.650197 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:52:49.650197 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 486/964] 10.222.0.57 () {56 vars in 1126 bytes} [Thu May 19 14:52:49 2022] GET /identity/v3/services/object-store => generated 93 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:49.651197 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86cec1c-9d53-4903-9117-e70c91aa9670 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:49.677680 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:49.685220 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad84c48-2ec2-4fdc-a3a1-550aae599f44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:49.686062 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad84c48-2ec2-4fdc-a3a1-550aae599f44 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:49.686354 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad84c48-2ec2-4fdc-a3a1-550aae599f44 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:49.686818 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad84c48-2ec2-4fdc-a3a1-550aae599f44 None admin] PATH_INFO: `/v3/services/compute` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:49.687862 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad84c48-2ec2-4fdc-a3a1-550aae599f44 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:49.689259 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad84c48-2ec2-4fdc-a3a1-550aae599f44 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:49.702150 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-1ad84c48-2ec2-4fdc-a3a1-550aae599f44 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 19 14:52:49.702150 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:49.702150 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:52:49.702150 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:52:49.702150 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:52:49.702150 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:52:49.702150 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:52:49.702150 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:52:49.702150 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 14:52:49.702150 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:52:49.702150 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:52:49.702150 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:52:49.702150 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:49.702150 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:49.702150 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:49.702150 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:49.702150 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:49.703570 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:49.703570 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:49.703570 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:49.703570 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:49.703570 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:49.703570 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:49.703570 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:52:49.703570 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:52:49.703570 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:52:49.703570 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:52:49.703570 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:49.703570 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:49.703570 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:49.703570 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:49.703570 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:49.703570 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:49.704606 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:49.704606 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:52:49.704606 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:49.704606 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:49.704606 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:49.704606 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:49.704606 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:49.704606 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:49.704606 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:49.704606 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:49.704606 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:52:49.704606 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:52:49.704606 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 14:52:49.704606 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:52:49.704606 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 479/965] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:52:49 2022] GET /identity/v3/services/compute => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:49.705557 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad84c48-2ec2-4fdc-a3a1-550aae599f44 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:49.777226 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86cec1c-9d53-4903-9117-e70c91aa9670 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:49.814047 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99a0a41-d829-4d84-bb99-09701bc5bb3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:49.815688 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99a0a41-d829-4d84-bb99-09701bc5bb3c None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:49.816198 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99a0a41-d829-4d84-bb99-09701bc5bb3c None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:49.816698 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99a0a41-d829-4d84-bb99-09701bc5bb3c None admin] PATH_INFO: `/v3/domains` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:49.817622 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99a0a41-d829-4d84-bb99-09701bc5bb3c None admin] RBAC: Adding query filter params (name=Default) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:49.818158 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99a0a41-d829-4d84-bb99-09701bc5bb3c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:49.819207 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad84c48-2ec2-4fdc-a3a1-550aae599f44 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:49.820211 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99a0a41-d829-4d84-bb99-09701bc5bb3c None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:49.833457 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 487/966] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:52:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:49.834644 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99a0a41-d829-4d84-bb99-09701bc5bb3c None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:49.862186 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31fd745-bc3c-43c0-b53a-b1f1044cefb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:49.862867 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31fd745-bc3c-43c0-b53a-b1f1044cefb4 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:49.863163 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31fd745-bc3c-43c0-b53a-b1f1044cefb4 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:49.863448 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31fd745-bc3c-43c0-b53a-b1f1044cefb4 None admin] PATH_INFO: `/v3/services` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:49.864171 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31fd745-bc3c-43c0-b53a-b1f1044cefb4 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:49.864482 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31fd745-bc3c-43c0-b53a-b1f1044cefb4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:49.865894 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31fd745-bc3c-43c0-b53a-b1f1044cefb4 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:49.867171 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:49.872746 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 480/967] 10.222.0.57 () {56 vars in 1122 bytes} [Thu May 19 14:52:49 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:49.873545 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31fd745-bc3c-43c0-b53a-b1f1044cefb4 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:49.936836 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:49.953458 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99a0a41-d829-4d84-bb99-09701bc5bb3c None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:49.984572 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31fd745-bc3c-43c0-b53a-b1f1044cefb4 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:49.991798 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d18070-9769-4051-9373-287f2d62e633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:49.992488 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d18070-9769-4051-9373-287f2d62e633 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:49.992782 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d18070-9769-4051-9373-287f2d62e633 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:49.993068 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d18070-9769-4051-9373-287f2d62e633 None admin] PATH_INFO: `/v3/services` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:49.994194 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d18070-9769-4051-9373-287f2d62e633 None admin] RBAC: Adding query filter params (name=compute) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:49.994515 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d18070-9769-4051-9373-287f2d62e633 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:49.996237 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d18070-9769-4051-9373-287f2d62e633 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:50.001239 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 488/968] 10.222.0.57 () {56 vars in 1112 bytes} [Thu May 19 14:52:49 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:50.002093 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d18070-9769-4051-9373-287f2d62e633 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:50.025051 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8383a618-0126-44de-84b4-b9e3df297c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:50.025874 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8383a618-0126-44de-84b4-b9e3df297c8d None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:50.026162 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8383a618-0126-44de-84b4-b9e3df297c8d None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:50.026442 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8383a618-0126-44de-84b4-b9e3df297c8d None admin] PATH_INFO: `/v3/projects/service` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:50.034544 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8383a618-0126-44de-84b4-b9e3df297c8d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:50.035930 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8383a618-0126-44de-84b4-b9e3df297c8d None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:50.051307 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-8383a618-0126-44de-84b4-b9e3df297c8d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:52:50.051307 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:50.051307 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:50.051307 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:50.051307 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:50.051307 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:50.051307 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:50.051307 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:50.051307 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:50.051307 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:50.051307 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:50.051307 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:50.051307 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:52:50.051307 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:52:50.051307 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:52:50.052461 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:52:50.052461 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:50.052461 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:50.052461 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:50.052461 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:50.052461 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:50.052461 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:50.052461 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:50.052461 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:52:50.052461 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:50.052461 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:50.052461 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:50.052461 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:50.052461 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:50.052461 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:50.052461 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:50.052461 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:50.053537 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:52:50.053537 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:52:50.053537 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:52:50.053537 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:52:50.053537 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:52:50.053537 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:52:50.053537 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:52:50.053537 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:52:50.053537 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 481/969] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:52:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:50.053537 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8383a618-0126-44de-84b4-b9e3df297c8d None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:50.106652 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:50.118047 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d18070-9769-4051-9373-287f2d62e633 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:50.156152 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf85fe0-68bb-4a13-b244-fac44a14f0b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:50.156918 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf85fe0-68bb-4a13-b244-fac44a14f0b0 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:50.157216 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf85fe0-68bb-4a13-b244-fac44a14f0b0 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:50.157504 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf85fe0-68bb-4a13-b244-fac44a14f0b0 None admin] PATH_INFO: `/v3/services` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:50.158260 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf85fe0-68bb-4a13-b244-fac44a14f0b0 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:50.158579 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf85fe0-68bb-4a13-b244-fac44a14f0b0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:50.158768 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8383a618-0126-44de-84b4-b9e3df297c8d None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:50.159956 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf85fe0-68bb-4a13-b244-fac44a14f0b0 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:50.170338 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 489/970] 10.222.0.57 () {56 vars in 1122 bytes} [Thu May 19 14:52:50 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:50.170338 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf85fe0-68bb-4a13-b244-fac44a14f0b0 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:50.182350 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:50.203802 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90f8b83-40da-4624-a267-e9681ce15f33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:50.204562 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90f8b83-40da-4624-a267-e9681ce15f33 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:50.204850 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90f8b83-40da-4624-a267-e9681ce15f33 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:50.205133 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90f8b83-40da-4624-a267-e9681ce15f33 None admin] PATH_INFO: `/v3/services` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:50.205849 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90f8b83-40da-4624-a267-e9681ce15f33 None admin] RBAC: Adding query filter params (type=compute) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:50.206174 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90f8b83-40da-4624-a267-e9681ce15f33 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:50.207552 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90f8b83-40da-4624-a267-e9681ce15f33 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:50.212384 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 482/971] 10.222.0.57 () {56 vars in 1112 bytes} [Thu May 19 14:52:50 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:50.222934 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90f8b83-40da-4624-a267-e9681ce15f33 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:50.307610 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf85fe0-68bb-4a13-b244-fac44a14f0b0 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:50.356776 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614ff54a-b2c4-4053-916b-47042f09b276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:50.357517 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614ff54a-b2c4-4053-916b-47042f09b276 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:50.357883 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614ff54a-b2c4-4053-916b-47042f09b276 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:50.358183 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614ff54a-b2c4-4053-916b-47042f09b276 None admin] PATH_INFO: `/v3/projects` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:50.359412 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614ff54a-b2c4-4053-916b-47042f09b276 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:50.359739 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614ff54a-b2c4-4053-916b-47042f09b276 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:50.361470 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614ff54a-b2c4-4053-916b-47042f09b276 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:50.378548 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90f8b83-40da-4624-a267-e9681ce15f33 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:50.382025 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 490/972] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:52:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:50.384930 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614ff54a-b2c4-4053-916b-47042f09b276 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:50.410712 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:50.431575 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f08aee-3070-40d9-8ece-f8c1b923600f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:50.432256 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f08aee-3070-40d9-8ece-f8c1b923600f None admin] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:50.432550 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f08aee-3070-40d9-8ece-f8c1b923600f None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:50.432834 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f08aee-3070-40d9-8ece-f8c1b923600f None admin] PATH_INFO: `/v3/endpoints` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:50.433594 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f08aee-3070-40d9-8ece-f8c1b923600f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:50.435009 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f08aee-3070-40d9-8ece-f8c1b923600f None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:50.473873 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 483/973] 10.222.0.57 () {60 vars in 1144 bytes} [Thu May 19 14:52:50 2022] POST /identity/v3/endpoints => generated 349 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:52:50.534842 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614ff54a-b2c4-4053-916b-47042f09b276 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:50.585812 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73453b0-786a-4622-a520-5faba8635192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:50.586431 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73453b0-786a-4622-a520-5faba8635192 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:50.586718 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73453b0-786a-4622-a520-5faba8635192 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:50.587001 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73453b0-786a-4622-a520-5faba8635192 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:50.587842 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73453b0-786a-4622-a520-5faba8635192 None admin] RBAC: Adding query filter params (role.id=d07cc726aa754554a8a566bac4b7d7fe, scope.project.id=fc99d742ae004954b6d841293b5fa79c, user.id=cd0da70fc3c1477d8202726187ab5fde) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:50.588157 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73453b0-786a-4622-a520-5faba8635192 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:50.589826 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73453b0-786a-4622-a520-5faba8635192 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:50.598519 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 491/974] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:52:50 2022] GET /identity/v3/role_assignments?user.id=cd0da70fc3c1477d8202726187ab5fde&scope.project.id=fc99d742ae004954b6d841293b5fa79c&role.id=d07cc726aa754554a8a566bac4b7d7fe => generated 608 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:51.627296 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5de842-ed00-4cb5-bb72-3db4b203fb87 None None] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:51.627841 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5de842-ed00-4cb5-bb72-3db4b203fb87 None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:51.627841 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5de842-ed00-4cb5-bb72-3db4b203fb87 None None] PATH_INFO: `/` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:51.629364 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 484/975] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:52:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:52:51.634917 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956b8adf-d1ca-4ac1-bbdf-644e661b2974 None None] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:51.635326 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956b8adf-d1ca-4ac1-bbdf-644e661b2974 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:51.635502 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956b8adf-d1ca-4ac1-bbdf-644e661b2974 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:51.655901 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.auth.core [None req-956b8adf-d1ca-4ac1-bbdf-644e661b2974 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:52:51.657227 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-956b8adf-d1ca-4ac1-bbdf-644e661b2974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:52:51.731870 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 492/976] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:52:51 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:52:51.758034 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5de842-ed00-4cb5-bb72-3db4b203fb87 None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:51.809879 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:51.879737 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5de842-ed00-4cb5-bb72-3db4b203fb87 None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:51.918341 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706656dc-06d3-4ca2-b536-b1d921208730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:51.919027 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706656dc-06d3-4ca2-b536-b1d921208730 None admin] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:51.919321 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706656dc-06d3-4ca2-b536-b1d921208730 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:51.919606 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706656dc-06d3-4ca2-b536-b1d921208730 None admin] PATH_INFO: `/v3/users` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:51.920337 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706656dc-06d3-4ca2-b536-b1d921208730 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:51.922495 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706656dc-06d3-4ca2-b536-b1d921208730 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:51.923820 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: WARNING keystone.server.flask.common [None req-706656dc-06d3-4ca2-b536-b1d921208730 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 19 14:52:51.938405 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.sql.core [None req-706656dc-06d3-4ca2-b536-b1d921208730 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 19 14:52:51.938405 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 19 14:52:51.938405 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [parameters: {'user_id': '6c0e405634a143db99be856675427496', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 19 14:52:51.938405 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=281786) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 19 14:52:51.956440 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-706656dc-06d3-4ca2-b536-b1d921208730 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 19 14:52:51.956440 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:51.956440 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 14:52:51.956440 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 14:52:51.956440 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 14:52:51.956440 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 14:52:51.956440 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 14:52:51.956440 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application result = self._query(query) May 19 14:52:51.956440 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 14:52:51.956440 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application conn.query(q) May 19 14:52:51.956440 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 14:52:51.956440 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 14:52:51.956440 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 14:52:51.956440 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application result.read() May 19 14:52:51.956440 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 14:52:51.957579 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 14:52:51.957579 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 14:52:51.957579 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 14:52:51.957579 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 14:52:51.957579 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 14:52:51.957579 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 14:52:51.957579 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 14:52:51.957579 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 19 14:52:51.957579 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:52:51.957579 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 19 14:52:51.957579 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:52:51.957579 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:51.957579 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 19 14:52:51.957579 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 19 14:52:51.957579 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user May 19 14:52:51.957579 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) May 19 14:52:51.957579 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 14:52:51.957579 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application next(self.gen) May 19 14:52:51.959714 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 19 14:52:51.959714 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application yield resource May 19 14:52:51.959714 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 14:52:51.959714 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application next(self.gen) May 19 14:52:51.959714 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 19 14:52:51.959714 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application self.session.rollback() May 19 14:52:51.959714 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 19 14:52:51.959714 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application self.force_reraise() May 19 14:52:51.959714 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 19 14:52:51.959714 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise self.value May 19 14:52:51.959714 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 19 14:52:51.959714 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 19 14:52:51.959714 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 19 14:52:51.959714 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application session.commit() May 19 14:52:51.959714 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 19 14:52:51.959714 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 19 14:52:51.959714 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 19 14:52:51.960864 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application self._prepare_impl() May 19 14:52:51.960864 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 19 14:52:51.960864 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application self.session.flush() May 19 14:52:51.960864 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 19 14:52:51.960864 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application self._flush(objects) May 19 14:52:51.960864 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 19 14:52:51.960864 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 19 14:52:51.960864 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 19 14:52:51.960864 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application compat.raise_( May 19 14:52:51.960864 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 14:52:51.960864 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception May 19 14:52:51.960864 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 19 14:52:51.960864 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application flush_context.execute() May 19 14:52:51.960864 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 19 14:52:51.960864 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rec.execute(self) May 19 14:52:51.960864 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 19 14:52:51.960864 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 19 14:52:51.962702 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 19 14:52:51.962702 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application _emit_insert_statements( May 19 14:52:51.962702 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements May 19 14:52:51.962702 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application result = connection._execute_20( May 19 14:52:51.962702 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 19 14:52:51.962702 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 19 14:52:51.962702 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 19 14:52:51.962702 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 19 14:52:51.962702 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 19 14:52:51.962702 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application ret = self._execute_context( May 19 14:52:51.962702 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 19 14:52:51.962702 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 19 14:52:51.962702 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 19 14:52:51.962702 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 19 14:52:51.962702 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 14:52:51.962702 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception May 19 14:52:51.964858 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 14:52:51.964858 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 14:52:51.964858 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 14:52:51.964858 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 14:52:51.964858 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 14:52:51.964858 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application result = self._query(query) May 19 14:52:51.964858 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 14:52:51.964858 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application conn.query(q) May 19 14:52:51.964858 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 14:52:51.964858 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 14:52:51.964858 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 14:52:51.964858 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application result.read() May 19 14:52:51.964858 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 14:52:51.964858 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 14:52:51.964858 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 14:52:51.964858 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 14:52:51.967093 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 14:52:51.967093 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 14:52:51.967093 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 14:52:51.967093 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 14:52:51.967093 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 19 14:52:51.967093 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 19 14:52:51.967093 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [parameters: {'user_id': '6c0e405634a143db99be856675427496', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 19 14:52:51.967093 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 19 14:52:51.967093 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:52:51.967093 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:52:51.967093 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:52:51.967093 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:51.967093 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:51.967093 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:51.967093 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:51.968358 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:51.968358 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:51.968358 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:51.968358 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:51.968358 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:51.968358 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:51.968358 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:51.968358 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post May 19 14:52:51.968358 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( May 19 14:52:51.968358 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:51.968358 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:51.968358 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:52:51.968358 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:52:51.968358 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:52:51.968358 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:52:51.968358 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user May 19 14:52:51.968358 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) May 19 14:52:51.969586 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects May 19 14:52:51.969586 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) May 19 14:52:51.969586 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 19 14:52:51.969586 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 19 14:52:51.969586 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 19 14:52:51.969586 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:52:51.969586 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 485/977] 10.222.0.57 () {60 vars in 1131 bytes} [Thu May 19 14:52:51 2022] POST /identity/v3/users => generated 160 bytes in 201 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:51.978264 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956b8adf-d1ca-4ac1-bbdf-644e661b2974 None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:51.978264 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:52.079465 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956b8adf-d1ca-4ac1-bbdf-644e661b2974 None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:52.116083 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a875cc-88fa-4f61-9a7a-d7e7c84c6dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:52.116894 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a875cc-88fa-4f61-9a7a-d7e7c84c6dc1 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:52.117191 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a875cc-88fa-4f61-9a7a-d7e7c84c6dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:52.117478 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a875cc-88fa-4f61-9a7a-d7e7c84c6dc1 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:52.125501 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a875cc-88fa-4f61-9a7a-d7e7c84c6dc1 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:52.126997 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a875cc-88fa-4f61-9a7a-d7e7c84c6dc1 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:52.142921 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-92a875cc-88fa-4f61-9a7a-d7e7c84c6dc1 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 14:52:52.142921 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:52.142921 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:52.142921 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:52.142921 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:52.142921 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:52.142921 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:52.142921 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:52.142921 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:52.142921 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:52.142921 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:52.142921 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:52.142921 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:52:52.142921 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:52:52.142921 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:52:52.142921 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:52:52.145294 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:52.145294 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:52.145294 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:52:52.145294 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:52:52.145294 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:52:52.145294 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:52:52.145294 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:52.145294 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:52.145294 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:52.145294 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:52.145294 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:52.145294 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:52:52.145294 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:52.145294 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:52.145294 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:52.145294 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:52.145294 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:52.145294 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:52.147822 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:52.147822 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:52.147822 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:52:52.147822 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:52:52.147822 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:52:52.147822 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:52:52.147822 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:52:52.147822 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:52:52.147822 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 14:52:52.147822 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:52:52.147822 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 493/978] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:52:51 2022] GET /identity/v3/users/cinder => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:52.153861 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706656dc-06d3-4ca2-b536-b1d921208730 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:52.262728 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:52.267726 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706656dc-06d3-4ca2-b536-b1d921208730 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:52.306804 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197acef3-dea2-40e6-bf16-d2d1427645de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:52.307492 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197acef3-dea2-40e6-bf16-d2d1427645de None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:52.307783 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197acef3-dea2-40e6-bf16-d2d1427645de None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:52.308069 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197acef3-dea2-40e6-bf16-d2d1427645de None admin] PATH_INFO: `/v3/users` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:52.308890 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197acef3-dea2-40e6-bf16-d2d1427645de None admin] RBAC: Adding query filter params (name=cinder) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:52.309206 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197acef3-dea2-40e6-bf16-d2d1427645de None admin] RBAC: Authorizing `identity:list_users()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:52.310884 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197acef3-dea2-40e6-bf16-d2d1427645de None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:52.351498 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 486/979] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 14:52:52 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:52.576714 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f54777-f777-41fb-a1ab-e4d37c834e37 None None] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:52.576714 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f54777-f777-41fb-a1ab-e4d37c834e37 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:52.576714 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f54777-f777-41fb-a1ab-e4d37c834e37 None None] PATH_INFO: `/` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:52.576714 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 494/980] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:52:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:52:52.583673 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8eb250-97b2-401b-8fd0-e10d8574ce84 None None] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:52.584716 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8eb250-97b2-401b-8fd0-e10d8574ce84 None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:52.585163 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8eb250-97b2-401b-8fd0-e10d8574ce84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:52.615623 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.auth.core [None req-6b8eb250-97b2-401b-8fd0-e10d8574ce84 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:52:52.617411 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.fernet_utils [None req-6b8eb250-97b2-401b-8fd0-e10d8574ce84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:52:52.724541 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 487/981] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:52:52 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:52:52.754741 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f54777-f777-41fb-a1ab-e4d37c834e37 None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:52.840201 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:52.886176 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f54777-f777-41fb-a1ab-e4d37c834e37 None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:52.924853 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfefe413-0011-45b9-89e5-d179a552f662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:52.925921 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfefe413-0011-45b9-89e5-d179a552f662 None admin] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:52.926465 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfefe413-0011-45b9-89e5-d179a552f662 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:52.927014 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfefe413-0011-45b9-89e5-d179a552f662 None admin] PATH_INFO: `/v3/services` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:52.928025 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfefe413-0011-45b9-89e5-d179a552f662 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:52.929645 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfefe413-0011-45b9-89e5-d179a552f662 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:52.943441 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b90f899-b94a-4ae9-8a1d-8ea776797e4d None None] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:52.943792 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b90f899-b94a-4ae9-8a1d-8ea776797e4d None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:52.944028 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b90f899-b94a-4ae9-8a1d-8ea776797e4d None None] PATH_INFO: `/` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:52.945743 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 488/982] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:52:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:52:52.957866 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1332cb96-60eb-47da-b710-0b61d4d3d7c9 None None] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:52.957866 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1332cb96-60eb-47da-b710-0b61d4d3d7c9 None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:52.957866 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1332cb96-60eb-47da-b710-0b61d4d3d7c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:52.958749 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 495/983] 10.222.0.57 () {60 vars in 1141 bytes} [Thu May 19 14:52:52 2022] POST /identity/v3/services => generated 242 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:52:52.982733 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.auth.core [None req-1332cb96-60eb-47da-b710-0b61d4d3d7c9 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:52:52.984695 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.fernet_utils [None req-1332cb96-60eb-47da-b710-0b61d4d3d7c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:52:53.048246 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a123a9da-9eff-4bb7-b8f6-b153cc6b9ea4 None None] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:53.048636 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a123a9da-9eff-4bb7-b8f6-b153cc6b9ea4 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:53.048829 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a123a9da-9eff-4bb7-b8f6-b153cc6b9ea4 None None] PATH_INFO: `/` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:53.050469 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 496/984] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:52:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:52:53.056160 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50e9a7d-1aee-4643-bc35-d63d8fef2078 None None] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:53.056458 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50e9a7d-1aee-4643-bc35-d63d8fef2078 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:53.056751 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50e9a7d-1aee-4643-bc35-d63d8fef2078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:53.074096 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 489/985] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:52:52 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:52:53.076050 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.auth.core [None req-a50e9a7d-1aee-4643-bc35-d63d8fef2078 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:52:53.077287 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-a50e9a7d-1aee-4643-bc35-d63d8fef2078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:52:53.100925 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1332cb96-60eb-47da-b710-0b61d4d3d7c9 None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:53.117663 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:53.163368 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 497/986] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:52:53 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:52:53.186202 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50e9a7d-1aee-4643-bc35-d63d8fef2078 None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:53.210701 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1332cb96-60eb-47da-b710-0b61d4d3d7c9 None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:53.243324 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:53.251530 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c462e850-241d-4b7f-b109-435acbdc937c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:53.252525 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c462e850-241d-4b7f-b109-435acbdc937c None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:53.253133 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c462e850-241d-4b7f-b109-435acbdc937c None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:53.253555 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c462e850-241d-4b7f-b109-435acbdc937c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:53.254399 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c462e850-241d-4b7f-b109-435acbdc937c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:53.255783 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c462e850-241d-4b7f-b109-435acbdc937c None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:53.271786 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-c462e850-241d-4b7f-b109-435acbdc937c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 14:52:53.271786 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:53.271786 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:52:53.271786 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:52:53.271786 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:52:53.271786 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:52:53.271786 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:52:53.271786 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:52:53.271786 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 14:52:53.271786 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:52:53.271786 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:52:53.271786 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:52:53.271786 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:53.271786 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:53.271786 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:53.271786 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:53.271786 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:53.273151 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:53.273151 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:53.273151 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:53.273151 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:53.273151 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:53.273151 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:53.273151 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:52:53.273151 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:52:53.273151 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:52:53.273151 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:52:53.273151 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:53.273151 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:53.273151 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:53.273151 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:53.273151 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:53.273151 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:53.274216 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:53.274216 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:52:53.274216 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:53.274216 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:53.274216 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:53.274216 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:53.274216 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:53.274216 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:53.274216 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:53.274216 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:53.274216 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:52:53.274216 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:52:53.274216 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 14:52:53.274216 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:52:53.274216 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 490/987] 10.222.0.57 () {56 vars in 1126 bytes} [Thu May 19 14:52:53 2022] GET /identity/v3/services/object-store => generated 93 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:52:53.279697 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c462e850-241d-4b7f-b109-435acbdc937c None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:53.299410 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50e9a7d-1aee-4643-bc35-d63d8fef2078 None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:53.341100 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee23432-c0b9-4b37-889e-c9a57b72d140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:53.341952 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee23432-c0b9-4b37-889e-c9a57b72d140 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:53.342249 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee23432-c0b9-4b37-889e-c9a57b72d140 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:53.342565 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee23432-c0b9-4b37-889e-c9a57b72d140 None admin] PATH_INFO: `/v3/services/image` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:53.343316 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee23432-c0b9-4b37-889e-c9a57b72d140 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:53.344680 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee23432-c0b9-4b37-889e-c9a57b72d140 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:53.357286 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-1ee23432-c0b9-4b37-889e-c9a57b72d140 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 19 14:52:53.357286 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:53.357286 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:52:53.357286 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:52:53.357286 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:52:53.357286 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:52:53.357286 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:52:53.357286 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:52:53.357286 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 14:52:53.357286 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:52:53.357286 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:52:53.357286 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:52:53.357286 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:53.357286 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:53.357286 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:53.357286 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:53.357286 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:53.358509 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:53.358509 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:53.358509 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:53.358509 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:53.358509 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:53.358509 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:53.358509 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:52:53.358509 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:52:53.358509 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:52:53.358509 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:52:53.358509 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:53.358509 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:53.358509 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:53.358509 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:53.358509 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:53.358509 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:53.359942 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:53.359942 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:52:53.359942 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:53.359942 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:53.359942 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:53.359942 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:53.359942 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:53.359942 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:53.359942 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:53.359942 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:53.359942 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:52:53.359942 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:52:53.359942 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 14:52:53.359942 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:52:53.363600 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 498/988] 10.222.0.57 () {56 vars in 1105 bytes} [Thu May 19 14:52:53 2022] GET /identity/v3/services/image => generated 86 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:52:53.364485 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee23432-c0b9-4b37-889e-c9a57b72d140 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:53.372831 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:53.399055 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c462e850-241d-4b7f-b109-435acbdc937c None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:53.434862 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3c1f43-73d3-4ad9-9200-90ff750bb843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:53.435544 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3c1f43-73d3-4ad9-9200-90ff750bb843 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:53.435840 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3c1f43-73d3-4ad9-9200-90ff750bb843 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:53.436126 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3c1f43-73d3-4ad9-9200-90ff750bb843 None admin] PATH_INFO: `/v3/services` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:53.437378 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3c1f43-73d3-4ad9-9200-90ff750bb843 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:53.437742 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3c1f43-73d3-4ad9-9200-90ff750bb843 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:53.439167 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3c1f43-73d3-4ad9-9200-90ff750bb843 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:53.445168 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 491/989] 10.222.0.57 () {56 vars in 1122 bytes} [Thu May 19 14:52:53 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:53.450831 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3c1f43-73d3-4ad9-9200-90ff750bb843 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:53.477226 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee23432-c0b9-4b37-889e-c9a57b72d140 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:53.498288 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:53.519052 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07807231-6adf-4f82-9445-cbbf7561e1d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:53.519822 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07807231-6adf-4f82-9445-cbbf7561e1d0 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:53.520229 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07807231-6adf-4f82-9445-cbbf7561e1d0 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:53.520532 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07807231-6adf-4f82-9445-cbbf7561e1d0 None admin] PATH_INFO: `/v3/services` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:53.521263 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07807231-6adf-4f82-9445-cbbf7561e1d0 None admin] RBAC: Adding query filter params (name=image) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:53.521626 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07807231-6adf-4f82-9445-cbbf7561e1d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:53.523078 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07807231-6adf-4f82-9445-cbbf7561e1d0 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:53.532785 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 499/990] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:52:53 2022] GET /identity/v3/services?name=image => generated 122 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:53.534155 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07807231-6adf-4f82-9445-cbbf7561e1d0 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:53.568758 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3c1f43-73d3-4ad9-9200-90ff750bb843 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:53.606460 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb9dcdc-6550-401d-be2a-0419ff920d64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:53.607203 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb9dcdc-6550-401d-be2a-0419ff920d64 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:53.607551 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb9dcdc-6550-401d-be2a-0419ff920d64 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:53.607832 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb9dcdc-6550-401d-be2a-0419ff920d64 None admin] PATH_INFO: `/v3/services` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:53.608555 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb9dcdc-6550-401d-be2a-0419ff920d64 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:53.609409 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb9dcdc-6550-401d-be2a-0419ff920d64 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:53.610809 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb9dcdc-6550-401d-be2a-0419ff920d64 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:53.620872 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 492/991] 10.222.0.57 () {56 vars in 1122 bytes} [Thu May 19 14:52:53 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:52:53.621928 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb9dcdc-6550-401d-be2a-0419ff920d64 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:53.633567 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:53.649468 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07807231-6adf-4f82-9445-cbbf7561e1d0 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:53.690064 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb60a07-7eff-463d-aa39-fed2d5a72cc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:53.691199 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb60a07-7eff-463d-aa39-fed2d5a72cc9 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:53.691613 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb60a07-7eff-463d-aa39-fed2d5a72cc9 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:53.691875 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb60a07-7eff-463d-aa39-fed2d5a72cc9 None admin] PATH_INFO: `/v3/services` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:53.692502 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb60a07-7eff-463d-aa39-fed2d5a72cc9 None admin] RBAC: Adding query filter params (type=image) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:53.692821 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb60a07-7eff-463d-aa39-fed2d5a72cc9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:53.694480 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb60a07-7eff-463d-aa39-fed2d5a72cc9 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:53.699525 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 500/992] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:52:53 2022] GET /identity/v3/services?type=image => generated 122 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:52:53.746397 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb9dcdc-6550-401d-be2a-0419ff920d64 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:53.786906 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ca8fe6-ce3e-45f7-9cf8-176a151c74d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:53.787607 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ca8fe6-ce3e-45f7-9cf8-176a151c74d3 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:53.787897 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ca8fe6-ce3e-45f7-9cf8-176a151c74d3 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:53.788180 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ca8fe6-ce3e-45f7-9cf8-176a151c74d3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:53.788997 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ca8fe6-ce3e-45f7-9cf8-176a151c74d3 None admin] RBAC: Adding query filter params (interface=admin, service_id=7165ab06af02447ebd3a0956ac881fd5, region_id=RegionOne) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:53.789321 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ca8fe6-ce3e-45f7-9cf8-176a151c74d3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:53.790760 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ca8fe6-ce3e-45f7-9cf8-176a151c74d3 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:53.801832 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 493/993] 10.222.0.57 () {56 vars in 1249 bytes} [Thu May 19 14:52:53 2022] GET /identity/v3/endpoints?service_id=7165ab06af02447ebd3a0956ac881fd5&interface=admin®ion_id=RegionOne => generated 193 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:53.803637 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb60a07-7eff-463d-aa39-fed2d5a72cc9 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:53.857176 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:53.918261 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb60a07-7eff-463d-aa39-fed2d5a72cc9 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:53.954713 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bada9770-20fe-4be8-b052-e72253376eb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:53.955634 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bada9770-20fe-4be8-b052-e72253376eb2 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:53.955934 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bada9770-20fe-4be8-b052-e72253376eb2 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:53.956222 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bada9770-20fe-4be8-b052-e72253376eb2 None admin] PATH_INFO: `/v3/services` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:53.956924 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bada9770-20fe-4be8-b052-e72253376eb2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:53.958348 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bada9770-20fe-4be8-b052-e72253376eb2 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:53.964806 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 501/994] 10.222.0.57 () {56 vars in 1087 bytes} [Thu May 19 14:52:53 2022] GET /identity/v3/services => generated 1727 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 14:52:55.252324 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecbf324-8bac-47b5-86fc-1c6653993c2b None None] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:55.252955 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecbf324-8bac-47b5-86fc-1c6653993c2b None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:55.252955 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecbf324-8bac-47b5-86fc-1c6653993c2b None None] PATH_INFO: `/` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:55.254461 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 494/995] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:52:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:52:55.259531 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc1cf2f-efda-45bb-bf0a-75052375abc2 None None] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:55.259931 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc1cf2f-efda-45bb-bf0a-75052375abc2 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:55.260126 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc1cf2f-efda-45bb-bf0a-75052375abc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:55.278105 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.auth.core [None req-0fc1cf2f-efda-45bb-bf0a-75052375abc2 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:52:55.279298 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-0fc1cf2f-efda-45bb-bf0a-75052375abc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:52:55.351002 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 502/996] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:52:55 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:52:55.373818 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecbf324-8bac-47b5-86fc-1c6653993c2b None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:55.450326 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:55.484552 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecbf324-8bac-47b5-86fc-1c6653993c2b None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:55.519893 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c43cde3-2ea0-4dd2-9fe2-a13bcac7cdd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:55.520572 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c43cde3-2ea0-4dd2-9fe2-a13bcac7cdd2 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:55.521038 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c43cde3-2ea0-4dd2-9fe2-a13bcac7cdd2 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:55.521328 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c43cde3-2ea0-4dd2-9fe2-a13bcac7cdd2 None admin] PATH_INFO: `/v3/services/compute` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:55.522027 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c43cde3-2ea0-4dd2-9fe2-a13bcac7cdd2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:55.523424 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c43cde3-2ea0-4dd2-9fe2-a13bcac7cdd2 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:55.535740 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-2c43cde3-2ea0-4dd2-9fe2-a13bcac7cdd2 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 19 14:52:55.535740 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:55.535740 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:52:55.535740 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:52:55.535740 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:52:55.535740 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:52:55.535740 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:52:55.535740 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:52:55.535740 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 14:52:55.535740 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:52:55.535740 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:52:55.535740 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:52:55.535740 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:55.535740 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:55.535740 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:55.535740 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:55.535740 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:55.536971 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:55.536971 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:55.536971 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:55.536971 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:55.536971 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:55.536971 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:55.536971 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:52:55.536971 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:52:55.536971 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:52:55.536971 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:52:55.536971 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:55.536971 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:55.536971 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:55.536971 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:55.536971 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:55.536971 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:55.538572 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:55.538572 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:52:55.538572 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:55.538572 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:55.538572 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:55.538572 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:55.538572 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:55.538572 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:55.538572 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:55.538572 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:55.538572 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:52:55.538572 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:52:55.538572 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 14:52:55.538572 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:52:55.538572 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 495/997] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:52:55 2022] GET /identity/v3/services/compute => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:55.543064 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc1cf2f-efda-45bb-bf0a-75052375abc2 None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:55.588946 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:55.649579 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc1cf2f-efda-45bb-bf0a-75052375abc2 None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:55.689246 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404b29ab-4020-48f0-99b1-91a52cbfd663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:55.689953 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404b29ab-4020-48f0-99b1-91a52cbfd663 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:55.690247 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404b29ab-4020-48f0-99b1-91a52cbfd663 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:55.690529 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404b29ab-4020-48f0-99b1-91a52cbfd663 None admin] PATH_INFO: `/v3/services` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:55.691250 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404b29ab-4020-48f0-99b1-91a52cbfd663 None admin] RBAC: Adding query filter params (name=compute) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:55.691560 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404b29ab-4020-48f0-99b1-91a52cbfd663 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:55.692930 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404b29ab-4020-48f0-99b1-91a52cbfd663 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:55.703478 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 503/998] 10.222.0.57 () {56 vars in 1112 bytes} [Thu May 19 14:52:55 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:55.704721 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c43cde3-2ea0-4dd2-9fe2-a13bcac7cdd2 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:55.816121 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c43cde3-2ea0-4dd2-9fe2-a13bcac7cdd2 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:55.853057 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16631d61-56c9-46ea-ad43-0fa572a783b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:55.853736 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16631d61-56c9-46ea-ad43-0fa572a783b6 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:55.854050 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16631d61-56c9-46ea-ad43-0fa572a783b6 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:55.854335 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16631d61-56c9-46ea-ad43-0fa572a783b6 None admin] PATH_INFO: `/v3/services` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:55.855058 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16631d61-56c9-46ea-ad43-0fa572a783b6 None admin] RBAC: Adding query filter params (type=compute) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:55.855370 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16631d61-56c9-46ea-ad43-0fa572a783b6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:55.856751 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16631d61-56c9-46ea-ad43-0fa572a783b6 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:55.858052 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:55.868451 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 496/999] 10.222.0.57 () {56 vars in 1112 bytes} [Thu May 19 14:52:55 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:55.870014 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404b29ab-4020-48f0-99b1-91a52cbfd663 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:55.975358 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404b29ab-4020-48f0-99b1-91a52cbfd663 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:55.977554 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812826f2-04f5-4738-a74e-74432b554e90 None None] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:55.977864 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812826f2-04f5-4738-a74e-74432b554e90 None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:55.978157 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812826f2-04f5-4738-a74e-74432b554e90 None None] PATH_INFO: `/` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:55.979545 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 497/1000] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:52:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:52:55.981660 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:55.985404 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6bb023-d749-4398-bc58-616a4aa02686 None None] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:55.985649 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6bb023-d749-4398-bc58-616a4aa02686 None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:55.987310 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6bb023-d749-4398-bc58-616a4aa02686 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:56.006758 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.auth.core [None req-9e6bb023-d749-4398-bc58-616a4aa02686 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:52:56.007970 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.fernet_utils [None req-9e6bb023-d749-4398-bc58-616a4aa02686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:52:56.019823 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9202eac8-55ee-4be0-88bd-3e8b71f28d4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:56.020500 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9202eac8-55ee-4be0-88bd-3e8b71f28d4c None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:56.020792 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9202eac8-55ee-4be0-88bd-3e8b71f28d4c None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:56.021077 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9202eac8-55ee-4be0-88bd-3e8b71f28d4c None admin] PATH_INFO: `/v3/endpoints` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:56.021905 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9202eac8-55ee-4be0-88bd-3e8b71f28d4c None admin] RBAC: Adding query filter params (interface=public, service_id=b175b1160e164039a5c0b2207d740d7c, region_id=RegionOne) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:56.022235 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9202eac8-55ee-4be0-88bd-3e8b71f28d4c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:56.023610 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9202eac8-55ee-4be0-88bd-3e8b71f28d4c None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:56.029890 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 504/1001] 10.222.0.57 () {56 vars in 1251 bytes} [Thu May 19 14:52:55 2022] GET /identity/v3/endpoints?service_id=b175b1160e164039a5c0b2207d740d7c&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:56.034810 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9202eac8-55ee-4be0-88bd-3e8b71f28d4c None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:56.090890 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 498/1002] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:52:55 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:52:56.113599 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6bb023-d749-4398-bc58-616a4aa02686 None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:56.154086 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9202eac8-55ee-4be0-88bd-3e8b71f28d4c None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:56.195233 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd2176f-d05e-4103-8d38-3d884a37ad71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:56.195913 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd2176f-d05e-4103-8d38-3d884a37ad71 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:56.196205 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd2176f-d05e-4103-8d38-3d884a37ad71 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:56.196489 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd2176f-d05e-4103-8d38-3d884a37ad71 None admin] PATH_INFO: `/v3/services` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:56.197188 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd2176f-d05e-4103-8d38-3d884a37ad71 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:56.198600 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd2176f-d05e-4103-8d38-3d884a37ad71 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:56.213874 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 505/1003] 10.222.0.57 () {56 vars in 1087 bytes} [Thu May 19 14:52:56 2022] GET /identity/v3/services => generated 1727 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 14:52:56.227435 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6bb023-d749-4398-bc58-616a4aa02686 None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:56.260059 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:56.261202 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3def44ae-ad2e-4f7a-9b08-d62914040dff None None] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:56.261625 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3def44ae-ad2e-4f7a-9b08-d62914040dff None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:56.261773 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3def44ae-ad2e-4f7a-9b08-d62914040dff None None] PATH_INFO: `/` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:56.263948 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 506/1004] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:52:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:52:56.271731 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d4da08-c2bb-42b0-b154-2acefa770a47 None None] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:56.271984 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2360034b-a111-46d0-bec2-4a95fffe48d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:56.272098 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d4da08-c2bb-42b0-b154-2acefa770a47 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:56.272337 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d4da08-c2bb-42b0-b154-2acefa770a47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:56.272637 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2360034b-a111-46d0-bec2-4a95fffe48d8 None admin] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:56.272926 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2360034b-a111-46d0-bec2-4a95fffe48d8 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:56.273208 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2360034b-a111-46d0-bec2-4a95fffe48d8 None admin] PATH_INFO: `/v3/services` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:56.273990 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2360034b-a111-46d0-bec2-4a95fffe48d8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:56.275438 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2360034b-a111-46d0-bec2-4a95fffe48d8 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:56.292705 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.auth.core [None req-c5d4da08-c2bb-42b0-b154-2acefa770a47 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:52:56.293873 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 499/1005] 10.222.0.57 () {60 vars in 1141 bytes} [Thu May 19 14:52:56 2022] POST /identity/v3/services => generated 242 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 14:52:56.294044 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-c5d4da08-c2bb-42b0-b154-2acefa770a47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:52:56.330447 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:56.393887 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 507/1006] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:52:56 2022] POST /identity/v3/auth/tokens => generated 1656 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:52:56.414093 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2360034b-a111-46d0-bec2-4a95fffe48d8 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:56.531731 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2360034b-a111-46d0-bec2-4a95fffe48d8 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:56.567753 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517ec902-1eba-49c6-85ae-afaf13e759c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:56.568585 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517ec902-1eba-49c6-85ae-afaf13e759c1 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:56.568874 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517ec902-1eba-49c6-85ae-afaf13e759c1 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:56.569412 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517ec902-1eba-49c6-85ae-afaf13e759c1 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:56.570105 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517ec902-1eba-49c6-85ae-afaf13e759c1 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:56.571609 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517ec902-1eba-49c6-85ae-afaf13e759c1 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:56.585583 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-517ec902-1eba-49c6-85ae-afaf13e759c1 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 14:52:56.585583 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:56.585583 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:52:56.585583 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:52:56.585583 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:52:56.585583 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:52:56.585583 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:52:56.585583 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:52:56.585583 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 14:52:56.585583 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:52:56.585583 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:52:56.585583 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:52:56.585583 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:56.585583 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:56.585583 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:56.585583 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:56.585583 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:56.586959 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:56.586959 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:56.586959 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:56.586959 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:56.586959 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:56.586959 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:56.586959 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:52:56.586959 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:52:56.586959 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:52:56.586959 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:52:56.586959 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:56.586959 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:56.586959 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:56.586959 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:56.586959 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:56.586959 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:56.587948 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:56.587948 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:52:56.587948 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:56.587948 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:56.587948 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:56.587948 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:56.587948 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:56.587948 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:56.587948 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:56.587948 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:56.587948 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:52:56.587948 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:52:56.587948 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 14:52:56.587948 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:52:56.587948 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 500/1007] 10.222.0.57 () {56 vars in 1126 bytes} [Thu May 19 14:52:56 2022] GET /identity/v3/services/object-store => generated 93 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:56.593509 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d4da08-c2bb-42b0-b154-2acefa770a47 None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:56.708469 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d4da08-c2bb-42b0-b154-2acefa770a47 None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:56.743650 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fbe383-9ae5-49b9-bfcc-96e215240c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:56.744329 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fbe383-9ae5-49b9-bfcc-96e215240c95 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:56.744623 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fbe383-9ae5-49b9-bfcc-96e215240c95 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:56.744922 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fbe383-9ae5-49b9-bfcc-96e215240c95 None admin] PATH_INFO: `/v3/services` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:56.745627 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fbe383-9ae5-49b9-bfcc-96e215240c95 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:56.745972 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fbe383-9ae5-49b9-bfcc-96e215240c95 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:56.747349 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fbe383-9ae5-49b9-bfcc-96e215240c95 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:56.757197 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 508/1008] 10.222.0.57 () {56 vars in 1122 bytes} [Thu May 19 14:52:56 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:56.758855 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517ec902-1eba-49c6-85ae-afaf13e759c1 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:56.814923 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:56.869301 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517ec902-1eba-49c6-85ae-afaf13e759c1 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:56.904746 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f3710f-830a-49d8-b55b-1c34f454e29d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:56.905455 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f3710f-830a-49d8-b55b-1c34f454e29d None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:56.906346 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f3710f-830a-49d8-b55b-1c34f454e29d None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:56.906630 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f3710f-830a-49d8-b55b-1c34f454e29d None admin] PATH_INFO: `/v3/services` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:56.907344 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f3710f-830a-49d8-b55b-1c34f454e29d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:56.907712 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f3710f-830a-49d8-b55b-1c34f454e29d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:56.909042 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f3710f-830a-49d8-b55b-1c34f454e29d None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:56.918802 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 501/1009] 10.222.0.57 () {56 vars in 1122 bytes} [Thu May 19 14:52:56 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:56.920364 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fbe383-9ae5-49b9-bfcc-96e215240c95 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:56.947672 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:57.029220 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fbe383-9ae5-49b9-bfcc-96e215240c95 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:57.067633 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc86ed5-8af1-431c-8891-d96bc2421fc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:57.068303 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc86ed5-8af1-431c-8891-d96bc2421fc9 None admin] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:57.068595 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc86ed5-8af1-431c-8891-d96bc2421fc9 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:57.068882 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc86ed5-8af1-431c-8891-d96bc2421fc9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:57.069919 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc86ed5-8af1-431c-8891-d96bc2421fc9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:57.071294 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc86ed5-8af1-431c-8891-d96bc2421fc9 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:57.093913 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 509/1010] 10.222.0.57 () {60 vars in 1144 bytes} [Thu May 19 14:52:56 2022] POST /identity/v3/endpoints => generated 325 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:52:58.514517 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d108b503-5266-40e4-a93b-992be4f62c95 None None] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:58.515123 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d108b503-5266-40e4-a93b-992be4f62c95 None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:58.515414 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d108b503-5266-40e4-a93b-992be4f62c95 None None] PATH_INFO: `/` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:58.516827 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 502/1011] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:52:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:52:58.522259 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec62b6c-c535-485e-8486-bc28e50ee885 None None] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:58.522683 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec62b6c-c535-485e-8486-bc28e50ee885 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:58.522877 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec62b6c-c535-485e-8486-bc28e50ee885 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:58.540350 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.auth.core [None req-bec62b6c-c535-485e-8486-bc28e50ee885 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:52:58.541556 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-bec62b6c-c535-485e-8486-bc28e50ee885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:52:58.603722 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f3279b-01aa-467a-88bd-08dc2ee81c6e None None] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:58.604025 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f3279b-01aa-467a-88bd-08dc2ee81c6e None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:58.604317 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f3279b-01aa-467a-88bd-08dc2ee81c6e None None] PATH_INFO: `/` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:58.605882 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 503/1012] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:52:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:52:58.611709 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693f52e5-8b82-4eda-951a-f1cb03ab66a6 None None] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:58.611998 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693f52e5-8b82-4eda-951a-f1cb03ab66a6 None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:58.612280 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693f52e5-8b82-4eda-951a-f1cb03ab66a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:58.619266 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 510/1013] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:52:58 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:52:58.634004 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.auth.core [None req-693f52e5-8b82-4eda-951a-f1cb03ab66a6 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:52:58.635205 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.fernet_utils [None req-693f52e5-8b82-4eda-951a-f1cb03ab66a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:52:58.642711 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec62b6c-c535-485e-8486-bc28e50ee885 None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:58.653233 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:58.723035 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 504/1014] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:52:58 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:52:58.746765 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693f52e5-8b82-4eda-951a-f1cb03ab66a6 None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:58.754096 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec62b6c-c535-485e-8486-bc28e50ee885 None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:58.791287 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b768106-6bb1-4556-97e8-32362eff04a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:58.792018 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b768106-6bb1-4556-97e8-32362eff04a7 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:58.792315 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b768106-6bb1-4556-97e8-32362eff04a7 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:58.792600 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b768106-6bb1-4556-97e8-32362eff04a7 None admin] PATH_INFO: `/v3/services/compute` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:58.793273 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b768106-6bb1-4556-97e8-32362eff04a7 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:58.795163 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b768106-6bb1-4556-97e8-32362eff04a7 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:58.802695 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:58.809613 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-4b768106-6bb1-4556-97e8-32362eff04a7 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 19 14:52:58.809613 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:58.809613 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:52:58.809613 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:52:58.809613 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:52:58.809613 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:52:58.809613 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:52:58.809613 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:52:58.809613 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 14:52:58.809613 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:52:58.809613 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:52:58.809613 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:52:58.809613 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:58.809613 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:58.809613 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:58.809613 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:58.809613 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:58.811079 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:58.811079 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:58.811079 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:58.811079 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:58.811079 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:58.811079 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:58.811079 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:52:58.811079 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:52:58.811079 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:52:58.811079 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:52:58.811079 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:58.811079 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:58.811079 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:58.811079 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:58.811079 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:58.811079 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:58.812417 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:58.812417 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:52:58.812417 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:58.812417 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:58.812417 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:58.812417 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:58.812417 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:58.812417 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:58.812417 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:58.812417 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:58.812417 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:52:58.812417 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:52:58.812417 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 14:52:58.812417 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:52:58.813323 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 511/1015] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:52:58 2022] GET /identity/v3/services/compute => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:52:58.819603 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b768106-6bb1-4556-97e8-32362eff04a7 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:58.867240 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693f52e5-8b82-4eda-951a-f1cb03ab66a6 None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:58.896835 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:58.905787 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c33291-bbf3-4764-8045-08f77ff3f75b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:58.906613 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c33291-bbf3-4764-8045-08f77ff3f75b None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:58.906901 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c33291-bbf3-4764-8045-08f77ff3f75b None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:58.907508 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c33291-bbf3-4764-8045-08f77ff3f75b None admin] PATH_INFO: `/v3/services/image` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:58.908180 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c33291-bbf3-4764-8045-08f77ff3f75b None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:58.909573 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c33291-bbf3-4764-8045-08f77ff3f75b None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:58.923367 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-c0c33291-bbf3-4764-8045-08f77ff3f75b None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 19 14:52:58.923367 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:58.923367 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:52:58.923367 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:52:58.923367 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:52:58.923367 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:52:58.923367 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:52:58.923367 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:52:58.923367 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 14:52:58.923367 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:52:58.923367 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:52:58.923367 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:52:58.923367 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:58.923367 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:58.923367 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:58.923367 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:58.923367 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:58.924570 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:58.924570 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:58.924570 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:58.924570 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:58.924570 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:58.924570 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:58.924570 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:52:58.924570 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:52:58.924570 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:52:58.924570 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:52:58.924570 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:58.924570 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:58.924570 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:58.924570 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:58.924570 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:58.924570 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:58.925582 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:58.925582 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:52:58.925582 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:58.925582 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:58.925582 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:58.925582 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:58.925582 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:58.925582 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:58.925582 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:58.925582 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:58.925582 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:52:58.925582 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:52:58.925582 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 14:52:58.925582 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:52:58.925582 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 505/1016] 10.222.0.57 () {56 vars in 1105 bytes} [Thu May 19 14:52:58 2022] GET /identity/v3/services/image => generated 86 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:52:58.930694 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c33291-bbf3-4764-8045-08f77ff3f75b None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:58.934620 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b768106-6bb1-4556-97e8-32362eff04a7 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:58.972905 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc7198e-13ba-481e-870d-73e4183b4665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:58.973587 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc7198e-13ba-481e-870d-73e4183b4665 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:58.973904 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc7198e-13ba-481e-870d-73e4183b4665 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:58.974191 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc7198e-13ba-481e-870d-73e4183b4665 None admin] PATH_INFO: `/v3/services` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:58.974916 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc7198e-13ba-481e-870d-73e4183b4665 None admin] RBAC: Adding query filter params (name=compute) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:58.975231 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc7198e-13ba-481e-870d-73e4183b4665 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:58.976608 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc7198e-13ba-481e-870d-73e4183b4665 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:58.982614 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 512/1017] 10.222.0.57 () {56 vars in 1112 bytes} [Thu May 19 14:52:58 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:52:58.989091 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc7198e-13ba-481e-870d-73e4183b4665 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:59.048639 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c33291-bbf3-4764-8045-08f77ff3f75b None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:59.059929 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:59.087568 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677330b8-7112-4621-94c4-ae967a4af929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:59.088258 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677330b8-7112-4621-94c4-ae967a4af929 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:59.088554 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677330b8-7112-4621-94c4-ae967a4af929 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:59.088839 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677330b8-7112-4621-94c4-ae967a4af929 None admin] PATH_INFO: `/v3/services` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:59.089560 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677330b8-7112-4621-94c4-ae967a4af929 None admin] RBAC: Adding query filter params (name=image) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:59.089900 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677330b8-7112-4621-94c4-ae967a4af929 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:59.091293 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677330b8-7112-4621-94c4-ae967a4af929 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:59.101721 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 506/1018] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:52:58 2022] GET /identity/v3/services?name=image => generated 122 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:52:59.102766 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677330b8-7112-4621-94c4-ae967a4af929 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:59.105993 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc7198e-13ba-481e-870d-73e4183b4665 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:59.146135 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036eab94-473a-4547-a1c5-64e685fb6044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:59.146832 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036eab94-473a-4547-a1c5-64e685fb6044 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:59.147127 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036eab94-473a-4547-a1c5-64e685fb6044 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:59.147415 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036eab94-473a-4547-a1c5-64e685fb6044 None admin] PATH_INFO: `/v3/services` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:59.148699 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036eab94-473a-4547-a1c5-64e685fb6044 None admin] RBAC: Adding query filter params (type=compute) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:59.149025 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036eab94-473a-4547-a1c5-64e685fb6044 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:59.151197 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036eab94-473a-4547-a1c5-64e685fb6044 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:59.153748 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:59.162884 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 513/1019] 10.222.0.57 () {56 vars in 1112 bytes} [Thu May 19 14:52:58 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:52:59.164157 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036eab94-473a-4547-a1c5-64e685fb6044 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:59.224920 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677330b8-7112-4621-94c4-ae967a4af929 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:59.267439 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312da448-985c-4e18-adc7-648673e3836d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:59.268361 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312da448-985c-4e18-adc7-648673e3836d None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:59.268730 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312da448-985c-4e18-adc7-648673e3836d None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:59.268922 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312da448-985c-4e18-adc7-648673e3836d None admin] PATH_INFO: `/v3/services` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:59.269823 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312da448-985c-4e18-adc7-648673e3836d None admin] RBAC: Adding query filter params (type=image) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:59.270470 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312da448-985c-4e18-adc7-648673e3836d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:59.271951 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312da448-985c-4e18-adc7-648673e3836d None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:59.277319 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 507/1020] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:52:59 2022] GET /identity/v3/services?type=image => generated 350 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:59.279688 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036eab94-473a-4547-a1c5-64e685fb6044 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:59.284092 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312da448-985c-4e18-adc7-648673e3836d None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:59.320008 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e4a3d2-a8b8-4d76-9087-220bfe715592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:59.320859 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e4a3d2-a8b8-4d76-9087-220bfe715592 None admin] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:59.321155 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e4a3d2-a8b8-4d76-9087-220bfe715592 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:59.322351 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e4a3d2-a8b8-4d76-9087-220bfe715592 None admin] PATH_INFO: `/v3/endpoints` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:59.323120 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e4a3d2-a8b8-4d76-9087-220bfe715592 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:59.324516 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e4a3d2-a8b8-4d76-9087-220bfe715592 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:59.326950 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:59.347090 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 514/1021] 10.222.0.57 () {60 vars in 1144 bytes} [Thu May 19 14:52:59 2022] POST /identity/v3/endpoints => generated 334 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:52:59.384473 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b24193b-7ce8-46dd-9d66-0edfeb39a278 None None] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:59.384834 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b24193b-7ce8-46dd-9d66-0edfeb39a278 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:59.385057 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b24193b-7ce8-46dd-9d66-0edfeb39a278 None None] PATH_INFO: `/` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:59.386912 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 515/1022] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:52:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:52:59.392582 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528840bf-34d3-42d7-bbae-b948e4539b3c None None] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:59.392885 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528840bf-34d3-42d7-bbae-b948e4539b3c None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:59.393177 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528840bf-34d3-42d7-bbae-b948e4539b3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:59.411436 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.auth.core [None req-528840bf-34d3-42d7-bbae-b948e4539b3c None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:52:59.411668 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312da448-985c-4e18-adc7-648673e3836d None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:59.412681 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-528840bf-34d3-42d7-bbae-b948e4539b3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:52:59.459469 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdaa959-d5e6-45cb-a52f-f53d0d9e29dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:59.460162 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdaa959-d5e6-45cb-a52f-f53d0d9e29dc None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:59.460447 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdaa959-d5e6-45cb-a52f-f53d0d9e29dc None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:59.460731 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdaa959-d5e6-45cb-a52f-f53d0d9e29dc None admin] PATH_INFO: `/v3/endpoints` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:59.461540 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdaa959-d5e6-45cb-a52f-f53d0d9e29dc None admin] RBAC: Adding query filter params (interface=public, service_id=3cac320e2f9a4f1290dcea385abee320, region_id=RegionOne) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:59.461886 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdaa959-d5e6-45cb-a52f-f53d0d9e29dc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:59.463315 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdaa959-d5e6-45cb-a52f-f53d0d9e29dc None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:59.469580 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 508/1023] 10.222.0.57 () {56 vars in 1251 bytes} [Thu May 19 14:52:59 2022] GET /identity/v3/endpoints?service_id=3cac320e2f9a4f1290dcea385abee320&interface=public®ion_id=RegionOne => generated 194 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:52:59.471454 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:59.475879 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdaa959-d5e6-45cb-a52f-f53d0d9e29dc None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:59.493394 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 516/1024] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:52:59 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:52:59.518935 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528840bf-34d3-42d7-bbae-b948e4539b3c None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:59.594650 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:59.599046 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdaa959-d5e6-45cb-a52f-f53d0d9e29dc None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:59.642641 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18ce3ff-946a-413b-918c-46db79f48793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:59.643750 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18ce3ff-946a-413b-918c-46db79f48793 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:59.643890 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528840bf-34d3-42d7-bbae-b948e4539b3c None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:59.644719 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18ce3ff-946a-413b-918c-46db79f48793 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:59.645230 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18ce3ff-946a-413b-918c-46db79f48793 None admin] PATH_INFO: `/v3/services` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:59.647176 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18ce3ff-946a-413b-918c-46db79f48793 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:59.648588 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18ce3ff-946a-413b-918c-46db79f48793 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:59.658377 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 509/1025] 10.222.0.57 () {56 vars in 1087 bytes} [Thu May 19 14:52:59 2022] GET /identity/v3/services => generated 1957 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 14:52:59.687570 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376036af-2b82-4941-86e7-457b52119b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:59.688406 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376036af-2b82-4941-86e7-457b52119b0a None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:59.688698 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376036af-2b82-4941-86e7-457b52119b0a None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:59.688986 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376036af-2b82-4941-86e7-457b52119b0a None admin] PATH_INFO: `/v3/domains/default` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:59.690354 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376036af-2b82-4941-86e7-457b52119b0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:59.691845 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376036af-2b82-4941-86e7-457b52119b0a None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:59.697929 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 517/1026] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:52:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:52:59.700142 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18ce3ff-946a-413b-918c-46db79f48793 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:59.818092 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18ce3ff-946a-413b-918c-46db79f48793 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:59.856728 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a71af3b-bc1a-4e74-aed3-5ab4315af695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:59.857328 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a71af3b-bc1a-4e74-aed3-5ab4315af695 None admin] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:59.857622 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a71af3b-bc1a-4e74-aed3-5ab4315af695 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:59.857942 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a71af3b-bc1a-4e74-aed3-5ab4315af695 None admin] PATH_INFO: `/v3/projects` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:59.859563 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a71af3b-bc1a-4e74-aed3-5ab4315af695 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:59.861071 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a71af3b-bc1a-4e74-aed3-5ab4315af695 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:59.890200 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 510/1027] 10.222.0.57 () {60 vars in 1141 bytes} [Thu May 19 14:52:59 2022] POST /identity/v3/projects => generated 311 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:53:01.712657 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f4f512-1bb9-4b4b-964b-ad15fd3ec3ec None None] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:01.713203 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f4f512-1bb9-4b4b-964b-ad15fd3ec3ec None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:01.713203 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f4f512-1bb9-4b4b-964b-ad15fd3ec3ec None None] PATH_INFO: `/` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:01.714759 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 518/1028] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:53:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:53:01.720780 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81277851-c313-4fa7-a072-b69b0282c5b6 None None] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:01.721328 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81277851-c313-4fa7-a072-b69b0282c5b6 None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:01.721328 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81277851-c313-4fa7-a072-b69b0282c5b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:01.739497 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.auth.core [None req-81277851-c313-4fa7-a072-b69b0282c5b6 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:53:01.740744 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.fernet_utils [None req-81277851-c313-4fa7-a072-b69b0282c5b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:53:01.753594 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:01.824553 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 511/1029] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:53:01 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:53:01.848883 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f4f512-1bb9-4b4b-964b-ad15fd3ec3ec None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:01.908866 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:01.923826 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c57cc5-25a4-4961-a21b-c92f325e958d None None] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:01.924069 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c57cc5-25a4-4961-a21b-c92f325e958d None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:01.924338 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c57cc5-25a4-4961-a21b-c92f325e958d None None] PATH_INFO: `/` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:01.925891 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 512/1030] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:53:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:53:01.930931 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88c2e8f-8288-4fed-8207-b0115ba2665a None None] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:01.931235 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88c2e8f-8288-4fed-8207-b0115ba2665a None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:01.931534 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88c2e8f-8288-4fed-8207-b0115ba2665a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:01.949361 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.auth.core [None req-f88c2e8f-8288-4fed-8207-b0115ba2665a None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:53:01.950608 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.fernet_utils [None req-f88c2e8f-8288-4fed-8207-b0115ba2665a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:53:01.959672 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f4f512-1bb9-4b4b-964b-ad15fd3ec3ec None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:01.998268 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9bfbae-a181-459a-9270-95fcdf7a3e23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:01.999242 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9bfbae-a181-459a-9270-95fcdf7a3e23 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:01.999532 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9bfbae-a181-459a-9270-95fcdf7a3e23 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:01.999813 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9bfbae-a181-459a-9270-95fcdf7a3e23 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:02.012793 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9bfbae-a181-459a-9270-95fcdf7a3e23 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:53:02.021581 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-6a9bfbae-a181-459a-9270-95fcdf7a3e23 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 14:53:02.021581 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:53:02.021581 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:53:02.021581 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:53:02.021581 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:53:02.021581 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:53:02.021581 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:53:02.021581 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:53:02.021581 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:53:02.021581 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:53:02.021581 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:53:02.021581 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:53:02.021581 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:53:02.021581 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:53:02.021581 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:53:02.021581 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:53:02.022988 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:53:02.022988 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:53:02.022988 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:53:02.022988 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:53:02.022988 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:53:02.022988 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:53:02.022988 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:53:02.022988 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:53:02.022988 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:53:02.022988 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:53:02.022988 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:53:02.022988 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:53:02.022988 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:53:02.022988 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:53:02.022988 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:53:02.022988 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:53:02.024749 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:53:02.024749 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:53:02.024749 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:53:02.024749 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:53:02.024749 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:53:02.024749 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:53:02.024749 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:53:02.024749 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:53:02.024749 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:53:02.024749 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:53:02.024749 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 14:53:02.024749 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:53:02.024749 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 519/1031] 10.222.0.57 () {56 vars in 1120 bytes} [Thu May 19 14:53:01 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:53:02.029708 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9bfbae-a181-459a-9270-95fcdf7a3e23 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:02.033893 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 513/1032] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:53:01 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:53:02.056614 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88c2e8f-8288-4fed-8207-b0115ba2665a None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:02.141395 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9bfbae-a181-459a-9270-95fcdf7a3e23 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:02.147146 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:02.161021 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:02.178545 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88c2e8f-8288-4fed-8207-b0115ba2665a None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:02.188347 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9b88fb-1712-4dce-bc14-9c54a354a62c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:02.190106 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9b88fb-1712-4dce-bc14-9c54a354a62c None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:02.190106 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9b88fb-1712-4dce-bc14-9c54a354a62c None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:02.190106 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9b88fb-1712-4dce-bc14-9c54a354a62c None admin] PATH_INFO: `/v3/roles` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:02.190548 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9b88fb-1712-4dce-bc14-9c54a354a62c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:53:02.190867 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9b88fb-1712-4dce-bc14-9c54a354a62c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:02.192301 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9b88fb-1712-4dce-bc14-9c54a354a62c None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:02.201663 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 520/1033] 10.222.0.57 () {56 vars in 1115 bytes} [Thu May 19 14:53:02 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:53:02.203823 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dce2a9-b9c9-486f-8e0b-7957b3c29502 None None] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:02.204115 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dce2a9-b9c9-486f-8e0b-7957b3c29502 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:02.204400 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dce2a9-b9c9-486f-8e0b-7957b3c29502 None None] PATH_INFO: `/` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:02.205777 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 521/1034] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:53:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:53:02.209023 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dce2a9-b9c9-486f-8e0b-7957b3c29502 None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:02.224004 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4119be3-ea61-4675-8861-d3c026b3afeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:02.224832 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4119be3-ea61-4675-8861-d3c026b3afeb None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:02.225126 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4119be3-ea61-4675-8861-d3c026b3afeb None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:02.225415 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4119be3-ea61-4675-8861-d3c026b3afeb None admin] PATH_INFO: `/v3/services/image` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:02.226127 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4119be3-ea61-4675-8861-d3c026b3afeb None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:02.227564 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4119be3-ea61-4675-8861-d3c026b3afeb None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:02.240368 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-e4119be3-ea61-4675-8861-d3c026b3afeb None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 19 14:53:02.240368 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:53:02.240368 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:53:02.240368 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:53:02.240368 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:53:02.240368 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:53:02.240368 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:53:02.240368 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:53:02.240368 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 14:53:02.240368 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:53:02.240368 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:53:02.240368 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:53:02.240368 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:53:02.240368 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:53:02.240368 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:53:02.240368 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:53:02.240368 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:53:02.241648 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:53:02.241648 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:53:02.241648 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:53:02.241648 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:53:02.241648 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:53:02.241648 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:53:02.241648 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:53:02.241648 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:53:02.241648 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:53:02.241648 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:53:02.241648 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:53:02.241648 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:53:02.241648 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:53:02.241648 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:53:02.241648 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:53:02.241648 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:53:02.242883 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:53:02.242883 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:53:02.242883 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:53:02.242883 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:53:02.242883 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:53:02.242883 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:53:02.242883 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:53:02.242883 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:53:02.242883 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:53:02.242883 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:53:02.242883 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:53:02.242883 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:53:02.242883 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 14:53:02.242883 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:53:02.246026 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 514/1035] 10.222.0.57 () {56 vars in 1105 bytes} [Thu May 19 14:53:02 2022] GET /identity/v3/services/image => generated 86 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:53:02.247471 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a229908-20d8-4e0e-870f-85f121edc1ce None None] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:02.247768 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a229908-20d8-4e0e-870f-85f121edc1ce None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:02.248049 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a229908-20d8-4e0e-870f-85f121edc1ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:02.267453 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.auth.core [None req-9a229908-20d8-4e0e-870f-85f121edc1ce None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:53:02.268865 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.fernet_utils [None req-9a229908-20d8-4e0e-870f-85f121edc1ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:53:02.320240 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dce2a9-b9c9-486f-8e0b-7957b3c29502 None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:02.338999 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 515/1036] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:53:02 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:53:02.339836 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a229908-20d8-4e0e-870f-85f121edc1ce None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:02.360846 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c8c22c-ccb7-4bc5-bdd9-5ae7a8fdd23e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:02.361639 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c8c22c-ccb7-4bc5-bdd9-5ae7a8fdd23e None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:02.361956 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c8c22c-ccb7-4bc5-bdd9-5ae7a8fdd23e None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:02.362241 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c8c22c-ccb7-4bc5-bdd9-5ae7a8fdd23e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:02.372660 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c8c22c-ccb7-4bc5-bdd9-5ae7a8fdd23e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:02.374810 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c8c22c-ccb7-4bc5-bdd9-5ae7a8fdd23e None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:02.393231 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-42c8c22c-ccb7-4bc5-bdd9-5ae7a8fdd23e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:53:02.393231 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:53:02.393231 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:53:02.393231 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:53:02.393231 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:53:02.393231 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:53:02.393231 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:53:02.393231 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:53:02.393231 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:53:02.393231 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:53:02.393231 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:53:02.393231 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:53:02.393231 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:53:02.393231 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:53:02.393231 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:53:02.394515 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:53:02.394515 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:53:02.394515 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:53:02.394515 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:53:02.394515 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:53:02.394515 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:53:02.394515 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:53:02.394515 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:53:02.394515 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:53:02.394515 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:53:02.394515 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:53:02.394515 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:53:02.394515 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:53:02.394515 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:53:02.394515 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:53:02.394515 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:53:02.394515 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:53:02.395669 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:53:02.395669 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:53:02.395669 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:53:02.395669 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:53:02.395669 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 522/1037] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:53:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:53:02.395669 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c8c22c-ccb7-4bc5-bdd9-5ae7a8fdd23e None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:02.413365 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:02.427487 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:02.454589 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a229908-20d8-4e0e-870f-85f121edc1ce None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:02.492300 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d55674d-8fbf-4dd2-8d4f-ca16cd88ebc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:02.493234 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d55674d-8fbf-4dd2-8d4f-ca16cd88ebc2 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:02.493522 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d55674d-8fbf-4dd2-8d4f-ca16cd88ebc2 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:02.493806 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d55674d-8fbf-4dd2-8d4f-ca16cd88ebc2 None admin] PATH_INFO: `/v3/services` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:02.494554 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d55674d-8fbf-4dd2-8d4f-ca16cd88ebc2 None admin] RBAC: Adding query filter params (name=image) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:53:02.494865 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d55674d-8fbf-4dd2-8d4f-ca16cd88ebc2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:02.496240 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d55674d-8fbf-4dd2-8d4f-ca16cd88ebc2 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:02.501810 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 516/1038] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:53:02 2022] GET /identity/v3/services?name=image => generated 122 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:53:02.502839 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d55674d-8fbf-4dd2-8d4f-ca16cd88ebc2 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:02.509524 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c8c22c-ccb7-4bc5-bdd9-5ae7a8fdd23e None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:02.544269 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60876393-7cfc-4e51-9f10-5b85ee8842f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:02.545072 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60876393-7cfc-4e51-9f10-5b85ee8842f8 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:02.545365 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60876393-7cfc-4e51-9f10-5b85ee8842f8 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:02.545651 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60876393-7cfc-4e51-9f10-5b85ee8842f8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:02.547434 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60876393-7cfc-4e51-9f10-5b85ee8842f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:02.548889 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60876393-7cfc-4e51-9f10-5b85ee8842f8 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:02.550618 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 523/1039] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:53:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:53:02.551392 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60876393-7cfc-4e51-9f10-5b85ee8842f8 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:02.603868 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d55674d-8fbf-4dd2-8d4f-ca16cd88ebc2 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:02.634023 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:02.639643 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2691e3cd-0159-42c4-918f-65397b859016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:02.640921 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2691e3cd-0159-42c4-918f-65397b859016 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:02.641209 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2691e3cd-0159-42c4-918f-65397b859016 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:02.641490 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2691e3cd-0159-42c4-918f-65397b859016 None admin] PATH_INFO: `/v3/domains` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:02.642279 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2691e3cd-0159-42c4-918f-65397b859016 None admin] RBAC: Adding query filter params (name=Default) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:53:02.642592 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2691e3cd-0159-42c4-918f-65397b859016 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:02.644004 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2691e3cd-0159-42c4-918f-65397b859016 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:02.650052 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:02.657963 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 517/1040] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:53:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:53:02.658161 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60876393-7cfc-4e51-9f10-5b85ee8842f8 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:02.659232 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2691e3cd-0159-42c4-918f-65397b859016 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:02.698221 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9997f6ca-b7e8-4009-848c-d51f41fea98f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:02.698894 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9997f6ca-b7e8-4009-848c-d51f41fea98f None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:02.699185 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9997f6ca-b7e8-4009-848c-d51f41fea98f None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:02.699476 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9997f6ca-b7e8-4009-848c-d51f41fea98f None admin] PATH_INFO: `/v3/services` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:02.700192 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9997f6ca-b7e8-4009-848c-d51f41fea98f None admin] RBAC: Adding query filter params (type=image) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:53:02.700558 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9997f6ca-b7e8-4009-848c-d51f41fea98f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:02.701984 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9997f6ca-b7e8-4009-848c-d51f41fea98f None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:02.707622 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 524/1041] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:53:02 2022] GET /identity/v3/services?type=image => generated 350 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:53:02.708354 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9997f6ca-b7e8-4009-848c-d51f41fea98f None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:02.772826 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2691e3cd-0159-42c4-918f-65397b859016 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:02.807534 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db55cbc4-aae6-4a42-b8f1-bb9e12ddd806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:02.808210 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db55cbc4-aae6-4a42-b8f1-bb9e12ddd806 None admin] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:02.808496 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db55cbc4-aae6-4a42-b8f1-bb9e12ddd806 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:02.808775 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db55cbc4-aae6-4a42-b8f1-bb9e12ddd806 None admin] PATH_INFO: `/v3/users` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:02.809508 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db55cbc4-aae6-4a42-b8f1-bb9e12ddd806 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:02.810978 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db55cbc4-aae6-4a42-b8f1-bb9e12ddd806 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:02.813953 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9997f6ca-b7e8-4009-848c-d51f41fea98f None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:02.832767 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 518/1042] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 19 14:53:02 2022] POST /identity/v3/users => generated 285 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:53:02.833913 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db55cbc4-aae6-4a42-b8f1-bb9e12ddd806 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:02.852963 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab6a813-bc1e-4e50-984a-f45b6dd7be14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:02.853787 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab6a813-bc1e-4e50-984a-f45b6dd7be14 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:02.854126 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab6a813-bc1e-4e50-984a-f45b6dd7be14 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:02.854388 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab6a813-bc1e-4e50-984a-f45b6dd7be14 None admin] PATH_INFO: `/v3/users/nova` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:02.863469 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab6a813-bc1e-4e50-984a-f45b6dd7be14 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:02.864949 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab6a813-bc1e-4e50-984a-f45b6dd7be14 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:02.881434 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-4ab6a813-bc1e-4e50-984a-f45b6dd7be14 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 14:53:02.881434 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:53:02.881434 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:53:02.881434 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:53:02.881434 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:53:02.881434 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:53:02.881434 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:53:02.881434 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:53:02.881434 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:53:02.881434 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:53:02.881434 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:53:02.881434 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:53:02.881434 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:53:02.881434 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:53:02.881434 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:53:02.881434 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:53:02.882823 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:53:02.882823 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:53:02.882823 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:53:02.882823 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:53:02.882823 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:53:02.882823 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:53:02.882823 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:53:02.882823 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:53:02.882823 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:53:02.882823 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:53:02.882823 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:53:02.882823 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:53:02.882823 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:53:02.882823 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:53:02.882823 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:53:02.882823 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:53:02.882823 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:53:02.882823 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:53:02.885061 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:53:02.885061 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:53:02.885061 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:53:02.885061 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:53:02.885061 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:53:02.885061 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:53:02.885061 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:53:02.885061 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:53:02.885061 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 14:53:02.885061 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:53:02.885061 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 525/1043] 10.222.0.57 () {56 vars in 1093 bytes} [Thu May 19 14:53:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:53:02.888710 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab6a813-bc1e-4e50-984a-f45b6dd7be14 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:02.897870 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:02.918110 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:02.960059 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db55cbc4-aae6-4a42-b8f1-bb9e12ddd806 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:02.997773 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747429ff-851f-43d8-8eb7-481d22ee52ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:02.998493 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747429ff-851f-43d8-8eb7-481d22ee52ec None admin] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:02.998795 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747429ff-851f-43d8-8eb7-481d22ee52ec None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:02.999676 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747429ff-851f-43d8-8eb7-481d22ee52ec None admin] PATH_INFO: `/v3/endpoints` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:03.000448 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747429ff-851f-43d8-8eb7-481d22ee52ec None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:03.002000 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747429ff-851f-43d8-8eb7-481d22ee52ec None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:03.013579 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab6a813-bc1e-4e50-984a-f45b6dd7be14 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:03.024725 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 519/1044] 10.222.0.57 () {60 vars in 1144 bytes} [Thu May 19 14:53:02 2022] POST /identity/v3/endpoints => generated 327 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:53:03.064563 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c34326-2a70-4847-a47b-46731885013e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:03.064563 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c34326-2a70-4847-a47b-46731885013e None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:03.065085 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c34326-2a70-4847-a47b-46731885013e None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:03.065313 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c34326-2a70-4847-a47b-46731885013e None admin] PATH_INFO: `/v3/users` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:03.066063 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c34326-2a70-4847-a47b-46731885013e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:53:03.066385 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c34326-2a70-4847-a47b-46731885013e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:03.067873 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c34326-2a70-4847-a47b-46731885013e None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:03.093374 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 526/1045] 10.222.0.57 () {56 vars in 1133 bytes} [Thu May 19 14:53:02 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:53:03.099290 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747429ff-851f-43d8-8eb7-481d22ee52ec None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:03.221123 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747429ff-851f-43d8-8eb7-481d22ee52ec None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:03.223702 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:03.259355 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de595037-9a27-4ee3-8faf-7cf319c21bdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:03.260018 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de595037-9a27-4ee3-8faf-7cf319c21bdc None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:03.260307 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de595037-9a27-4ee3-8faf-7cf319c21bdc None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:03.260589 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de595037-9a27-4ee3-8faf-7cf319c21bdc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:03.333351 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de595037-9a27-4ee3-8faf-7cf319c21bdc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:03.334856 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de595037-9a27-4ee3-8faf-7cf319c21bdc None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:03.439653 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 520/1046] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:53:03 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:53:03.441006 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c34326-2a70-4847-a47b-46731885013e None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:03.523804 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:03.550686 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c34326-2a70-4847-a47b-46731885013e None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:03.587322 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4581c79-da63-456f-a24a-4d60def47ae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:03.588127 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4581c79-da63-456f-a24a-4d60def47ae0 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:03.588421 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4581c79-da63-456f-a24a-4d60def47ae0 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:03.588703 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4581c79-da63-456f-a24a-4d60def47ae0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:03.599909 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4581c79-da63-456f-a24a-4d60def47ae0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:03.601372 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4581c79-da63-456f-a24a-4d60def47ae0 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:03.619799 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-c4581c79-da63-456f-a24a-4d60def47ae0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:53:03.619799 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:53:03.619799 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:53:03.619799 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:53:03.619799 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:53:03.619799 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:53:03.619799 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:53:03.619799 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:53:03.619799 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:53:03.619799 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:53:03.619799 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:53:03.619799 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:53:03.619799 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:53:03.619799 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:53:03.619799 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:53:03.620912 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:53:03.620912 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:53:03.620912 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:53:03.620912 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:53:03.620912 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:53:03.620912 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:53:03.620912 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:53:03.620912 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:53:03.620912 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:53:03.620912 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:53:03.620912 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:53:03.620912 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:53:03.620912 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:53:03.620912 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:53:03.620912 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:53:03.620912 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:53:03.620912 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:53:03.627097 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:53:03.627097 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:53:03.627097 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:53:03.627097 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:53:03.627097 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 527/1047] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:53:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:53:03.629131 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de595037-9a27-4ee3-8faf-7cf319c21bdc None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:03.632399 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:03.740698 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de595037-9a27-4ee3-8faf-7cf319c21bdc None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:03.776729 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6debe3f-be7e-48e3-84a9-39638da1c939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:03.777381 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6debe3f-be7e-48e3-84a9-39638da1c939 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:03.777616 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6debe3f-be7e-48e3-84a9-39638da1c939 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:03.777903 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6debe3f-be7e-48e3-84a9-39638da1c939 None admin] PATH_INFO: `/v3/domains` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:03.778638 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6debe3f-be7e-48e3-84a9-39638da1c939 None admin] RBAC: Adding query filter params (name=Default) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:53:03.778952 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6debe3f-be7e-48e3-84a9-39638da1c939 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:03.781105 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6debe3f-be7e-48e3-84a9-39638da1c939 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:03.797818 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 521/1048] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:53:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:53:03.799186 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4581c79-da63-456f-a24a-4d60def47ae0 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:03.909773 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4581c79-da63-456f-a24a-4d60def47ae0 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:03.937531 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:03.948001 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2a6790-425e-41a5-b222-f301c513fe92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:03.948808 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2a6790-425e-41a5-b222-f301c513fe92 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:03.949097 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2a6790-425e-41a5-b222-f301c513fe92 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:03.949379 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2a6790-425e-41a5-b222-f301c513fe92 None admin] PATH_INFO: `/v3/projects/service` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:03.956896 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2a6790-425e-41a5-b222-f301c513fe92 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:03.958400 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2a6790-425e-41a5-b222-f301c513fe92 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:03.972009 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-6b2a6790-425e-41a5-b222-f301c513fe92 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:53:03.972009 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:53:03.972009 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:53:03.972009 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:53:03.972009 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:53:03.972009 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:53:03.972009 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:53:03.972009 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:53:03.972009 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:53:03.972009 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:53:03.972009 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:53:03.972009 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:53:03.972009 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:53:03.972009 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:53:03.972009 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:53:03.973082 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:53:03.973082 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:53:03.973082 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:53:03.973082 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:53:03.973082 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:53:03.973082 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:53:03.973082 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:53:03.973082 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:53:03.973082 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:53:03.973082 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:53:03.973082 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:53:03.973082 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:53:03.973082 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:53:03.973082 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:53:03.973082 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:53:03.973082 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:53:03.973082 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:53:03.978936 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:53:03.978936 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:53:03.978936 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:53:03.978936 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:53:03.978936 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:53:03.978936 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:53:03.978936 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:53:03.978936 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:53:03.978936 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 528/1049] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:53:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:53:03.980948 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6debe3f-be7e-48e3-84a9-39638da1c939 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:04.046783 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:04.089935 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6debe3f-be7e-48e3-84a9-39638da1c939 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:04.125206 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f193a8c3-f687-4716-8eca-0fc4c45caa0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:04.126165 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f193a8c3-f687-4716-8eca-0fc4c45caa0e None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:04.126461 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f193a8c3-f687-4716-8eca-0fc4c45caa0e None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:04.126743 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f193a8c3-f687-4716-8eca-0fc4c45caa0e None admin] PATH_INFO: `/v3/projects` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:04.127538 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f193a8c3-f687-4716-8eca-0fc4c45caa0e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:53:04.127857 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f193a8c3-f687-4716-8eca-0fc4c45caa0e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:04.129298 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f193a8c3-f687-4716-8eca-0fc4c45caa0e None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:04.141894 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 522/1050] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:53:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:53:04.148193 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2a6790-425e-41a5-b222-f301c513fe92 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:04.259351 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2a6790-425e-41a5-b222-f301c513fe92 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:04.294443 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6743a6-dd4f-442c-8eb4-be60c6b0358e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:04.295117 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6743a6-dd4f-442c-8eb4-be60c6b0358e None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:04.295408 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6743a6-dd4f-442c-8eb4-be60c6b0358e None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:04.296250 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6743a6-dd4f-442c-8eb4-be60c6b0358e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:04.297339 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6743a6-dd4f-442c-8eb4-be60c6b0358e None admin] RBAC: Adding query filter params (role.id=d07cc726aa754554a8a566bac4b7d7fe, scope.project.id=fc99d742ae004954b6d841293b5fa79c, user.id=5d2aa8c8870a49d98f23f26e036695ef) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:53:04.297612 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6743a6-dd4f-442c-8eb4-be60c6b0358e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:04.299139 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6743a6-dd4f-442c-8eb4-be60c6b0358e None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:04.313941 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 529/1051] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:53:04 2022] GET /identity/v3/role_assignments?user.id=5d2aa8c8870a49d98f23f26e036695ef&scope.project.id=fc99d742ae004954b6d841293b5fa79c&role.id=d07cc726aa754554a8a566bac4b7d7fe => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:53:05.218379 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6477fbeb-649e-4adc-a854-30e677228428 None None] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:05.218954 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6477fbeb-649e-4adc-a854-30e677228428 None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:05.218954 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6477fbeb-649e-4adc-a854-30e677228428 None None] PATH_INFO: `/` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:05.220440 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 523/1052] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:53:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:53:05.226183 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5226de-7659-4ac3-9573-38ea612348b2 None None] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:05.226604 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5226de-7659-4ac3-9573-38ea612348b2 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:05.226779 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5226de-7659-4ac3-9573-38ea612348b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:05.243728 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.auth.core [None req-9d5226de-7659-4ac3-9573-38ea612348b2 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:53:05.244922 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-9d5226de-7659-4ac3-9573-38ea612348b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:53:05.295333 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:05.319707 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 530/1053] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:53:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:53:05.344422 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6477fbeb-649e-4adc-a854-30e677228428 None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:05.353791 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c885bd7-6d56-4d5c-a310-03acfb88a5ed None None] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:05.354132 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c885bd7-6d56-4d5c-a310-03acfb88a5ed None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:05.354429 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c885bd7-6d56-4d5c-a310-03acfb88a5ed None None] PATH_INFO: `/` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:05.355977 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 531/1054] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:53:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:53:05.360942 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acf60f3-4257-4e7a-9d6c-06fefad7501e None None] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:05.361241 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acf60f3-4257-4e7a-9d6c-06fefad7501e None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:05.361531 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acf60f3-4257-4e7a-9d6c-06fefad7501e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:05.378825 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.auth.core [None req-2acf60f3-4257-4e7a-9d6c-06fefad7501e None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:53:05.380038 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-2acf60f3-4257-4e7a-9d6c-06fefad7501e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:53:05.457464 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 532/1055] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:53:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:53:05.458854 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6477fbeb-649e-4adc-a854-30e677228428 None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:05.480075 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acf60f3-4257-4e7a-9d6c-06fefad7501e None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:05.487377 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:05.501215 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2968cc55-c74b-4b97-9c52-cb474702851b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:05.502058 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2968cc55-c74b-4b97-9c52-cb474702851b None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:05.502353 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2968cc55-c74b-4b97-9c52-cb474702851b None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:05.503089 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2968cc55-c74b-4b97-9c52-cb474702851b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:05.514227 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2968cc55-c74b-4b97-9c52-cb474702851b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:53:05.522568 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-2968cc55-c74b-4b97-9c52-cb474702851b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:53:05.522568 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:53:05.522568 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:53:05.522568 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:53:05.522568 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:53:05.522568 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:53:05.522568 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:53:05.522568 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:53:05.522568 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:53:05.522568 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:53:05.522568 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:53:05.522568 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:53:05.522568 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:53:05.522568 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:53:05.522568 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:53:05.522568 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:53:05.523750 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:53:05.523750 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:53:05.523750 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:53:05.523750 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:53:05.523750 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:53:05.523750 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:53:05.523750 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:53:05.523750 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:53:05.523750 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:53:05.523750 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:53:05.523750 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:53:05.523750 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:53:05.523750 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:53:05.523750 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:53:05.523750 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:53:05.523750 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:53:05.524803 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:53:05.524803 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:53:05.524803 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:53:05.524803 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:53:05.524803 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:53:05.524803 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:53:05.524803 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:53:05.524803 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:53:05.524803 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:53:05.524803 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:53:05.524803 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:53:05.524803 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:53:05.524803 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 524/1056] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:53:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:53:05.529765 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2968cc55-c74b-4b97-9c52-cb474702851b None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:05.588642 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acf60f3-4257-4e7a-9d6c-06fefad7501e None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:05.600590 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:05.629120 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a311a45a-14b5-465f-b8bc-89af21cbc336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:05.629790 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a311a45a-14b5-465f-b8bc-89af21cbc336 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:05.630112 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a311a45a-14b5-465f-b8bc-89af21cbc336 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:05.630396 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a311a45a-14b5-465f-b8bc-89af21cbc336 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:05.642648 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2968cc55-c74b-4b97-9c52-cb474702851b None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:05.679762 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fb325c-4c29-494a-8f11-861b2d3f70bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:05.680444 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fb325c-4c29-494a-8f11-861b2d3f70bf None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:05.680736 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fb325c-4c29-494a-8f11-861b2d3f70bf None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:05.681019 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fb325c-4c29-494a-8f11-861b2d3f70bf None admin] PATH_INFO: `/v3/roles` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:05.681772 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fb325c-4c29-494a-8f11-861b2d3f70bf None admin] RBAC: Adding query filter params (name=admin) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:53:05.682121 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fb325c-4c29-494a-8f11-861b2d3f70bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:05.683495 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fb325c-4c29-494a-8f11-861b2d3f70bf None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:05.697151 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 525/1057] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:53:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:53:05.698715 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fb325c-4c29-494a-8f11-861b2d3f70bf None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:05.716797 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a311a45a-14b5-465f-b8bc-89af21cbc336 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:05.718627 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a311a45a-14b5-465f-b8bc-89af21cbc336 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:05.750040 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:05.811733 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fb325c-4c29-494a-8f11-861b2d3f70bf None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:05.834293 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 533/1058] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:53:05 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 19 14:53:05.839039 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a311a45a-14b5-465f-b8bc-89af21cbc336 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:05.850052 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06c9ab0-e6c7-4b49-a7e9-7345286e4996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:05.850954 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06c9ab0-e6c7-4b49-a7e9-7345286e4996 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:05.851615 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06c9ab0-e6c7-4b49-a7e9-7345286e4996 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:05.851904 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06c9ab0-e6c7-4b49-a7e9-7345286e4996 None admin] PATH_INFO: `/v3/users/09568ec3718e40608461688918cb922b` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:05.852752 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:05.867414 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06c9ab0-e6c7-4b49-a7e9-7345286e4996 None admin] RBAC: Authorizing `identity:get_user(user_id=09568ec3718e40608461688918cb922b)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:05.869182 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06c9ab0-e6c7-4b49-a7e9-7345286e4996 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:05.876201 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 526/1059] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:53:05 2022] GET /identity/v3/users/09568ec3718e40608461688918cb922b => generated 285 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:53:05.877974 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06c9ab0-e6c7-4b49-a7e9-7345286e4996 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:05.954642 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a311a45a-14b5-465f-b8bc-89af21cbc336 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:05.992447 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06c9ab0-e6c7-4b49-a7e9-7345286e4996 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:05.992758 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244c0e39-6951-49b1-a57e-97c9604f5944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:05.993526 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244c0e39-6951-49b1-a57e-97c9604f5944 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:05.993819 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244c0e39-6951-49b1-a57e-97c9604f5944 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:05.994233 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244c0e39-6951-49b1-a57e-97c9604f5944 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:05.995042 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:06.007882 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244c0e39-6951-49b1-a57e-97c9604f5944 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:06.009277 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244c0e39-6951-49b1-a57e-97c9604f5944 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:06.028391 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-244c0e39-6951-49b1-a57e-97c9604f5944 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:53:06.028391 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:53:06.028391 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:53:06.028391 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:53:06.028391 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:53:06.028391 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:53:06.028391 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:53:06.028391 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:53:06.028391 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:53:06.028391 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:53:06.028391 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:53:06.028391 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:53:06.028391 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:53:06.028391 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:53:06.028391 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:53:06.029460 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:53:06.029460 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:53:06.029460 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:53:06.029460 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:53:06.029460 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:53:06.029460 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:53:06.029460 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:53:06.029460 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:53:06.029460 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:53:06.029460 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:53:06.029460 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:53:06.029460 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:53:06.029460 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:53:06.029460 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:53:06.029460 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:53:06.029460 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:53:06.029460 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:53:06.031026 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:53:06.031026 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:53:06.031026 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:53:06.031026 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:53:06.031026 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 534/1060] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:53:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:53:06.033379 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db404acb-c775-4b13-b4d9-56d891e8835f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:06.034082 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db404acb-c775-4b13-b4d9-56d891e8835f None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:06.034376 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db404acb-c775-4b13-b4d9-56d891e8835f None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:06.034657 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db404acb-c775-4b13-b4d9-56d891e8835f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:06.035975 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244c0e39-6951-49b1-a57e-97c9604f5944 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:06.105862 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:06.113988 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db404acb-c775-4b13-b4d9-56d891e8835f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:06.115468 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db404acb-c775-4b13-b4d9-56d891e8835f None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:06.150445 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244c0e39-6951-49b1-a57e-97c9604f5944 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:06.186647 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abe9400-76c0-4894-bf09-c28ee8ecfd3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:06.187323 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abe9400-76c0-4894-bf09-c28ee8ecfd3b None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:06.187747 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abe9400-76c0-4894-bf09-c28ee8ecfd3b None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:06.188045 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abe9400-76c0-4894-bf09-c28ee8ecfd3b None admin] PATH_INFO: `/v3/domains` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:06.188796 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abe9400-76c0-4894-bf09-c28ee8ecfd3b None admin] RBAC: Adding query filter params (name=Default) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:53:06.189105 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abe9400-76c0-4894-bf09-c28ee8ecfd3b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:06.190508 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abe9400-76c0-4894-bf09-c28ee8ecfd3b None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:06.213868 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 535/1061] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:53:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:53:06.240510 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 527/1062] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:53:05 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 19 14:53:06.246649 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abe9400-76c0-4894-bf09-c28ee8ecfd3b None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:06.364310 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abe9400-76c0-4894-bf09-c28ee8ecfd3b None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:06.390976 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:06.404337 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da66c9e2-5617-404b-a714-265dbf435005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:06.405250 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da66c9e2-5617-404b-a714-265dbf435005 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:06.405536 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da66c9e2-5617-404b-a714-265dbf435005 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:06.405816 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da66c9e2-5617-404b-a714-265dbf435005 None admin] PATH_INFO: `/v3/projects/57b3750f90cb43a1a798f128a15fe685` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:06.407036 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da66c9e2-5617-404b-a714-265dbf435005 None admin] RBAC: Authorizing `identity:get_project(project_id=57b3750f90cb43a1a798f128a15fe685)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:06.408473 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da66c9e2-5617-404b-a714-265dbf435005 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:06.410596 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 536/1063] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:53:06 2022] GET /identity/v3/projects/57b3750f90cb43a1a798f128a15fe685 => generated 311 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:53:06.416289 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db404acb-c775-4b13-b4d9-56d891e8835f None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:06.421729 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:06.528256 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db404acb-c775-4b13-b4d9-56d891e8835f None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:06.542504 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb421854-8aca-4ed4-8a35-b9373f132fb4 None None] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:06.542792 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb421854-8aca-4ed4-8a35-b9373f132fb4 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:06.543033 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb421854-8aca-4ed4-8a35-b9373f132fb4 None None] PATH_INFO: `/` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:06.544694 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 537/1064] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:53:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:53:06.550895 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346d9282-af70-42da-93d2-b9255a4b9fc3 None None] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:06.551188 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346d9282-af70-42da-93d2-b9255a4b9fc3 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:06.551470 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346d9282-af70-42da-93d2-b9255a4b9fc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:06.566847 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b060571-ea64-490e-9d98-387437eba5be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:06.567512 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b060571-ea64-490e-9d98-387437eba5be None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:06.567803 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b060571-ea64-490e-9d98-387437eba5be None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:06.568075 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b060571-ea64-490e-9d98-387437eba5be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:06.568627 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.auth.core [None req-346d9282-af70-42da-93d2-b9255a4b9fc3 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:53:06.569158 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b060571-ea64-490e-9d98-387437eba5be None admin] RBAC: Adding query filter params (role.id=028379e01f9245e9b8fc56e8b9e8a6ad, scope.project.id=57b3750f90cb43a1a798f128a15fe685, user.id=09568ec3718e40608461688918cb922b) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:53:06.569475 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b060571-ea64-490e-9d98-387437eba5be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:06.570110 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-346d9282-af70-42da-93d2-b9255a4b9fc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:53:06.570947 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b060571-ea64-490e-9d98-387437eba5be None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:06.576526 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 528/1065] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:53:06 2022] GET /identity/v3/role_assignments?user.id=09568ec3718e40608461688918cb922b&scope.project.id=57b3750f90cb43a1a798f128a15fe685&role.id=028379e01f9245e9b8fc56e8b9e8a6ad => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:53:06.644141 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 538/1066] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:53:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:53:06.667734 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b060571-ea64-490e-9d98-387437eba5be None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:06.741326 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:06.777101 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b060571-ea64-490e-9d98-387437eba5be None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:06.811472 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a841b2fc-a315-4c1a-a925-645746adecc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:06.812296 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a841b2fc-a315-4c1a-a925-645746adecc3 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:06.812589 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a841b2fc-a315-4c1a-a925-645746adecc3 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:06.812873 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a841b2fc-a315-4c1a-a925-645746adecc3 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:06.823250 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a841b2fc-a315-4c1a-a925-645746adecc3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:53:06.831651 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-a841b2fc-a315-4c1a-a925-645746adecc3 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 14:53:06.831651 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:53:06.831651 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:53:06.831651 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:53:06.831651 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:53:06.831651 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:53:06.831651 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:53:06.831651 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:53:06.831651 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:53:06.831651 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:53:06.831651 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:53:06.831651 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:53:06.831651 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:53:06.831651 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:53:06.831651 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:53:06.831651 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:53:06.832801 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:53:06.832801 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:53:06.832801 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:53:06.832801 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:53:06.832801 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:53:06.832801 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:53:06.832801 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:53:06.832801 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:53:06.832801 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:53:06.832801 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:53:06.832801 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:53:06.832801 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:53:06.832801 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:53:06.832801 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:53:06.832801 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:53:06.832801 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:53:06.834391 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:53:06.834391 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:53:06.834391 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:53:06.834391 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:53:06.834391 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:53:06.834391 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:53:06.834391 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:53:06.834391 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:53:06.834391 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:53:06.834391 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:53:06.834391 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 14:53:06.834391 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:53:06.834391 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 529/1067] 10.222.0.57 () {56 vars in 1120 bytes} [Thu May 19 14:53:06 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:53:06.838794 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346d9282-af70-42da-93d2-b9255a4b9fc3 None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:06.942126 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346d9282-af70-42da-93d2-b9255a4b9fc3 None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:06.965000 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:06.980667 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baec67c-83bb-465f-8689-831885a2bc7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:06.981346 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baec67c-83bb-465f-8689-831885a2bc7e None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:06.981636 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baec67c-83bb-465f-8689-831885a2bc7e None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:06.981947 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baec67c-83bb-465f-8689-831885a2bc7e None admin] PATH_INFO: `/v3/roles` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:06.982707 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1baec67c-83bb-465f-8689-831885a2bc7e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:53:06.983021 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1baec67c-83bb-465f-8689-831885a2bc7e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:06.984562 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1baec67c-83bb-465f-8689-831885a2bc7e None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:06.992638 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 539/1068] 10.222.0.57 () {56 vars in 1115 bytes} [Thu May 19 14:53:06 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:53:06.998686 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a841b2fc-a315-4c1a-a925-645746adecc3 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:07.102863 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a841b2fc-a315-4c1a-a925-645746adecc3 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:07.130100 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:07.138277 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5c718a-4525-498d-9b8c-33de82b657e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:07.139078 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5c718a-4525-498d-9b8c-33de82b657e9 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:07.139419 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5c718a-4525-498d-9b8c-33de82b657e9 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:07.139806 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5c718a-4525-498d-9b8c-33de82b657e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:07.150664 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5c718a-4525-498d-9b8c-33de82b657e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:07.152121 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5c718a-4525-498d-9b8c-33de82b657e9 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:07.169214 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-5d5c718a-4525-498d-9b8c-33de82b657e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:53:07.169214 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:53:07.169214 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:53:07.169214 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:53:07.169214 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:53:07.169214 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:53:07.169214 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:53:07.169214 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:53:07.169214 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:53:07.169214 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:53:07.169214 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:53:07.169214 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:53:07.169214 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:53:07.169214 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:53:07.169214 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:53:07.170486 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:53:07.170486 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:53:07.170486 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:53:07.170486 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:53:07.170486 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:53:07.170486 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:53:07.170486 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:53:07.170486 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:53:07.170486 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:53:07.170486 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:53:07.170486 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:53:07.170486 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:53:07.170486 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:53:07.170486 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:53:07.170486 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:53:07.170486 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:53:07.170486 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:53:07.172347 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:53:07.172347 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:53:07.172347 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:53:07.172347 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:53:07.172347 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 530/1069] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:53:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:53:07.176833 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baec67c-83bb-465f-8689-831885a2bc7e None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:07.278611 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baec67c-83bb-465f-8689-831885a2bc7e None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:07.312713 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71ef7d9-0e5d-491b-8111-33c53e9002c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:07.313388 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71ef7d9-0e5d-491b-8111-33c53e9002c7 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:07.313678 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71ef7d9-0e5d-491b-8111-33c53e9002c7 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:07.314000 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71ef7d9-0e5d-491b-8111-33c53e9002c7 None admin] PATH_INFO: `/v3/domains` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:07.314726 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71ef7d9-0e5d-491b-8111-33c53e9002c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:53:07.315038 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71ef7d9-0e5d-491b-8111-33c53e9002c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:07.316410 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71ef7d9-0e5d-491b-8111-33c53e9002c7 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:07.329320 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 540/1070] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:53:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:53:07.335332 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5c718a-4525-498d-9b8c-33de82b657e9 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:07.435839 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5c718a-4525-498d-9b8c-33de82b657e9 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:07.469591 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b4aa93-99eb-4efb-ad95-19c38a6354fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:07.470401 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b4aa93-99eb-4efb-ad95-19c38a6354fe None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:07.470689 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b4aa93-99eb-4efb-ad95-19c38a6354fe None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:07.470968 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b4aa93-99eb-4efb-ad95-19c38a6354fe None admin] PATH_INFO: `/v3/users/nova` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:07.478348 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b4aa93-99eb-4efb-ad95-19c38a6354fe None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:07.479784 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b4aa93-99eb-4efb-ad95-19c38a6354fe None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:07.493993 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-47b4aa93-99eb-4efb-ad95-19c38a6354fe None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 14:53:07.493993 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:53:07.493993 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:53:07.493993 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:53:07.493993 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:53:07.493993 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:53:07.493993 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:53:07.493993 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:53:07.493993 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:53:07.493993 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:53:07.493993 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:53:07.493993 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:53:07.493993 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:53:07.493993 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:53:07.493993 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:53:07.493993 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:53:07.495263 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:53:07.495263 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:53:07.495263 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:53:07.495263 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:53:07.495263 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:53:07.495263 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:53:07.495263 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:53:07.495263 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:53:07.495263 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:53:07.495263 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:53:07.495263 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:53:07.495263 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:53:07.495263 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:53:07.495263 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:53:07.495263 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:53:07.495263 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:53:07.495263 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:53:07.495263 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:53:07.496382 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:53:07.496382 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:53:07.496382 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:53:07.496382 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:53:07.496382 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:53:07.496382 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:53:07.496382 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:53:07.496382 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:53:07.496382 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 14:53:07.496382 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:53:07.496382 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 531/1071] 10.222.0.57 () {56 vars in 1093 bytes} [Thu May 19 14:53:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:53:07.501615 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71ef7d9-0e5d-491b-8111-33c53e9002c7 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:07.544941 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:07.602296 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71ef7d9-0e5d-491b-8111-33c53e9002c7 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:07.643176 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ab4e5f-c11c-4dfa-afa1-b39cfea5b8ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:07.643855 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ab4e5f-c11c-4dfa-afa1-b39cfea5b8ce None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:07.644142 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ab4e5f-c11c-4dfa-afa1-b39cfea5b8ce None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:07.644420 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ab4e5f-c11c-4dfa-afa1-b39cfea5b8ce None admin] PATH_INFO: `/v3/users` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:07.645271 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ab4e5f-c11c-4dfa-afa1-b39cfea5b8ce None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:53:07.645589 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ab4e5f-c11c-4dfa-afa1-b39cfea5b8ce None admin] RBAC: Authorizing `identity:list_users()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:07.647071 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ab4e5f-c11c-4dfa-afa1-b39cfea5b8ce None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:07.664923 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 541/1072] 10.222.0.57 () {56 vars in 1133 bytes} [Thu May 19 14:53:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:53:07.670520 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b4aa93-99eb-4efb-ad95-19c38a6354fe None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:07.715530 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:07.775030 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b4aa93-99eb-4efb-ad95-19c38a6354fe None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:07.808803 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e14b635-00a9-46ac-87b4-caea263569e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:07.809534 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e14b635-00a9-46ac-87b4-caea263569e2 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:07.809825 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e14b635-00a9-46ac-87b4-caea263569e2 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:07.810136 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e14b635-00a9-46ac-87b4-caea263569e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:07.821075 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e14b635-00a9-46ac-87b4-caea263569e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:07.822574 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e14b635-00a9-46ac-87b4-caea263569e2 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:07.840477 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-4e14b635-00a9-46ac-87b4-caea263569e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:53:07.840477 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:53:07.840477 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:53:07.840477 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:53:07.840477 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:53:07.840477 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:53:07.840477 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:53:07.840477 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:53:07.840477 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:53:07.840477 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:53:07.840477 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:53:07.840477 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:53:07.840477 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:53:07.840477 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:53:07.840477 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:53:07.841810 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:53:07.841810 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:53:07.841810 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:53:07.841810 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:53:07.841810 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:53:07.841810 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:53:07.841810 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:53:07.841810 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:53:07.841810 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:53:07.841810 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:53:07.841810 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:53:07.841810 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:53:07.841810 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:53:07.841810 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:53:07.841810 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:53:07.841810 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:53:07.841810 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:53:07.843604 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:53:07.843604 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:53:07.843604 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:53:07.843604 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:53:07.843604 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 532/1073] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:53:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:53:07.848300 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ab4e5f-c11c-4dfa-afa1-b39cfea5b8ce None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:07.948037 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:07.951282 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ab4e5f-c11c-4dfa-afa1-b39cfea5b8ce None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:07.987408 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b61f3a3-84cd-4811-8add-854bfc1dbb51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:07.988076 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b61f3a3-84cd-4811-8add-854bfc1dbb51 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:07.988372 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b61f3a3-84cd-4811-8add-854bfc1dbb51 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:07.988653 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b61f3a3-84cd-4811-8add-854bfc1dbb51 None admin] PATH_INFO: `/v3/domains` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:07.989378 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b61f3a3-84cd-4811-8add-854bfc1dbb51 None admin] RBAC: Adding query filter params (name=Default) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:53:07.989699 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b61f3a3-84cd-4811-8add-854bfc1dbb51 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:07.991097 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b61f3a3-84cd-4811-8add-854bfc1dbb51 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:08.003006 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 542/1074] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:53:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:53:08.008616 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e14b635-00a9-46ac-87b4-caea263569e2 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:08.102792 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:08.112423 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e14b635-00a9-46ac-87b4-caea263569e2 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:08.145992 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4127a2d3-ba43-4e0e-94ad-24f1e525eff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:08.146789 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4127a2d3-ba43-4e0e-94ad-24f1e525eff5 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:08.147079 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4127a2d3-ba43-4e0e-94ad-24f1e525eff5 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:08.147362 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4127a2d3-ba43-4e0e-94ad-24f1e525eff5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:08.154025 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4127a2d3-ba43-4e0e-94ad-24f1e525eff5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:08.156154 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4127a2d3-ba43-4e0e-94ad-24f1e525eff5 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:08.170394 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-4127a2d3-ba43-4e0e-94ad-24f1e525eff5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:53:08.170394 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:53:08.170394 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:53:08.170394 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:53:08.170394 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:53:08.170394 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:53:08.170394 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:53:08.170394 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:53:08.170394 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:53:08.170394 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:53:08.170394 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:53:08.170394 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:53:08.170394 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:53:08.170394 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:53:08.170394 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:53:08.172049 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:53:08.172049 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:53:08.172049 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:53:08.172049 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:53:08.172049 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:53:08.172049 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:53:08.172049 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:53:08.172049 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:53:08.172049 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:53:08.172049 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:53:08.172049 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:53:08.172049 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:53:08.172049 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:53:08.172049 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:53:08.172049 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:53:08.172049 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:53:08.172049 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:53:08.173525 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:53:08.173525 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:53:08.173525 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:53:08.173525 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:53:08.173525 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:53:08.173525 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:53:08.173525 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:53:08.173525 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:53:08.173525 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 533/1075] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:53:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:53:08.178136 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b61f3a3-84cd-4811-8add-854bfc1dbb51 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:08.278935 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b61f3a3-84cd-4811-8add-854bfc1dbb51 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:08.313548 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b1a197-e1fc-4f7f-b3aa-4b5259130da8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:08.314243 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b1a197-e1fc-4f7f-b3aa-4b5259130da8 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:08.314533 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b1a197-e1fc-4f7f-b3aa-4b5259130da8 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:08.314814 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b1a197-e1fc-4f7f-b3aa-4b5259130da8 None admin] PATH_INFO: `/v3/projects` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:08.315606 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b1a197-e1fc-4f7f-b3aa-4b5259130da8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:53:08.315921 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b1a197-e1fc-4f7f-b3aa-4b5259130da8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:08.317351 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b1a197-e1fc-4f7f-b3aa-4b5259130da8 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:08.330364 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 543/1076] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:53:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:53:08.336482 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4127a2d3-ba43-4e0e-94ad-24f1e525eff5 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:08.440753 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4127a2d3-ba43-4e0e-94ad-24f1e525eff5 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:08.476721 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8986ed9-251d-4725-8735-fd847bcc3e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:08.477621 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8986ed9-251d-4725-8735-fd847bcc3e06 None admin] REQUEST_METHOD: `PUT` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:08.477949 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8986ed9-251d-4725-8735-fd847bcc3e06 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:08.478239 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8986ed9-251d-4725-8735-fd847bcc3e06 None admin] PATH_INFO: `/v3/projects/fc99d742ae004954b6d841293b5fa79c/users/5d2aa8c8870a49d98f23f26e036695ef/roles/d07cc726aa754554a8a566bac4b7d7fe` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:08.480487 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8986ed9-251d-4725-8735-fd847bcc3e06 None admin] RBAC: Authorizing `identity:create_grant(project_id=fc99d742ae004954b6d841293b5fa79c, user_id=5d2aa8c8870a49d98f23f26e036695ef, role_id=d07cc726aa754554a8a566bac4b7d7fe)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:08.481970 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8986ed9-251d-4725-8735-fd847bcc3e06 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:08.495085 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 534/1077] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 14:53:08 2022] PUT /identity/v3/projects/fc99d742ae004954b6d841293b5fa79c/users/5d2aa8c8870a49d98f23f26e036695ef/roles/d07cc726aa754554a8a566bac4b7d7fe => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:53:08.816783 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351b0b20-0eac-46d6-9e29-f8c4cac48f7f None None] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:08.816981 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351b0b20-0eac-46d6-9e29-f8c4cac48f7f None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:08.817276 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351b0b20-0eac-46d6-9e29-f8c4cac48f7f None None] PATH_INFO: `/` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:08.819847 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 544/1078] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:53:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:53:08.824326 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89736b98-0ff4-480b-80b4-ae21954d17b5 None None] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:08.824619 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89736b98-0ff4-480b-80b4-ae21954d17b5 None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:08.824907 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89736b98-0ff4-480b-80b4-ae21954d17b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:08.842191 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.auth.core [None req-89736b98-0ff4-480b-80b4-ae21954d17b5 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:53:08.844191 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.fernet_utils [None req-89736b98-0ff4-480b-80b4-ae21954d17b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:53:08.851770 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:08.914789 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 535/1079] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:53:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:53:08.937994 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351b0b20-0eac-46d6-9e29-f8c4cac48f7f None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:08.960619 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:09.043451 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351b0b20-0eac-46d6-9e29-f8c4cac48f7f None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:09.079635 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8dc12d-6acc-4739-8be4-6a86c829c320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:09.080360 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8dc12d-6acc-4739-8be4-6a86c829c320 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:09.080649 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8dc12d-6acc-4739-8be4-6a86c829c320 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:09.080928 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8dc12d-6acc-4739-8be4-6a86c829c320 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:09.090992 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8dc12d-6acc-4739-8be4-6a86c829c320 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:53:09.098762 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-3f8dc12d-6acc-4739-8be4-6a86c829c320 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:53:09.098762 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:53:09.098762 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:53:09.098762 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:53:09.098762 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:53:09.098762 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:53:09.098762 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:53:09.098762 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:53:09.098762 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:53:09.098762 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:53:09.098762 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:53:09.098762 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:53:09.098762 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:53:09.098762 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:53:09.098762 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:53:09.098762 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:53:09.099942 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:53:09.099942 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:53:09.099942 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:53:09.099942 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:53:09.099942 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:53:09.099942 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:53:09.099942 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:53:09.099942 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:53:09.099942 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:53:09.099942 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:53:09.099942 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:53:09.099942 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:53:09.099942 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:53:09.099942 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:53:09.099942 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:53:09.099942 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:53:09.100945 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:53:09.100945 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:53:09.100945 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:53:09.100945 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:53:09.100945 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:53:09.100945 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:53:09.100945 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:53:09.100945 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:53:09.100945 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:53:09.100945 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:53:09.100945 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:53:09.100945 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:53:09.100945 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 545/1080] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:53:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:53:09.105979 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89736b98-0ff4-480b-80b4-ae21954d17b5 None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:09.204558 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89736b98-0ff4-480b-80b4-ae21954d17b5 None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:09.238237 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9939a2-6f73-4256-8260-6254caffb4bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:09.238907 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9939a2-6f73-4256-8260-6254caffb4bb None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:09.239190 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9939a2-6f73-4256-8260-6254caffb4bb None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:09.239466 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9939a2-6f73-4256-8260-6254caffb4bb None admin] PATH_INFO: `/v3/roles` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:09.240216 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9939a2-6f73-4256-8260-6254caffb4bb None admin] RBAC: Adding query filter params (name=admin) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:53:09.240525 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9939a2-6f73-4256-8260-6254caffb4bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:09.241936 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9939a2-6f73-4256-8260-6254caffb4bb None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:09.251068 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 536/1081] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:53:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:53:09.257039 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8dc12d-6acc-4739-8be4-6a86c829c320 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:09.340119 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:09.360167 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8dc12d-6acc-4739-8be4-6a86c829c320 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:09.394594 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0521e8fd-9b4e-424d-ba54-3b08b25196ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:09.395469 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0521e8fd-9b4e-424d-ba54-3b08b25196ef None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:09.395758 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0521e8fd-9b4e-424d-ba54-3b08b25196ef None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:09.396041 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0521e8fd-9b4e-424d-ba54-3b08b25196ef None admin] PATH_INFO: `/v3/users/09568ec3718e40608461688918cb922b` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:09.397235 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0521e8fd-9b4e-424d-ba54-3b08b25196ef None admin] RBAC: Authorizing `identity:get_user(user_id=09568ec3718e40608461688918cb922b)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:09.398709 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0521e8fd-9b4e-424d-ba54-3b08b25196ef None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:09.400356 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 546/1082] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:53:09 2022] GET /identity/v3/users/09568ec3718e40608461688918cb922b => generated 285 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:53:09.406805 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9939a2-6f73-4256-8260-6254caffb4bb None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:09.410625 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:09.507725 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9939a2-6f73-4256-8260-6254caffb4bb None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:09.541685 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43b7546-a2d3-4146-bbe9-2222ccf705ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:09.542570 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43b7546-a2d3-4146-bbe9-2222ccf705ce None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:09.542854 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43b7546-a2d3-4146-bbe9-2222ccf705ce None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:09.543133 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43b7546-a2d3-4146-bbe9-2222ccf705ce None admin] PATH_INFO: `/v3/projects/57b3750f90cb43a1a798f128a15fe685` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:09.544308 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43b7546-a2d3-4146-bbe9-2222ccf705ce None admin] RBAC: Authorizing `identity:get_project(project_id=57b3750f90cb43a1a798f128a15fe685)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:09.545743 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43b7546-a2d3-4146-bbe9-2222ccf705ce None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:09.547548 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 537/1083] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:53:09 2022] GET /identity/v3/projects/57b3750f90cb43a1a798f128a15fe685 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:53:09.553735 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0521e8fd-9b4e-424d-ba54-3b08b25196ef None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:09.659807 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0521e8fd-9b4e-424d-ba54-3b08b25196ef None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:09.694305 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d903ecff-3031-4123-ac8d-3a165c8f894b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:09.695196 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d903ecff-3031-4123-ac8d-3a165c8f894b None admin] REQUEST_METHOD: `PUT` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:09.695480 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d903ecff-3031-4123-ac8d-3a165c8f894b None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:09.695761 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d903ecff-3031-4123-ac8d-3a165c8f894b None admin] PATH_INFO: `/v3/projects/57b3750f90cb43a1a798f128a15fe685/users/09568ec3718e40608461688918cb922b/roles/028379e01f9245e9b8fc56e8b9e8a6ad` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:09.697727 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d903ecff-3031-4123-ac8d-3a165c8f894b None admin] RBAC: Authorizing `identity:create_grant(project_id=57b3750f90cb43a1a798f128a15fe685, user_id=09568ec3718e40608461688918cb922b, role_id=028379e01f9245e9b8fc56e8b9e8a6ad)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:09.699182 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d903ecff-3031-4123-ac8d-3a165c8f894b None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:09.713122 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 547/1084] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 14:53:09 2022] PUT /identity/v3/projects/57b3750f90cb43a1a798f128a15fe685/users/09568ec3718e40608461688918cb922b/roles/028379e01f9245e9b8fc56e8b9e8a6ad => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:53:10.756152 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579eb5bb-d349-47b3-99dc-5f148a44603f None None] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:10.756709 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579eb5bb-d349-47b3-99dc-5f148a44603f None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:10.756709 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579eb5bb-d349-47b3-99dc-5f148a44603f None None] PATH_INFO: `/` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:10.758146 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 538/1085] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:53:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:53:10.763638 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba14ca78-7499-4b1a-8dd8-b3b20315939c None None] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:10.764078 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba14ca78-7499-4b1a-8dd8-b3b20315939c None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:10.764806 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba14ca78-7499-4b1a-8dd8-b3b20315939c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:10.783487 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.auth.core [None req-ba14ca78-7499-4b1a-8dd8-b3b20315939c None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:53:10.784779 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-ba14ca78-7499-4b1a-8dd8-b3b20315939c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:53:10.786858 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:10.856272 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 548/1086] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:53:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:53:10.880602 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579eb5bb-d349-47b3-99dc-5f148a44603f None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:10.952942 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:10.985038 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579eb5bb-d349-47b3-99dc-5f148a44603f None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:11.019402 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e655b7-828e-4fd0-8ff0-f774997db2be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:11.020220 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e655b7-828e-4fd0-8ff0-f774997db2be None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:11.020513 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e655b7-828e-4fd0-8ff0-f774997db2be None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:11.020798 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e655b7-828e-4fd0-8ff0-f774997db2be None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:11.032372 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e655b7-828e-4fd0-8ff0-f774997db2be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:53:11.040613 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-31e655b7-828e-4fd0-8ff0-f774997db2be None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 14:53:11.040613 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:53:11.040613 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:53:11.040613 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:53:11.040613 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:53:11.040613 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:53:11.040613 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:53:11.040613 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:53:11.040613 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:53:11.040613 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:53:11.040613 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:53:11.040613 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:53:11.040613 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:53:11.040613 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:53:11.040613 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:53:11.040613 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:53:11.041890 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:53:11.041890 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:53:11.041890 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:53:11.041890 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:53:11.041890 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:53:11.041890 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:53:11.041890 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:53:11.041890 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:53:11.041890 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:53:11.041890 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:53:11.041890 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:53:11.041890 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:53:11.041890 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:53:11.041890 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:53:11.041890 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:53:11.041890 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:53:11.043075 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:53:11.043075 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:53:11.043075 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:53:11.043075 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:53:11.043075 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:53:11.043075 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:53:11.043075 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:53:11.043075 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:53:11.043075 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:53:11.043075 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:53:11.043075 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 14:53:11.043075 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:53:11.043075 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 539/1087] 10.222.0.57 () {56 vars in 1120 bytes} [Thu May 19 14:53:10 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:53:11.048554 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba14ca78-7499-4b1a-8dd8-b3b20315939c None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:11.148520 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba14ca78-7499-4b1a-8dd8-b3b20315939c None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:11.182904 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f79fed-2c24-478c-ac64-986da4faf009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:11.183577 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f79fed-2c24-478c-ac64-986da4faf009 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:11.183870 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f79fed-2c24-478c-ac64-986da4faf009 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:11.184153 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f79fed-2c24-478c-ac64-986da4faf009 None admin] PATH_INFO: `/v3/roles` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:11.184911 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f79fed-2c24-478c-ac64-986da4faf009 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:53:11.185229 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f79fed-2c24-478c-ac64-986da4faf009 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:11.186694 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f79fed-2c24-478c-ac64-986da4faf009 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:11.191696 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:11.196710 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 549/1088] 10.222.0.57 () {56 vars in 1115 bytes} [Thu May 19 14:53:11 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:53:11.201056 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e655b7-828e-4fd0-8ff0-f774997db2be None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:11.302251 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e655b7-828e-4fd0-8ff0-f774997db2be None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:11.332233 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:11.337894 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddad355c-da3d-4987-9300-cd2e6bdef39e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:11.338700 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddad355c-da3d-4987-9300-cd2e6bdef39e None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:11.338990 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddad355c-da3d-4987-9300-cd2e6bdef39e None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:11.339271 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddad355c-da3d-4987-9300-cd2e6bdef39e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:11.349967 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddad355c-da3d-4987-9300-cd2e6bdef39e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:11.351414 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddad355c-da3d-4987-9300-cd2e6bdef39e None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:11.369165 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-ddad355c-da3d-4987-9300-cd2e6bdef39e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:53:11.369165 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:53:11.369165 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:53:11.369165 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:53:11.369165 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:53:11.369165 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:53:11.369165 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:53:11.369165 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:53:11.369165 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:53:11.369165 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:53:11.369165 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:53:11.369165 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:53:11.369165 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:53:11.369165 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:53:11.369165 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:53:11.370457 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:53:11.370457 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:53:11.370457 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:53:11.370457 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:53:11.370457 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:53:11.370457 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:53:11.370457 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:53:11.370457 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:53:11.370457 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:53:11.370457 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:53:11.370457 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:53:11.370457 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:53:11.370457 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:53:11.370457 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:53:11.370457 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:53:11.370457 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:53:11.370457 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:53:11.371528 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:53:11.371528 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:53:11.371528 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:53:11.371528 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:53:11.371528 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 540/1089] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:53:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:53:11.376477 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f79fed-2c24-478c-ac64-986da4faf009 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:11.479862 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f79fed-2c24-478c-ac64-986da4faf009 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:11.513505 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779d38f5-1f56-42eb-b9b6-04f814b3e0a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:11.514768 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779d38f5-1f56-42eb-b9b6-04f814b3e0a5 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:11.515192 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779d38f5-1f56-42eb-b9b6-04f814b3e0a5 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:11.515481 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779d38f5-1f56-42eb-b9b6-04f814b3e0a5 None admin] PATH_INFO: `/v3/domains` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:11.516216 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779d38f5-1f56-42eb-b9b6-04f814b3e0a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:53:11.516534 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779d38f5-1f56-42eb-b9b6-04f814b3e0a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:11.517928 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779d38f5-1f56-42eb-b9b6-04f814b3e0a5 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:11.530830 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 550/1090] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:53:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:53:11.536622 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddad355c-da3d-4987-9300-cd2e6bdef39e None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:11.638948 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddad355c-da3d-4987-9300-cd2e6bdef39e None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:11.677235 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9afbef-cac0-4e1d-9b48-e23c2a15d53d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:11.678073 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9afbef-cac0-4e1d-9b48-e23c2a15d53d None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:11.678367 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9afbef-cac0-4e1d-9b48-e23c2a15d53d None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:11.678646 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9afbef-cac0-4e1d-9b48-e23c2a15d53d None admin] PATH_INFO: `/v3/users/nova` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:11.687105 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9afbef-cac0-4e1d-9b48-e23c2a15d53d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:11.688566 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9afbef-cac0-4e1d-9b48-e23c2a15d53d None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:11.702911 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-1b9afbef-cac0-4e1d-9b48-e23c2a15d53d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 14:53:11.702911 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:53:11.702911 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:53:11.702911 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:53:11.702911 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:53:11.702911 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:53:11.702911 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:53:11.702911 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:53:11.702911 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:53:11.702911 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:53:11.702911 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:53:11.702911 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:53:11.702911 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:53:11.702911 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:53:11.702911 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:53:11.702911 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:53:11.704043 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:53:11.704043 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:53:11.704043 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:53:11.704043 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:53:11.704043 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:53:11.704043 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:53:11.704043 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:53:11.704043 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:53:11.704043 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:53:11.704043 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:53:11.704043 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:53:11.704043 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:53:11.704043 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:53:11.704043 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:53:11.704043 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:53:11.704043 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:53:11.704043 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:53:11.704043 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:53:11.705724 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:53:11.705724 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:53:11.705724 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:53:11.705724 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:53:11.705724 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:53:11.705724 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:53:11.705724 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:53:11.705724 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:53:11.705724 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 14:53:11.705724 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:53:11.705724 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 541/1091] 10.222.0.57 () {56 vars in 1093 bytes} [Thu May 19 14:53:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:53:11.709951 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779d38f5-1f56-42eb-b9b6-04f814b3e0a5 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:11.770529 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:11.816668 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779d38f5-1f56-42eb-b9b6-04f814b3e0a5 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:11.851616 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8db025d-c100-46dd-b9db-779fe9c28a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:11.852292 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8db025d-c100-46dd-b9db-779fe9c28a0b None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:11.852585 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8db025d-c100-46dd-b9db-779fe9c28a0b None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:11.852872 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8db025d-c100-46dd-b9db-779fe9c28a0b None admin] PATH_INFO: `/v3/users` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:11.853718 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8db025d-c100-46dd-b9db-779fe9c28a0b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:53:11.854068 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8db025d-c100-46dd-b9db-779fe9c28a0b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:11.855515 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8db025d-c100-46dd-b9db-779fe9c28a0b None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:11.878098 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 551/1092] 10.222.0.57 () {56 vars in 1133 bytes} [Thu May 19 14:53:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:53:11.879958 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9afbef-cac0-4e1d-9b48-e23c2a15d53d None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:11.926934 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:11.976101 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b29ed2c-4ceb-489f-afd9-bd10acbc6764 None None] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:11.976394 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b29ed2c-4ceb-489f-afd9-bd10acbc6764 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:11.976673 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b29ed2c-4ceb-489f-afd9-bd10acbc6764 None None] PATH_INFO: `/` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:11.978101 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 552/1093] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:53:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:53:11.983239 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e28f5d-4486-4733-a8db-ed63b166cbb9 None None] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:11.983557 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e28f5d-4486-4733-a8db-ed63b166cbb9 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:11.983847 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e28f5d-4486-4733-a8db-ed63b166cbb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:11.983972 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9afbef-cac0-4e1d-9b48-e23c2a15d53d None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:12.002052 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.auth.core [None req-91e28f5d-4486-4733-a8db-ed63b166cbb9 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:53:12.003264 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-91e28f5d-4486-4733-a8db-ed63b166cbb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:53:12.020424 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214acf2f-171c-4c1a-b814-07db1d7d795f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:12.021143 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214acf2f-171c-4c1a-b814-07db1d7d795f None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:12.021543 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214acf2f-171c-4c1a-b814-07db1d7d795f None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:12.021988 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214acf2f-171c-4c1a-b814-07db1d7d795f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:12.074078 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 553/1094] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:53:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:53:12.090491 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214acf2f-171c-4c1a-b814-07db1d7d795f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:12.091976 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214acf2f-171c-4c1a-b814-07db1d7d795f None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:12.098325 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e28f5d-4486-4733-a8db-ed63b166cbb9 None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:12.137779 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:12.148302 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:12.195577 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 542/1095] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:53:11 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:53:12.199325 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e28f5d-4486-4733-a8db-ed63b166cbb9 None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:12.201384 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214acf2f-171c-4c1a-b814-07db1d7d795f None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:12.234769 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d572121-8ad0-4b72-8aa8-22c55a75cb06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:12.235658 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d572121-8ad0-4b72-8aa8-22c55a75cb06 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:12.235883 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d572121-8ad0-4b72-8aa8-22c55a75cb06 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:12.236206 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d572121-8ad0-4b72-8aa8-22c55a75cb06 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:12.246639 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d572121-8ad0-4b72-8aa8-22c55a75cb06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:53:12.255052 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-6d572121-8ad0-4b72-8aa8-22c55a75cb06 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:53:12.255052 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:53:12.255052 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:53:12.255052 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:53:12.255052 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:53:12.255052 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:53:12.255052 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:53:12.255052 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:53:12.255052 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:53:12.255052 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:53:12.255052 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:53:12.255052 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:53:12.255052 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:53:12.255052 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:53:12.255052 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:53:12.255052 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:53:12.261475 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:53:12.261475 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:53:12.261475 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:53:12.261475 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:53:12.261475 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:53:12.261475 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:53:12.261475 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:53:12.261475 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:53:12.261475 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:53:12.261475 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:53:12.261475 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:53:12.261475 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:53:12.261475 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:53:12.261475 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:53:12.261475 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:53:12.261475 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:53:12.262523 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:53:12.262523 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:53:12.262523 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:53:12.262523 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:53:12.262523 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:53:12.262523 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:53:12.262523 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:53:12.262523 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:53:12.262523 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:53:12.262523 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:53:12.262523 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:53:12.262523 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:53:12.262523 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 554/1096] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:53:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:53:12.262523 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d572121-8ad0-4b72-8aa8-22c55a75cb06 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:12.302601 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214acf2f-171c-4c1a-b814-07db1d7d795f None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:12.337003 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5ca9e7-41b5-4d53-b72b-c5ea6f64344a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:12.337801 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5ca9e7-41b5-4d53-b72b-c5ea6f64344a None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:12.338126 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5ca9e7-41b5-4d53-b72b-c5ea6f64344a None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:12.338412 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5ca9e7-41b5-4d53-b72b-c5ea6f64344a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:12.348527 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5ca9e7-41b5-4d53-b72b-c5ea6f64344a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:12.350026 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5ca9e7-41b5-4d53-b72b-c5ea6f64344a None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:12.352508 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:12.366151 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d572121-8ad0-4b72-8aa8-22c55a75cb06 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:12.369066 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-fa5ca9e7-41b5-4d53-b72b-c5ea6f64344a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:53:12.369066 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:53:12.369066 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:53:12.369066 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:53:12.369066 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:53:12.369066 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:53:12.369066 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:53:12.369066 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:53:12.369066 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:53:12.369066 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:53:12.369066 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:53:12.369066 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:53:12.369066 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:53:12.369066 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:53:12.369066 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:53:12.370325 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:53:12.370325 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:53:12.370325 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:53:12.370325 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:53:12.370325 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:53:12.370325 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:53:12.370325 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:53:12.370325 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:53:12.370325 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:53:12.370325 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:53:12.370325 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:53:12.370325 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:53:12.370325 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:53:12.370325 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:53:12.370325 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:53:12.370325 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:53:12.370325 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:53:12.371452 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:53:12.371452 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:53:12.371452 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:53:12.371452 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:53:12.371452 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 543/1097] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:53:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:53:12.376530 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5ca9e7-41b5-4d53-b72b-c5ea6f64344a None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:12.377857 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:12.403350 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55af2d8a-a3df-44d1-817d-9e39c65d00f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:12.404016 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55af2d8a-a3df-44d1-817d-9e39c65d00f1 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:12.404305 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55af2d8a-a3df-44d1-817d-9e39c65d00f1 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:12.404583 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55af2d8a-a3df-44d1-817d-9e39c65d00f1 None admin] PATH_INFO: `/v3/roles` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:12.405724 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55af2d8a-a3df-44d1-817d-9e39c65d00f1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:53:12.406066 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55af2d8a-a3df-44d1-817d-9e39c65d00f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:12.407613 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55af2d8a-a3df-44d1-817d-9e39c65d00f1 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:12.415692 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 555/1098] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:53:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:53:12.422173 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55af2d8a-a3df-44d1-817d-9e39c65d00f1 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:12.475973 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5ca9e7-41b5-4d53-b72b-c5ea6f64344a None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:12.511117 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5631b1-2595-4a05-99d0-9360b3ad3a0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:12.511792 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5631b1-2595-4a05-99d0-9360b3ad3a0a None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:12.512082 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5631b1-2595-4a05-99d0-9360b3ad3a0a None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:12.512367 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5631b1-2595-4a05-99d0-9360b3ad3a0a None admin] PATH_INFO: `/v3/domains` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:12.513096 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5631b1-2595-4a05-99d0-9360b3ad3a0a None admin] RBAC: Adding query filter params (name=Default) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:53:12.513412 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5631b1-2595-4a05-99d0-9360b3ad3a0a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:12.515530 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5631b1-2595-4a05-99d0-9360b3ad3a0a None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:12.524293 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55af2d8a-a3df-44d1-817d-9e39c65d00f1 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:12.526914 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 544/1099] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:53:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:53:12.533414 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5631b1-2595-4a05-99d0-9360b3ad3a0a None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:12.559872 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d0a38e-14df-439f-bf4e-2d6bcb964176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:12.560750 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d0a38e-14df-439f-bf4e-2d6bcb964176 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:12.561064 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d0a38e-14df-439f-bf4e-2d6bcb964176 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:12.561473 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d0a38e-14df-439f-bf4e-2d6bcb964176 None admin] PATH_INFO: `/v3/users/09568ec3718e40608461688918cb922b` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:12.563115 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d0a38e-14df-439f-bf4e-2d6bcb964176 None admin] RBAC: Authorizing `identity:get_user(user_id=09568ec3718e40608461688918cb922b)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:12.564557 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d0a38e-14df-439f-bf4e-2d6bcb964176 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:12.566245 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 556/1100] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:53:12 2022] GET /identity/v3/users/09568ec3718e40608461688918cb922b => generated 285 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:53:12.572078 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d0a38e-14df-439f-bf4e-2d6bcb964176 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:12.592263 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:12.612781 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:12.638945 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5631b1-2595-4a05-99d0-9360b3ad3a0a None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:12.675862 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4dca54-a3a9-47e6-9e16-cdda8c3f4424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:12.676671 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4dca54-a3a9-47e6-9e16-cdda8c3f4424 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:12.676960 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4dca54-a3a9-47e6-9e16-cdda8c3f4424 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:12.677241 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d0a38e-14df-439f-bf4e-2d6bcb964176 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:12.677369 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4dca54-a3a9-47e6-9e16-cdda8c3f4424 None admin] PATH_INFO: `/v3/projects/service` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:12.684093 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4dca54-a3a9-47e6-9e16-cdda8c3f4424 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:12.685695 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4dca54-a3a9-47e6-9e16-cdda8c3f4424 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:12.700573 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-7c4dca54-a3a9-47e6-9e16-cdda8c3f4424 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:53:12.700573 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:53:12.700573 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:53:12.700573 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:53:12.700573 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:53:12.700573 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:53:12.700573 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:53:12.700573 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:53:12.700573 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:53:12.700573 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:53:12.700573 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:53:12.700573 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:53:12.700573 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:53:12.700573 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:53:12.700573 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:53:12.701664 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:53:12.701664 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:53:12.701664 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:53:12.701664 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:53:12.701664 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:53:12.701664 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:53:12.701664 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:53:12.701664 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:53:12.701664 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:53:12.701664 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:53:12.701664 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:53:12.701664 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:53:12.701664 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:53:12.701664 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:53:12.701664 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:53:12.701664 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:53:12.701664 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:53:12.703727 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:53:12.703727 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:53:12.703727 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:53:12.703727 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:53:12.703727 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:53:12.703727 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:53:12.703727 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:53:12.703727 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:53:12.705991 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 545/1101] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:53:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:53:12.708002 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4dca54-a3a9-47e6-9e16-cdda8c3f4424 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:12.714389 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8420cfcb-6606-4f67-9dc0-37485634958e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:12.715040 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8420cfcb-6606-4f67-9dc0-37485634958e None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:12.715326 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8420cfcb-6606-4f67-9dc0-37485634958e None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:12.715605 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8420cfcb-6606-4f67-9dc0-37485634958e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:12.785202 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8420cfcb-6606-4f67-9dc0-37485634958e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:12.786609 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8420cfcb-6606-4f67-9dc0-37485634958e None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:12.807226 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4dca54-a3a9-47e6-9e16-cdda8c3f4424 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:12.824716 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:12.827310 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:12.844151 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4d87fc-d393-48ba-a48f-242f7e648231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:12.844822 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4d87fc-d393-48ba-a48f-242f7e648231 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:12.845110 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4d87fc-d393-48ba-a48f-242f7e648231 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:12.845390 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4d87fc-d393-48ba-a48f-242f7e648231 None admin] PATH_INFO: `/v3/projects` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:12.846212 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4d87fc-d393-48ba-a48f-242f7e648231 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:53:12.846531 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4d87fc-d393-48ba-a48f-242f7e648231 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:12.847967 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4d87fc-d393-48ba-a48f-242f7e648231 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:12.860092 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 546/1102] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:53:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:53:12.866059 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4d87fc-d393-48ba-a48f-242f7e648231 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:12.890491 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 557/1103] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:53:12 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 19 14:53:12.896302 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8420cfcb-6606-4f67-9dc0-37485634958e None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:12.966677 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4d87fc-d393-48ba-a48f-242f7e648231 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:12.995658 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8420cfcb-6606-4f67-9dc0-37485634958e None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:13.001457 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0993852a-df9a-4479-b1c1-e9df5855f7d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:13.002158 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0993852a-df9a-4479-b1c1-e9df5855f7d8 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:13.002453 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0993852a-df9a-4479-b1c1-e9df5855f7d8 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:13.002739 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0993852a-df9a-4479-b1c1-e9df5855f7d8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:13.003577 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0993852a-df9a-4479-b1c1-e9df5855f7d8 None admin] RBAC: Adding query filter params (role.id=d07cc726aa754554a8a566bac4b7d7fe, scope.project.id=fc99d742ae004954b6d841293b5fa79c, user.id=5d2aa8c8870a49d98f23f26e036695ef) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:53:13.003903 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0993852a-df9a-4479-b1c1-e9df5855f7d8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:13.005358 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0993852a-df9a-4479-b1c1-e9df5855f7d8 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:13.011332 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 547/1104] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:53:12 2022] GET /identity/v3/role_assignments?user.id=5d2aa8c8870a49d98f23f26e036695ef&scope.project.id=fc99d742ae004954b6d841293b5fa79c&role.id=d07cc726aa754554a8a566bac4b7d7fe => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:53:13.034573 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea84bc3-7fd8-412b-b803-ad35e73d7bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:13.035521 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea84bc3-7fd8-412b-b803-ad35e73d7bc5 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:13.035759 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea84bc3-7fd8-412b-b803-ad35e73d7bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:13.036043 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea84bc3-7fd8-412b-b803-ad35e73d7bc5 None admin] PATH_INFO: `/v3/projects/57b3750f90cb43a1a798f128a15fe685` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:13.037318 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea84bc3-7fd8-412b-b803-ad35e73d7bc5 None admin] RBAC: Authorizing `identity:get_project(project_id=57b3750f90cb43a1a798f128a15fe685)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:13.038826 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea84bc3-7fd8-412b-b803-ad35e73d7bc5 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:13.040641 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 558/1105] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:53:12 2022] GET /identity/v3/projects/57b3750f90cb43a1a798f128a15fe685 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:53:13.047891 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0993852a-df9a-4479-b1c1-e9df5855f7d8 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:13.095901 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:13.158259 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0993852a-df9a-4479-b1c1-e9df5855f7d8 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:13.194834 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc1504b-6e0c-47e8-87d7-49568f0410c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:13.195498 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc1504b-6e0c-47e8-87d7-49568f0410c4 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:13.195785 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc1504b-6e0c-47e8-87d7-49568f0410c4 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:13.196064 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc1504b-6e0c-47e8-87d7-49568f0410c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:13.196908 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc1504b-6e0c-47e8-87d7-49568f0410c4 None admin] RBAC: Adding query filter params (role.id=028379e01f9245e9b8fc56e8b9e8a6ad, scope.project.id=57b3750f90cb43a1a798f128a15fe685, user.id=09568ec3718e40608461688918cb922b) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:53:13.197224 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc1504b-6e0c-47e8-87d7-49568f0410c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:13.198696 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc1504b-6e0c-47e8-87d7-49568f0410c4 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:13.209865 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 548/1106] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:53:13 2022] GET /identity/v3/role_assignments?user.id=09568ec3718e40608461688918cb922b&scope.project.id=57b3750f90cb43a1a798f128a15fe685&role.id=028379e01f9245e9b8fc56e8b9e8a6ad => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:53:15.466573 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878bfb32-6444-42a3-a8c2-d9d327ce7329 None None] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:15.467356 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878bfb32-6444-42a3-a8c2-d9d327ce7329 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:15.467356 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878bfb32-6444-42a3-a8c2-d9d327ce7329 None None] PATH_INFO: `/` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:15.468826 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 559/1107] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:53:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:53:15.474036 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb62a4a-a6cd-47a6-8d5d-68c9031c117b None None] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:15.474534 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb62a4a-a6cd-47a6-8d5d-68c9031c117b None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:15.474649 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb62a4a-a6cd-47a6-8d5d-68c9031c117b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:15.491928 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.auth.core [None req-1bb62a4a-a6cd-47a6-8d5d-68c9031c117b None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:53:15.493107 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.fernet_utils [None req-1bb62a4a-a6cd-47a6-8d5d-68c9031c117b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:53:15.558842 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 549/1108] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:53:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:53:15.584598 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878bfb32-6444-42a3-a8c2-d9d327ce7329 None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:15.594694 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:15.687889 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878bfb32-6444-42a3-a8c2-d9d327ce7329 None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:15.722428 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33eee0ae-c5e0-4f12-b8cd-d8d1a59b59bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:15.723164 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eee0ae-c5e0-4f12-b8cd-d8d1a59b59bc None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:15.723460 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eee0ae-c5e0-4f12-b8cd-d8d1a59b59bc None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:15.723745 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eee0ae-c5e0-4f12-b8cd-d8d1a59b59bc None admin] PATH_INFO: `/v3/domains/default` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:15.724935 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33eee0ae-c5e0-4f12-b8cd-d8d1a59b59bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:15.726409 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33eee0ae-c5e0-4f12-b8cd-d8d1a59b59bc None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:15.728041 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 560/1109] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:53:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:53:15.734642 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb62a4a-a6cd-47a6-8d5d-68c9031c117b None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:15.767521 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:15.833117 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb62a4a-a6cd-47a6-8d5d-68c9031c117b None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:15.866973 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f78668-9a43-4dc8-86c0-94cacf963c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:15.867637 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f78668-9a43-4dc8-86c0-94cacf963c6e None admin] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:15.867928 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f78668-9a43-4dc8-86c0-94cacf963c6e None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:15.868213 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f78668-9a43-4dc8-86c0-94cacf963c6e None admin] PATH_INFO: `/v3/users` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:15.868922 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f78668-9a43-4dc8-86c0-94cacf963c6e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:15.870405 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f78668-9a43-4dc8-86c0-94cacf963c6e None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:15.889445 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 550/1110] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 19 14:53:15 2022] POST /identity/v3/users => generated 286 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 14:53:18.165383 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bd7cf2-4643-48dd-a5f7-54f84716ccbb None None] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:18.165968 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bd7cf2-4643-48dd-a5f7-54f84716ccbb None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:18.167818 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bd7cf2-4643-48dd-a5f7-54f84716ccbb None None] PATH_INFO: `/` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:18.168473 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 561/1111] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:53:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:53:18.174536 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa711c0-d5a7-4b30-b45d-be960b082f9d None None] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:18.175004 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa711c0-d5a7-4b30-b45d-be960b082f9d None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:18.175107 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa711c0-d5a7-4b30-b45d-be960b082f9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:18.191758 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.auth.core [None req-2aa711c0-d5a7-4b30-b45d-be960b082f9d None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:53:18.192967 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.fernet_utils [None req-2aa711c0-d5a7-4b30-b45d-be960b082f9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:53:18.259216 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 551/1112] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:53:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:53:18.283916 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bd7cf2-4643-48dd-a5f7-54f84716ccbb None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:18.359574 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:18.385540 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bd7cf2-4643-48dd-a5f7-54f84716ccbb None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:18.420020 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b132f4c6-c49a-4a31-b903-9eac67c586fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:18.421002 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b132f4c6-c49a-4a31-b903-9eac67c586fa None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:18.421295 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b132f4c6-c49a-4a31-b903-9eac67c586fa None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:18.421577 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b132f4c6-c49a-4a31-b903-9eac67c586fa None admin] PATH_INFO: `/v3/roles/3b9c36a010504ba7bc63a8ccf0a9bca0` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:18.423099 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b132f4c6-c49a-4a31-b903-9eac67c586fa None admin] RBAC: Authorizing `identity:get_role(role_id=3b9c36a010504ba7bc63a8ccf0a9bca0)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:18.424483 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b132f4c6-c49a-4a31-b903-9eac67c586fa None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:18.425863 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 562/1113] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:53:18 2022] GET /identity/v3/roles/3b9c36a010504ba7bc63a8ccf0a9bca0 => generated 223 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:53:18.431679 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa711c0-d5a7-4b30-b45d-be960b082f9d None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:18.453700 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:18.532068 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa711c0-d5a7-4b30-b45d-be960b082f9d None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:18.565108 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f92d83-3eaf-4c59-8d1b-09c7f50c855d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:18.566008 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f92d83-3eaf-4c59-8d1b-09c7f50c855d None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:18.566302 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f92d83-3eaf-4c59-8d1b-09c7f50c855d None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:18.566585 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f92d83-3eaf-4c59-8d1b-09c7f50c855d None admin] PATH_INFO: `/v3/users/ee33154b249c451cb2c0bda957431c3e` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:18.580398 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f92d83-3eaf-4c59-8d1b-09c7f50c855d None admin] RBAC: Authorizing `identity:get_user(user_id=ee33154b249c451cb2c0bda957431c3e)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:18.581871 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f92d83-3eaf-4c59-8d1b-09c7f50c855d None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:18.583536 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 552/1114] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:53:18 2022] GET /identity/v3/users/ee33154b249c451cb2c0bda957431c3e => generated 286 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:53:18.589299 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b132f4c6-c49a-4a31-b903-9eac67c586fa None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:18.690663 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b132f4c6-c49a-4a31-b903-9eac67c586fa None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:18.724524 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496083c0-0e05-4309-83cf-1324eee383e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:18.725172 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496083c0-0e05-4309-83cf-1324eee383e0 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:18.725463 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496083c0-0e05-4309-83cf-1324eee383e0 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:18.725750 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496083c0-0e05-4309-83cf-1324eee383e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:18.737926 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:18.792819 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496083c0-0e05-4309-83cf-1324eee383e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:18.794301 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496083c0-0e05-4309-83cf-1324eee383e0 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:18.892456 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 563/1115] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:53:18 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:53:18.898385 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f92d83-3eaf-4c59-8d1b-09c7f50c855d None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:18.976734 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:18.997765 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f92d83-3eaf-4c59-8d1b-09c7f50c855d None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:19.032408 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc0dde8-4ff8-46fc-89b2-df7a31510571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:19.033250 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc0dde8-4ff8-46fc-89b2-df7a31510571 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:19.033536 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc0dde8-4ff8-46fc-89b2-df7a31510571 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:19.033811 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc0dde8-4ff8-46fc-89b2-df7a31510571 None admin] PATH_INFO: `/v3/projects/57b3750f90cb43a1a798f128a15fe685` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:19.035015 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc0dde8-4ff8-46fc-89b2-df7a31510571 None admin] RBAC: Authorizing `identity:get_project(project_id=57b3750f90cb43a1a798f128a15fe685)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:19.036434 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc0dde8-4ff8-46fc-89b2-df7a31510571 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:19.038136 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 553/1116] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:53:18 2022] GET /identity/v3/projects/57b3750f90cb43a1a798f128a15fe685 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:53:19.044300 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496083c0-0e05-4309-83cf-1324eee383e0 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:19.087981 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:19.145088 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496083c0-0e05-4309-83cf-1324eee383e0 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:19.179423 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34b68f6-bf39-4c27-92d8-62fa0ba732a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:19.180080 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34b68f6-bf39-4c27-92d8-62fa0ba732a0 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:19.180374 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34b68f6-bf39-4c27-92d8-62fa0ba732a0 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:19.180659 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34b68f6-bf39-4c27-92d8-62fa0ba732a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:19.181494 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34b68f6-bf39-4c27-92d8-62fa0ba732a0 None admin] RBAC: Adding query filter params (role.id=3b9c36a010504ba7bc63a8ccf0a9bca0, scope.project.id=57b3750f90cb43a1a798f128a15fe685, user.id=ee33154b249c451cb2c0bda957431c3e) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:53:19.181814 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34b68f6-bf39-4c27-92d8-62fa0ba732a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:19.183291 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34b68f6-bf39-4c27-92d8-62fa0ba732a0 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:19.188841 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 564/1117] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:53:19 2022] GET /identity/v3/role_assignments?user.id=ee33154b249c451cb2c0bda957431c3e&scope.project.id=57b3750f90cb43a1a798f128a15fe685&role.id=3b9c36a010504ba7bc63a8ccf0a9bca0 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:53:21.403937 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e99faf-63c8-42d5-8057-45434614db54 None None] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:21.404579 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e99faf-63c8-42d5-8057-45434614db54 None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:21.404579 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e99faf-63c8-42d5-8057-45434614db54 None None] PATH_INFO: `/` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:21.405980 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 554/1118] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:53:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:53:21.411205 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ca80ee-7389-4dbd-adc0-c9d10f53ef3f None None] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:21.411643 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ca80ee-7389-4dbd-adc0-c9d10f53ef3f None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:21.411781 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ca80ee-7389-4dbd-adc0-c9d10f53ef3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:21.428923 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.auth.core [None req-30ca80ee-7389-4dbd-adc0-c9d10f53ef3f None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:53:21.430176 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-30ca80ee-7389-4dbd-adc0-c9d10f53ef3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:53:21.496862 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 565/1119] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:53:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:53:21.520170 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e99faf-63c8-42d5-8057-45434614db54 None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:21.620663 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e99faf-63c8-42d5-8057-45434614db54 None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:21.655004 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fa0f18-824d-47a3-bf0b-fb2323b91132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:21.655796 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fa0f18-824d-47a3-bf0b-fb2323b91132 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:21.656081 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fa0f18-824d-47a3-bf0b-fb2323b91132 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:21.656359 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fa0f18-824d-47a3-bf0b-fb2323b91132 None admin] PATH_INFO: `/v3/roles/3b9c36a010504ba7bc63a8ccf0a9bca0` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:21.658244 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fa0f18-824d-47a3-bf0b-fb2323b91132 None admin] RBAC: Authorizing `identity:get_role(role_id=3b9c36a010504ba7bc63a8ccf0a9bca0)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:21.659631 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fa0f18-824d-47a3-bf0b-fb2323b91132 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:21.661040 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 555/1120] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:53:21 2022] GET /identity/v3/roles/3b9c36a010504ba7bc63a8ccf0a9bca0 => generated 223 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:53:21.666791 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ca80ee-7389-4dbd-adc0-c9d10f53ef3f None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:21.700949 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:21.766461 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ca80ee-7389-4dbd-adc0-c9d10f53ef3f None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:21.799479 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54206095-9804-443d-b3df-54091a1f906b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:21.800352 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54206095-9804-443d-b3df-54091a1f906b None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:21.800643 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54206095-9804-443d-b3df-54091a1f906b None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:21.800927 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54206095-9804-443d-b3df-54091a1f906b None admin] PATH_INFO: `/v3/users/ee33154b249c451cb2c0bda957431c3e` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:21.802141 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54206095-9804-443d-b3df-54091a1f906b None admin] RBAC: Authorizing `identity:get_user(user_id=ee33154b249c451cb2c0bda957431c3e)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:21.803587 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54206095-9804-443d-b3df-54091a1f906b None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:21.805226 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 566/1121] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:53:21 2022] GET /identity/v3/users/ee33154b249c451cb2c0bda957431c3e => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:53:21.811825 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fa0f18-824d-47a3-bf0b-fb2323b91132 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:21.828310 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:21.912045 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fa0f18-824d-47a3-bf0b-fb2323b91132 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:21.946750 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134fb061-b350-451e-b0d6-af751c23f4a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:21.947612 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134fb061-b350-451e-b0d6-af751c23f4a8 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:21.947901 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134fb061-b350-451e-b0d6-af751c23f4a8 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:21.948178 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134fb061-b350-451e-b0d6-af751c23f4a8 None admin] PATH_INFO: `/v3/projects/57b3750f90cb43a1a798f128a15fe685` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:21.949368 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134fb061-b350-451e-b0d6-af751c23f4a8 None admin] RBAC: Authorizing `identity:get_project(project_id=57b3750f90cb43a1a798f128a15fe685)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:21.950833 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134fb061-b350-451e-b0d6-af751c23f4a8 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:21.953237 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 556/1122] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:53:21 2022] GET /identity/v3/projects/57b3750f90cb43a1a798f128a15fe685 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:53:21.959416 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54206095-9804-443d-b3df-54091a1f906b None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:22.058334 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54206095-9804-443d-b3df-54091a1f906b None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:22.060607 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:22.093732 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c845673-0078-422c-8145-f88cd979ffe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:22.094674 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c845673-0078-422c-8145-f88cd979ffe5 None admin] REQUEST_METHOD: `PUT` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:22.094945 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c845673-0078-422c-8145-f88cd979ffe5 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:22.095231 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c845673-0078-422c-8145-f88cd979ffe5 None admin] PATH_INFO: `/v3/projects/57b3750f90cb43a1a798f128a15fe685/users/ee33154b249c451cb2c0bda957431c3e/roles/3b9c36a010504ba7bc63a8ccf0a9bca0` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:22.097531 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c845673-0078-422c-8145-f88cd979ffe5 None admin] RBAC: Authorizing `identity:create_grant(project_id=57b3750f90cb43a1a798f128a15fe685, user_id=ee33154b249c451cb2c0bda957431c3e, role_id=3b9c36a010504ba7bc63a8ccf0a9bca0)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:22.099003 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c845673-0078-422c-8145-f88cd979ffe5 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:22.112189 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 567/1123] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 14:53:21 2022] PUT /identity/v3/projects/57b3750f90cb43a1a798f128a15fe685/users/ee33154b249c451cb2c0bda957431c3e/roles/3b9c36a010504ba7bc63a8ccf0a9bca0 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:53:24.327422 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ead2151-f017-40e3-bc60-0906e4642946 None None] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:24.327997 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ead2151-f017-40e3-bc60-0906e4642946 None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:24.327997 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ead2151-f017-40e3-bc60-0906e4642946 None None] PATH_INFO: `/` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:24.330279 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 557/1124] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:53:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:53:24.335571 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15c19b7-adca-49eb-b0cc-267714d53a7e None None] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:24.335961 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15c19b7-adca-49eb-b0cc-267714d53a7e None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:24.336132 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15c19b7-adca-49eb-b0cc-267714d53a7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:24.353389 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.auth.core [None req-a15c19b7-adca-49eb-b0cc-267714d53a7e None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:53:24.354598 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-a15c19b7-adca-49eb-b0cc-267714d53a7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:53:24.420136 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 568/1125] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:53:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:53:24.443822 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ead2151-f017-40e3-bc60-0906e4642946 None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:24.525160 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:24.544491 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ead2151-f017-40e3-bc60-0906e4642946 None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:24.577900 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0acf225-9473-4540-bdc6-ccaa1ff7850d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:24.578822 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0acf225-9473-4540-bdc6-ccaa1ff7850d None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:24.579103 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0acf225-9473-4540-bdc6-ccaa1ff7850d None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:24.579373 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0acf225-9473-4540-bdc6-ccaa1ff7850d None admin] PATH_INFO: `/v3/roles/3b9c36a010504ba7bc63a8ccf0a9bca0` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:24.580845 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0acf225-9473-4540-bdc6-ccaa1ff7850d None admin] RBAC: Authorizing `identity:get_role(role_id=3b9c36a010504ba7bc63a8ccf0a9bca0)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:24.582245 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0acf225-9473-4540-bdc6-ccaa1ff7850d None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:24.583613 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 558/1126] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:53:24 2022] GET /identity/v3/roles/3b9c36a010504ba7bc63a8ccf0a9bca0 => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:53:24.589510 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15c19b7-adca-49eb-b0cc-267714d53a7e None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:24.681240 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:24.689494 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15c19b7-adca-49eb-b0cc-267714d53a7e None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:24.724751 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308340f6-bd50-4c00-9950-15c3ef366366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:24.725615 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308340f6-bd50-4c00-9950-15c3ef366366 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:24.725929 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308340f6-bd50-4c00-9950-15c3ef366366 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:24.726214 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308340f6-bd50-4c00-9950-15c3ef366366 None admin] PATH_INFO: `/v3/users/ee33154b249c451cb2c0bda957431c3e` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:24.727405 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308340f6-bd50-4c00-9950-15c3ef366366 None admin] RBAC: Authorizing `identity:get_user(user_id=ee33154b249c451cb2c0bda957431c3e)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:24.728840 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308340f6-bd50-4c00-9950-15c3ef366366 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:24.730655 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 569/1127] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:53:24 2022] GET /identity/v3/users/ee33154b249c451cb2c0bda957431c3e => generated 286 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:53:24.736579 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0acf225-9473-4540-bdc6-ccaa1ff7850d None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:24.835651 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0acf225-9473-4540-bdc6-ccaa1ff7850d None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:24.868621 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795d9641-c8b0-4ddc-9105-4895f87c121c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:24.869271 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795d9641-c8b0-4ddc-9105-4895f87c121c None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:24.869558 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795d9641-c8b0-4ddc-9105-4895f87c121c None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:24.869835 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795d9641-c8b0-4ddc-9105-4895f87c121c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:24.889897 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:24.938155 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795d9641-c8b0-4ddc-9105-4895f87c121c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:24.939605 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795d9641-c8b0-4ddc-9105-4895f87c121c None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:25.036569 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 559/1128] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:53:24 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:53:25.042531 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308340f6-bd50-4c00-9950-15c3ef366366 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:25.142071 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308340f6-bd50-4c00-9950-15c3ef366366 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:25.175937 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11c130a-6290-4fed-bfd5-4ca2b78e1ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:25.176812 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11c130a-6290-4fed-bfd5-4ca2b78e1ee4 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:25.177098 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11c130a-6290-4fed-bfd5-4ca2b78e1ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:25.177378 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11c130a-6290-4fed-bfd5-4ca2b78e1ee4 None admin] PATH_INFO: `/v3/projects/57b3750f90cb43a1a798f128a15fe685` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:25.178571 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11c130a-6290-4fed-bfd5-4ca2b78e1ee4 None admin] RBAC: Authorizing `identity:get_project(project_id=57b3750f90cb43a1a798f128a15fe685)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:25.179993 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11c130a-6290-4fed-bfd5-4ca2b78e1ee4 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:25.181683 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 570/1129] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:53:25 2022] GET /identity/v3/projects/57b3750f90cb43a1a798f128a15fe685 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:53:25.188537 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795d9641-c8b0-4ddc-9105-4895f87c121c None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:25.238031 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:25.287969 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795d9641-c8b0-4ddc-9105-4895f87c121c None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:25.322042 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5f7ead-af43-49ab-a5c5-392e8fc929d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:25.322685 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5f7ead-af43-49ab-a5c5-392e8fc929d3 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:25.322970 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5f7ead-af43-49ab-a5c5-392e8fc929d3 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:25.323246 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5f7ead-af43-49ab-a5c5-392e8fc929d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:25.324069 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5f7ead-af43-49ab-a5c5-392e8fc929d3 None admin] RBAC: Adding query filter params (role.id=3b9c36a010504ba7bc63a8ccf0a9bca0, scope.project.id=57b3750f90cb43a1a798f128a15fe685, user.id=ee33154b249c451cb2c0bda957431c3e) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:53:25.324380 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5f7ead-af43-49ab-a5c5-392e8fc929d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:25.325798 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5f7ead-af43-49ab-a5c5-392e8fc929d3 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:25.331934 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 560/1130] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:53:25 2022] GET /identity/v3/role_assignments?user.id=ee33154b249c451cb2c0bda957431c3e&scope.project.id=57b3750f90cb43a1a798f128a15fe685&role.id=3b9c36a010504ba7bc63a8ccf0a9bca0 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:53:27.579430 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceed25d-b6bd-4f0f-8308-4dd64afef1d0 None None] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:27.580038 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceed25d-b6bd-4f0f-8308-4dd64afef1d0 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:27.580038 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceed25d-b6bd-4f0f-8308-4dd64afef1d0 None None] PATH_INFO: `/` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:27.581382 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 571/1131] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:53:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:53:27.586568 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467f6781-6700-46e3-ba4b-f4f5b8c694e5 None None] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:27.586978 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467f6781-6700-46e3-ba4b-f4f5b8c694e5 None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:27.587131 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467f6781-6700-46e3-ba4b-f4f5b8c694e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:27.603780 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.auth.core [None req-467f6781-6700-46e3-ba4b-f4f5b8c694e5 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:53:27.604957 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.fernet_utils [None req-467f6781-6700-46e3-ba4b-f4f5b8c694e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:53:27.672230 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 561/1132] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:53:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:53:27.696869 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ceed25d-b6bd-4f0f-8308-4dd64afef1d0 None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:27.727794 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:27.800644 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ceed25d-b6bd-4f0f-8308-4dd64afef1d0 None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:27.834795 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e153244-755d-4368-9360-c442ef6adb86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:27.835532 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e153244-755d-4368-9360-c442ef6adb86 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:27.835825 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e153244-755d-4368-9360-c442ef6adb86 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:27.836110 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e153244-755d-4368-9360-c442ef6adb86 None admin] PATH_INFO: `/v3/domains/default` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:27.837297 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e153244-755d-4368-9360-c442ef6adb86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:27.838773 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e153244-755d-4368-9360-c442ef6adb86 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:27.840408 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 572/1133] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:53:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:53:27.846322 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467f6781-6700-46e3-ba4b-f4f5b8c694e5 None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:27.890575 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:27.944765 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467f6781-6700-46e3-ba4b-f4f5b8c694e5 None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:27.979216 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef2a945-0082-48f0-9656-a3720fae7621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:27.979882 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef2a945-0082-48f0-9656-a3720fae7621 None admin] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:27.980174 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef2a945-0082-48f0-9656-a3720fae7621 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:27.980460 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef2a945-0082-48f0-9656-a3720fae7621 None admin] PATH_INFO: `/v3/projects` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:27.981170 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef2a945-0082-48f0-9656-a3720fae7621 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:27.982925 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef2a945-0082-48f0-9656-a3720fae7621 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:28.007867 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 562/1134] 10.222.0.57 () {60 vars in 1141 bytes} [Thu May 19 14:53:27 2022] POST /identity/v3/projects => generated 311 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:53:30.276500 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f189d2bb-02a0-41e2-9d36-286f248767ca None None] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:30.277076 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f189d2bb-02a0-41e2-9d36-286f248767ca None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:30.277076 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f189d2bb-02a0-41e2-9d36-286f248767ca None None] PATH_INFO: `/` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:30.278521 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 573/1135] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:53:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:53:30.284044 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acb3703-299c-4453-b533-4e901c133890 None None] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:30.284493 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acb3703-299c-4453-b533-4e901c133890 None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:30.284606 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acb3703-299c-4453-b533-4e901c133890 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:30.302091 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.auth.core [None req-0acb3703-299c-4453-b533-4e901c133890 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:53:30.303294 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.fernet_utils [None req-0acb3703-299c-4453-b533-4e901c133890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:53:30.369534 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 563/1136] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:53:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:53:30.395327 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f189d2bb-02a0-41e2-9d36-286f248767ca None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:30.492883 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:30.498974 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f189d2bb-02a0-41e2-9d36-286f248767ca None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:30.532171 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3599b27c-1bc6-4cbf-9437-4617d8b60642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:30.532985 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3599b27c-1bc6-4cbf-9437-4617d8b60642 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:30.533285 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3599b27c-1bc6-4cbf-9437-4617d8b60642 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:30.533571 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3599b27c-1bc6-4cbf-9437-4617d8b60642 None admin] PATH_INFO: `/v3/domains/default` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:30.534790 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3599b27c-1bc6-4cbf-9437-4617d8b60642 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:30.536229 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3599b27c-1bc6-4cbf-9437-4617d8b60642 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:30.537866 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 574/1137] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:53:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:53:30.543606 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acb3703-299c-4453-b533-4e901c133890 None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:30.570748 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:30.643011 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acb3703-299c-4453-b533-4e901c133890 None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:30.676427 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d2f4da-cce0-4764-a1cc-e016cf906abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:30.677103 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d2f4da-cce0-4764-a1cc-e016cf906abe None admin] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:30.677390 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d2f4da-cce0-4764-a1cc-e016cf906abe None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:30.677669 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d2f4da-cce0-4764-a1cc-e016cf906abe None admin] PATH_INFO: `/v3/users` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:30.678409 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d2f4da-cce0-4764-a1cc-e016cf906abe None admin] RBAC: Authorizing `identity:create_user()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:30.679842 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d2f4da-cce0-4764-a1cc-e016cf906abe None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:30.699900 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 564/1138] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 19 14:53:30 2022] POST /identity/v3/users => generated 286 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:53:32.964587 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33952177-af62-4d2d-ad08-e93cca7ac8b4 None None] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:32.965126 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33952177-af62-4d2d-ad08-e93cca7ac8b4 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:32.965126 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33952177-af62-4d2d-ad08-e93cca7ac8b4 None None] PATH_INFO: `/` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:32.966528 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 575/1139] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:53:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:53:32.972060 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee72481-bcd4-4974-851b-33404d83e4f4 None None] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:32.972453 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee72481-bcd4-4974-851b-33404d83e4f4 None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:32.972640 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee72481-bcd4-4974-851b-33404d83e4f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:32.989653 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.auth.core [None req-4ee72481-bcd4-4974-851b-33404d83e4f4 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:53:32.990899 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.fernet_utils [None req-4ee72481-bcd4-4974-851b-33404d83e4f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:53:33.057548 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 565/1140] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:53:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:53:33.082012 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33952177-af62-4d2d-ad08-e93cca7ac8b4 None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:33.184964 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33952177-af62-4d2d-ad08-e93cca7ac8b4 None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:33.221833 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0630f12-3bfb-443b-90d0-a3b6b585df6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:33.222592 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0630f12-3bfb-443b-90d0-a3b6b585df6a None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:33.222886 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0630f12-3bfb-443b-90d0-a3b6b585df6a None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:33.223169 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0630f12-3bfb-443b-90d0-a3b6b585df6a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:33.233863 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0630f12-3bfb-443b-90d0-a3b6b585df6a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:53:33.241519 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-b0630f12-3bfb-443b-90d0-a3b6b585df6a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:53:33.241519 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:53:33.241519 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:53:33.241519 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:53:33.241519 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:53:33.241519 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:53:33.241519 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:53:33.241519 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:53:33.241519 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:53:33.241519 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:53:33.241519 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:53:33.241519 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:53:33.241519 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:53:33.241519 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:53:33.241519 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:53:33.241519 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:53:33.242935 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:53:33.242935 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:53:33.242935 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:53:33.242935 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:53:33.242935 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:53:33.242935 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:53:33.242935 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:53:33.242935 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:53:33.242935 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:53:33.242935 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:53:33.242935 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:53:33.242935 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:53:33.242935 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:53:33.242935 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:53:33.242935 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:53:33.242935 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:53:33.244230 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:53:33.244230 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:53:33.244230 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:53:33.244230 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:53:33.244230 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:53:33.244230 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:53:33.244230 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:53:33.244230 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:53:33.244230 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:53:33.244230 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:53:33.244230 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:53:33.244230 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:53:33.244230 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 576/1141] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:53:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:53:33.249321 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee72481-bcd4-4974-851b-33404d83e4f4 None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:33.263155 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:33.347294 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee72481-bcd4-4974-851b-33404d83e4f4 None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:33.380976 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3ba949-b677-4697-8089-f305029db50d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:33.381634 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3ba949-b677-4697-8089-f305029db50d None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:33.381947 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3ba949-b677-4697-8089-f305029db50d None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:33.382227 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3ba949-b677-4697-8089-f305029db50d None admin] PATH_INFO: `/v3/roles` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:33.382964 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3ba949-b677-4697-8089-f305029db50d None admin] RBAC: Adding query filter params (name=admin) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:53:33.383272 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3ba949-b677-4697-8089-f305029db50d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:33.384639 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3ba949-b677-4697-8089-f305029db50d None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:33.392793 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 566/1142] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:53:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:53:33.398622 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0630f12-3bfb-443b-90d0-a3b6b585df6a None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:33.426481 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:33.499208 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0630f12-3bfb-443b-90d0-a3b6b585df6a None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:33.531723 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390c8a98-2cd3-4f80-9b42-0413ac65e791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:33.532580 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390c8a98-2cd3-4f80-9b42-0413ac65e791 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:33.532870 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390c8a98-2cd3-4f80-9b42-0413ac65e791 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:33.533152 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390c8a98-2cd3-4f80-9b42-0413ac65e791 None admin] PATH_INFO: `/v3/users/7eb32406c8894738ab3dc398c0834cff` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:33.547246 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390c8a98-2cd3-4f80-9b42-0413ac65e791 None admin] RBAC: Authorizing `identity:get_user(user_id=7eb32406c8894738ab3dc398c0834cff)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:33.548686 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390c8a98-2cd3-4f80-9b42-0413ac65e791 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:33.550382 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 577/1143] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:53:33 2022] GET /identity/v3/users/7eb32406c8894738ab3dc398c0834cff => generated 286 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:53:33.556147 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3ba949-b677-4697-8089-f305029db50d None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:33.630601 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:33.655398 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3ba949-b677-4697-8089-f305029db50d None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:33.688966 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053938c2-eca1-4e4e-825b-b24a16df0cca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:33.689616 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053938c2-eca1-4e4e-825b-b24a16df0cca None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:33.689938 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053938c2-eca1-4e4e-825b-b24a16df0cca None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:33.690231 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053938c2-eca1-4e4e-825b-b24a16df0cca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:33.756099 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053938c2-eca1-4e4e-825b-b24a16df0cca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:33.757489 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053938c2-eca1-4e4e-825b-b24a16df0cca None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:33.829039 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:33.854429 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 567/1144] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:53:33 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:53:33.861295 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390c8a98-2cd3-4f80-9b42-0413ac65e791 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:33.944680 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:33.960906 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390c8a98-2cd3-4f80-9b42-0413ac65e791 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:33.994650 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec012047-305f-44ac-a5ac-fb4973f32562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:33.995516 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec012047-305f-44ac-a5ac-fb4973f32562 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:33.995807 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec012047-305f-44ac-a5ac-fb4973f32562 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:33.996089 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec012047-305f-44ac-a5ac-fb4973f32562 None admin] PATH_INFO: `/v3/projects/7e4fa214c98649658d05c1db40029db1` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:33.997278 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec012047-305f-44ac-a5ac-fb4973f32562 None admin] RBAC: Authorizing `identity:get_project(project_id=7e4fa214c98649658d05c1db40029db1)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:33.999008 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec012047-305f-44ac-a5ac-fb4973f32562 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:34.000723 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 578/1145] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:53:33 2022] GET /identity/v3/projects/7e4fa214c98649658d05c1db40029db1 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:53:34.006962 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053938c2-eca1-4e4e-825b-b24a16df0cca None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:34.104772 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053938c2-eca1-4e4e-825b-b24a16df0cca None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:34.139000 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b51739-7538-44c4-b971-3cc2a7fab380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:34.139653 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b51739-7538-44c4-b971-3cc2a7fab380 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:34.139943 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b51739-7538-44c4-b971-3cc2a7fab380 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:34.140227 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b51739-7538-44c4-b971-3cc2a7fab380 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:34.141058 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b51739-7538-44c4-b971-3cc2a7fab380 None admin] RBAC: Adding query filter params (role.id=028379e01f9245e9b8fc56e8b9e8a6ad, scope.project.id=7e4fa214c98649658d05c1db40029db1, user.id=7eb32406c8894738ab3dc398c0834cff) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:53:34.141376 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b51739-7538-44c4-b971-3cc2a7fab380 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:34.142849 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b51739-7538-44c4-b971-3cc2a7fab380 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:34.148293 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 568/1146] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:53:34 2022] GET /identity/v3/role_assignments?user.id=7eb32406c8894738ab3dc398c0834cff&scope.project.id=7e4fa214c98649658d05c1db40029db1&role.id=028379e01f9245e9b8fc56e8b9e8a6ad => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:53:36.415741 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425a493d-1cd0-49d8-ac04-651a80bf867e None None] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:36.416318 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425a493d-1cd0-49d8-ac04-651a80bf867e None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:36.416318 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425a493d-1cd0-49d8-ac04-651a80bf867e None None] PATH_INFO: `/` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:36.417684 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 579/1147] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:53:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:53:36.423681 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cc74c1-3d9d-4c2a-8305-af9a4144b932 None None] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:36.424121 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cc74c1-3d9d-4c2a-8305-af9a4144b932 None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:36.424235 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cc74c1-3d9d-4c2a-8305-af9a4144b932 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:36.441531 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.auth.core [None req-46cc74c1-3d9d-4c2a-8305-af9a4144b932 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:53:36.442766 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.fernet_utils [None req-46cc74c1-3d9d-4c2a-8305-af9a4144b932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:53:36.472567 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:36.510137 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 569/1148] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:53:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:53:36.533074 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425a493d-1cd0-49d8-ac04-651a80bf867e None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:36.633833 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425a493d-1cd0-49d8-ac04-651a80bf867e None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:36.667405 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dde6768-fdf3-4aca-b4a3-42564a060439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:36.668146 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dde6768-fdf3-4aca-b4a3-42564a060439 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:36.668440 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dde6768-fdf3-4aca-b4a3-42564a060439 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:36.668724 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dde6768-fdf3-4aca-b4a3-42564a060439 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:36.679796 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dde6768-fdf3-4aca-b4a3-42564a060439 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:53:36.687718 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-2dde6768-fdf3-4aca-b4a3-42564a060439 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:53:36.687718 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:53:36.687718 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:53:36.687718 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:53:36.687718 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:53:36.687718 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:53:36.687718 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:53:36.687718 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:53:36.687718 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:53:36.687718 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:53:36.687718 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:53:36.687718 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:53:36.687718 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:53:36.687718 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:53:36.687718 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:53:36.687718 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:53:36.689002 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:53:36.689002 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:53:36.689002 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:53:36.689002 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:53:36.689002 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:53:36.689002 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:53:36.689002 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:53:36.689002 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:53:36.689002 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:53:36.689002 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:53:36.689002 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:53:36.689002 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:53:36.689002 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:53:36.689002 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:53:36.689002 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:53:36.689002 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:53:36.690391 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:53:36.690391 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:53:36.690391 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:53:36.690391 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:53:36.690391 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:53:36.690391 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:53:36.690391 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:53:36.690391 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:53:36.690391 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:53:36.690391 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:53:36.690391 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:53:36.690391 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:53:36.690391 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 580/1149] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:53:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:53:36.695281 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cc74c1-3d9d-4c2a-8305-af9a4144b932 None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:36.792825 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cc74c1-3d9d-4c2a-8305-af9a4144b932 None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:36.827694 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da64cf57-2129-49c8-88e0-992529bf7d3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:36.828360 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da64cf57-2129-49c8-88e0-992529bf7d3f None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:36.828652 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da64cf57-2129-49c8-88e0-992529bf7d3f None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:36.828933 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da64cf57-2129-49c8-88e0-992529bf7d3f None admin] PATH_INFO: `/v3/roles` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:36.829684 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da64cf57-2129-49c8-88e0-992529bf7d3f None admin] RBAC: Adding query filter params (name=admin) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:53:36.830028 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da64cf57-2129-49c8-88e0-992529bf7d3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:36.831407 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da64cf57-2129-49c8-88e0-992529bf7d3f None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:36.839382 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 570/1150] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:53:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:53:36.845986 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dde6768-fdf3-4aca-b4a3-42564a060439 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:36.861805 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:36.946472 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dde6768-fdf3-4aca-b4a3-42564a060439 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:36.980660 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fbdd2c-5e1a-4288-9c9b-2251a6772c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:36.981539 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fbdd2c-5e1a-4288-9c9b-2251a6772c2d None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:36.981875 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fbdd2c-5e1a-4288-9c9b-2251a6772c2d None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:36.982270 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fbdd2c-5e1a-4288-9c9b-2251a6772c2d None admin] PATH_INFO: `/v3/users/7eb32406c8894738ab3dc398c0834cff` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:36.983488 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fbdd2c-5e1a-4288-9c9b-2251a6772c2d None admin] RBAC: Authorizing `identity:get_user(user_id=7eb32406c8894738ab3dc398c0834cff)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:36.984938 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fbdd2c-5e1a-4288-9c9b-2251a6772c2d None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:36.986640 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 581/1151] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:53:36 2022] GET /identity/v3/users/7eb32406c8894738ab3dc398c0834cff => generated 286 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:53:36.992577 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da64cf57-2129-49c8-88e0-992529bf7d3f None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:37.023752 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:37.093720 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da64cf57-2129-49c8-88e0-992529bf7d3f None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:37.126785 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0107e544-b5fe-4f50-af85-ffaa94e684ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:37.127656 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0107e544-b5fe-4f50-af85-ffaa94e684ef None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:37.127948 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0107e544-b5fe-4f50-af85-ffaa94e684ef None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:37.128230 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0107e544-b5fe-4f50-af85-ffaa94e684ef None admin] PATH_INFO: `/v3/projects/7e4fa214c98649658d05c1db40029db1` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:37.129418 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0107e544-b5fe-4f50-af85-ffaa94e684ef None admin] RBAC: Authorizing `identity:get_project(project_id=7e4fa214c98649658d05c1db40029db1)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:37.130890 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0107e544-b5fe-4f50-af85-ffaa94e684ef None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:37.132593 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 571/1152] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:53:36 2022] GET /identity/v3/projects/7e4fa214c98649658d05c1db40029db1 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:53:37.139139 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fbdd2c-5e1a-4288-9c9b-2251a6772c2d None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:37.219516 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:37.239464 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fbdd2c-5e1a-4288-9c9b-2251a6772c2d None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:37.273718 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fecacc-2292-46c2-9f3c-f2ecce6cae00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:37.274642 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fecacc-2292-46c2-9f3c-f2ecce6cae00 None admin] REQUEST_METHOD: `PUT` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:37.274935 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fecacc-2292-46c2-9f3c-f2ecce6cae00 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:37.275219 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fecacc-2292-46c2-9f3c-f2ecce6cae00 None admin] PATH_INFO: `/v3/projects/7e4fa214c98649658d05c1db40029db1/users/7eb32406c8894738ab3dc398c0834cff/roles/028379e01f9245e9b8fc56e8b9e8a6ad` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:37.277190 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fecacc-2292-46c2-9f3c-f2ecce6cae00 None admin] RBAC: Authorizing `identity:create_grant(project_id=7e4fa214c98649658d05c1db40029db1, user_id=7eb32406c8894738ab3dc398c0834cff, role_id=028379e01f9245e9b8fc56e8b9e8a6ad)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:37.278667 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fecacc-2292-46c2-9f3c-f2ecce6cae00 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:37.291729 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 582/1153] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 14:53:37 2022] PUT /identity/v3/projects/7e4fa214c98649658d05c1db40029db1/users/7eb32406c8894738ab3dc398c0834cff/roles/028379e01f9245e9b8fc56e8b9e8a6ad => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:53:39.512789 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65cf328-a7da-4bf0-b2c9-762c532b10a8 None None] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:39.513949 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65cf328-a7da-4bf0-b2c9-762c532b10a8 None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:39.514241 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65cf328-a7da-4bf0-b2c9-762c532b10a8 None None] PATH_INFO: `/` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:39.515629 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 572/1154] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:53:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:53:39.521055 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62a0709-a5e1-4933-908b-160215dc4820 None None] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:39.521497 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62a0709-a5e1-4933-908b-160215dc4820 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:39.521609 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62a0709-a5e1-4933-908b-160215dc4820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:39.539988 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.auth.core [None req-a62a0709-a5e1-4933-908b-160215dc4820 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:53:39.541182 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-a62a0709-a5e1-4933-908b-160215dc4820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:53:39.608031 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 583/1155] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:53:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:53:39.632504 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65cf328-a7da-4bf0-b2c9-762c532b10a8 None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:39.730055 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:39.735047 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65cf328-a7da-4bf0-b2c9-762c532b10a8 None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:39.768152 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14135f5-6763-445a-9259-9defca734e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:39.768909 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14135f5-6763-445a-9259-9defca734e71 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:39.769196 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14135f5-6763-445a-9259-9defca734e71 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:39.769477 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14135f5-6763-445a-9259-9defca734e71 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:39.779869 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14135f5-6763-445a-9259-9defca734e71 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:53:39.787351 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-d14135f5-6763-445a-9259-9defca734e71 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:53:39.787351 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:53:39.787351 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:53:39.787351 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:53:39.787351 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:53:39.787351 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:53:39.787351 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:53:39.787351 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:53:39.787351 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:53:39.787351 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:53:39.787351 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:53:39.787351 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:53:39.787351 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:53:39.787351 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:53:39.787351 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:53:39.787351 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:53:39.788623 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:53:39.788623 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:53:39.788623 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:53:39.788623 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:53:39.788623 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:53:39.788623 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:53:39.788623 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:53:39.788623 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:53:39.788623 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:53:39.788623 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:53:39.788623 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:53:39.788623 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:53:39.788623 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:53:39.788623 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:53:39.788623 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:53:39.788623 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:53:39.790074 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:53:39.790074 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:53:39.790074 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:53:39.790074 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:53:39.790074 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:53:39.790074 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:53:39.790074 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:53:39.790074 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:53:39.790074 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:53:39.790074 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:53:39.790074 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:53:39.790074 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:53:39.790074 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 573/1156] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:53:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:53:39.794826 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62a0709-a5e1-4933-908b-160215dc4820 None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:39.862382 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:39.893147 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62a0709-a5e1-4933-908b-160215dc4820 None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:39.925437 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26810d67-1ed0-44c0-b4f0-52e1bbf59535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:39.926138 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26810d67-1ed0-44c0-b4f0-52e1bbf59535 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:39.926426 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26810d67-1ed0-44c0-b4f0-52e1bbf59535 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:39.926706 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26810d67-1ed0-44c0-b4f0-52e1bbf59535 None admin] PATH_INFO: `/v3/roles` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:39.927449 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26810d67-1ed0-44c0-b4f0-52e1bbf59535 None admin] RBAC: Adding query filter params (name=admin) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:53:39.927758 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26810d67-1ed0-44c0-b4f0-52e1bbf59535 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:39.929125 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26810d67-1ed0-44c0-b4f0-52e1bbf59535 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:39.937596 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 584/1157] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:53:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:53:39.943717 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14135f5-6763-445a-9259-9defca734e71 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:40.042120 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14135f5-6763-445a-9259-9defca734e71 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:40.076485 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304813b6-9285-4fab-aeba-9bb51820a376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:40.077346 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304813b6-9285-4fab-aeba-9bb51820a376 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:40.077707 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304813b6-9285-4fab-aeba-9bb51820a376 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:40.077946 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304813b6-9285-4fab-aeba-9bb51820a376 None admin] PATH_INFO: `/v3/users/7eb32406c8894738ab3dc398c0834cff` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:40.079147 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304813b6-9285-4fab-aeba-9bb51820a376 None admin] RBAC: Authorizing `identity:get_user(user_id=7eb32406c8894738ab3dc398c0834cff)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:40.080593 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304813b6-9285-4fab-aeba-9bb51820a376 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:40.082281 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 574/1158] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:53:39 2022] GET /identity/v3/users/7eb32406c8894738ab3dc398c0834cff => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:53:40.088088 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26810d67-1ed0-44c0-b4f0-52e1bbf59535 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:40.189389 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26810d67-1ed0-44c0-b4f0-52e1bbf59535 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:40.218614 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:40.223813 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1646ec5-ee04-4cbc-a1bd-849087e41cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:40.224466 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1646ec5-ee04-4cbc-a1bd-849087e41cff None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:40.224757 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1646ec5-ee04-4cbc-a1bd-849087e41cff None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:40.225039 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1646ec5-ee04-4cbc-a1bd-849087e41cff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:40.295164 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1646ec5-ee04-4cbc-a1bd-849087e41cff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:40.296554 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1646ec5-ee04-4cbc-a1bd-849087e41cff None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:40.393931 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 585/1159] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:53:40 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:53:40.399841 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304813b6-9285-4fab-aeba-9bb51820a376 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:40.426202 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:40.501060 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304813b6-9285-4fab-aeba-9bb51820a376 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:40.535081 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061642c4-19e6-4e37-bc5e-3139d60e3de5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:40.535941 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061642c4-19e6-4e37-bc5e-3139d60e3de5 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:40.536231 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061642c4-19e6-4e37-bc5e-3139d60e3de5 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:40.536512 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061642c4-19e6-4e37-bc5e-3139d60e3de5 None admin] PATH_INFO: `/v3/projects/7e4fa214c98649658d05c1db40029db1` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:40.537691 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061642c4-19e6-4e37-bc5e-3139d60e3de5 None admin] RBAC: Authorizing `identity:get_project(project_id=7e4fa214c98649658d05c1db40029db1)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:40.539182 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061642c4-19e6-4e37-bc5e-3139d60e3de5 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:40.540888 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 575/1160] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:53:40 2022] GET /identity/v3/projects/7e4fa214c98649658d05c1db40029db1 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:53:40.547241 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1646ec5-ee04-4cbc-a1bd-849087e41cff None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:40.578540 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:40.646999 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1646ec5-ee04-4cbc-a1bd-849087e41cff None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:40.680866 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313a83d8-5c7b-4a78-9751-312a7bb961a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:40.681526 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313a83d8-5c7b-4a78-9751-312a7bb961a2 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:40.681817 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313a83d8-5c7b-4a78-9751-312a7bb961a2 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:40.682135 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313a83d8-5c7b-4a78-9751-312a7bb961a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:40.682967 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313a83d8-5c7b-4a78-9751-312a7bb961a2 None admin] RBAC: Adding query filter params (role.id=028379e01f9245e9b8fc56e8b9e8a6ad, scope.project.id=7e4fa214c98649658d05c1db40029db1, user.id=7eb32406c8894738ab3dc398c0834cff) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:53:40.683287 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313a83d8-5c7b-4a78-9751-312a7bb961a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:40.685564 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313a83d8-5c7b-4a78-9751-312a7bb961a2 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:40.691609 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 586/1161] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:53:40 2022] GET /identity/v3/role_assignments?user.id=7eb32406c8894738ab3dc398c0834cff&scope.project.id=7e4fa214c98649658d05c1db40029db1&role.id=028379e01f9245e9b8fc56e8b9e8a6ad => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:53:42.932891 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd55badd-5c9b-44d5-bcf4-73ad746497b9 None None] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:42.933425 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd55badd-5c9b-44d5-bcf4-73ad746497b9 None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:42.933425 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd55badd-5c9b-44d5-bcf4-73ad746497b9 None None] PATH_INFO: `/` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:42.935322 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 576/1162] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:53:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:53:42.940643 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c5f852-698b-4658-9348-4df656566d61 None None] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:42.941095 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c5f852-698b-4658-9348-4df656566d61 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:42.941204 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c5f852-698b-4658-9348-4df656566d61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:42.959225 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.auth.core [None req-29c5f852-698b-4658-9348-4df656566d61 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:53:42.960413 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-29c5f852-698b-4658-9348-4df656566d61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:53:43.027664 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 587/1163] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:53:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:53:43.050955 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd55badd-5c9b-44d5-bcf4-73ad746497b9 None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:43.145052 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:43.152796 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd55badd-5c9b-44d5-bcf4-73ad746497b9 None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:43.187627 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181f8eaf-10f3-473a-a99f-2ae69eb33ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:43.188203 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181f8eaf-10f3-473a-a99f-2ae69eb33ded None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:43.188483 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181f8eaf-10f3-473a-a99f-2ae69eb33ded None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:43.188756 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181f8eaf-10f3-473a-a99f-2ae69eb33ded None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:43.256032 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181f8eaf-10f3-473a-a99f-2ae69eb33ded None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:43.257467 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181f8eaf-10f3-473a-a99f-2ae69eb33ded None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:43.347238 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:43.354986 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 577/1164] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:53:43 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:53:43.361954 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c5f852-698b-4658-9348-4df656566d61 None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:43.387624 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:43.458558 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c5f852-698b-4658-9348-4df656566d61 None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:43.492051 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbbc445-becd-4733-9565-505e14244e5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:43.492843 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbbc445-becd-4733-9565-505e14244e5a None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:43.493133 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbbc445-becd-4733-9565-505e14244e5a None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:43.493420 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbbc445-becd-4733-9565-505e14244e5a None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:43.500309 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbbc445-becd-4733-9565-505e14244e5a None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:43.501753 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbbc445-becd-4733-9565-505e14244e5a None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:43.515846 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-2cbbc445-becd-4733-9565-505e14244e5a None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. May 19 14:53:43.515846 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:53:43.515846 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 19 14:53:43.515846 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 19 14:53:43.515846 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:53:43.515846 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:53:43.515846 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:53:43.515846 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:53:43.515846 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. May 19 14:53:43.515846 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:53:43.515846 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:53:43.515846 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:53:43.515846 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:53:43.515846 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:53:43.515846 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:53:43.515846 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:53:43.515846 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:53:43.517219 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:53:43.517219 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:53:43.517219 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:53:43.517219 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:53:43.517219 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:53:43.517219 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:53:43.517219 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:53:43.517219 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:53:43.517219 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:53:43.517219 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:53:43.517219 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:53:43.517219 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:53:43.517219 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:53:43.517219 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:53:43.517219 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:53:43.517219 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:53:43.518294 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:53:43.518294 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:53:43.518294 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:53:43.518294 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:53:43.518294 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:53:43.518294 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:53:43.518294 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:53:43.518294 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:53:43.518294 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:53:43.518294 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:53:43.518294 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 19 14:53:43.518294 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:53:43.518294 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. May 19 14:53:43.518294 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:53:43.518294 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 588/1165] 10.222.0.57 () {56 vars in 1117 bytes} [Thu May 19 14:53:43 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:53:43.523112 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181f8eaf-10f3-473a-a99f-2ae69eb33ded None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:43.621290 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181f8eaf-10f3-473a-a99f-2ae69eb33ded None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:43.654936 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72bb945-03c3-4ec3-afd1-89fc4472e19f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:43.655603 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72bb945-03c3-4ec3-afd1-89fc4472e19f None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:43.655892 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72bb945-03c3-4ec3-afd1-89fc4472e19f None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:43.656172 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72bb945-03c3-4ec3-afd1-89fc4472e19f None admin] PATH_INFO: `/v3/domains` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:43.656896 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72bb945-03c3-4ec3-afd1-89fc4472e19f None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:53:43.657205 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72bb945-03c3-4ec3-afd1-89fc4472e19f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:43.658705 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72bb945-03c3-4ec3-afd1-89fc4472e19f None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:43.667176 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 578/1166] 10.222.0.57 () {56 vars in 1115 bytes} [Thu May 19 14:53:43 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:53:45.882415 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2d8ecf-3d31-41dd-b1a2-77178919b822 None None] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:45.882959 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2d8ecf-3d31-41dd-b1a2-77178919b822 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:45.883081 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2d8ecf-3d31-41dd-b1a2-77178919b822 None None] PATH_INFO: `/` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:45.884380 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 589/1167] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:53:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:53:45.890446 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac758c3-59ff-4e1e-8ae6-231fa354ad22 None None] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:45.891198 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac758c3-59ff-4e1e-8ae6-231fa354ad22 None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:45.891392 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac758c3-59ff-4e1e-8ae6-231fa354ad22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:45.909150 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.auth.core [None req-5ac758c3-59ff-4e1e-8ae6-231fa354ad22 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:53:45.910375 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.fernet_utils [None req-5ac758c3-59ff-4e1e-8ae6-231fa354ad22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:53:45.962438 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:45.976655 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 579/1168] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:53:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:53:46.000370 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2d8ecf-3d31-41dd-b1a2-77178919b822 None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:46.087600 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:46.104080 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2d8ecf-3d31-41dd-b1a2-77178919b822 None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:46.138776 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f49c574-a13e-4242-a443-a9302f979632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:46.139435 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f49c574-a13e-4242-a443-a9302f979632 None admin] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:46.139729 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f49c574-a13e-4242-a443-a9302f979632 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:46.140097 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f49c574-a13e-4242-a443-a9302f979632 None admin] PATH_INFO: `/v3/domains` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:46.140852 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f49c574-a13e-4242-a443-a9302f979632 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:46.142248 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f49c574-a13e-4242-a443-a9302f979632 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:46.156736 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 590/1169] 10.222.0.57 () {60 vars in 1138 bytes} [Thu May 19 14:53:45 2022] POST /identity/v3/domains => generated 267 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:53:48.398650 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87935b9b-c450-4e61-99d8-d842fde06ddd None None] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:48.399211 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87935b9b-c450-4e61-99d8-d842fde06ddd None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:48.399211 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87935b9b-c450-4e61-99d8-d842fde06ddd None None] PATH_INFO: `/` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:48.400614 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 580/1170] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:53:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:53:48.406639 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef56093-7f1c-42d5-8d24-9e81d4008bf9 None None] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:48.407062 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef56093-7f1c-42d5-8d24-9e81d4008bf9 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:48.407218 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef56093-7f1c-42d5-8d24-9e81d4008bf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:48.423868 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.auth.core [None req-cef56093-7f1c-42d5-8d24-9e81d4008bf9 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:53:48.425062 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-cef56093-7f1c-42d5-8d24-9e81d4008bf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:53:48.491267 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 591/1171] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:53:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:53:48.516447 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87935b9b-c450-4e61-99d8-d842fde06ddd None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:48.616828 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87935b9b-c450-4e61-99d8-d842fde06ddd None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:48.652822 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9312ee-e947-49ef-9acc-d7951175eba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:48.653885 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9312ee-e947-49ef-9acc-d7951175eba7 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:48.654249 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9312ee-e947-49ef-9acc-d7951175eba7 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:48.654355 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9312ee-e947-49ef-9acc-d7951175eba7 None admin] PATH_INFO: `/v3/domains/3bfbc82e9ef34278a6e261cb9686cb68` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:48.665136 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9312ee-e947-49ef-9acc-d7951175eba7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=3bfbc82e9ef34278a6e261cb9686cb68)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:48.666635 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9312ee-e947-49ef-9acc-d7951175eba7 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:48.668318 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 581/1172] 10.222.0.57 () {56 vars in 1183 bytes} [Thu May 19 14:53:48 2022] GET /identity/v3/domains/3bfbc82e9ef34278a6e261cb9686cb68 => generated 267 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:53:48.674724 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef56093-7f1c-42d5-8d24-9e81d4008bf9 None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:48.751191 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:48.774391 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef56093-7f1c-42d5-8d24-9e81d4008bf9 None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:48.807997 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edc65b5-b0f2-4bbc-b80a-9bac48ff2bc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:48.808661 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edc65b5-b0f2-4bbc-b80a-9bac48ff2bc0 None admin] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:48.808953 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edc65b5-b0f2-4bbc-b80a-9bac48ff2bc0 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:48.809236 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edc65b5-b0f2-4bbc-b80a-9bac48ff2bc0 None admin] PATH_INFO: `/v3/projects` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:48.810567 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edc65b5-b0f2-4bbc-b80a-9bac48ff2bc0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:48.812013 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edc65b5-b0f2-4bbc-b80a-9bac48ff2bc0 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:48.836452 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 592/1173] 10.222.0.57 () {60 vars in 1141 bytes} [Thu May 19 14:53:48 2022] POST /identity/v3/projects => generated 361 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:53:51.088909 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062fa443-cdd6-4895-b1e9-ec42cc10718e None None] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:51.089449 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062fa443-cdd6-4895-b1e9-ec42cc10718e None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:51.089449 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062fa443-cdd6-4895-b1e9-ec42cc10718e None None] PATH_INFO: `/` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:51.090879 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 582/1174] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:53:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:53:51.096363 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91441ef2-88c3-4468-b463-5d3532edfa7f None None] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:51.096794 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91441ef2-88c3-4468-b463-5d3532edfa7f None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:51.096934 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91441ef2-88c3-4468-b463-5d3532edfa7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:51.114236 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.auth.core [None req-91441ef2-88c3-4468-b463-5d3532edfa7f None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:53:51.115433 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-91441ef2-88c3-4468-b463-5d3532edfa7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:53:51.181207 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 593/1175] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:53:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:53:51.206845 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062fa443-cdd6-4895-b1e9-ec42cc10718e None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:51.258327 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:51.311347 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062fa443-cdd6-4895-b1e9-ec42cc10718e None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:51.345099 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c043396-215d-452b-9f45-af4eab0778ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:51.345918 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c043396-215d-452b-9f45-af4eab0778ee None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:51.346212 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c043396-215d-452b-9f45-af4eab0778ee None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:51.346495 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c043396-215d-452b-9f45-af4eab0778ee None admin] PATH_INFO: `/v3/domains/3bfbc82e9ef34278a6e261cb9686cb68` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:51.347686 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c043396-215d-452b-9f45-af4eab0778ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=3bfbc82e9ef34278a6e261cb9686cb68)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:51.349139 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c043396-215d-452b-9f45-af4eab0778ee None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:51.350806 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 583/1176] 10.222.0.57 () {56 vars in 1183 bytes} [Thu May 19 14:53:51 2022] GET /identity/v3/domains/3bfbc82e9ef34278a6e261cb9686cb68 => generated 267 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:53:51.357617 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91441ef2-88c3-4468-b463-5d3532edfa7f None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:51.415426 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:51.458132 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91441ef2-88c3-4468-b463-5d3532edfa7f None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:51.492024 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06bc7e1-14f0-4815-9ae2-54d7f6f4386a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:51.492697 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06bc7e1-14f0-4815-9ae2-54d7f6f4386a None admin] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:51.492990 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06bc7e1-14f0-4815-9ae2-54d7f6f4386a None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:51.493276 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06bc7e1-14f0-4815-9ae2-54d7f6f4386a None admin] PATH_INFO: `/v3/users` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:51.494035 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06bc7e1-14f0-4815-9ae2-54d7f6f4386a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:51.495502 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06bc7e1-14f0-4815-9ae2-54d7f6f4386a None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:51.515097 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 594/1177] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 19 14:53:51 2022] POST /identity/v3/users => generated 311 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:53:53.794055 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d27091-95e6-494d-9b71-e9f17ae4c5cd None None] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:53.794625 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d27091-95e6-494d-9b71-e9f17ae4c5cd None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:53.794625 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d27091-95e6-494d-9b71-e9f17ae4c5cd None None] PATH_INFO: `/` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:53.796035 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 584/1178] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:53:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:53:53.801350 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54b3148-53a2-40ee-b8b2-13117dd68836 None None] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:53.801769 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54b3148-53a2-40ee-b8b2-13117dd68836 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:53.801967 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54b3148-53a2-40ee-b8b2-13117dd68836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:53.819386 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.auth.core [None req-e54b3148-53a2-40ee-b8b2-13117dd68836 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:53:53.820589 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-e54b3148-53a2-40ee-b8b2-13117dd68836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:53:53.887091 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 595/1179] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:53:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:53:53.918039 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d27091-95e6-494d-9b71-e9f17ae4c5cd None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:54.021305 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d27091-95e6-494d-9b71-e9f17ae4c5cd None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:54.037330 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:54.055435 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7937b30c-fc45-4bd9-b388-1bd7ec5ab09f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:54.056235 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7937b30c-fc45-4bd9-b388-1bd7ec5ab09f None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:54.056527 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7937b30c-fc45-4bd9-b388-1bd7ec5ab09f None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:54.056811 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7937b30c-fc45-4bd9-b388-1bd7ec5ab09f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:54.068537 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7937b30c-fc45-4bd9-b388-1bd7ec5ab09f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:53:54.076323 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-7937b30c-fc45-4bd9-b388-1bd7ec5ab09f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:53:54.076323 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:53:54.076323 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:53:54.076323 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:53:54.076323 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:53:54.076323 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:53:54.076323 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:53:54.076323 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:53:54.076323 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:53:54.076323 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:53:54.076323 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:53:54.076323 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:53:54.076323 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:53:54.076323 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:53:54.076323 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:53:54.076323 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:53:54.077582 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:53:54.077582 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:53:54.077582 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:53:54.077582 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:53:54.077582 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:53:54.077582 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:53:54.077582 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:53:54.077582 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:53:54.077582 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:53:54.077582 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:53:54.077582 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:53:54.077582 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:53:54.077582 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:53:54.077582 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:53:54.077582 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:53:54.077582 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:53:54.078760 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:53:54.078760 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:53:54.078760 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:53:54.078760 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:53:54.078760 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:53:54.078760 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:53:54.078760 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:53:54.078760 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:53:54.078760 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:53:54.078760 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:53:54.078760 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:53:54.078760 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:53:54.078760 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 585/1180] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:53:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:53:54.083656 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54b3148-53a2-40ee-b8b2-13117dd68836 None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:54.129086 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:54.183669 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54b3148-53a2-40ee-b8b2-13117dd68836 None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:54.219232 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef1ea4f-f551-4ee3-bf37-7c2bc9894544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:54.219910 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef1ea4f-f551-4ee3-bf37-7c2bc9894544 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:54.220204 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef1ea4f-f551-4ee3-bf37-7c2bc9894544 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:54.220487 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef1ea4f-f551-4ee3-bf37-7c2bc9894544 None admin] PATH_INFO: `/v3/roles` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:54.221240 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef1ea4f-f551-4ee3-bf37-7c2bc9894544 None admin] RBAC: Adding query filter params (name=admin) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:53:54.221556 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef1ea4f-f551-4ee3-bf37-7c2bc9894544 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:54.222969 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef1ea4f-f551-4ee3-bf37-7c2bc9894544 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:54.231063 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 596/1181] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:53:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:53:54.237680 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7937b30c-fc45-4bd9-b388-1bd7ec5ab09f None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:54.337009 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7937b30c-fc45-4bd9-b388-1bd7ec5ab09f None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:54.370929 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6883e9b3-0448-40b2-aed4-7ba6c20c6956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:54.371843 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6883e9b3-0448-40b2-aed4-7ba6c20c6956 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:54.372412 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6883e9b3-0448-40b2-aed4-7ba6c20c6956 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:54.372687 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6883e9b3-0448-40b2-aed4-7ba6c20c6956 None admin] PATH_INFO: `/v3/users/23af7addee2640d3a2fe209ac343e817` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:54.385520 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6883e9b3-0448-40b2-aed4-7ba6c20c6956 None admin] RBAC: Authorizing `identity:get_user(user_id=23af7addee2640d3a2fe209ac343e817)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:54.387087 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6883e9b3-0448-40b2-aed4-7ba6c20c6956 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:54.388942 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 586/1182] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:53:54 2022] GET /identity/v3/users/23af7addee2640d3a2fe209ac343e817 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:53:54.394862 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef1ea4f-f551-4ee3-bf37-7c2bc9894544 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:54.495441 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef1ea4f-f551-4ee3-bf37-7c2bc9894544 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:54.502396 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:54.530302 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2d224d-6a86-413c-a47e-4be4bb018d9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:54.530944 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2d224d-6a86-413c-a47e-4be4bb018d9f None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:54.531236 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2d224d-6a86-413c-a47e-4be4bb018d9f None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:54.531517 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2d224d-6a86-413c-a47e-4be4bb018d9f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:54.597085 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2d224d-6a86-413c-a47e-4be4bb018d9f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:54.598576 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2d224d-6a86-413c-a47e-4be4bb018d9f None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:54.695333 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 597/1183] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:53:54 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:53:54.701828 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6883e9b3-0448-40b2-aed4-7ba6c20c6956 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:54.736228 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:54.800849 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6883e9b3-0448-40b2-aed4-7ba6c20c6956 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:54.834540 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818eeab9-1528-4802-b153-aef635cdee5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:54.835391 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818eeab9-1528-4802-b153-aef635cdee5a None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:54.835677 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818eeab9-1528-4802-b153-aef635cdee5a None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:54.835955 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818eeab9-1528-4802-b153-aef635cdee5a None admin] PATH_INFO: `/v3/projects/2be0e4247f5c41d2b9a5c22d443e0dbd` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:54.837121 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818eeab9-1528-4802-b153-aef635cdee5a None admin] RBAC: Authorizing `identity:get_project(project_id=2be0e4247f5c41d2b9a5c22d443e0dbd)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:54.838591 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818eeab9-1528-4802-b153-aef635cdee5a None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:54.840292 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 587/1184] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:53:54 2022] GET /identity/v3/projects/2be0e4247f5c41d2b9a5c22d443e0dbd => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:53:54.847170 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2d224d-6a86-413c-a47e-4be4bb018d9f None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:54.855276 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:54.945598 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2d224d-6a86-413c-a47e-4be4bb018d9f None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:54.980691 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07383c61-570c-429c-8dfb-b3613b432507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:54.981429 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07383c61-570c-429c-8dfb-b3613b432507 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:54.981986 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07383c61-570c-429c-8dfb-b3613b432507 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:54.982327 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07383c61-570c-429c-8dfb-b3613b432507 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:54.983290 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07383c61-570c-429c-8dfb-b3613b432507 None admin] RBAC: Adding query filter params (role.id=028379e01f9245e9b8fc56e8b9e8a6ad, scope.project.id=2be0e4247f5c41d2b9a5c22d443e0dbd, user.id=23af7addee2640d3a2fe209ac343e817) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:53:54.983805 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07383c61-570c-429c-8dfb-b3613b432507 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:54.985165 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07383c61-570c-429c-8dfb-b3613b432507 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:54.990688 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 598/1185] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:53:54 2022] GET /identity/v3/role_assignments?user.id=23af7addee2640d3a2fe209ac343e817&scope.project.id=2be0e4247f5c41d2b9a5c22d443e0dbd&role.id=028379e01f9245e9b8fc56e8b9e8a6ad => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:53:57.205863 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b05129-4c02-4388-934d-3d9342a0c3bb None None] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:57.206436 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b05129-4c02-4388-934d-3d9342a0c3bb None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:57.206436 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b05129-4c02-4388-934d-3d9342a0c3bb None None] PATH_INFO: `/` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:57.207819 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 588/1186] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:53:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:53:57.213122 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcc32bd-51d1-4cef-b546-1f492b90ae13 None None] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:57.213553 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcc32bd-51d1-4cef-b546-1f492b90ae13 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:57.213698 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcc32bd-51d1-4cef-b546-1f492b90ae13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:57.231107 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.auth.core [None req-1dcc32bd-51d1-4cef-b546-1f492b90ae13 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:53:57.232902 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-1dcc32bd-51d1-4cef-b546-1f492b90ae13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:53:57.296456 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:57.301776 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 599/1187] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:53:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:53:57.324714 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b05129-4c02-4388-934d-3d9342a0c3bb None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:57.424372 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b05129-4c02-4388-934d-3d9342a0c3bb None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:57.425911 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:57.458855 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923b4eed-7560-4b35-99b2-50bd83ecee0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:57.459578 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923b4eed-7560-4b35-99b2-50bd83ecee0f None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:57.459864 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923b4eed-7560-4b35-99b2-50bd83ecee0f None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:57.460142 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923b4eed-7560-4b35-99b2-50bd83ecee0f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:57.471917 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923b4eed-7560-4b35-99b2-50bd83ecee0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:53:57.479533 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-923b4eed-7560-4b35-99b2-50bd83ecee0f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:53:57.479533 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:53:57.479533 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:53:57.479533 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:53:57.479533 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:53:57.479533 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:53:57.479533 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:53:57.479533 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:53:57.479533 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:53:57.479533 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:53:57.479533 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:53:57.479533 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:53:57.479533 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:53:57.479533 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:53:57.479533 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:53:57.479533 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:53:57.480759 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:53:57.480759 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:53:57.480759 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:53:57.480759 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:53:57.480759 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:53:57.480759 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:53:57.480759 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:53:57.480759 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:53:57.480759 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:53:57.480759 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:53:57.480759 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:53:57.480759 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:53:57.480759 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:53:57.480759 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:53:57.480759 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:53:57.480759 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:53:57.481989 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:53:57.481989 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:53:57.481989 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:53:57.481989 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:53:57.481989 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:53:57.481989 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:53:57.481989 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:53:57.481989 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:53:57.481989 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:53:57.481989 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:53:57.481989 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:53:57.481989 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:53:57.481989 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 589/1188] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:53:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:53:57.487308 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcc32bd-51d1-4cef-b546-1f492b90ae13 None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:57.584096 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcc32bd-51d1-4cef-b546-1f492b90ae13 None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:57.617832 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11566a7e-ac8c-431b-ad19-5d38bbb8fac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:57.618595 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11566a7e-ac8c-431b-ad19-5d38bbb8fac5 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:57.618887 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11566a7e-ac8c-431b-ad19-5d38bbb8fac5 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:57.619170 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11566a7e-ac8c-431b-ad19-5d38bbb8fac5 None admin] PATH_INFO: `/v3/roles` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:57.619922 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11566a7e-ac8c-431b-ad19-5d38bbb8fac5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:53:57.620237 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11566a7e-ac8c-431b-ad19-5d38bbb8fac5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:57.621622 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11566a7e-ac8c-431b-ad19-5d38bbb8fac5 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:57.629585 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 600/1189] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:53:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:53:57.635501 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923b4eed-7560-4b35-99b2-50bd83ecee0f None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:57.736036 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923b4eed-7560-4b35-99b2-50bd83ecee0f None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:57.768959 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2717ca-2868-458a-9fc0-218e7af4682c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:57.769806 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2717ca-2868-458a-9fc0-218e7af4682c None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:57.770122 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2717ca-2868-458a-9fc0-218e7af4682c None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:57.770399 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2717ca-2868-458a-9fc0-218e7af4682c None admin] PATH_INFO: `/v3/users/23af7addee2640d3a2fe209ac343e817` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:57.771576 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2717ca-2868-458a-9fc0-218e7af4682c None admin] RBAC: Authorizing `identity:get_user(user_id=23af7addee2640d3a2fe209ac343e817)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:57.773002 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2717ca-2868-458a-9fc0-218e7af4682c None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:57.774637 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 590/1190] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:53:57 2022] GET /identity/v3/users/23af7addee2640d3a2fe209ac343e817 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:53:57.781011 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11566a7e-ac8c-431b-ad19-5d38bbb8fac5 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:57.842205 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:57.880691 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11566a7e-ac8c-431b-ad19-5d38bbb8fac5 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:57.914507 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2326cb0-4b6d-4fe9-a235-3be1bbbf48ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:57.915370 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2326cb0-4b6d-4fe9-a235-3be1bbbf48ae None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:57.915663 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2326cb0-4b6d-4fe9-a235-3be1bbbf48ae None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:57.915948 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2326cb0-4b6d-4fe9-a235-3be1bbbf48ae None admin] PATH_INFO: `/v3/projects/2be0e4247f5c41d2b9a5c22d443e0dbd` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:57.917128 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2326cb0-4b6d-4fe9-a235-3be1bbbf48ae None admin] RBAC: Authorizing `identity:get_project(project_id=2be0e4247f5c41d2b9a5c22d443e0dbd)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:57.918643 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2326cb0-4b6d-4fe9-a235-3be1bbbf48ae None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:57.920341 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 601/1191] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:53:57 2022] GET /identity/v3/projects/2be0e4247f5c41d2b9a5c22d443e0dbd => generated 361 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:53:57.926252 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2717ca-2868-458a-9fc0-218e7af4682c None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:57.956702 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:58.026259 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2717ca-2868-458a-9fc0-218e7af4682c None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:58.058640 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc526ad-4ab8-4ddd-b5f1-211ccc04e559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:58.060152 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc526ad-4ab8-4ddd-b5f1-211ccc04e559 None admin] REQUEST_METHOD: `PUT` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:58.060477 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc526ad-4ab8-4ddd-b5f1-211ccc04e559 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:58.060757 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc526ad-4ab8-4ddd-b5f1-211ccc04e559 None admin] PATH_INFO: `/v3/projects/2be0e4247f5c41d2b9a5c22d443e0dbd/users/23af7addee2640d3a2fe209ac343e817/roles/028379e01f9245e9b8fc56e8b9e8a6ad` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:58.062726 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc526ad-4ab8-4ddd-b5f1-211ccc04e559 None admin] RBAC: Authorizing `identity:create_grant(project_id=2be0e4247f5c41d2b9a5c22d443e0dbd, user_id=23af7addee2640d3a2fe209ac343e817, role_id=028379e01f9245e9b8fc56e8b9e8a6ad)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:58.064172 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc526ad-4ab8-4ddd-b5f1-211ccc04e559 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:58.077593 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 591/1192] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 14:53:57 2022] PUT /identity/v3/projects/2be0e4247f5c41d2b9a5c22d443e0dbd/users/23af7addee2640d3a2fe209ac343e817/roles/028379e01f9245e9b8fc56e8b9e8a6ad => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:54:00.310593 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3941a88c-6e19-4ef8-b632-00fd96eba11f None None] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:54:00.311180 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3941a88c-6e19-4ef8-b632-00fd96eba11f None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:54:00.311180 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3941a88c-6e19-4ef8-b632-00fd96eba11f None None] PATH_INFO: `/` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:54:00.312543 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 602/1193] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:54:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:54:00.318078 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67782d82-c852-4de2-99ba-df61aecef2b8 None None] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:54:00.318533 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67782d82-c852-4de2-99ba-df61aecef2b8 None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:54:00.318645 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67782d82-c852-4de2-99ba-df61aecef2b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:54:00.335561 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.auth.core [None req-67782d82-c852-4de2-99ba-df61aecef2b8 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:54:00.336769 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.fernet_utils [None req-67782d82-c852-4de2-99ba-df61aecef2b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:54:00.402404 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 592/1194] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:54:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:54:00.426992 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3941a88c-6e19-4ef8-b632-00fd96eba11f None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:54:00.528823 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3941a88c-6e19-4ef8-b632-00fd96eba11f None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:54:00.556955 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:54:00.563318 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18fee85-79f1-44e5-a96a-cb924a85ec45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:54:00.564052 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18fee85-79f1-44e5-a96a-cb924a85ec45 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:54:00.564342 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18fee85-79f1-44e5-a96a-cb924a85ec45 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:54:00.564623 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18fee85-79f1-44e5-a96a-cb924a85ec45 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:54:00.574990 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18fee85-79f1-44e5-a96a-cb924a85ec45 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:54:00.582613 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-f18fee85-79f1-44e5-a96a-cb924a85ec45 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:54:00.582613 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:54:00.582613 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:54:00.582613 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:54:00.582613 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:54:00.582613 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:54:00.582613 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:54:00.582613 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:54:00.582613 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:54:00.582613 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:54:00.582613 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:54:00.582613 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:54:00.582613 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:54:00.582613 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:54:00.582613 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:54:00.582613 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:54:00.583867 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:54:00.583867 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:54:00.583867 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:54:00.583867 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:54:00.583867 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:54:00.583867 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:54:00.583867 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:54:00.583867 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:54:00.583867 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:54:00.583867 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:54:00.583867 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:54:00.583867 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:54:00.583867 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:54:00.583867 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:54:00.583867 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:54:00.583867 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:54:00.585092 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:54:00.585092 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:54:00.585092 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:54:00.585092 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:54:00.585092 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:54:00.585092 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:54:00.585092 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:54:00.585092 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:54:00.585092 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:54:00.585092 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:54:00.585092 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:54:00.585092 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:54:00.585092 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 603/1195] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:54:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:54:00.590000 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67782d82-c852-4de2-99ba-df61aecef2b8 None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:54:00.611925 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:54:00.689223 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67782d82-c852-4de2-99ba-df61aecef2b8 None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:54:00.723072 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048325fa-db24-4a50-8c69-b5b201fe0cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:54:00.723735 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048325fa-db24-4a50-8c69-b5b201fe0cb1 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:54:00.724020 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048325fa-db24-4a50-8c69-b5b201fe0cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:54:00.724295 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048325fa-db24-4a50-8c69-b5b201fe0cb1 None admin] PATH_INFO: `/v3/roles` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:54:00.725031 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048325fa-db24-4a50-8c69-b5b201fe0cb1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:54:00.725336 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048325fa-db24-4a50-8c69-b5b201fe0cb1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:54:00.726722 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048325fa-db24-4a50-8c69-b5b201fe0cb1 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:54:00.734903 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 593/1196] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:54:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:54:00.740715 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18fee85-79f1-44e5-a96a-cb924a85ec45 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:54:00.841449 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18fee85-79f1-44e5-a96a-cb924a85ec45 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:54:00.873856 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5744a2-d0ab-4a0c-8b07-ee6fef489034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:54:00.874718 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5744a2-d0ab-4a0c-8b07-ee6fef489034 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:54:00.875009 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5744a2-d0ab-4a0c-8b07-ee6fef489034 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:54:00.875291 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5744a2-d0ab-4a0c-8b07-ee6fef489034 None admin] PATH_INFO: `/v3/users/23af7addee2640d3a2fe209ac343e817` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:54:00.876489 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5744a2-d0ab-4a0c-8b07-ee6fef489034 None admin] RBAC: Authorizing `identity:get_user(user_id=23af7addee2640d3a2fe209ac343e817)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:54:00.877964 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5744a2-d0ab-4a0c-8b07-ee6fef489034 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:54:00.879590 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 604/1197] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 14:54:00 2022] GET /identity/v3/users/23af7addee2640d3a2fe209ac343e817 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:54:00.885273 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048325fa-db24-4a50-8c69-b5b201fe0cb1 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:54:00.966270 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:54:00.984572 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048325fa-db24-4a50-8c69-b5b201fe0cb1 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:54:01.018450 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f1bcc9-99a7-4260-ac6a-3045d14033c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:54:01.019091 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f1bcc9-99a7-4260-ac6a-3045d14033c7 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:54:01.019377 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f1bcc9-99a7-4260-ac6a-3045d14033c7 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:54:01.019655 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f1bcc9-99a7-4260-ac6a-3045d14033c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:54:01.085090 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f1bcc9-99a7-4260-ac6a-3045d14033c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:54:01.086563 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f1bcc9-99a7-4260-ac6a-3045d14033c7 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:54:01.170217 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:54:01.189350 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 594/1198] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:54:00 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:54:01.195554 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5744a2-d0ab-4a0c-8b07-ee6fef489034 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:54:01.253284 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:54:01.294930 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5744a2-d0ab-4a0c-8b07-ee6fef489034 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:54:01.328725 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72df819e-71cb-4db0-b522-ecbb3ecc75d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:54:01.329589 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72df819e-71cb-4db0-b522-ecbb3ecc75d1 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:54:01.329943 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72df819e-71cb-4db0-b522-ecbb3ecc75d1 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:54:01.330192 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72df819e-71cb-4db0-b522-ecbb3ecc75d1 None admin] PATH_INFO: `/v3/projects/2be0e4247f5c41d2b9a5c22d443e0dbd` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:54:01.331375 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72df819e-71cb-4db0-b522-ecbb3ecc75d1 None admin] RBAC: Authorizing `identity:get_project(project_id=2be0e4247f5c41d2b9a5c22d443e0dbd)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:54:01.332808 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72df819e-71cb-4db0-b522-ecbb3ecc75d1 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:54:01.334527 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 605/1199] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 14:54:01 2022] GET /identity/v3/projects/2be0e4247f5c41d2b9a5c22d443e0dbd => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:54:01.340848 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f1bcc9-99a7-4260-ac6a-3045d14033c7 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:54:01.440008 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f1bcc9-99a7-4260-ac6a-3045d14033c7 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:54:01.473642 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f961fb-78b5-468b-9df3-682a999e8c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:54:01.474320 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f961fb-78b5-468b-9df3-682a999e8c59 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:54:01.474612 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f961fb-78b5-468b-9df3-682a999e8c59 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:54:01.474898 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f961fb-78b5-468b-9df3-682a999e8c59 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:54:01.475726 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f961fb-78b5-468b-9df3-682a999e8c59 None admin] RBAC: Adding query filter params (role.id=028379e01f9245e9b8fc56e8b9e8a6ad, scope.project.id=2be0e4247f5c41d2b9a5c22d443e0dbd, user.id=23af7addee2640d3a2fe209ac343e817) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:54:01.476045 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f961fb-78b5-468b-9df3-682a999e8c59 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:54:01.477495 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f961fb-78b5-468b-9df3-682a999e8c59 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:54:01.484043 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 595/1200] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:54:01 2022] GET /identity/v3/role_assignments?user.id=23af7addee2640d3a2fe209ac343e817&scope.project.id=2be0e4247f5c41d2b9a5c22d443e0dbd&role.id=028379e01f9245e9b8fc56e8b9e8a6ad => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:54:11.323881 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=37 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:54:11.480692 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=77 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:54:59.977594 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eb32a8-0bb0-483b-8882-86d2a472633e None None] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:54:59.978413 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eb32a8-0bb0-483b-8882-86d2a472633e None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:54:59.978413 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eb32a8-0bb0-483b-8882-86d2a472633e None None] PATH_INFO: `/` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:54:59.979640 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 606/1201] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:54:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:54:59.985257 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887934d2-64cf-4a50-b42e-5f4dcf47099c None None] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:54:59.985728 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887934d2-64cf-4a50-b42e-5f4dcf47099c None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:54:59.985833 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887934d2-64cf-4a50-b42e-5f4dcf47099c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:55:00.004116 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.auth.core [None req-887934d2-64cf-4a50-b42e-5f4dcf47099c None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:55:00.005333 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.fernet_utils [None req-887934d2-64cf-4a50-b42e-5f4dcf47099c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:55:00.078802 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 596/1202] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:54:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:55:00.103544 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eb32a8-0bb0-483b-8882-86d2a472633e None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:55:00.231893 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eb32a8-0bb0-483b-8882-86d2a472633e None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:55:00.267960 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8966b57-2ebf-420a-8f7a-f72457a2cb14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:55:00.268766 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8966b57-2ebf-420a-8f7a-f72457a2cb14 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:55:00.269056 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8966b57-2ebf-420a-8f7a-f72457a2cb14 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:55:00.269336 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8966b57-2ebf-420a-8f7a-f72457a2cb14 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:55:00.280684 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8966b57-2ebf-420a-8f7a-f72457a2cb14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:55:00.282195 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8966b57-2ebf-420a-8f7a-f72457a2cb14 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:55:00.302535 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-f8966b57-2ebf-420a-8f7a-f72457a2cb14 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:55:00.302535 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:55:00.302535 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:55:00.302535 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:55:00.302535 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:55:00.302535 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:55:00.302535 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:55:00.302535 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:55:00.302535 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:55:00.302535 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:55:00.302535 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:55:00.302535 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:55:00.302535 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:55:00.302535 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:55:00.302535 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:55:00.303775 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:55:00.303775 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:55:00.303775 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:55:00.303775 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:55:00.303775 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:55:00.303775 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:55:00.303775 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:55:00.303775 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:55:00.303775 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:55:00.303775 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:55:00.303775 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:55:00.303775 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:55:00.303775 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:55:00.303775 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:55:00.303775 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:55:00.303775 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:55:00.303775 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:55:00.304991 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:55:00.304991 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:55:00.304991 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:55:00.304991 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:55:00.306051 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 607/1203] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:55:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:55:00.311011 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887934d2-64cf-4a50-b42e-5f4dcf47099c None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:55:00.436165 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887934d2-64cf-4a50-b42e-5f4dcf47099c None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:55:00.468084 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:55:00.474740 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1b5f66-4499-4c33-a020-bbd67aaa3394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:55:00.475419 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1b5f66-4499-4c33-a020-bbd67aaa3394 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:55:00.475697 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1b5f66-4499-4c33-a020-bbd67aaa3394 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:55:00.475976 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1b5f66-4499-4c33-a020-bbd67aaa3394 None admin] PATH_INFO: `/v3/domains` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:55:00.476830 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1b5f66-4499-4c33-a020-bbd67aaa3394 None admin] RBAC: Adding query filter params (name=Default) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:55:00.477039 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1b5f66-4499-4c33-a020-bbd67aaa3394 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:55:00.479038 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1b5f66-4499-4c33-a020-bbd67aaa3394 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:55:00.491439 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 597/1204] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:55:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:55:00.497993 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8966b57-2ebf-420a-8f7a-f72457a2cb14 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:55:00.561078 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:55:00.625457 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8966b57-2ebf-420a-8f7a-f72457a2cb14 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:55:00.684956 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aba63a-731f-484b-8686-fcce5e2bfd50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:55:00.685645 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aba63a-731f-484b-8686-fcce5e2bfd50 None admin] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:55:00.685993 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aba63a-731f-484b-8686-fcce5e2bfd50 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:55:00.686280 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aba63a-731f-484b-8686-fcce5e2bfd50 None admin] PATH_INFO: `/v3/users` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:55:00.686999 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57aba63a-731f-484b-8686-fcce5e2bfd50 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:55:00.688441 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57aba63a-731f-484b-8686-fcce5e2bfd50 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:55:00.715272 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 608/1205] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 19 14:55:00 2022] POST /identity/v3/users => generated 251 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:55:03.134444 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd8d807-38a8-4bd9-89c3-c067796632f2 None None] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:55:03.135164 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd8d807-38a8-4bd9-89c3-c067796632f2 None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:55:03.135164 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd8d807-38a8-4bd9-89c3-c067796632f2 None None] PATH_INFO: `/` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:55:03.136695 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 598/1206] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:55:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:55:03.142271 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8211f0-ea61-4a67-ac0e-3918ed0786b8 None None] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:55:03.142744 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8211f0-ea61-4a67-ac0e-3918ed0786b8 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:55:03.142847 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8211f0-ea61-4a67-ac0e-3918ed0786b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:55:03.161295 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.auth.core [None req-8f8211f0-ea61-4a67-ac0e-3918ed0786b8 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:55:03.162528 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-8f8211f0-ea61-4a67-ac0e-3918ed0786b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:55:03.243994 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 609/1207] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:55:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:55:03.269076 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd8d807-38a8-4bd9-89c3-c067796632f2 None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:55:03.395344 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd8d807-38a8-4bd9-89c3-c067796632f2 None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:55:03.436878 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f64e4d-b139-4241-ac99-cc5f55f25bf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:55:03.437748 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f64e4d-b139-4241-ac99-cc5f55f25bf3 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:55:03.438069 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f64e4d-b139-4241-ac99-cc5f55f25bf3 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:55:03.438352 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f64e4d-b139-4241-ac99-cc5f55f25bf3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:55:03.452386 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f64e4d-b139-4241-ac99-cc5f55f25bf3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:55:03.459961 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-72f64e4d-b139-4241-ac99-cc5f55f25bf3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:55:03.459961 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:55:03.459961 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:55:03.459961 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:55:03.459961 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:55:03.459961 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:55:03.459961 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:55:03.459961 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:55:03.459961 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:55:03.459961 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:55:03.459961 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:55:03.459961 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:55:03.459961 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:55:03.459961 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:55:03.459961 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:55:03.459961 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:55:03.461169 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:55:03.461169 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:55:03.461169 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:55:03.461169 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:55:03.461169 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:55:03.461169 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:55:03.461169 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:55:03.461169 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:55:03.461169 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:55:03.461169 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:55:03.461169 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:55:03.461169 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:55:03.461169 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:55:03.461169 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:55:03.461169 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:55:03.461169 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:55:03.467827 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:55:03.467827 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:55:03.467827 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:55:03.467827 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:55:03.467827 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:55:03.467827 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:55:03.467827 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:55:03.467827 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:55:03.467827 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:55:03.467827 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:55:03.467827 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:55:03.467827 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:55:03.467827 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 599/1208] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 14:55:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:55:03.469141 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8211f0-ea61-4a67-ac0e-3918ed0786b8 None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:55:03.520563 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:55:03.593110 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8211f0-ea61-4a67-ac0e-3918ed0786b8 None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:55:03.632291 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e535ee84-9f24-4e1a-9739-148be7ccd9e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:55:03.632972 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e535ee84-9f24-4e1a-9739-148be7ccd9e7 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:55:03.633266 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e535ee84-9f24-4e1a-9739-148be7ccd9e7 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:55:03.633553 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e535ee84-9f24-4e1a-9739-148be7ccd9e7 None admin] PATH_INFO: `/v3/roles` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:55:03.634479 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e535ee84-9f24-4e1a-9739-148be7ccd9e7 None admin] RBAC: Adding query filter params (name=service) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:55:03.635293 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e535ee84-9f24-4e1a-9739-148be7ccd9e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:55:03.636684 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e535ee84-9f24-4e1a-9739-148be7ccd9e7 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:55:03.646526 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 610/1209] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 19 14:55:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:55:03.654148 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f64e4d-b139-4241-ac99-cc5f55f25bf3 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:55:03.721832 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:55:03.789228 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f64e4d-b139-4241-ac99-cc5f55f25bf3 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:55:03.832654 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eae8cb9-e0ff-400d-a061-509da580104a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:55:03.833472 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eae8cb9-e0ff-400d-a061-509da580104a None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:55:03.833764 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eae8cb9-e0ff-400d-a061-509da580104a None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:55:03.834082 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eae8cb9-e0ff-400d-a061-509da580104a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:55:03.846153 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eae8cb9-e0ff-400d-a061-509da580104a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:55:03.847633 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eae8cb9-e0ff-400d-a061-509da580104a None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:55:03.866951 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-2eae8cb9-e0ff-400d-a061-509da580104a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:55:03.866951 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:55:03.866951 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:55:03.866951 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:55:03.866951 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:55:03.866951 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:55:03.866951 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:55:03.866951 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:55:03.866951 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:55:03.866951 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:55:03.866951 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:55:03.866951 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:55:03.866951 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:55:03.866951 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:55:03.866951 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:55:03.868344 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:55:03.868344 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:55:03.868344 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:55:03.868344 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:55:03.868344 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:55:03.868344 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:55:03.868344 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:55:03.868344 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:55:03.868344 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:55:03.868344 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:55:03.868344 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:55:03.868344 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:55:03.868344 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:55:03.868344 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:55:03.868344 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:55:03.868344 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:55:03.868344 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:55:03.869697 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:55:03.869697 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:55:03.869697 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:55:03.869697 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:55:03.869697 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 600/1210] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:55:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:55:03.874217 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e535ee84-9f24-4e1a-9739-148be7ccd9e7 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:55:03.995837 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:55:03.999542 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e535ee84-9f24-4e1a-9739-148be7ccd9e7 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:55:04.043873 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972c7807-c7d1-4dbf-a82a-d698e5d24b46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:55:04.044829 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972c7807-c7d1-4dbf-a82a-d698e5d24b46 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:55:04.045295 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972c7807-c7d1-4dbf-a82a-d698e5d24b46 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:55:04.045673 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972c7807-c7d1-4dbf-a82a-d698e5d24b46 None admin] PATH_INFO: `/v3/domains` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:55:04.046531 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972c7807-c7d1-4dbf-a82a-d698e5d24b46 None admin] RBAC: Adding query filter params (name=Default) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:55:04.047069 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972c7807-c7d1-4dbf-a82a-d698e5d24b46 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:55:04.048872 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972c7807-c7d1-4dbf-a82a-d698e5d24b46 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:55:04.068262 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 611/1211] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:55:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:55:04.075507 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eae8cb9-e0ff-400d-a061-509da580104a None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:55:04.209815 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eae8cb9-e0ff-400d-a061-509da580104a None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:55:04.211540 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:55:04.248674 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2035d8c-5f40-4396-a3b8-c8edf2aa8b90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:55:04.249458 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2035d8c-5f40-4396-a3b8-c8edf2aa8b90 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:55:04.249686 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2035d8c-5f40-4396-a3b8-c8edf2aa8b90 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:55:04.250083 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2035d8c-5f40-4396-a3b8-c8edf2aa8b90 None admin] PATH_INFO: `/v3/users/placement` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:55:04.257761 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2035d8c-5f40-4396-a3b8-c8edf2aa8b90 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:55:04.259287 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2035d8c-5f40-4396-a3b8-c8edf2aa8b90 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:55:04.276819 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-d2035d8c-5f40-4396-a3b8-c8edf2aa8b90 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 14:55:04.276819 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:55:04.276819 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:55:04.276819 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:55:04.276819 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:55:04.276819 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:55:04.276819 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:55:04.276819 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:55:04.276819 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:55:04.276819 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:55:04.276819 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:55:04.276819 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:55:04.276819 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:55:04.276819 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:55:04.276819 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:55:04.276819 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:55:04.278455 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:55:04.278455 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:55:04.278455 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:55:04.278455 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:55:04.278455 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:55:04.278455 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:55:04.278455 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:55:04.278455 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:55:04.278455 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:55:04.278455 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:55:04.278455 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:55:04.278455 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:55:04.278455 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:55:04.278455 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:55:04.278455 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:55:04.278455 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:55:04.278455 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:55:04.278455 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:55:04.279856 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:55:04.279856 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:55:04.279856 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:55:04.279856 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:55:04.279856 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:55:04.279856 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:55:04.279856 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:55:04.279856 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:55:04.279856 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 14:55:04.279856 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:55:04.279856 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 601/1212] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:55:04 2022] GET /identity/v3/users/placement => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:55:04.284825 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972c7807-c7d1-4dbf-a82a-d698e5d24b46 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:55:04.411156 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972c7807-c7d1-4dbf-a82a-d698e5d24b46 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:55:04.456983 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b081c44-d365-4b0c-ad28-f739fb91ac5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:55:04.456983 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b081c44-d365-4b0c-ad28-f739fb91ac5c None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:55:04.457439 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b081c44-d365-4b0c-ad28-f739fb91ac5c None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:55:04.457819 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b081c44-d365-4b0c-ad28-f739fb91ac5c None admin] PATH_INFO: `/v3/users` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:55:04.458900 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b081c44-d365-4b0c-ad28-f739fb91ac5c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:55:04.459378 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b081c44-d365-4b0c-ad28-f739fb91ac5c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:55:04.461053 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b081c44-d365-4b0c-ad28-f739fb91ac5c None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:55:04.482361 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 612/1213] 10.222.0.57 () {56 vars in 1143 bytes} [Thu May 19 14:55:04 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:55:04.488697 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2035d8c-5f40-4396-a3b8-c8edf2aa8b90 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:55:04.625291 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2035d8c-5f40-4396-a3b8-c8edf2aa8b90 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:55:04.674877 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f4c3c0-dc49-4279-b814-bcf6850b0bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:55:04.675587 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f4c3c0-dc49-4279-b814-bcf6850b0bbf None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:55:04.675947 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f4c3c0-dc49-4279-b814-bcf6850b0bbf None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:55:04.676255 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f4c3c0-dc49-4279-b814-bcf6850b0bbf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:55:04.713628 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:55:04.759860 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f4c3c0-dc49-4279-b814-bcf6850b0bbf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:55:04.761595 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f4c3c0-dc49-4279-b814-bcf6850b0bbf None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:55:04.878634 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 602/1214] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:55:04 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:55:04.885266 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b081c44-d365-4b0c-ad28-f739fb91ac5c None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:55:04.909366 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:55:05.003932 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b081c44-d365-4b0c-ad28-f739fb91ac5c None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:55:05.044474 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e99e28-aa32-4181-aca9-3091c21ad7a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:55:05.045306 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e99e28-aa32-4181-aca9-3091c21ad7a8 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:55:05.045575 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e99e28-aa32-4181-aca9-3091c21ad7a8 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:55:05.045871 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e99e28-aa32-4181-aca9-3091c21ad7a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:55:05.058608 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e99e28-aa32-4181-aca9-3091c21ad7a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:55:05.060304 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e99e28-aa32-4181-aca9-3091c21ad7a8 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:55:05.081091 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-77e99e28-aa32-4181-aca9-3091c21ad7a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:55:05.081091 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:55:05.081091 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:55:05.081091 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:55:05.081091 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:55:05.081091 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:55:05.081091 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:55:05.081091 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:55:05.081091 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:55:05.081091 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:55:05.081091 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:55:05.081091 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:55:05.081091 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:55:05.081091 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:55:05.081091 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:55:05.082553 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:55:05.082553 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:55:05.082553 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:55:05.082553 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:55:05.082553 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:55:05.082553 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:55:05.082553 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:55:05.082553 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:55:05.082553 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:55:05.082553 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:55:05.082553 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:55:05.082553 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:55:05.082553 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:55:05.082553 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:55:05.082553 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:55:05.082553 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:55:05.082553 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:55:05.083811 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:55:05.083811 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:55:05.083811 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:55:05.083811 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:55:05.086002 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 613/1215] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:55:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:55:05.089000 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f4c3c0-dc49-4279-b814-bcf6850b0bbf None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:55:05.166098 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:55:05.214748 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f4c3c0-dc49-4279-b814-bcf6850b0bbf None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:55:05.261341 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155335f6-48c6-4d6d-ac08-42127c7b2152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:55:05.262069 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155335f6-48c6-4d6d-ac08-42127c7b2152 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:55:05.262361 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155335f6-48c6-4d6d-ac08-42127c7b2152 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:55:05.262647 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155335f6-48c6-4d6d-ac08-42127c7b2152 None admin] PATH_INFO: `/v3/domains` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:55:05.263623 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155335f6-48c6-4d6d-ac08-42127c7b2152 None admin] RBAC: Adding query filter params (name=Default) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:55:05.263949 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155335f6-48c6-4d6d-ac08-42127c7b2152 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:55:05.265472 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155335f6-48c6-4d6d-ac08-42127c7b2152 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:55:05.282187 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 603/1216] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:55:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:55:05.289115 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e99e28-aa32-4181-aca9-3091c21ad7a8 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:55:05.382675 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:55:05.434516 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e99e28-aa32-4181-aca9-3091c21ad7a8 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:55:05.490392 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc80c1f-a67c-4381-be2b-f6d5fa06a3d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:55:05.491307 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc80c1f-a67c-4381-be2b-f6d5fa06a3d5 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:55:05.491629 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc80c1f-a67c-4381-be2b-f6d5fa06a3d5 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:55:05.491950 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc80c1f-a67c-4381-be2b-f6d5fa06a3d5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:55:05.501204 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc80c1f-a67c-4381-be2b-f6d5fa06a3d5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:55:05.502906 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc80c1f-a67c-4381-be2b-f6d5fa06a3d5 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:55:05.519412 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-dbc80c1f-a67c-4381-be2b-f6d5fa06a3d5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:55:05.519412 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:55:05.519412 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:55:05.519412 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:55:05.519412 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:55:05.519412 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:55:05.519412 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:55:05.519412 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:55:05.519412 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:55:05.519412 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:55:05.519412 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:55:05.519412 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:55:05.519412 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:55:05.519412 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:55:05.519412 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:55:05.520883 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:55:05.520883 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:55:05.520883 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:55:05.520883 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:55:05.520883 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:55:05.520883 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:55:05.520883 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:55:05.520883 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:55:05.520883 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:55:05.520883 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:55:05.520883 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:55:05.520883 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:55:05.520883 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:55:05.520883 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:55:05.520883 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:55:05.520883 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:55:05.520883 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:55:05.522322 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:55:05.522322 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:55:05.522322 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:55:05.522322 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:55:05.522322 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:55:05.522322 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:55:05.522322 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:55:05.522322 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:55:05.523881 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 614/1217] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:55:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:55:05.530899 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155335f6-48c6-4d6d-ac08-42127c7b2152 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:55:05.671613 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155335f6-48c6-4d6d-ac08-42127c7b2152 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:55:05.699822 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:55:05.717681 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a2d314-b846-4209-9e0b-2086aab4d0fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:55:05.718546 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a2d314-b846-4209-9e0b-2086aab4d0fa None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:55:05.718925 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a2d314-b846-4209-9e0b-2086aab4d0fa None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:55:05.719301 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a2d314-b846-4209-9e0b-2086aab4d0fa None admin] PATH_INFO: `/v3/projects` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:55:05.720199 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a2d314-b846-4209-9e0b-2086aab4d0fa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:55:05.720616 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a2d314-b846-4209-9e0b-2086aab4d0fa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:55:05.722205 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a2d314-b846-4209-9e0b-2086aab4d0fa None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:55:05.738786 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 604/1218] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:55:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:55:05.745637 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc80c1f-a67c-4381-be2b-f6d5fa06a3d5 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:55:05.861401 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc80c1f-a67c-4381-be2b-f6d5fa06a3d5 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:55:05.894479 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:55:05.903002 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e86614e-2b5f-448f-a0ca-f39057052fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:55:05.903702 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e86614e-2b5f-448f-a0ca-f39057052fd7 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:55:05.904003 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e86614e-2b5f-448f-a0ca-f39057052fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:55:05.904271 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e86614e-2b5f-448f-a0ca-f39057052fd7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:55:05.905127 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e86614e-2b5f-448f-a0ca-f39057052fd7 None admin] RBAC: Adding query filter params (role.id=08f16502894349708527e30032247468, scope.project.id=fc99d742ae004954b6d841293b5fa79c, user.id=e80cdca4cf964f308a7618f89cce7b1a) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:55:05.905446 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e86614e-2b5f-448f-a0ca-f39057052fd7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:55:05.906948 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e86614e-2b5f-448f-a0ca-f39057052fd7 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:55:05.913176 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 615/1219] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:55:05 2022] GET /identity/v3/role_assignments?user.id=e80cdca4cf964f308a7618f89cce7b1a&scope.project.id=fc99d742ae004954b6d841293b5fa79c&role.id=08f16502894349708527e30032247468 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:55:08.231390 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4ff222-5681-48b0-9523-d93afcdd0516 None None] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:55:08.231951 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4ff222-5681-48b0-9523-d93afcdd0516 None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:55:08.231951 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4ff222-5681-48b0-9523-d93afcdd0516 None None] PATH_INFO: `/` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:55:08.233509 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 605/1220] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:55:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:55:08.238587 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8910c60b-630b-492d-ac3c-4b3399880279 None None] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:55:08.239034 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8910c60b-630b-492d-ac3c-4b3399880279 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:55:08.239173 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8910c60b-630b-492d-ac3c-4b3399880279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:55:08.258250 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.auth.core [None req-8910c60b-630b-492d-ac3c-4b3399880279 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:55:08.259667 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-8910c60b-630b-492d-ac3c-4b3399880279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:55:08.349098 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 616/1221] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:55:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:55:08.375458 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4ff222-5681-48b0-9523-d93afcdd0516 None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:55:08.498678 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4ff222-5681-48b0-9523-d93afcdd0516 None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:55:08.534868 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23609882-f859-4ab1-a5e9-6742ec4fa878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:55:08.536757 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23609882-f859-4ab1-a5e9-6742ec4fa878 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:55:08.536757 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23609882-f859-4ab1-a5e9-6742ec4fa878 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:55:08.536757 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23609882-f859-4ab1-a5e9-6742ec4fa878 None admin] PATH_INFO: `/v3/roles/service` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:55:08.547997 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23609882-f859-4ab1-a5e9-6742ec4fa878 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:55:08.557110 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-23609882-f859-4ab1-a5e9-6742ec4fa878 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:55:08.557110 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:55:08.557110 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:55:08.557110 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:55:08.557110 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:55:08.557110 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:55:08.557110 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:55:08.557110 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:55:08.557110 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:55:08.557110 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:55:08.557110 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:55:08.557110 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:55:08.557110 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:55:08.557110 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:55:08.557110 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:55:08.557110 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:55:08.558475 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:55:08.558475 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:55:08.558475 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:55:08.558475 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:55:08.558475 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:55:08.558475 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:55:08.558475 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:55:08.558475 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:55:08.558475 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:55:08.558475 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:55:08.558475 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:55:08.558475 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:55:08.558475 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:55:08.558475 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:55:08.558475 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:55:08.558475 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:55:08.559880 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:55:08.559880 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:55:08.559880 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:55:08.559880 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:55:08.559880 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:55:08.559880 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:55:08.559880 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:55:08.559880 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:55:08.559880 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:55:08.559880 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:55:08.559880 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:55:08.559880 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:55:08.559880 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 606/1222] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 14:55:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:55:08.564639 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8910c60b-630b-492d-ac3c-4b3399880279 None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:55:08.678180 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8910c60b-630b-492d-ac3c-4b3399880279 None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:55:08.705895 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:55:08.717658 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dd879b-d534-48c3-a392-40765c0e926c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:55:08.718377 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dd879b-d534-48c3-a392-40765c0e926c None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:55:08.718672 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dd879b-d534-48c3-a392-40765c0e926c None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:55:08.718956 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dd879b-d534-48c3-a392-40765c0e926c None admin] PATH_INFO: `/v3/roles` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:55:08.719725 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dd879b-d534-48c3-a392-40765c0e926c None admin] RBAC: Adding query filter params (name=service) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:55:08.720040 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dd879b-d534-48c3-a392-40765c0e926c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:55:08.721422 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dd879b-d534-48c3-a392-40765c0e926c None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:55:08.730556 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 617/1223] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 19 14:55:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:55:08.736602 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23609882-f859-4ab1-a5e9-6742ec4fa878 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:55:08.786553 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:55:08.850315 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23609882-f859-4ab1-a5e9-6742ec4fa878 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:55:08.888457 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89896c97-6f4d-4f17-8754-0971cf3ca3f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:55:08.888992 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89896c97-6f4d-4f17-8754-0971cf3ca3f8 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:55:08.889279 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89896c97-6f4d-4f17-8754-0971cf3ca3f8 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:55:08.889569 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89896c97-6f4d-4f17-8754-0971cf3ca3f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:55:08.902432 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89896c97-6f4d-4f17-8754-0971cf3ca3f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:55:08.904026 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89896c97-6f4d-4f17-8754-0971cf3ca3f8 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:55:08.922931 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-89896c97-6f4d-4f17-8754-0971cf3ca3f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:55:08.922931 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:55:08.922931 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:55:08.922931 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:55:08.922931 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:55:08.922931 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:55:08.922931 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:55:08.922931 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:55:08.922931 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:55:08.922931 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:55:08.922931 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:55:08.922931 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:55:08.922931 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:55:08.922931 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:55:08.922931 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:55:08.924261 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:55:08.924261 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:55:08.924261 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:55:08.924261 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:55:08.924261 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:55:08.924261 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:55:08.924261 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:55:08.924261 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:55:08.924261 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:55:08.924261 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:55:08.924261 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:55:08.924261 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:55:08.924261 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:55:08.924261 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:55:08.924261 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:55:08.924261 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:55:08.924261 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:55:08.925557 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:55:08.925557 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:55:08.925557 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:55:08.925557 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:55:08.925557 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 607/1224] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:55:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:55:08.930048 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dd879b-d534-48c3-a392-40765c0e926c None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:55:09.038535 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dd879b-d534-48c3-a392-40765c0e926c None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:55:09.076460 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457fad17-a54d-47cf-8f78-7b5e35da519e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:55:09.077182 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457fad17-a54d-47cf-8f78-7b5e35da519e None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:55:09.077423 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457fad17-a54d-47cf-8f78-7b5e35da519e None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:55:09.077727 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457fad17-a54d-47cf-8f78-7b5e35da519e None admin] PATH_INFO: `/v3/domains` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:55:09.078478 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457fad17-a54d-47cf-8f78-7b5e35da519e None admin] RBAC: Adding query filter params (name=Default) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:55:09.078799 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457fad17-a54d-47cf-8f78-7b5e35da519e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:55:09.080178 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457fad17-a54d-47cf-8f78-7b5e35da519e None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:55:09.093137 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 618/1225] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:55:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:55:09.099068 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89896c97-6f4d-4f17-8754-0971cf3ca3f8 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:55:09.204601 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:55:09.211742 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89896c97-6f4d-4f17-8754-0971cf3ca3f8 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:55:09.250348 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3818fe2a-2cc3-4105-bdb1-0a83bb93d6c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:55:09.251169 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3818fe2a-2cc3-4105-bdb1-0a83bb93d6c2 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:55:09.251460 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3818fe2a-2cc3-4105-bdb1-0a83bb93d6c2 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:55:09.251743 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3818fe2a-2cc3-4105-bdb1-0a83bb93d6c2 None admin] PATH_INFO: `/v3/users/placement` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:55:09.261483 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3818fe2a-2cc3-4105-bdb1-0a83bb93d6c2 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:55:09.263164 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3818fe2a-2cc3-4105-bdb1-0a83bb93d6c2 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:55:09.281973 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-3818fe2a-2cc3-4105-bdb1-0a83bb93d6c2 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 14:55:09.281973 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:55:09.281973 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:55:09.281973 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:55:09.281973 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:55:09.281973 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:55:09.281973 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:55:09.281973 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:55:09.281973 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:55:09.281973 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:55:09.281973 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:55:09.281973 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:55:09.281973 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:55:09.281973 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:55:09.281973 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:55:09.281973 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:55:09.283494 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:55:09.283494 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:55:09.283494 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:55:09.283494 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:55:09.283494 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:55:09.283494 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:55:09.283494 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:55:09.283494 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:55:09.283494 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:55:09.283494 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:55:09.283494 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:55:09.283494 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:55:09.283494 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:55:09.283494 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:55:09.283494 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:55:09.283494 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:55:09.283494 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:55:09.283494 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:55:09.284862 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:55:09.284862 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:55:09.284862 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:55:09.284862 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:55:09.284862 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:55:09.284862 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:55:09.284862 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:55:09.284862 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:55:09.284862 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 14:55:09.284862 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:55:09.284862 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 608/1226] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:55:09 2022] GET /identity/v3/users/placement => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:55:09.290012 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457fad17-a54d-47cf-8f78-7b5e35da519e None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:55:09.344430 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:55:09.401122 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457fad17-a54d-47cf-8f78-7b5e35da519e None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:55:09.436584 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44269d58-880f-4696-b86c-a9e1e9e840f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:55:09.437291 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44269d58-880f-4696-b86c-a9e1e9e840f5 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:55:09.437643 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44269d58-880f-4696-b86c-a9e1e9e840f5 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:55:09.437905 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44269d58-880f-4696-b86c-a9e1e9e840f5 None admin] PATH_INFO: `/v3/users` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:55:09.438803 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44269d58-880f-4696-b86c-a9e1e9e840f5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:55:09.439147 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44269d58-880f-4696-b86c-a9e1e9e840f5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:55:09.440895 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44269d58-880f-4696-b86c-a9e1e9e840f5 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:55:09.459689 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 619/1227] 10.222.0.57 () {56 vars in 1143 bytes} [Thu May 19 14:55:09 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:55:09.466752 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3818fe2a-2cc3-4105-bdb1-0a83bb93d6c2 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:55:09.576157 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3818fe2a-2cc3-4105-bdb1-0a83bb93d6c2 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:55:09.613529 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5313ad-26ce-4158-99ec-bfa1bca953e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:55:09.614365 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5313ad-26ce-4158-99ec-bfa1bca953e9 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:55:09.614660 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5313ad-26ce-4158-99ec-bfa1bca953e9 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:55:09.614945 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5313ad-26ce-4158-99ec-bfa1bca953e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:55:09.626355 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5313ad-26ce-4158-99ec-bfa1bca953e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:55:09.627974 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5313ad-26ce-4158-99ec-bfa1bca953e9 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:55:09.648446 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-1c5313ad-26ce-4158-99ec-bfa1bca953e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:55:09.648446 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:55:09.648446 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:55:09.648446 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:55:09.648446 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:55:09.648446 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:55:09.648446 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:55:09.648446 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:55:09.648446 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:55:09.648446 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:55:09.648446 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:55:09.648446 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:55:09.648446 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:55:09.648446 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:55:09.648446 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:55:09.649784 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:55:09.649784 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:55:09.649784 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:55:09.649784 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:55:09.649784 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:55:09.649784 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:55:09.649784 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:55:09.649784 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:55:09.649784 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:55:09.649784 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:55:09.649784 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:55:09.649784 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:55:09.649784 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:55:09.649784 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:55:09.649784 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:55:09.649784 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:55:09.649784 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:55:09.651073 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:55:09.651073 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:55:09.651073 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:55:09.651073 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:55:09.651073 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 609/1228] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:55:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:55:09.655913 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44269d58-880f-4696-b86c-a9e1e9e840f5 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:55:09.761318 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44269d58-880f-4696-b86c-a9e1e9e840f5 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:55:09.763770 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:55:09.799542 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f0d5df-3a96-4c50-b01b-113e343e904b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:55:09.800211 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f0d5df-3a96-4c50-b01b-113e343e904b None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:55:09.800500 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f0d5df-3a96-4c50-b01b-113e343e904b None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:55:09.800777 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f0d5df-3a96-4c50-b01b-113e343e904b None admin] PATH_INFO: `/v3/domains` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:55:09.801495 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f0d5df-3a96-4c50-b01b-113e343e904b None admin] RBAC: Adding query filter params (name=Default) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:55:09.801804 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f0d5df-3a96-4c50-b01b-113e343e904b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:55:09.803210 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f0d5df-3a96-4c50-b01b-113e343e904b None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:55:09.815088 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 620/1229] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:55:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:55:09.821025 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5313ad-26ce-4158-99ec-bfa1bca953e9 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:55:09.830673 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:55:09.933243 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5313ad-26ce-4158-99ec-bfa1bca953e9 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:55:09.969396 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d4c710-96b8-4be0-8666-2ca0eece1cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:55:09.970419 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d4c710-96b8-4be0-8666-2ca0eece1cda None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:55:09.970556 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d4c710-96b8-4be0-8666-2ca0eece1cda None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:55:09.970885 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d4c710-96b8-4be0-8666-2ca0eece1cda None admin] PATH_INFO: `/v3/projects/service` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:55:09.979132 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d4c710-96b8-4be0-8666-2ca0eece1cda None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:55:09.980918 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d4c710-96b8-4be0-8666-2ca0eece1cda None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:55:09.997194 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-40d4c710-96b8-4be0-8666-2ca0eece1cda None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:55:09.997194 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:55:09.997194 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:55:09.997194 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:55:09.997194 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:55:09.997194 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:55:09.997194 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:55:09.997194 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:55:09.997194 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:55:09.997194 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:55:09.997194 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:55:09.997194 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:55:09.997194 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:55:09.997194 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:55:09.997194 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:55:09.998603 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:55:09.998603 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:55:09.998603 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:55:09.998603 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:55:09.998603 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:55:09.998603 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:55:09.998603 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:55:09.998603 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:55:09.998603 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:55:09.998603 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:55:09.998603 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:55:09.998603 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:55:09.998603 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:55:09.998603 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:55:09.998603 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:55:09.998603 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:55:09.998603 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:55:09.999912 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:55:09.999912 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:55:09.999912 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:55:09.999912 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:55:09.999912 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:55:09.999912 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:55:09.999912 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:55:09.999912 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:55:09.999912 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 610/1230] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:55:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:55:10.004551 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f0d5df-3a96-4c50-b01b-113e343e904b None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:55:10.117785 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f0d5df-3a96-4c50-b01b-113e343e904b None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:55:10.160101 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfab94cd-0e1f-40fc-aad4-97d4b31e95aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:55:10.160792 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfab94cd-0e1f-40fc-aad4-97d4b31e95aa None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:55:10.161067 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfab94cd-0e1f-40fc-aad4-97d4b31e95aa None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:55:10.161354 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfab94cd-0e1f-40fc-aad4-97d4b31e95aa None admin] PATH_INFO: `/v3/projects` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:55:10.162216 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfab94cd-0e1f-40fc-aad4-97d4b31e95aa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:55:10.162521 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfab94cd-0e1f-40fc-aad4-97d4b31e95aa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:55:10.163943 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfab94cd-0e1f-40fc-aad4-97d4b31e95aa None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:55:10.177186 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 621/1231] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:55:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:55:10.183965 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d4c710-96b8-4be0-8666-2ca0eece1cda None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:55:10.254929 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:55:10.298204 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d4c710-96b8-4be0-8666-2ca0eece1cda None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:55:10.338942 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31adc0f6-024a-41c0-9eeb-c5397042e924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:55:10.339875 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31adc0f6-024a-41c0-9eeb-c5397042e924 None admin] REQUEST_METHOD: `PUT` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:55:10.340200 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31adc0f6-024a-41c0-9eeb-c5397042e924 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:55:10.340480 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31adc0f6-024a-41c0-9eeb-c5397042e924 None admin] PATH_INFO: `/v3/projects/fc99d742ae004954b6d841293b5fa79c/users/e80cdca4cf964f308a7618f89cce7b1a/roles/08f16502894349708527e30032247468` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:55:10.358136 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31adc0f6-024a-41c0-9eeb-c5397042e924 None admin] RBAC: Authorizing `identity:create_grant(project_id=fc99d742ae004954b6d841293b5fa79c, user_id=e80cdca4cf964f308a7618f89cce7b1a, role_id=08f16502894349708527e30032247468)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:55:10.359860 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31adc0f6-024a-41c0-9eeb-c5397042e924 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:55:10.378909 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 611/1232] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 14:55:10 2022] PUT /identity/v3/projects/fc99d742ae004954b6d841293b5fa79c/users/e80cdca4cf964f308a7618f89cce7b1a/roles/08f16502894349708527e30032247468 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:55:12.789767 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d61910-e336-4135-aee0-597c9b79ab09 None None] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:55:12.790467 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d61910-e336-4135-aee0-597c9b79ab09 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:55:12.790669 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d61910-e336-4135-aee0-597c9b79ab09 None None] PATH_INFO: `/` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:55:12.792661 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 622/1233] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:55:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:55:12.798159 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0a5319-0f82-46cb-a116-01fac85a4f50 None None] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:55:12.798635 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0a5319-0f82-46cb-a116-01fac85a4f50 None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:55:12.798751 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0a5319-0f82-46cb-a116-01fac85a4f50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:55:12.815244 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.auth.core [None req-1a0a5319-0f82-46cb-a116-01fac85a4f50 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:55:12.816689 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.fernet_utils [None req-1a0a5319-0f82-46cb-a116-01fac85a4f50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:55:12.888887 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 612/1234] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:55:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:55:12.914345 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d61910-e336-4135-aee0-597c9b79ab09 None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:55:12.941361 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:55:13.034473 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d61910-e336-4135-aee0-597c9b79ab09 None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:55:13.070644 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33de1fa1-50c0-4dfe-a83d-e3bd5c5b680e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:55:13.071438 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33de1fa1-50c0-4dfe-a83d-e3bd5c5b680e None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:55:13.072217 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33de1fa1-50c0-4dfe-a83d-e3bd5c5b680e None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:55:13.072495 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33de1fa1-50c0-4dfe-a83d-e3bd5c5b680e None admin] PATH_INFO: `/v3/roles/service` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:55:13.085490 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33de1fa1-50c0-4dfe-a83d-e3bd5c5b680e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:55:13.094742 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-33de1fa1-50c0-4dfe-a83d-e3bd5c5b680e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:55:13.094742 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:55:13.094742 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:55:13.094742 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:55:13.094742 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:55:13.094742 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:55:13.094742 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:55:13.094742 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:55:13.094742 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:55:13.094742 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:55:13.094742 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:55:13.094742 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:55:13.094742 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:55:13.094742 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:55:13.094742 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:55:13.094742 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:55:13.095886 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:55:13.095886 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:55:13.095886 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:55:13.095886 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:55:13.095886 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:55:13.095886 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:55:13.095886 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:55:13.095886 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:55:13.095886 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:55:13.095886 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:55:13.095886 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:55:13.095886 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:55:13.095886 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:55:13.095886 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:55:13.095886 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:55:13.095886 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:55:13.096909 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:55:13.096909 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:55:13.096909 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:55:13.096909 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:55:13.096909 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:55:13.096909 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:55:13.096909 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:55:13.096909 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:55:13.096909 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:55:13.096909 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:55:13.096909 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:55:13.096909 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:55:13.096909 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 623/1235] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 14:55:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:55:13.102248 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0a5319-0f82-46cb-a116-01fac85a4f50 None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:55:13.151555 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:55:13.213132 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0a5319-0f82-46cb-a116-01fac85a4f50 None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:55:13.248902 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a3e5f4-7cec-49f5-83f9-3a9389806c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:55:13.249580 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a3e5f4-7cec-49f5-83f9-3a9389806c45 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:55:13.249892 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a3e5f4-7cec-49f5-83f9-3a9389806c45 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:55:13.250179 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a3e5f4-7cec-49f5-83f9-3a9389806c45 None admin] PATH_INFO: `/v3/roles` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:55:13.250931 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a3e5f4-7cec-49f5-83f9-3a9389806c45 None admin] RBAC: Adding query filter params (name=service) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:55:13.251244 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a3e5f4-7cec-49f5-83f9-3a9389806c45 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:55:13.252629 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a3e5f4-7cec-49f5-83f9-3a9389806c45 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:55:13.262759 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 613/1236] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 19 14:55:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:55:13.269064 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33de1fa1-50c0-4dfe-a83d-e3bd5c5b680e None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:55:13.372918 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:55:13.393884 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33de1fa1-50c0-4dfe-a83d-e3bd5c5b680e None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:55:13.434016 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7940af1e-ae23-47ec-9e53-53573ab86714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:55:13.435053 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7940af1e-ae23-47ec-9e53-53573ab86714 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:55:13.435392 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7940af1e-ae23-47ec-9e53-53573ab86714 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:55:13.435756 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7940af1e-ae23-47ec-9e53-53573ab86714 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:55:13.449095 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7940af1e-ae23-47ec-9e53-53573ab86714 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:55:13.451060 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7940af1e-ae23-47ec-9e53-53573ab86714 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:55:13.475070 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-7940af1e-ae23-47ec-9e53-53573ab86714 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:55:13.475070 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:55:13.475070 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:55:13.475070 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:55:13.475070 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:55:13.475070 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:55:13.475070 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:55:13.475070 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:55:13.475070 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:55:13.475070 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:55:13.475070 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:55:13.475070 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:55:13.475070 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:55:13.475070 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:55:13.475070 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:55:13.476453 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:55:13.476453 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:55:13.476453 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:55:13.476453 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:55:13.476453 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:55:13.476453 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:55:13.476453 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:55:13.476453 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:55:13.476453 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:55:13.476453 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:55:13.476453 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:55:13.476453 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:55:13.476453 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:55:13.476453 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:55:13.476453 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:55:13.476453 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:55:13.476453 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:55:13.477575 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:55:13.477575 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:55:13.477575 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:55:13.477575 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:55:13.477575 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 624/1237] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:55:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:55:13.482701 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a3e5f4-7cec-49f5-83f9-3a9389806c45 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:55:13.615792 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a3e5f4-7cec-49f5-83f9-3a9389806c45 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:55:13.628483 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:55:13.666190 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017da5cd-891a-4d27-9ca3-3364f65e4d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:55:13.666981 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017da5cd-891a-4d27-9ca3-3364f65e4d69 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:55:13.667367 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017da5cd-891a-4d27-9ca3-3364f65e4d69 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:55:13.667750 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017da5cd-891a-4d27-9ca3-3364f65e4d69 None admin] PATH_INFO: `/v3/domains` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:55:13.668609 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017da5cd-891a-4d27-9ca3-3364f65e4d69 None admin] RBAC: Adding query filter params (name=Default) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:55:13.669016 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017da5cd-891a-4d27-9ca3-3364f65e4d69 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:55:13.670596 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017da5cd-891a-4d27-9ca3-3364f65e4d69 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:55:13.683476 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 614/1238] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:55:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:55:13.689941 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7940af1e-ae23-47ec-9e53-53573ab86714 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:55:13.803901 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7940af1e-ae23-47ec-9e53-53573ab86714 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:55:13.843458 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dd8528-d689-4ea1-981f-a1af6619b645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:55:13.844283 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dd8528-d689-4ea1-981f-a1af6619b645 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:55:13.844573 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dd8528-d689-4ea1-981f-a1af6619b645 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:55:13.844841 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dd8528-d689-4ea1-981f-a1af6619b645 None admin] PATH_INFO: `/v3/users/placement` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:55:13.848064 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:55:13.855093 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dd8528-d689-4ea1-981f-a1af6619b645 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:55:13.856789 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dd8528-d689-4ea1-981f-a1af6619b645 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:55:13.872236 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-a6dd8528-d689-4ea1-981f-a1af6619b645 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 14:55:13.872236 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:55:13.872236 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:55:13.872236 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:55:13.872236 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:55:13.872236 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:55:13.872236 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:55:13.872236 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:55:13.872236 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:55:13.872236 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:55:13.872236 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:55:13.872236 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:55:13.872236 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:55:13.872236 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:55:13.872236 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:55:13.872236 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:55:13.873677 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:55:13.873677 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:55:13.873677 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:55:13.873677 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:55:13.873677 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:55:13.873677 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:55:13.873677 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:55:13.873677 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:55:13.873677 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:55:13.873677 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:55:13.873677 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:55:13.873677 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:55:13.873677 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:55:13.873677 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:55:13.873677 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:55:13.873677 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:55:13.873677 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:55:13.873677 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:55:13.875080 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:55:13.875080 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:55:13.875080 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:55:13.875080 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:55:13.875080 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:55:13.875080 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:55:13.875080 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:55:13.875080 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:55:13.875080 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 14:55:13.875080 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:55:13.875080 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 625/1239] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:55:13 2022] GET /identity/v3/users/placement => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:55:13.879655 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017da5cd-891a-4d27-9ca3-3364f65e4d69 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:55:13.993768 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017da5cd-891a-4d27-9ca3-3364f65e4d69 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:55:14.033666 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af06159-34f4-4611-9cb6-a88c875ab104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:55:14.034582 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af06159-34f4-4611-9cb6-a88c875ab104 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:55:14.035094 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af06159-34f4-4611-9cb6-a88c875ab104 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:55:14.035444 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af06159-34f4-4611-9cb6-a88c875ab104 None admin] PATH_INFO: `/v3/users` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:55:14.036488 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af06159-34f4-4611-9cb6-a88c875ab104 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:55:14.036904 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af06159-34f4-4611-9cb6-a88c875ab104 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:55:14.038920 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af06159-34f4-4611-9cb6-a88c875ab104 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:55:14.059488 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 615/1240] 10.222.0.57 () {56 vars in 1143 bytes} [Thu May 19 14:55:13 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:55:14.065359 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dd8528-d689-4ea1-981f-a1af6619b645 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:55:14.169581 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dd8528-d689-4ea1-981f-a1af6619b645 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:55:14.206577 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5703a845-5fc1-4da5-a535-606d3b35f690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:55:14.207244 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5703a845-5fc1-4da5-a535-606d3b35f690 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:55:14.207534 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5703a845-5fc1-4da5-a535-606d3b35f690 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:55:14.207822 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5703a845-5fc1-4da5-a535-606d3b35f690 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:55:14.284246 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:55:14.294886 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5703a845-5fc1-4da5-a535-606d3b35f690 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:55:14.296845 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5703a845-5fc1-4da5-a535-606d3b35f690 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:55:14.412846 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 626/1241] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:55:14 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 19 14:55:14.418933 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af06159-34f4-4611-9cb6-a88c875ab104 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:55:14.452354 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:55:14.533919 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af06159-34f4-4611-9cb6-a88c875ab104 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:55:14.571972 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858e198a-d3d9-48bb-8747-70ffb1d0cb56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:55:14.572775 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858e198a-d3d9-48bb-8747-70ffb1d0cb56 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:55:14.573063 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858e198a-d3d9-48bb-8747-70ffb1d0cb56 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:55:14.573342 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858e198a-d3d9-48bb-8747-70ffb1d0cb56 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:55:14.585101 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858e198a-d3d9-48bb-8747-70ffb1d0cb56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:55:14.586628 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858e198a-d3d9-48bb-8747-70ffb1d0cb56 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:55:14.606187 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-858e198a-d3d9-48bb-8747-70ffb1d0cb56 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:55:14.606187 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:55:14.606187 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:55:14.606187 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:55:14.606187 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:55:14.606187 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:55:14.606187 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:55:14.606187 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:55:14.606187 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:55:14.606187 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:55:14.606187 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:55:14.606187 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:55:14.606187 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:55:14.606187 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:55:14.606187 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:55:14.607354 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:55:14.607354 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:55:14.607354 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:55:14.607354 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:55:14.607354 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:55:14.607354 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:55:14.607354 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:55:14.607354 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:55:14.607354 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:55:14.607354 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:55:14.607354 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:55:14.607354 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:55:14.607354 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:55:14.607354 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:55:14.607354 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:55:14.607354 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:55:14.607354 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:55:14.608892 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:55:14.608892 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:55:14.608892 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:55:14.608892 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:55:14.608892 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 616/1242] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:55:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:55:14.617029 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5703a845-5fc1-4da5-a535-606d3b35f690 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:55:14.729229 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:55:14.736108 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5703a845-5fc1-4da5-a535-606d3b35f690 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:55:14.775208 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15098a77-d39f-42ba-ba4b-dc4b683be4ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:55:14.776278 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15098a77-d39f-42ba-ba4b-dc4b683be4ff None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:55:14.776976 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15098a77-d39f-42ba-ba4b-dc4b683be4ff None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:55:14.777503 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15098a77-d39f-42ba-ba4b-dc4b683be4ff None admin] PATH_INFO: `/v3/domains` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:55:14.778781 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15098a77-d39f-42ba-ba4b-dc4b683be4ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:55:14.779299 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15098a77-d39f-42ba-ba4b-dc4b683be4ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:55:14.780945 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15098a77-d39f-42ba-ba4b-dc4b683be4ff None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:55:14.796845 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 627/1243] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:55:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:55:14.802734 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858e198a-d3d9-48bb-8747-70ffb1d0cb56 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:55:14.894926 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:55:14.922155 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858e198a-d3d9-48bb-8747-70ffb1d0cb56 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:55:14.959716 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45ea70a-3e59-4879-b8a4-8b014375edf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:55:14.960637 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45ea70a-3e59-4879-b8a4-8b014375edf5 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:55:14.960924 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45ea70a-3e59-4879-b8a4-8b014375edf5 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:55:14.961268 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45ea70a-3e59-4879-b8a4-8b014375edf5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:55:14.969571 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45ea70a-3e59-4879-b8a4-8b014375edf5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:55:14.971254 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45ea70a-3e59-4879-b8a4-8b014375edf5 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:55:14.987027 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-c45ea70a-3e59-4879-b8a4-8b014375edf5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:55:14.987027 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:55:14.987027 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:55:14.987027 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:55:14.987027 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:55:14.987027 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:55:14.987027 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:55:14.987027 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:55:14.987027 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:55:14.987027 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:55:14.987027 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:55:14.987027 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:55:14.987027 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:55:14.987027 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:55:14.987027 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:55:14.988757 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:55:14.988757 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:55:14.988757 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:55:14.988757 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:55:14.988757 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:55:14.988757 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:55:14.988757 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:55:14.988757 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:55:14.988757 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:55:14.988757 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:55:14.988757 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:55:14.988757 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:55:14.988757 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:55:14.988757 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:55:14.988757 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:55:14.988757 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:55:14.988757 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:55:14.990068 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:55:14.990068 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:55:14.990068 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:55:14.990068 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:55:14.990068 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:55:14.990068 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:55:14.990068 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:55:14.990068 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:55:14.990068 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 617/1244] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:55:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:55:14.994369 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15098a77-d39f-42ba-ba4b-dc4b683be4ff None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:55:15.111553 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15098a77-d39f-42ba-ba4b-dc4b683be4ff None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:55:15.146963 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21da45d-57c1-4f72-82de-ee7c2641f6f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:55:15.147697 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21da45d-57c1-4f72-82de-ee7c2641f6f8 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:55:15.148002 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21da45d-57c1-4f72-82de-ee7c2641f6f8 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:55:15.148316 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21da45d-57c1-4f72-82de-ee7c2641f6f8 None admin] PATH_INFO: `/v3/projects` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:55:15.149143 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21da45d-57c1-4f72-82de-ee7c2641f6f8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:55:15.149468 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21da45d-57c1-4f72-82de-ee7c2641f6f8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:55:15.151827 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21da45d-57c1-4f72-82de-ee7c2641f6f8 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:55:15.164090 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 628/1245] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:55:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:55:15.171248 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45ea70a-3e59-4879-b8a4-8b014375edf5 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:55:15.286405 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45ea70a-3e59-4879-b8a4-8b014375edf5 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:55:15.326339 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64707cf0-c59b-43fa-8a97-479bc95f36ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:55:15.327118 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64707cf0-c59b-43fa-8a97-479bc95f36ac None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:55:15.327426 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64707cf0-c59b-43fa-8a97-479bc95f36ac None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:55:15.327823 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64707cf0-c59b-43fa-8a97-479bc95f36ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:55:15.328696 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64707cf0-c59b-43fa-8a97-479bc95f36ac None admin] RBAC: Adding query filter params (role.id=08f16502894349708527e30032247468, scope.project.id=fc99d742ae004954b6d841293b5fa79c, user.id=e80cdca4cf964f308a7618f89cce7b1a) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:55:15.329057 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64707cf0-c59b-43fa-8a97-479bc95f36ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:55:15.330655 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64707cf0-c59b-43fa-8a97-479bc95f36ac None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:55:15.336669 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 618/1246] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:55:15 2022] GET /identity/v3/role_assignments?user.id=e80cdca4cf964f308a7618f89cce7b1a&scope.project.id=fc99d742ae004954b6d841293b5fa79c&role.id=08f16502894349708527e30032247468 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:55:17.902034 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c31a960-9ccd-4504-b3c0-ed5c7fb22682 None None] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:55:17.902862 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c31a960-9ccd-4504-b3c0-ed5c7fb22682 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:55:17.902862 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c31a960-9ccd-4504-b3c0-ed5c7fb22682 None None] PATH_INFO: `/` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:55:17.904231 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 629/1247] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:55:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:55:17.911028 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb0cad6-07fa-4a83-a08b-1951a67da170 None None] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:55:17.911635 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb0cad6-07fa-4a83-a08b-1951a67da170 None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:55:17.911764 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb0cad6-07fa-4a83-a08b-1951a67da170 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:55:17.917587 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:55:17.932169 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.auth.core [None req-bbb0cad6-07fa-4a83-a08b-1951a67da170 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:55:17.933623 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.fernet_utils [None req-bbb0cad6-07fa-4a83-a08b-1951a67da170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:55:18.011272 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 619/1248] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:55:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:55:18.036097 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c31a960-9ccd-4504-b3c0-ed5c7fb22682 None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:55:18.058909 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:55:18.158289 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c31a960-9ccd-4504-b3c0-ed5c7fb22682 None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:55:18.199346 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ea24b3-4bc7-468d-9738-1d475e7980d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:55:18.199964 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ea24b3-4bc7-468d-9738-1d475e7980d8 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:55:18.200249 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ea24b3-4bc7-468d-9738-1d475e7980d8 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:55:18.200528 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ea24b3-4bc7-468d-9738-1d475e7980d8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:55:18.213569 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ea24b3-4bc7-468d-9738-1d475e7980d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:55:18.222654 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-27ea24b3-4bc7-468d-9738-1d475e7980d8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:55:18.222654 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:55:18.222654 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:55:18.222654 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:55:18.222654 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:55:18.222654 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:55:18.222654 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:55:18.222654 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:55:18.222654 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:55:18.222654 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:55:18.222654 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:55:18.222654 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:55:18.222654 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:55:18.222654 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:55:18.222654 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:55:18.222654 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:55:18.224259 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:55:18.224259 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:55:18.224259 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:55:18.224259 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:55:18.224259 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:55:18.224259 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:55:18.224259 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:55:18.224259 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:55:18.224259 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:55:18.224259 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:55:18.224259 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:55:18.224259 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:55:18.224259 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:55:18.224259 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:55:18.224259 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:55:18.224259 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:55:18.225579 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:55:18.225579 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:55:18.225579 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:55:18.225579 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:55:18.225579 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:55:18.225579 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:55:18.225579 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:55:18.225579 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:55:18.225579 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:55:18.225579 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:55:18.225579 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:55:18.225579 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:55:18.225579 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 630/1249] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:55:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:55:18.231241 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb0cad6-07fa-4a83-a08b-1951a67da170 None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:55:18.360260 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb0cad6-07fa-4a83-a08b-1951a67da170 None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:55:18.398617 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:55:18.406208 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0920dc26-e479-47c0-a9b2-0e63f79f332b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:55:18.407101 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0920dc26-e479-47c0-a9b2-0e63f79f332b None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:55:18.407765 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0920dc26-e479-47c0-a9b2-0e63f79f332b None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:55:18.408243 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0920dc26-e479-47c0-a9b2-0e63f79f332b None admin] PATH_INFO: `/v3/roles` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:55:18.409261 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0920dc26-e479-47c0-a9b2-0e63f79f332b None admin] RBAC: Adding query filter params (name=admin) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:55:18.409751 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0920dc26-e479-47c0-a9b2-0e63f79f332b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:55:18.411441 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0920dc26-e479-47c0-a9b2-0e63f79f332b None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:55:18.423324 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 620/1250] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:55:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:55:18.429512 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ea24b3-4bc7-468d-9738-1d475e7980d8 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:55:18.529156 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:55:18.559640 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ea24b3-4bc7-468d-9738-1d475e7980d8 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:55:18.597171 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8411de12-1b26-46d9-b2a4-404ef158cbcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:55:18.598239 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8411de12-1b26-46d9-b2a4-404ef158cbcc None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:55:18.598239 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8411de12-1b26-46d9-b2a4-404ef158cbcc None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:55:18.598697 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8411de12-1b26-46d9-b2a4-404ef158cbcc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:55:18.611103 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8411de12-1b26-46d9-b2a4-404ef158cbcc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:55:18.612894 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8411de12-1b26-46d9-b2a4-404ef158cbcc None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:55:18.641559 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-8411de12-1b26-46d9-b2a4-404ef158cbcc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:55:18.641559 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:55:18.641559 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:55:18.641559 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:55:18.641559 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:55:18.641559 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:55:18.641559 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:55:18.641559 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:55:18.641559 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:55:18.641559 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:55:18.641559 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:55:18.641559 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:55:18.641559 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:55:18.641559 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:55:18.641559 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:55:18.643077 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:55:18.643077 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:55:18.643077 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:55:18.643077 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:55:18.643077 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:55:18.643077 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:55:18.643077 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:55:18.643077 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:55:18.643077 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:55:18.643077 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:55:18.643077 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:55:18.643077 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:55:18.643077 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:55:18.643077 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:55:18.643077 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:55:18.643077 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:55:18.643077 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:55:18.644314 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:55:18.644314 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:55:18.644314 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:55:18.644314 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:55:18.644314 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 631/1251] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:55:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:55:18.649212 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0920dc26-e479-47c0-a9b2-0e63f79f332b None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:55:18.771826 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0920dc26-e479-47c0-a9b2-0e63f79f332b None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:55:18.809878 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1856ee2-3272-40b3-8b5d-e1ff6b24c1e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:55:18.810529 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1856ee2-3272-40b3-8b5d-e1ff6b24c1e7 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:55:18.810779 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1856ee2-3272-40b3-8b5d-e1ff6b24c1e7 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:55:18.811060 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1856ee2-3272-40b3-8b5d-e1ff6b24c1e7 None admin] PATH_INFO: `/v3/domains` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:55:18.811783 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1856ee2-3272-40b3-8b5d-e1ff6b24c1e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:55:18.812092 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1856ee2-3272-40b3-8b5d-e1ff6b24c1e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:55:18.813468 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1856ee2-3272-40b3-8b5d-e1ff6b24c1e7 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:55:18.826507 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 621/1252] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:55:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:55:18.832427 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8411de12-1b26-46d9-b2a4-404ef158cbcc None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:55:18.942492 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8411de12-1b26-46d9-b2a4-404ef158cbcc None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:55:18.972952 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:55:18.982717 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8250494-f5d9-4b47-bab6-69634ba04968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:55:18.983734 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8250494-f5d9-4b47-bab6-69634ba04968 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:55:18.984164 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8250494-f5d9-4b47-bab6-69634ba04968 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:55:18.984454 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8250494-f5d9-4b47-bab6-69634ba04968 None admin] PATH_INFO: `/v3/users/placement` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:55:18.995850 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8250494-f5d9-4b47-bab6-69634ba04968 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:55:18.997584 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8250494-f5d9-4b47-bab6-69634ba04968 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:55:19.015035 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-d8250494-f5d9-4b47-bab6-69634ba04968 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 14:55:19.015035 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:55:19.015035 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:55:19.015035 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:55:19.015035 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:55:19.015035 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:55:19.015035 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:55:19.015035 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:55:19.015035 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:55:19.015035 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:55:19.015035 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:55:19.015035 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:55:19.015035 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:55:19.015035 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:55:19.015035 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:55:19.015035 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:55:19.016452 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:55:19.016452 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:55:19.016452 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:55:19.016452 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:55:19.016452 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:55:19.016452 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:55:19.016452 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:55:19.016452 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:55:19.016452 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:55:19.016452 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:55:19.016452 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:55:19.016452 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:55:19.016452 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:55:19.016452 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:55:19.016452 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:55:19.016452 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:55:19.016452 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:55:19.016452 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:55:19.017651 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:55:19.017651 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:55:19.017651 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:55:19.017651 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:55:19.017651 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:55:19.017651 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:55:19.017651 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:55:19.017651 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:55:19.017651 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 14:55:19.017651 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:55:19.017651 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 632/1253] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:55:18 2022] GET /identity/v3/users/placement => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:55:19.022653 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1856ee2-3272-40b3-8b5d-e1ff6b24c1e7 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:55:19.084275 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:55:19.133655 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1856ee2-3272-40b3-8b5d-e1ff6b24c1e7 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:55:19.174212 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1af32c9-be4c-4930-afb2-1e29d3343686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:55:19.174897 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1af32c9-be4c-4930-afb2-1e29d3343686 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:55:19.175184 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1af32c9-be4c-4930-afb2-1e29d3343686 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:55:19.175506 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1af32c9-be4c-4930-afb2-1e29d3343686 None admin] PATH_INFO: `/v3/users` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:55:19.176483 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1af32c9-be4c-4930-afb2-1e29d3343686 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:55:19.176801 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1af32c9-be4c-4930-afb2-1e29d3343686 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:55:19.178363 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1af32c9-be4c-4930-afb2-1e29d3343686 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:55:19.196208 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 622/1254] 10.222.0.57 () {56 vars in 1143 bytes} [Thu May 19 14:55:19 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:55:19.202730 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8250494-f5d9-4b47-bab6-69634ba04968 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:55:19.330461 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8250494-f5d9-4b47-bab6-69634ba04968 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:55:19.373300 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b8ea72-6c19-4524-9f9d-a8c058bb081e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:55:19.374078 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b8ea72-6c19-4524-9f9d-a8c058bb081e None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:55:19.374542 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b8ea72-6c19-4524-9f9d-a8c058bb081e None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:55:19.374907 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b8ea72-6c19-4524-9f9d-a8c058bb081e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:55:19.440192 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:55:19.455472 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b8ea72-6c19-4524-9f9d-a8c058bb081e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:55:19.457117 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b8ea72-6c19-4524-9f9d-a8c058bb081e None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:55:19.588716 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 633/1255] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:55:19 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 19 14:55:19.590186 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1af32c9-be4c-4930-afb2-1e29d3343686 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:55:19.705383 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1af32c9-be4c-4930-afb2-1e29d3343686 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:55:19.720873 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:55:19.746364 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d63d26-1d76-4d6e-950e-6d124d67750c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:55:19.747168 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d63d26-1d76-4d6e-950e-6d124d67750c None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:55:19.747455 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d63d26-1d76-4d6e-950e-6d124d67750c None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:55:19.747736 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d63d26-1d76-4d6e-950e-6d124d67750c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:55:19.759251 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d63d26-1d76-4d6e-950e-6d124d67750c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:55:19.760975 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d63d26-1d76-4d6e-950e-6d124d67750c None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:55:19.783721 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-85d63d26-1d76-4d6e-950e-6d124d67750c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:55:19.783721 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:55:19.783721 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:55:19.783721 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:55:19.783721 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:55:19.783721 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:55:19.783721 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:55:19.783721 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:55:19.783721 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:55:19.783721 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:55:19.783721 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:55:19.783721 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:55:19.783721 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:55:19.783721 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:55:19.783721 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:55:19.785049 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:55:19.785049 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:55:19.785049 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:55:19.785049 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:55:19.785049 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:55:19.785049 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:55:19.785049 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:55:19.785049 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:55:19.785049 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:55:19.785049 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:55:19.785049 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:55:19.785049 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:55:19.785049 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:55:19.785049 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:55:19.785049 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:55:19.785049 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:55:19.785049 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:55:19.786373 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:55:19.786373 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:55:19.786373 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:55:19.786373 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:55:19.786373 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 623/1256] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:55:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:55:19.792904 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b8ea72-6c19-4524-9f9d-a8c058bb081e None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:55:19.887853 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:55:19.903297 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b8ea72-6c19-4524-9f9d-a8c058bb081e None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:55:19.939115 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c56a17-e894-4bd5-b346-04ada351dc12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:55:19.939510 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c56a17-e894-4bd5-b346-04ada351dc12 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:55:19.939792 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c56a17-e894-4bd5-b346-04ada351dc12 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:55:19.940069 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c56a17-e894-4bd5-b346-04ada351dc12 None admin] PATH_INFO: `/v3/domains` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:55:19.941055 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c56a17-e894-4bd5-b346-04ada351dc12 None admin] RBAC: Adding query filter params (name=Default) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:55:19.941378 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c56a17-e894-4bd5-b346-04ada351dc12 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:55:19.942760 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c56a17-e894-4bd5-b346-04ada351dc12 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:55:19.955211 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 634/1257] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:55:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:55:19.960925 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d63d26-1d76-4d6e-950e-6d124d67750c None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:55:20.078185 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d63d26-1d76-4d6e-950e-6d124d67750c None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:55:20.118488 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089a61a3-0c1f-406f-a06a-a5aa3f8757c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:55:20.120107 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089a61a3-0c1f-406f-a06a-a5aa3f8757c1 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:55:20.120584 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089a61a3-0c1f-406f-a06a-a5aa3f8757c1 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:55:20.120893 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089a61a3-0c1f-406f-a06a-a5aa3f8757c1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:55:20.130425 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089a61a3-0c1f-406f-a06a-a5aa3f8757c1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:55:20.132206 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089a61a3-0c1f-406f-a06a-a5aa3f8757c1 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:55:20.146711 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-089a61a3-0c1f-406f-a06a-a5aa3f8757c1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:55:20.146711 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:55:20.146711 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:55:20.146711 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:55:20.146711 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:55:20.146711 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:55:20.146711 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:55:20.146711 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:55:20.146711 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:55:20.146711 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:55:20.146711 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:55:20.146711 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:55:20.146711 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:55:20.146711 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:55:20.146711 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:55:20.148084 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:55:20.148084 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:55:20.148084 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:55:20.148084 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:55:20.148084 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:55:20.148084 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:55:20.148084 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:55:20.148084 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:55:20.148084 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:55:20.148084 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:55:20.148084 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:55:20.148084 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:55:20.148084 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:55:20.148084 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:55:20.148084 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:55:20.148084 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:55:20.148084 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:55:20.149168 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:55:20.149168 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:55:20.149168 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:55:20.149168 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:55:20.149168 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:55:20.149168 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:55:20.149168 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:55:20.149168 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:55:20.149168 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 624/1258] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:55:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:55:20.154347 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c56a17-e894-4bd5-b346-04ada351dc12 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:55:20.261460 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c56a17-e894-4bd5-b346-04ada351dc12 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:55:20.302915 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d2743e-0d5e-47ff-86d0-4f63c531df8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:55:20.303606 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d2743e-0d5e-47ff-86d0-4f63c531df8d None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:55:20.303894 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d2743e-0d5e-47ff-86d0-4f63c531df8d None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:55:20.304174 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d2743e-0d5e-47ff-86d0-4f63c531df8d None admin] PATH_INFO: `/v3/projects` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:55:20.304984 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d2743e-0d5e-47ff-86d0-4f63c531df8d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:55:20.305298 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d2743e-0d5e-47ff-86d0-4f63c531df8d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:55:20.306968 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d2743e-0d5e-47ff-86d0-4f63c531df8d None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:55:20.321536 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 635/1259] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:55:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:55:20.328967 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089a61a3-0c1f-406f-a06a-a5aa3f8757c1 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:55:20.359468 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:55:20.443979 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089a61a3-0c1f-406f-a06a-a5aa3f8757c1 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:55:20.482851 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bbd60d-7c65-4ba3-9189-3e33660a70e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:55:20.483536 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bbd60d-7c65-4ba3-9189-3e33660a70e9 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:55:20.483826 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bbd60d-7c65-4ba3-9189-3e33660a70e9 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:55:20.484104 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bbd60d-7c65-4ba3-9189-3e33660a70e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:55:20.484953 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bbd60d-7c65-4ba3-9189-3e33660a70e9 None admin] RBAC: Adding query filter params (role.id=028379e01f9245e9b8fc56e8b9e8a6ad, scope.project.id=fc99d742ae004954b6d841293b5fa79c, user.id=e80cdca4cf964f308a7618f89cce7b1a) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:55:20.485271 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bbd60d-7c65-4ba3-9189-3e33660a70e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:55:20.486780 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bbd60d-7c65-4ba3-9189-3e33660a70e9 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:55:20.493089 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 625/1260] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:55:20 2022] GET /identity/v3/role_assignments?user.id=e80cdca4cf964f308a7618f89cce7b1a&scope.project.id=fc99d742ae004954b6d841293b5fa79c&role.id=028379e01f9245e9b8fc56e8b9e8a6ad => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:55:22.763442 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff70140-4285-40b9-9adf-6ad4e56ca975 None None] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:55:22.763997 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff70140-4285-40b9-9adf-6ad4e56ca975 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:55:22.763997 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff70140-4285-40b9-9adf-6ad4e56ca975 None None] PATH_INFO: `/` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:55:22.765399 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 636/1261] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:55:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:55:22.770759 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e46f3ae-3532-442d-9827-c2cdcc5c41ad None None] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:55:22.771221 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e46f3ae-3532-442d-9827-c2cdcc5c41ad None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:55:22.771338 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e46f3ae-3532-442d-9827-c2cdcc5c41ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:55:22.788098 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.auth.core [None req-7e46f3ae-3532-442d-9827-c2cdcc5c41ad None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:55:22.789287 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.fernet_utils [None req-7e46f3ae-3532-442d-9827-c2cdcc5c41ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:55:22.855776 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 626/1262] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:55:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:55:22.878925 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff70140-4285-40b9-9adf-6ad4e56ca975 None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:55:22.890018 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:55:22.981595 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff70140-4285-40b9-9adf-6ad4e56ca975 None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:55:23.015550 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583c0fca-6ff0-4925-a255-26d7cc4d3399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:55:23.016338 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583c0fca-6ff0-4925-a255-26d7cc4d3399 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:55:23.016632 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583c0fca-6ff0-4925-a255-26d7cc4d3399 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:55:23.016915 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583c0fca-6ff0-4925-a255-26d7cc4d3399 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:55:23.027283 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583c0fca-6ff0-4925-a255-26d7cc4d3399 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:55:23.034988 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-583c0fca-6ff0-4925-a255-26d7cc4d3399 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:55:23.034988 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:55:23.034988 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:55:23.034988 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:55:23.034988 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:55:23.034988 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:55:23.034988 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:55:23.034988 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:55:23.034988 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:55:23.034988 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:55:23.034988 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:55:23.034988 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:55:23.034988 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:55:23.034988 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:55:23.034988 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:55:23.034988 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:55:23.036370 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:55:23.036370 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:55:23.036370 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:55:23.036370 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:55:23.036370 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:55:23.036370 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:55:23.036370 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:55:23.036370 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:55:23.036370 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:55:23.036370 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:55:23.036370 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:55:23.036370 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:55:23.036370 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:55:23.036370 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:55:23.036370 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:55:23.036370 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:55:23.037584 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:55:23.037584 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:55:23.037584 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:55:23.037584 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:55:23.037584 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:55:23.037584 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:55:23.037584 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:55:23.037584 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:55:23.037584 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:55:23.037584 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:55:23.037584 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:55:23.037584 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:55:23.037584 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 637/1263] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:55:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:55:23.042848 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e46f3ae-3532-442d-9827-c2cdcc5c41ad None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:55:23.061769 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:55:23.142339 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e46f3ae-3532-442d-9827-c2cdcc5c41ad None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:55:23.176648 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90c085c-1f31-4972-832e-f03e6cc50cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:55:23.177347 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90c085c-1f31-4972-832e-f03e6cc50cfd None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:55:23.177638 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90c085c-1f31-4972-832e-f03e6cc50cfd None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:55:23.177949 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90c085c-1f31-4972-832e-f03e6cc50cfd None admin] PATH_INFO: `/v3/roles` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:55:23.178700 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90c085c-1f31-4972-832e-f03e6cc50cfd None admin] RBAC: Adding query filter params (name=admin) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:55:23.179012 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90c085c-1f31-4972-832e-f03e6cc50cfd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:55:23.180454 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90c085c-1f31-4972-832e-f03e6cc50cfd None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:55:23.188599 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 627/1264] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:55:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:55:23.194821 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583c0fca-6ff0-4925-a255-26d7cc4d3399 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:55:23.261288 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:55:23.298523 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583c0fca-6ff0-4925-a255-26d7cc4d3399 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:55:23.336691 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f58c73-247c-406f-9877-14603e518c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:55:23.338191 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f58c73-247c-406f-9877-14603e518c5d None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:55:23.338526 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f58c73-247c-406f-9877-14603e518c5d None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:55:23.338814 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f58c73-247c-406f-9877-14603e518c5d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:55:23.350478 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f58c73-247c-406f-9877-14603e518c5d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:55:23.352034 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f58c73-247c-406f-9877-14603e518c5d None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:55:23.370207 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-70f58c73-247c-406f-9877-14603e518c5d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:55:23.370207 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:55:23.370207 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:55:23.370207 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:55:23.370207 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:55:23.370207 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:55:23.370207 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:55:23.370207 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:55:23.370207 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:55:23.370207 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:55:23.370207 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:55:23.370207 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:55:23.370207 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:55:23.370207 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:55:23.370207 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:55:23.371395 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:55:23.371395 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:55:23.371395 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:55:23.371395 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:55:23.371395 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:55:23.371395 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:55:23.371395 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:55:23.371395 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:55:23.371395 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:55:23.371395 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:55:23.371395 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:55:23.371395 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:55:23.371395 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:55:23.371395 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:55:23.371395 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:55:23.371395 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:55:23.371395 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:55:23.372613 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:55:23.372613 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:55:23.372613 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:55:23.372613 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:55:23.372613 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 638/1265] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:55:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:55:23.377232 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90c085c-1f31-4972-832e-f03e6cc50cfd None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:55:23.458168 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:55:23.478238 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90c085c-1f31-4972-832e-f03e6cc50cfd None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:55:23.514162 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd24c10d-6102-43a1-b78f-bad35562efb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:55:23.514770 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd24c10d-6102-43a1-b78f-bad35562efb8 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:55:23.515044 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd24c10d-6102-43a1-b78f-bad35562efb8 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:55:23.515324 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd24c10d-6102-43a1-b78f-bad35562efb8 None admin] PATH_INFO: `/v3/domains` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:55:23.516073 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd24c10d-6102-43a1-b78f-bad35562efb8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:55:23.516355 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd24c10d-6102-43a1-b78f-bad35562efb8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:55:23.517741 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd24c10d-6102-43a1-b78f-bad35562efb8 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:55:23.530487 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 628/1266] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:55:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:55:23.536281 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f58c73-247c-406f-9877-14603e518c5d None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:55:23.643242 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f58c73-247c-406f-9877-14603e518c5d None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:55:23.666708 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:55:23.683283 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10952bcd-e00f-428c-81b9-bca517a9229f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:55:23.684098 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10952bcd-e00f-428c-81b9-bca517a9229f None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:55:23.684390 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10952bcd-e00f-428c-81b9-bca517a9229f None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:55:23.684674 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10952bcd-e00f-428c-81b9-bca517a9229f None admin] PATH_INFO: `/v3/users/placement` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:55:23.694484 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10952bcd-e00f-428c-81b9-bca517a9229f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:55:23.695979 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10952bcd-e00f-428c-81b9-bca517a9229f None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:55:23.711269 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-10952bcd-e00f-428c-81b9-bca517a9229f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 14:55:23.711269 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:55:23.711269 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:55:23.711269 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:55:23.711269 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:55:23.711269 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:55:23.711269 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:55:23.711269 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:55:23.711269 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:55:23.711269 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:55:23.711269 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:55:23.711269 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:55:23.711269 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:55:23.711269 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:55:23.711269 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:55:23.711269 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:55:23.715766 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:55:23.715766 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:55:23.715766 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:55:23.715766 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:55:23.715766 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:55:23.715766 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:55:23.715766 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:55:23.715766 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:55:23.715766 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:55:23.715766 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:55:23.715766 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:55:23.715766 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:55:23.715766 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:55:23.715766 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:55:23.715766 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:55:23.715766 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:55:23.715766 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:55:23.715766 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:55:23.717233 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:55:23.717233 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:55:23.717233 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:55:23.717233 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:55:23.717233 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:55:23.717233 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:55:23.717233 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:55:23.717233 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:55:23.717233 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 14:55:23.717233 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:55:23.717233 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 639/1267] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:55:23 2022] GET /identity/v3/users/placement => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:55:23.719167 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd24c10d-6102-43a1-b78f-bad35562efb8 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:55:23.820157 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd24c10d-6102-43a1-b78f-bad35562efb8 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:55:23.855982 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e0333a-1e9a-4be8-9b63-df114187c6b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:55:23.856663 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e0333a-1e9a-4be8-9b63-df114187c6b4 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:55:23.856953 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e0333a-1e9a-4be8-9b63-df114187c6b4 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:55:23.857238 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e0333a-1e9a-4be8-9b63-df114187c6b4 None admin] PATH_INFO: `/v3/users` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:55:23.858114 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e0333a-1e9a-4be8-9b63-df114187c6b4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:55:23.858435 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e0333a-1e9a-4be8-9b63-df114187c6b4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:55:23.859882 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e0333a-1e9a-4be8-9b63-df114187c6b4 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:55:23.869550 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:55:23.878804 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 629/1268] 10.222.0.57 () {56 vars in 1143 bytes} [Thu May 19 14:55:23 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:55:23.885099 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10952bcd-e00f-428c-81b9-bca517a9229f None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:55:23.987999 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10952bcd-e00f-428c-81b9-bca517a9229f None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:55:24.022903 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe65c74-8f9c-460c-a045-b4215b175421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:55:24.023698 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe65c74-8f9c-460c-a045-b4215b175421 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:55:24.023986 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe65c74-8f9c-460c-a045-b4215b175421 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:55:24.024267 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe65c74-8f9c-460c-a045-b4215b175421 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:55:24.035275 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe65c74-8f9c-460c-a045-b4215b175421 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:55:24.036714 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe65c74-8f9c-460c-a045-b4215b175421 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:55:24.053519 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-1fe65c74-8f9c-460c-a045-b4215b175421 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:55:24.053519 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:55:24.053519 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:55:24.053519 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:55:24.053519 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:55:24.053519 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:55:24.053519 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:55:24.053519 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:55:24.053519 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:55:24.053519 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:55:24.053519 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:55:24.053519 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:55:24.053519 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:55:24.053519 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:55:24.053519 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:55:24.054755 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:55:24.054755 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:55:24.054755 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:55:24.054755 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:55:24.054755 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:55:24.054755 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:55:24.054755 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:55:24.054755 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:55:24.054755 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:55:24.054755 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:55:24.054755 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:55:24.054755 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:55:24.054755 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:55:24.054755 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:55:24.054755 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:55:24.054755 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:55:24.054755 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:55:24.056588 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:55:24.056588 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:55:24.056588 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:55:24.056588 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:55:24.056588 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 640/1269] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:55:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:55:24.060979 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e0333a-1e9a-4be8-9b63-df114187c6b4 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:55:24.159479 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e0333a-1e9a-4be8-9b63-df114187c6b4 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:55:24.193925 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f40bf74-8088-4b0f-bbff-f9162aa2b04e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:55:24.194586 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f40bf74-8088-4b0f-bbff-f9162aa2b04e None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:55:24.194872 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f40bf74-8088-4b0f-bbff-f9162aa2b04e None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:55:24.195154 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f40bf74-8088-4b0f-bbff-f9162aa2b04e None admin] PATH_INFO: `/v3/domains` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:55:24.195875 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f40bf74-8088-4b0f-bbff-f9162aa2b04e None admin] RBAC: Adding query filter params (name=Default) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:55:24.196184 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f40bf74-8088-4b0f-bbff-f9162aa2b04e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:55:24.198125 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f40bf74-8088-4b0f-bbff-f9162aa2b04e None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:55:24.209045 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 630/1270] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:55:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:55:24.215079 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe65c74-8f9c-460c-a045-b4215b175421 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:55:24.247174 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:55:24.321748 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe65c74-8f9c-460c-a045-b4215b175421 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:55:24.355690 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d64c9f-ff34-4454-b728-76fd4151d719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:55:24.356487 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d64c9f-ff34-4454-b728-76fd4151d719 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:55:24.356774 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d64c9f-ff34-4454-b728-76fd4151d719 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:55:24.357055 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d64c9f-ff34-4454-b728-76fd4151d719 None admin] PATH_INFO: `/v3/projects/service` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:55:24.363652 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d64c9f-ff34-4454-b728-76fd4151d719 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:55:24.365090 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d64c9f-ff34-4454-b728-76fd4151d719 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:55:24.379900 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-b8d64c9f-ff34-4454-b728-76fd4151d719 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:55:24.379900 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:55:24.379900 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:55:24.379900 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:55:24.379900 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:55:24.379900 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:55:24.379900 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:55:24.379900 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:55:24.379900 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:55:24.379900 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:55:24.379900 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:55:24.379900 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:55:24.379900 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:55:24.379900 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:55:24.379900 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:55:24.381193 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:55:24.381193 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:55:24.381193 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:55:24.381193 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:55:24.381193 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:55:24.381193 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:55:24.381193 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:55:24.381193 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:55:24.381193 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:55:24.381193 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:55:24.381193 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:55:24.381193 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:55:24.381193 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:55:24.381193 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:55:24.381193 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:55:24.381193 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:55:24.381193 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:55:24.382320 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:55:24.382320 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:55:24.382320 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:55:24.382320 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:55:24.382320 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:55:24.382320 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:55:24.382320 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:55:24.382320 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:55:24.382320 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 641/1271] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:55:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:55:24.387424 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f40bf74-8088-4b0f-bbff-f9162aa2b04e None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:55:24.446463 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:55:24.488591 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f40bf74-8088-4b0f-bbff-f9162aa2b04e None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:55:24.522444 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad51dcfc-5eca-4d94-9fe3-7045fa21dbf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:55:24.523102 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad51dcfc-5eca-4d94-9fe3-7045fa21dbf3 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:55:24.523389 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad51dcfc-5eca-4d94-9fe3-7045fa21dbf3 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:55:24.523668 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad51dcfc-5eca-4d94-9fe3-7045fa21dbf3 None admin] PATH_INFO: `/v3/projects` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:55:24.524486 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad51dcfc-5eca-4d94-9fe3-7045fa21dbf3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:55:24.524800 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad51dcfc-5eca-4d94-9fe3-7045fa21dbf3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:55:24.526263 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad51dcfc-5eca-4d94-9fe3-7045fa21dbf3 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:55:24.538102 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 631/1272] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:55:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:55:24.544900 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d64c9f-ff34-4454-b728-76fd4151d719 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:55:24.636718 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:55:24.646620 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d64c9f-ff34-4454-b728-76fd4151d719 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:55:24.682387 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e2eda0-ac1f-4068-8283-6a3be64c92d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:55:24.683290 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e2eda0-ac1f-4068-8283-6a3be64c92d1 None admin] REQUEST_METHOD: `PUT` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:55:24.683581 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e2eda0-ac1f-4068-8283-6a3be64c92d1 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:55:24.683863 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e2eda0-ac1f-4068-8283-6a3be64c92d1 None admin] PATH_INFO: `/v3/projects/fc99d742ae004954b6d841293b5fa79c/users/e80cdca4cf964f308a7618f89cce7b1a/roles/028379e01f9245e9b8fc56e8b9e8a6ad` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:55:24.685864 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e2eda0-ac1f-4068-8283-6a3be64c92d1 None admin] RBAC: Authorizing `identity:create_grant(project_id=fc99d742ae004954b6d841293b5fa79c, user_id=e80cdca4cf964f308a7618f89cce7b1a, role_id=028379e01f9245e9b8fc56e8b9e8a6ad)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:55:24.687307 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e2eda0-ac1f-4068-8283-6a3be64c92d1 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:55:24.700492 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 642/1273] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 14:55:24 2022] PUT /identity/v3/projects/fc99d742ae004954b6d841293b5fa79c/users/e80cdca4cf964f308a7618f89cce7b1a/roles/028379e01f9245e9b8fc56e8b9e8a6ad => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:55:26.909738 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe7b9ec-c56e-4a44-8259-7fded758063d None None] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:55:26.910356 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe7b9ec-c56e-4a44-8259-7fded758063d None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:55:26.910356 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe7b9ec-c56e-4a44-8259-7fded758063d None None] PATH_INFO: `/` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:55:26.911769 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 632/1274] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:55:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:55:26.917023 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77ea458-fd98-4750-9684-c501213ac84c None None] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:55:26.917427 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77ea458-fd98-4750-9684-c501213ac84c None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:55:26.917591 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77ea458-fd98-4750-9684-c501213ac84c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:55:26.934726 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.auth.core [None req-a77ea458-fd98-4750-9684-c501213ac84c None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:55:26.935915 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-a77ea458-fd98-4750-9684-c501213ac84c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:55:27.002194 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 643/1275] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:55:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:55:27.026724 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe7b9ec-c56e-4a44-8259-7fded758063d None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:55:27.129440 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe7b9ec-c56e-4a44-8259-7fded758063d None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:55:27.147043 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:55:27.164270 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfaab61-f9af-4d37-b655-5ef9d6dfff7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:55:27.165035 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfaab61-f9af-4d37-b655-5ef9d6dfff7c None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:55:27.165329 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfaab61-f9af-4d37-b655-5ef9d6dfff7c None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:55:27.165606 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfaab61-f9af-4d37-b655-5ef9d6dfff7c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:55:27.175737 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfaab61-f9af-4d37-b655-5ef9d6dfff7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:55:27.183994 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-9cfaab61-f9af-4d37-b655-5ef9d6dfff7c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:55:27.183994 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:55:27.183994 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:55:27.183994 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:55:27.183994 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:55:27.183994 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:55:27.183994 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:55:27.183994 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:55:27.183994 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:55:27.183994 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:55:27.183994 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:55:27.183994 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:55:27.183994 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:55:27.183994 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:55:27.183994 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:55:27.183994 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:55:27.185311 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:55:27.185311 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:55:27.185311 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:55:27.185311 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:55:27.185311 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:55:27.185311 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:55:27.185311 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:55:27.185311 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:55:27.185311 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:55:27.185311 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:55:27.185311 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:55:27.185311 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:55:27.185311 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:55:27.185311 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:55:27.185311 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:55:27.185311 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:55:27.188065 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:55:27.188065 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:55:27.188065 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:55:27.188065 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:55:27.188065 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:55:27.188065 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:55:27.188065 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:55:27.188065 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:55:27.188065 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:55:27.188065 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:55:27.188065 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:55:27.188065 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:55:27.188065 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 633/1276] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 14:55:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:55:27.191662 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77ea458-fd98-4750-9684-c501213ac84c None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:55:27.272541 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:55:27.298385 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77ea458-fd98-4750-9684-c501213ac84c None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:55:27.337617 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0874c2f8-5217-4a8e-8565-31058e8f185a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:55:27.338342 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0874c2f8-5217-4a8e-8565-31058e8f185a None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:55:27.338637 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0874c2f8-5217-4a8e-8565-31058e8f185a None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:55:27.338917 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0874c2f8-5217-4a8e-8565-31058e8f185a None admin] PATH_INFO: `/v3/roles` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:55:27.339668 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0874c2f8-5217-4a8e-8565-31058e8f185a None admin] RBAC: Adding query filter params (name=admin) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:55:27.339982 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0874c2f8-5217-4a8e-8565-31058e8f185a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:55:27.341355 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0874c2f8-5217-4a8e-8565-31058e8f185a None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:55:27.349387 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 644/1277] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 14:55:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:55:27.355848 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfaab61-f9af-4d37-b655-5ef9d6dfff7c None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:55:27.456218 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfaab61-f9af-4d37-b655-5ef9d6dfff7c None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:55:27.489378 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e699a9dd-0280-461c-b643-0b242a7be581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:55:27.490197 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e699a9dd-0280-461c-b643-0b242a7be581 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:55:27.490487 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e699a9dd-0280-461c-b643-0b242a7be581 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:55:27.490769 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e699a9dd-0280-461c-b643-0b242a7be581 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:55:27.501721 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e699a9dd-0280-461c-b643-0b242a7be581 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:55:27.503204 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e699a9dd-0280-461c-b643-0b242a7be581 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:55:27.520281 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-e699a9dd-0280-461c-b643-0b242a7be581 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:55:27.520281 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:55:27.520281 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:55:27.520281 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:55:27.520281 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:55:27.520281 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:55:27.520281 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:55:27.520281 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:55:27.520281 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:55:27.520281 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:55:27.520281 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:55:27.520281 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:55:27.520281 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:55:27.520281 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:55:27.520281 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:55:27.521521 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:55:27.521521 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:55:27.521521 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:55:27.521521 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:55:27.521521 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:55:27.521521 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:55:27.521521 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:55:27.521521 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:55:27.521521 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:55:27.521521 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:55:27.521521 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:55:27.521521 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:55:27.521521 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:55:27.521521 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:55:27.521521 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:55:27.521521 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:55:27.521521 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:55:27.522865 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:55:27.522865 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:55:27.522865 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:55:27.522865 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:55:27.522865 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 634/1278] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:55:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:55:27.527294 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0874c2f8-5217-4a8e-8565-31058e8f185a None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:55:27.627817 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0874c2f8-5217-4a8e-8565-31058e8f185a None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:55:27.661665 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2ac251-f0c0-4b71-a39b-7624b169f721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:55:27.662120 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2ac251-f0c0-4b71-a39b-7624b169f721 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:55:27.662411 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2ac251-f0c0-4b71-a39b-7624b169f721 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:55:27.662692 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2ac251-f0c0-4b71-a39b-7624b169f721 None admin] PATH_INFO: `/v3/domains` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:55:27.663413 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2ac251-f0c0-4b71-a39b-7624b169f721 None admin] RBAC: Adding query filter params (name=Default) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:55:27.663728 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2ac251-f0c0-4b71-a39b-7624b169f721 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:55:27.666270 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2ac251-f0c0-4b71-a39b-7624b169f721 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:55:27.678137 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 645/1279] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:55:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:55:27.684354 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e699a9dd-0280-461c-b643-0b242a7be581 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:55:27.716183 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:55:27.784097 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e699a9dd-0280-461c-b643-0b242a7be581 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:55:27.817443 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d68b735-4351-483a-a2c2-c199da849a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:55:27.818271 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d68b735-4351-483a-a2c2-c199da849a6a None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:55:27.818556 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d68b735-4351-483a-a2c2-c199da849a6a None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:55:27.818829 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d68b735-4351-483a-a2c2-c199da849a6a None admin] PATH_INFO: `/v3/users/placement` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:55:27.826319 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d68b735-4351-483a-a2c2-c199da849a6a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:55:27.827759 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d68b735-4351-483a-a2c2-c199da849a6a None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:55:27.842340 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-3d68b735-4351-483a-a2c2-c199da849a6a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 14:55:27.842340 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:55:27.842340 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:55:27.842340 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:55:27.842340 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:55:27.842340 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:55:27.842340 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:55:27.842340 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:55:27.842340 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:55:27.842340 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:55:27.842340 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:55:27.842340 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:55:27.842340 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:55:27.842340 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:55:27.842340 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:55:27.842340 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:55:27.843661 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:55:27.843661 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:55:27.843661 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:55:27.843661 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:55:27.843661 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:55:27.843661 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:55:27.843661 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:55:27.843661 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:55:27.843661 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:55:27.843661 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:55:27.843661 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:55:27.843661 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:55:27.843661 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:55:27.843661 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:55:27.843661 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:55:27.843661 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:55:27.843661 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:55:27.843661 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:55:27.844848 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:55:27.844848 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:55:27.844848 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:55:27.844848 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:55:27.844848 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:55:27.844848 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:55:27.844848 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:55:27.844848 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:55:27.844848 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 14:55:27.844848 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:55:27.844848 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 635/1280] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:55:27 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:55:27.849323 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2ac251-f0c0-4b71-a39b-7624b169f721 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:55:27.853863 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:55:27.947625 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2ac251-f0c0-4b71-a39b-7624b169f721 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:55:27.981984 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc110c0-e37a-4bc4-a72c-7e2bc09aa44b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:55:27.982585 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc110c0-e37a-4bc4-a72c-7e2bc09aa44b None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:55:27.982876 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc110c0-e37a-4bc4-a72c-7e2bc09aa44b None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:55:27.983160 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc110c0-e37a-4bc4-a72c-7e2bc09aa44b None admin] PATH_INFO: `/v3/users` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:55:27.983997 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc110c0-e37a-4bc4-a72c-7e2bc09aa44b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:55:27.984321 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc110c0-e37a-4bc4-a72c-7e2bc09aa44b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:55:27.985761 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc110c0-e37a-4bc4-a72c-7e2bc09aa44b None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:55:28.002560 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 646/1281] 10.222.0.57 () {56 vars in 1143 bytes} [Thu May 19 14:55:27 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:55:28.008308 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d68b735-4351-483a-a2c2-c199da849a6a None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:55:28.098638 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:55:28.108887 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d68b735-4351-483a-a2c2-c199da849a6a None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:55:28.142929 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d8d3f1-527c-44ba-9e39-c21cf80502c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:55:28.143571 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d8d3f1-527c-44ba-9e39-c21cf80502c5 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:55:28.143855 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d8d3f1-527c-44ba-9e39-c21cf80502c5 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:55:28.144132 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d8d3f1-527c-44ba-9e39-c21cf80502c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:55:28.209480 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d8d3f1-527c-44ba-9e39-c21cf80502c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:55:28.211075 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d8d3f1-527c-44ba-9e39-c21cf80502c5 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:55:28.299509 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:55:28.312156 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 636/1282] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 14:55:28 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:55:28.318264 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc110c0-e37a-4bc4-a72c-7e2bc09aa44b None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:55:28.380232 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:55:28.418723 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc110c0-e37a-4bc4-a72c-7e2bc09aa44b None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:55:28.453377 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913a6db5-315a-4f7f-8c26-4ee94a48da8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:55:28.454187 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913a6db5-315a-4f7f-8c26-4ee94a48da8d None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:55:28.454474 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913a6db5-315a-4f7f-8c26-4ee94a48da8d None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:55:28.454750 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913a6db5-315a-4f7f-8c26-4ee94a48da8d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:55:28.464863 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913a6db5-315a-4f7f-8c26-4ee94a48da8d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:55:28.466334 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913a6db5-315a-4f7f-8c26-4ee94a48da8d None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:55:28.483848 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-913a6db5-315a-4f7f-8c26-4ee94a48da8d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:55:28.483848 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:55:28.483848 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:55:28.483848 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:55:28.483848 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:55:28.483848 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:55:28.483848 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:55:28.483848 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:55:28.483848 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:55:28.483848 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:55:28.483848 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:55:28.483848 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:55:28.483848 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:55:28.483848 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:55:28.483848 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:55:28.485069 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:55:28.485069 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:55:28.485069 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:55:28.485069 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:55:28.485069 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:55:28.485069 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:55:28.485069 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:55:28.485069 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:55:28.485069 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:55:28.485069 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:55:28.485069 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:55:28.485069 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:55:28.485069 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:55:28.485069 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:55:28.485069 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:55:28.485069 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:55:28.485069 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:55:28.486480 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:55:28.486480 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:55:28.486480 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:55:28.486480 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:55:28.486480 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 647/1283] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 14:55:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:55:28.490764 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d8d3f1-527c-44ba-9e39-c21cf80502c5 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:55:28.589751 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d8d3f1-527c-44ba-9e39-c21cf80502c5 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:55:28.623010 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca13b58-eab4-4640-aed4-b99b32659146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:55:28.623673 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca13b58-eab4-4640-aed4-b99b32659146 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:55:28.623960 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca13b58-eab4-4640-aed4-b99b32659146 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:55:28.624238 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca13b58-eab4-4640-aed4-b99b32659146 None admin] PATH_INFO: `/v3/domains` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:55:28.624955 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca13b58-eab4-4640-aed4-b99b32659146 None admin] RBAC: Adding query filter params (name=Default) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:55:28.625268 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca13b58-eab4-4640-aed4-b99b32659146 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:55:28.626740 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca13b58-eab4-4640-aed4-b99b32659146 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:55:28.638452 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 637/1284] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 14:55:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:55:28.644134 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913a6db5-315a-4f7f-8c26-4ee94a48da8d None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:55:28.744911 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913a6db5-315a-4f7f-8c26-4ee94a48da8d None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:55:28.758519 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:55:28.778778 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6115a2-be53-41a8-a336-d36326251c2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:55:28.779569 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6115a2-be53-41a8-a336-d36326251c2a None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:55:28.779858 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6115a2-be53-41a8-a336-d36326251c2a None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:55:28.780140 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6115a2-be53-41a8-a336-d36326251c2a None admin] PATH_INFO: `/v3/projects/service` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:55:28.786714 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6115a2-be53-41a8-a336-d36326251c2a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:55:28.788152 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6115a2-be53-41a8-a336-d36326251c2a None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:55:28.802165 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application [None req-1c6115a2-be53-41a8-a336-d36326251c2a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:55:28.802165 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:55:28.802165 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:55:28.802165 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:55:28.802165 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:55:28.802165 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:55:28.802165 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:55:28.802165 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:55:28.802165 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:55:28.802165 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:55:28.802165 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:55:28.802165 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:55:28.802165 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:55:28.802165 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:55:28.802165 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:55:28.803367 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:55:28.803367 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:55:28.803367 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:55:28.803367 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:55:28.803367 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:55:28.803367 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:55:28.803367 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:55:28.803367 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:55:28.803367 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application with Lock( May 19 14:55:28.803367 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:55:28.803367 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._enter() May 19 14:55:28.803367 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:55:28.803367 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:55:28.803367 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:55:28.803367 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.creator() May 19 14:55:28.803367 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:55:28.803367 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application created_value = creator( May 19 14:55:28.804672 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:55:28.804672 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:55:28.804672 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:55:28.804672 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:55:28.804672 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:55:28.804672 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:55:28.804672 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:55:28.804672 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: ERROR keystone.server.flask.application May 19 14:55:28.804672 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 648/1285] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 14:55:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:55:28.809771 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca13b58-eab4-4640-aed4-b99b32659146 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:55:28.859220 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:55:28.908909 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca13b58-eab4-4640-aed4-b99b32659146 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:55:28.942659 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be02f722-cf48-4d26-9a2e-40276b92b8c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:55:28.943314 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be02f722-cf48-4d26-9a2e-40276b92b8c8 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:55:28.943602 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be02f722-cf48-4d26-9a2e-40276b92b8c8 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:55:28.943882 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be02f722-cf48-4d26-9a2e-40276b92b8c8 None admin] PATH_INFO: `/v3/projects` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:55:28.944664 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be02f722-cf48-4d26-9a2e-40276b92b8c8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:55:28.944976 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be02f722-cf48-4d26-9a2e-40276b92b8c8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:55:28.946569 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be02f722-cf48-4d26-9a2e-40276b92b8c8 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:55:28.958419 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 638/1286] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 14:55:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:55:28.964773 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6115a2-be53-41a8-a336-d36326251c2a None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:55:29.066512 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6115a2-be53-41a8-a336-d36326251c2a None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:55:29.099137 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975d0509-ef3c-4af6-8d74-fb9744e88a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:55:29.099789 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975d0509-ef3c-4af6-8d74-fb9744e88a2b None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:55:29.100080 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975d0509-ef3c-4af6-8d74-fb9744e88a2b None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:55:29.100363 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975d0509-ef3c-4af6-8d74-fb9744e88a2b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:55:29.101191 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975d0509-ef3c-4af6-8d74-fb9744e88a2b None admin] RBAC: Adding query filter params (role.id=028379e01f9245e9b8fc56e8b9e8a6ad, scope.project.id=fc99d742ae004954b6d841293b5fa79c, user.id=e80cdca4cf964f308a7618f89cce7b1a) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:55:29.101508 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975d0509-ef3c-4af6-8d74-fb9744e88a2b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:55:29.103576 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975d0509-ef3c-4af6-8d74-fb9744e88a2b None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:55:29.109370 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 649/1287] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 14:55:28 2022] GET /identity/v3/role_assignments?user.id=e80cdca4cf964f308a7618f89cce7b1a&scope.project.id=fc99d742ae004954b6d841293b5fa79c&role.id=028379e01f9245e9b8fc56e8b9e8a6ad => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:55:31.406277 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7553bc-3a1b-48a3-978d-c728da6df70a None None] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:55:31.406832 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7553bc-3a1b-48a3-978d-c728da6df70a None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:55:31.406832 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7553bc-3a1b-48a3-978d-c728da6df70a None None] PATH_INFO: `/` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:55:31.408202 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 639/1288] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:55:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:55:31.413368 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb6f9df-316a-4aee-aa75-aa712dc2825d None None] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:55:31.413835 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb6f9df-316a-4aee-aa75-aa712dc2825d None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:55:31.414011 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb6f9df-316a-4aee-aa75-aa712dc2825d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:55:31.431445 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.auth.core [None req-cdb6f9df-316a-4aee-aa75-aa712dc2825d None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:55:31.432640 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-cdb6f9df-316a-4aee-aa75-aa712dc2825d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:55:31.462461 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:55:31.501057 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 650/1289] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:55:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:55:31.523058 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7553bc-3a1b-48a3-978d-c728da6df70a None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:55:31.625430 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7553bc-3a1b-48a3-978d-c728da6df70a None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:55:31.636007 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:55:31.659759 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa16ad7-0c5c-4a17-b188-5bfe48cc1e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:55:31.660487 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa16ad7-0c5c-4a17-b188-5bfe48cc1e80 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:55:31.660772 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa16ad7-0c5c-4a17-b188-5bfe48cc1e80 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:55:31.661050 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa16ad7-0c5c-4a17-b188-5bfe48cc1e80 None admin] PATH_INFO: `/v3/services/placement` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:55:31.661707 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa16ad7-0c5c-4a17-b188-5bfe48cc1e80 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:55:31.663129 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa16ad7-0c5c-4a17-b188-5bfe48cc1e80 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:55:31.675663 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-efa16ad7-0c5c-4a17-b188-5bfe48cc1e80 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 19 14:55:31.675663 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:55:31.675663 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:55:31.675663 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:55:31.675663 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:55:31.675663 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:55:31.675663 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:55:31.675663 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:55:31.675663 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 14:55:31.675663 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:55:31.675663 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:55:31.675663 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:55:31.675663 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:55:31.675663 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:55:31.675663 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:55:31.675663 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:55:31.675663 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:55:31.677069 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:55:31.677069 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:55:31.677069 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:55:31.677069 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:55:31.677069 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:55:31.677069 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:55:31.677069 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:55:31.677069 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:55:31.677069 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:55:31.677069 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:55:31.677069 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:55:31.677069 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:55:31.677069 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:55:31.677069 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:55:31.677069 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:55:31.677069 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:55:31.678426 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:55:31.678426 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:55:31.678426 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:55:31.678426 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:55:31.678426 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:55:31.678426 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:55:31.678426 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:55:31.678426 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:55:31.678426 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:55:31.678426 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:55:31.678426 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:55:31.678426 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:55:31.678426 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 14:55:31.678426 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:55:31.678426 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 640/1290] 10.222.0.57 () {56 vars in 1117 bytes} [Thu May 19 14:55:31 2022] GET /identity/v3/services/placement => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:55:31.683060 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb6f9df-316a-4aee-aa75-aa712dc2825d None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:55:31.782951 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb6f9df-316a-4aee-aa75-aa712dc2825d None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:55:31.816861 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a204ed-7100-47ee-8168-d45c96768cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:55:31.817523 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a204ed-7100-47ee-8168-d45c96768cba None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:55:31.817812 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a204ed-7100-47ee-8168-d45c96768cba None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:55:31.818124 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a204ed-7100-47ee-8168-d45c96768cba None admin] PATH_INFO: `/v3/services` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:55:31.818834 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a204ed-7100-47ee-8168-d45c96768cba None admin] RBAC: Adding query filter params (name=placement) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:55:31.819141 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a204ed-7100-47ee-8168-d45c96768cba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:55:31.820520 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a204ed-7100-47ee-8168-d45c96768cba None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:55:31.826277 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 651/1291] 10.222.0.57 () {56 vars in 1116 bytes} [Thu May 19 14:55:31 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:55:31.832168 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa16ad7-0c5c-4a17-b188-5bfe48cc1e80 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:55:31.930447 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa16ad7-0c5c-4a17-b188-5bfe48cc1e80 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:55:31.963880 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee35460f-28e8-4827-a51e-60ba01c855fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:55:31.964534 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee35460f-28e8-4827-a51e-60ba01c855fe None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:55:31.964820 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee35460f-28e8-4827-a51e-60ba01c855fe None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:55:31.965101 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee35460f-28e8-4827-a51e-60ba01c855fe None admin] PATH_INFO: `/v3/services` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:55:31.965802 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee35460f-28e8-4827-a51e-60ba01c855fe None admin] RBAC: Adding query filter params (type=placement) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:55:31.966146 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee35460f-28e8-4827-a51e-60ba01c855fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:55:31.967527 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee35460f-28e8-4827-a51e-60ba01c855fe None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:55:31.971915 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 641/1292] 10.222.0.57 () {56 vars in 1116 bytes} [Thu May 19 14:55:31 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:55:34.209478 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40e3acf-2c8b-4f15-a9f7-9a2bc786fe54 None None] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:55:34.210052 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40e3acf-2c8b-4f15-a9f7-9a2bc786fe54 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:55:34.210157 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40e3acf-2c8b-4f15-a9f7-9a2bc786fe54 None None] PATH_INFO: `/` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:55:34.211475 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 652/1293] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:55:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:55:34.217248 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f2df0d-0580-420a-8a25-0920cc53083d None None] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:55:34.217691 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f2df0d-0580-420a-8a25-0920cc53083d None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:55:34.217802 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f2df0d-0580-420a-8a25-0920cc53083d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:55:34.234561 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.auth.core [None req-35f2df0d-0580-420a-8a25-0920cc53083d None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:55:34.235749 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.fernet_utils [None req-35f2df0d-0580-420a-8a25-0920cc53083d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:55:34.265055 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:55:34.306435 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 642/1294] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:55:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:55:34.332021 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40e3acf-2c8b-4f15-a9f7-9a2bc786fe54 None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:55:34.368608 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:55:34.440576 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40e3acf-2c8b-4f15-a9f7-9a2bc786fe54 None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:55:34.474234 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128d3e15-4555-49cc-800e-fea5f445f5f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:55:34.474828 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128d3e15-4555-49cc-800e-fea5f445f5f3 None admin] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:55:34.475118 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128d3e15-4555-49cc-800e-fea5f445f5f3 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:55:34.475408 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128d3e15-4555-49cc-800e-fea5f445f5f3 None admin] PATH_INFO: `/v3/services` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:55:34.476133 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128d3e15-4555-49cc-800e-fea5f445f5f3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:55:34.477767 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128d3e15-4555-49cc-800e-fea5f445f5f3 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:55:34.490737 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 653/1295] 10.222.0.57 () {60 vars in 1141 bytes} [Thu May 19 14:55:34 2022] POST /identity/v3/services => generated 246 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 14:55:36.757022 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58774ee9-eaff-43de-a2a4-881dfed7abce None None] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:55:36.757636 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58774ee9-eaff-43de-a2a4-881dfed7abce None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:55:36.757636 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58774ee9-eaff-43de-a2a4-881dfed7abce None None] PATH_INFO: `/` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:55:36.759055 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 643/1296] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:55:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:55:36.764454 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7933cb5f-f97e-4c17-a368-4f424b07f8bb None None] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:55:36.764905 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7933cb5f-f97e-4c17-a368-4f424b07f8bb None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:55:36.765017 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7933cb5f-f97e-4c17-a368-4f424b07f8bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:55:36.782227 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.auth.core [None req-7933cb5f-f97e-4c17-a368-4f424b07f8bb None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:55:36.783434 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-7933cb5f-f97e-4c17-a368-4f424b07f8bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:55:36.857388 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 654/1297] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:55:36 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:55:36.880077 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58774ee9-eaff-43de-a2a4-881dfed7abce None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:55:36.982540 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58774ee9-eaff-43de-a2a4-881dfed7abce None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:55:37.015683 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80620518-be33-4078-ab83-f75e0f2fe7e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:55:37.016477 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80620518-be33-4078-ab83-f75e0f2fe7e4 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:55:37.016762 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80620518-be33-4078-ab83-f75e0f2fe7e4 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:55:37.017038 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80620518-be33-4078-ab83-f75e0f2fe7e4 None admin] PATH_INFO: `/v3/services/placement` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:55:37.017690 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80620518-be33-4078-ab83-f75e0f2fe7e4 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:55:37.019088 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80620518-be33-4078-ab83-f75e0f2fe7e4 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:55:37.031414 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-80620518-be33-4078-ab83-f75e0f2fe7e4 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 19 14:55:37.031414 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:55:37.031414 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:55:37.031414 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:55:37.031414 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:55:37.031414 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:55:37.031414 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:55:37.031414 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:55:37.031414 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 14:55:37.031414 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:55:37.031414 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:55:37.031414 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:55:37.031414 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:55:37.031414 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:55:37.031414 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:55:37.031414 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:55:37.031414 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:55:37.032840 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:55:37.032840 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:55:37.032840 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:55:37.032840 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:55:37.032840 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:55:37.032840 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:55:37.032840 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:55:37.032840 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:55:37.032840 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:55:37.032840 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:55:37.032840 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:55:37.032840 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:55:37.032840 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:55:37.032840 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:55:37.032840 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:55:37.032840 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:55:37.034185 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:55:37.034185 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:55:37.034185 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:55:37.034185 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:55:37.034185 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:55:37.034185 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:55:37.034185 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:55:37.034185 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:55:37.034185 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:55:37.034185 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:55:37.034185 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:55:37.034185 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:55:37.034185 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 14:55:37.034185 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:55:37.034185 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 644/1298] 10.222.0.57 () {56 vars in 1117 bytes} [Thu May 19 14:55:36 2022] GET /identity/v3/services/placement => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:55:37.038928 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7933cb5f-f97e-4c17-a368-4f424b07f8bb None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:55:37.057044 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:55:37.138779 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7933cb5f-f97e-4c17-a368-4f424b07f8bb None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:55:37.172099 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87db5201-80a7-4cd0-86fd-165308220c65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:55:37.172759 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87db5201-80a7-4cd0-86fd-165308220c65 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:55:37.173050 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87db5201-80a7-4cd0-86fd-165308220c65 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:55:37.173332 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87db5201-80a7-4cd0-86fd-165308220c65 None admin] PATH_INFO: `/v3/services` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:55:37.174069 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87db5201-80a7-4cd0-86fd-165308220c65 None admin] RBAC: Adding query filter params (name=placement) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:55:37.174383 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87db5201-80a7-4cd0-86fd-165308220c65 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:55:37.175752 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87db5201-80a7-4cd0-86fd-165308220c65 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:55:37.180872 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 655/1299] 10.222.0.57 () {56 vars in 1116 bytes} [Thu May 19 14:55:37 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:55:37.187162 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80620518-be33-4078-ab83-f75e0f2fe7e4 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:55:37.219632 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:55:37.288593 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80620518-be33-4078-ab83-f75e0f2fe7e4 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:55:37.321577 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f1ba79-54ca-40b2-b161-7e7a08851a82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:55:37.322259 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f1ba79-54ca-40b2-b161-7e7a08851a82 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:55:37.322547 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f1ba79-54ca-40b2-b161-7e7a08851a82 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:55:37.322829 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f1ba79-54ca-40b2-b161-7e7a08851a82 None admin] PATH_INFO: `/v3/endpoints` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:55:37.323619 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f1ba79-54ca-40b2-b161-7e7a08851a82 None admin] RBAC: Adding query filter params (interface=public, service_id=21b5584db32340418f9e0894b37344de, region_id=RegionOne) {{(pid=281786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:55:37.323940 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f1ba79-54ca-40b2-b161-7e7a08851a82 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:55:37.325313 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f1ba79-54ca-40b2-b161-7e7a08851a82 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:55:37.330383 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 645/1300] 10.222.0.57 () {56 vars in 1251 bytes} [Thu May 19 14:55:37 2022] GET /identity/v3/endpoints?service_id=21b5584db32340418f9e0894b37344de&interface=public®ion_id=RegionOne => generated 194 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:55:37.336098 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87db5201-80a7-4cd0-86fd-165308220c65 None admin] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:55:37.419116 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:55:37.438619 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87db5201-80a7-4cd0-86fd-165308220c65 None admin] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:55:37.473176 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b00940-ab8f-482b-8c1c-8cc5b00ea0d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:55:37.473863 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b00940-ab8f-482b-8c1c-8cc5b00ea0d3 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:55:37.474161 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b00940-ab8f-482b-8c1c-8cc5b00ea0d3 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:55:37.474447 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b00940-ab8f-482b-8c1c-8cc5b00ea0d3 None admin] PATH_INFO: `/v3/services` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:55:37.475138 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b00940-ab8f-482b-8c1c-8cc5b00ea0d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:55:37.476508 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b00940-ab8f-482b-8c1c-8cc5b00ea0d3 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:55:37.483581 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 656/1301] 10.222.0.57 () {56 vars in 1087 bytes} [Thu May 19 14:55:37 2022] GET /identity/v3/services => generated 2191 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 14:55:39.705803 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ecb48c-7181-472f-89a4-9b9f62a8154d None None] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:55:39.706374 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ecb48c-7181-472f-89a4-9b9f62a8154d None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:55:39.706374 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ecb48c-7181-472f-89a4-9b9f62a8154d None None] PATH_INFO: `/` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:55:39.707769 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 646/1302] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:55:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:55:39.712918 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba98e53b-99f1-4d1d-8b19-9c74b63281fd None None] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:55:39.713369 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba98e53b-99f1-4d1d-8b19-9c74b63281fd None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:55:39.713480 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba98e53b-99f1-4d1d-8b19-9c74b63281fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:55:39.730565 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.auth.core [None req-ba98e53b-99f1-4d1d-8b19-9c74b63281fd None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:55:39.731763 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-ba98e53b-99f1-4d1d-8b19-9c74b63281fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:55:39.798224 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 657/1303] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:55:39 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:55:39.822937 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ecb48c-7181-472f-89a4-9b9f62a8154d None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:55:39.916590 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:55:39.923478 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ecb48c-7181-472f-89a4-9b9f62a8154d None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:55:39.956993 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d38a60f-6acf-4b9a-9a08-3026f06eb7b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:55:39.957773 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d38a60f-6acf-4b9a-9a08-3026f06eb7b0 None admin] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:55:39.958094 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d38a60f-6acf-4b9a-9a08-3026f06eb7b0 None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:55:39.958371 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d38a60f-6acf-4b9a-9a08-3026f06eb7b0 None admin] PATH_INFO: `/v3/services/placement` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:55:39.959020 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d38a60f-6acf-4b9a-9a08-3026f06eb7b0 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:55:39.960409 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d38a60f-6acf-4b9a-9a08-3026f06eb7b0 None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:55:39.971988 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application [None req-0d38a60f-6acf-4b9a-9a08-3026f06eb7b0 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 19 14:55:39.971988 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:55:39.971988 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:55:39.971988 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:55:39.971988 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:55:39.971988 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:55:39.971988 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:55:39.971988 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:55:39.971988 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 14:55:39.971988 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:55:39.971988 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:55:39.971988 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:55:39.971988 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:55:39.971988 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:55:39.971988 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:55:39.971988 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:55:39.971988 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:55:39.973352 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:55:39.973352 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:55:39.973352 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:55:39.973352 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:55:39.973352 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:55:39.973352 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:55:39.973352 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:55:39.973352 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:55:39.973352 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:55:39.973352 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:55:39.973352 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:55:39.973352 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:55:39.973352 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:55:39.973352 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:55:39.973352 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:55:39.973352 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:55:39.974590 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:55:39.974590 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application with Lock( May 19 14:55:39.974590 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:55:39.974590 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self._enter() May 19 14:55:39.974590 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:55:39.974590 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:55:39.974590 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:55:39.974590 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application return self.creator() May 19 14:55:39.974590 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:55:39.974590 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application created_value = creator( May 19 14:55:39.974590 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:55:39.974590 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:55:39.974590 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 14:55:39.974590 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: ERROR keystone.server.flask.application May 19 14:55:39.974590 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 647/1304] 10.222.0.57 () {56 vars in 1117 bytes} [Thu May 19 14:55:39 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:55:39.980092 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba98e53b-99f1-4d1d-8b19-9c74b63281fd None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:55:40.056481 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:55:40.080169 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba98e53b-99f1-4d1d-8b19-9c74b63281fd None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:55:40.114257 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af667d22-7723-44fa-8e3d-39d99ead8733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:55:40.114920 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af667d22-7723-44fa-8e3d-39d99ead8733 None admin] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:55:40.115213 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af667d22-7723-44fa-8e3d-39d99ead8733 None admin] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:55:40.115496 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af667d22-7723-44fa-8e3d-39d99ead8733 None admin] PATH_INFO: `/v3/services` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:55:40.116211 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af667d22-7723-44fa-8e3d-39d99ead8733 None admin] RBAC: Adding query filter params (name=placement) {{(pid=281785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:55:40.116524 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af667d22-7723-44fa-8e3d-39d99ead8733 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:55:40.118657 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af667d22-7723-44fa-8e3d-39d99ead8733 None admin] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:55:40.123906 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 658/1305] 10.222.0.57 () {56 vars in 1116 bytes} [Thu May 19 14:55:39 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:55:40.129713 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d38a60f-6acf-4b9a-9a08-3026f06eb7b0 None admin] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:55:40.228401 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d38a60f-6acf-4b9a-9a08-3026f06eb7b0 None admin] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:55:40.261563 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fbea11-5d52-4ad9-aa05-27a4b1ccf3db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c487f3640744d568515b03847007a04', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:55:40.262249 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fbea11-5d52-4ad9-aa05-27a4b1ccf3db None admin] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:55:40.262544 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fbea11-5d52-4ad9-aa05-27a4b1ccf3db None admin] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:55:40.262853 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fbea11-5d52-4ad9-aa05-27a4b1ccf3db None admin] PATH_INFO: `/v3/endpoints` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:55:40.263607 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fbea11-5d52-4ad9-aa05-27a4b1ccf3db None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:55:40.264999 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fbea11-5d52-4ad9-aa05-27a4b1ccf3db None admin] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:55:40.283365 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 648/1306] 10.222.0.57 () {60 vars in 1144 bytes} [Thu May 19 14:55:40 2022] POST /identity/v3/endpoints => generated 331 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:55:50.121099 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=30 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:55:50.278910 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:56:47.262094 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b320d8-c2ba-4c37-acab-9738f529effd None None] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:56:47.262719 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b320d8-c2ba-4c37-acab-9738f529effd None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:56:47.262719 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b320d8-c2ba-4c37-acab-9738f529effd None None] PATH_INFO: `/` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:56:47.264060 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 659/1307] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:56:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:56:47.269995 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f87be16-9081-46a0-9499-70b206c724cf None None] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:56:47.271032 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f87be16-9081-46a0-9499-70b206c724cf None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:56:47.271563 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f87be16-9081-46a0-9499-70b206c724cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:56:47.455315 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.auth.core [None req-2f87be16-9081-46a0-9499-70b206c724cf None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:56:47.457048 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.fernet_utils [None req-2f87be16-9081-46a0-9499-70b206c724cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:56:47.682586 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 649/1308] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:56:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 414 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:56:47.756348 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7474ffc-9607-494f-9a57-13510b8547a2 None None] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:56:47.756783 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7474ffc-9607-494f-9a57-13510b8547a2 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:56:47.757082 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7474ffc-9607-494f-9a57-13510b8547a2 None None] PATH_INFO: `/` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:56:47.758659 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 660/1309] 10.222.0.57 () {54 vars in 964 bytes} [Thu May 19 14:56:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:56:47.770156 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e71beb5-fead-4126-bec6-9a40b65b3d8c None None] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:56:47.770612 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e71beb5-fead-4126-bec6-9a40b65b3d8c None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:56:47.771012 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e71beb5-fead-4126-bec6-9a40b65b3d8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:56:47.923864 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.auth.core [None req-9e71beb5-fead-4126-bec6-9a40b65b3d8c None None] MFA Rules not processed for user `8de553016ea641cdbf9c4a8e08c96a67`. Rule list: `[]` (Enabled: `True`). {{(pid=281786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:56:47.925180 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.fernet_utils [None req-9e71beb5-fead-4126-bec6-9a40b65b3d8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:56:47.980378 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:56:48.092529 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 650/1310] 10.222.0.57 () {58 vars in 1061 bytes} [Thu May 19 14:56:47 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:56:48.102386 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7474ffc-9607-494f-9a57-13510b8547a2 None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:56:48.104130 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-e7474ffc-9607-494f-9a57-13510b8547a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:56:48.344386 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7474ffc-9607-494f-9a57-13510b8547a2 None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:56:48.440192 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f5a57a-b95e-457f-8840-ffb7efa9dede service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8de553016ea641cdbf9c4a8e08c96a67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc99d742ae004954b6d841293b5fa79c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:56:48.440868 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f5a57a-b95e-457f-8840-ffb7efa9dede service neutron] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:56:48.441160 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f5a57a-b95e-457f-8840-ffb7efa9dede service neutron] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:56:48.441444 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f5a57a-b95e-457f-8840-ffb7efa9dede service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:56:48.443209 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-a1f5a57a-b95e-457f-8840-ffb7efa9dede service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:56:48.515504 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:56:48.585278 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f5a57a-b95e-457f-8840-ffb7efa9dede service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:56:48.587428 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f5a57a-b95e-457f-8840-ffb7efa9dede service neutron] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:56:48.610605 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 661/1311] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 14:56:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 509 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:56:51.370344 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808ff02d-373f-4e38-ad8e-60c5f98c6c9f None None] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:56:51.370955 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808ff02d-373f-4e38-ad8e-60c5f98c6c9f None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:56:51.370955 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808ff02d-373f-4e38-ad8e-60c5f98c6c9f None None] PATH_INFO: `/` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:56:51.372306 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 651/1312] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:56:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:56:51.377595 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746ed295-9fda-4663-b66e-5a1a4d3f4cc2 None None] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:56:51.378554 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746ed295-9fda-4663-b66e-5a1a4d3f4cc2 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:56:51.379040 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746ed295-9fda-4663-b66e-5a1a4d3f4cc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:56:51.467136 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.auth.core [None req-746ed295-9fda-4663-b66e-5a1a4d3f4cc2 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:56:51.468353 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-746ed295-9fda-4663-b66e-5a1a4d3f4cc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:56:51.571797 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 662/1313] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:56:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:56:51.610500 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808ff02d-373f-4e38-ad8e-60c5f98c6c9f None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:56:51.612065 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.fernet_utils [None req-808ff02d-373f-4e38-ad8e-60c5f98c6c9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:56:51.730305 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808ff02d-373f-4e38-ad8e-60c5f98c6c9f None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:56:51.808123 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:56:51.809945 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405ac345-738f-44f6-97c6-c83819d1f35c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8de553016ea641cdbf9c4a8e08c96a67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc99d742ae004954b6d841293b5fa79c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:56:51.810611 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405ac345-738f-44f6-97c6-c83819d1f35c service neutron] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:56:51.810911 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405ac345-738f-44f6-97c6-c83819d1f35c service neutron] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:56:51.811231 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405ac345-738f-44f6-97c6-c83819d1f35c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:56:51.813727 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.fernet_utils [None req-405ac345-738f-44f6-97c6-c83819d1f35c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:56:51.936707 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405ac345-738f-44f6-97c6-c83819d1f35c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:56:51.938467 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405ac345-738f-44f6-97c6-c83819d1f35c service neutron] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:56:51.958225 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 652/1314] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 14:56:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:56:54.586579 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846e6b6d-3671-4bab-9c9c-cc11dd184589 None None] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:56:54.587200 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846e6b6d-3671-4bab-9c9c-cc11dd184589 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:56:54.587200 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846e6b6d-3671-4bab-9c9c-cc11dd184589 None None] PATH_INFO: `/` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:56:54.588525 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 663/1315] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:56:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:56:54.593973 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75688f4d-07ca-4d3b-9e27-b9d30139a484 None None] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:56:54.594396 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75688f4d-07ca-4d3b-9e27-b9d30139a484 None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:56:54.594543 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75688f4d-07ca-4d3b-9e27-b9d30139a484 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:56:54.659575 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.auth.core [None req-75688f4d-07ca-4d3b-9e27-b9d30139a484 None None] MFA Rules not processed for user `2950d06ae84d46c081f68af26db145d4`. Rule list: `[]` (Enabled: `True`). {{(pid=281786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:56:54.660795 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.fernet_utils [None req-75688f4d-07ca-4d3b-9e27-b9d30139a484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:56:54.712963 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:56:54.758292 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 653/1316] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:56:54 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 14:56:54.797525 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846e6b6d-3671-4bab-9c9c-cc11dd184589 None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:56:54.799116 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-846e6b6d-3671-4bab-9c9c-cc11dd184589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:56:54.809608 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:56:54.904778 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846e6b6d-3671-4bab-9c9c-cc11dd184589 None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:56:54.979526 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a58df7d-069c-4dfe-9c2c-984190150809 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8de553016ea641cdbf9c4a8e08c96a67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc99d742ae004954b6d841293b5fa79c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:56:54.980175 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a58df7d-069c-4dfe-9c2c-984190150809 service neutron] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:56:54.980467 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a58df7d-069c-4dfe-9c2c-984190150809 service neutron] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:56:54.980755 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a58df7d-069c-4dfe-9c2c-984190150809 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:56:54.982455 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-7a58df7d-069c-4dfe-9c2c-984190150809 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:56:55.046962 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:56:55.094323 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a58df7d-069c-4dfe-9c2c-984190150809 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:56:55.095778 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a58df7d-069c-4dfe-9c2c-984190150809 service neutron] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:56:55.116132 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 664/1317] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 14:56:54 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:56:57.936634 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60926dd-8b6b-4d3d-bb2a-92d3c9cab769 None None] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:56:57.937176 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60926dd-8b6b-4d3d-bb2a-92d3c9cab769 None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:56:57.937176 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60926dd-8b6b-4d3d-bb2a-92d3c9cab769 None None] PATH_INFO: `/` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:56:57.938813 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 654/1318] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:56:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:56:57.944676 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c278fde6-14d4-4001-958b-ef091a1ac81b None None] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:56:57.945116 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c278fde6-14d4-4001-958b-ef091a1ac81b None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:56:57.945241 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c278fde6-14d4-4001-958b-ef091a1ac81b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:56:58.010732 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.auth.core [None req-c278fde6-14d4-4001-958b-ef091a1ac81b None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:56:58.011955 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-c278fde6-14d4-4001-958b-ef091a1ac81b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:56:58.110254 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 665/1319] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:56:57 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:56:58.149327 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60926dd-8b6b-4d3d-bb2a-92d3c9cab769 None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:56:58.150934 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.fernet_utils [None req-d60926dd-8b6b-4d3d-bb2a-92d3c9cab769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:56:58.255558 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60926dd-8b6b-4d3d-bb2a-92d3c9cab769 None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:56:58.332368 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2594b9c6-c0cf-423b-b08d-0d36e8c167fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8de553016ea641cdbf9c4a8e08c96a67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc99d742ae004954b6d841293b5fa79c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:56:58.333027 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2594b9c6-c0cf-423b-b08d-0d36e8c167fc service neutron] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:56:58.333318 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2594b9c6-c0cf-423b-b08d-0d36e8c167fc service neutron] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:56:58.333602 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2594b9c6-c0cf-423b-b08d-0d36e8c167fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:56:58.335541 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.fernet_utils [None req-2594b9c6-c0cf-423b-b08d-0d36e8c167fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:56:58.343120 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:56:58.453984 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2594b9c6-c0cf-423b-b08d-0d36e8c167fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:56:58.455443 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2594b9c6-c0cf-423b-b08d-0d36e8c167fc service neutron] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:56:58.473441 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 655/1320] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 14:56:58 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:57:01.392184 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3a4329-c4b4-4b22-991b-46029515c137 None None] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:57:01.392731 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3a4329-c4b4-4b22-991b-46029515c137 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:57:01.392930 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3a4329-c4b4-4b22-991b-46029515c137 None None] PATH_INFO: `/` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:57:01.394215 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 666/1321] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:57:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:57:01.399787 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9bd1f3-a128-4fd0-beb1-adfd4775d274 None None] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:57:01.400276 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9bd1f3-a128-4fd0-beb1-adfd4775d274 None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:57:01.400375 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9bd1f3-a128-4fd0-beb1-adfd4775d274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:57:01.466794 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.auth.core [None req-3a9bd1f3-a128-4fd0-beb1-adfd4775d274 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:57:01.468005 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.fernet_utils [None req-3a9bd1f3-a128-4fd0-beb1-adfd4775d274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:57:01.528438 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:57:01.569318 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 656/1322] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:57:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:57:01.604649 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3a4329-c4b4-4b22-991b-46029515c137 None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:57:01.606289 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-ea3a4329-c4b4-4b22-991b-46029515c137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:57:01.626272 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:57:01.714751 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3a4329-c4b4-4b22-991b-46029515c137 None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:57:01.790294 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fd38b5-9f4a-4d0b-8cd3-d240012fc07f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8de553016ea641cdbf9c4a8e08c96a67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc99d742ae004954b6d841293b5fa79c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:57:01.790966 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fd38b5-9f4a-4d0b-8cd3-d240012fc07f service neutron] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:57:01.791240 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fd38b5-9f4a-4d0b-8cd3-d240012fc07f service neutron] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:57:01.791519 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fd38b5-9f4a-4d0b-8cd3-d240012fc07f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:57:01.793191 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-76fd38b5-9f4a-4d0b-8cd3-d240012fc07f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:57:01.866180 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:57:01.906492 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fd38b5-9f4a-4d0b-8cd3-d240012fc07f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:57:01.907948 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fd38b5-9f4a-4d0b-8cd3-d240012fc07f service neutron] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:57:01.928740 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 667/1323] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 14:57:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:57:04.239211 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507c14d0-2c48-4327-820b-0da5fe26a107 None None] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:57:04.240041 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507c14d0-2c48-4327-820b-0da5fe26a107 None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:57:04.240041 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507c14d0-2c48-4327-820b-0da5fe26a107 None None] PATH_INFO: `/` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:57:04.241490 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 657/1324] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:57:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:57:04.247035 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29e98a9-3785-4de7-84dd-cf9aa6d8d396 None None] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:57:04.247510 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29e98a9-3785-4de7-84dd-cf9aa6d8d396 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:57:04.247612 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29e98a9-3785-4de7-84dd-cf9aa6d8d396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:57:04.320010 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.auth.core [None req-f29e98a9-3785-4de7-84dd-cf9aa6d8d396 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:57:04.321242 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-f29e98a9-3785-4de7-84dd-cf9aa6d8d396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:57:04.426034 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 668/1325] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:57:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:57:04.479123 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9434078-cea5-420a-832e-2241b6679bb8 None None] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:57:04.479460 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9434078-cea5-420a-832e-2241b6679bb8 None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:57:04.479780 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9434078-cea5-420a-832e-2241b6679bb8 None None] PATH_INFO: `/` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:57:04.481180 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 658/1326] 10.222.0.57 () {54 vars in 964 bytes} [Thu May 19 14:57:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:57:04.487357 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9066bb10-d617-4073-916e-24b11a81e772 None None] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:57:04.487728 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9066bb10-d617-4073-916e-24b11a81e772 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:57:04.488080 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9066bb10-d617-4073-916e-24b11a81e772 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:57:04.518472 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:57:04.564650 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.auth.core [None req-9066bb10-d617-4073-916e-24b11a81e772 None None] MFA Rules not processed for user `8de553016ea641cdbf9c4a8e08c96a67`. Rule list: `[]` (Enabled: `True`). {{(pid=281785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:57:04.566002 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-9066bb10-d617-4073-916e-24b11a81e772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:57:04.643699 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 669/1327] 10.222.0.57 () {58 vars in 1061 bytes} [Thu May 19 14:57:04 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:57:04.653314 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9434078-cea5-420a-832e-2241b6679bb8 None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:57:04.654909 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.fernet_utils [None req-e9434078-cea5-420a-832e-2241b6679bb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:57:04.766321 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9434078-cea5-420a-832e-2241b6679bb8 None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:57:04.844024 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62579321-37e4-4a38-8da9-0fd9d388440a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8de553016ea641cdbf9c4a8e08c96a67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc99d742ae004954b6d841293b5fa79c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:57:04.844579 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62579321-37e4-4a38-8da9-0fd9d388440a service neutron] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:57:04.844870 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62579321-37e4-4a38-8da9-0fd9d388440a service neutron] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:57:04.845155 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62579321-37e4-4a38-8da9-0fd9d388440a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:57:04.846864 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.fernet_utils [None req-62579321-37e4-4a38-8da9-0fd9d388440a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:57:04.865499 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:57:04.963786 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62579321-37e4-4a38-8da9-0fd9d388440a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:57:04.965258 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62579321-37e4-4a38-8da9-0fd9d388440a service neutron] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:57:04.983977 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 659/1328] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 14:57:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:57:07.482429 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7f57e1-d046-4eea-a43e-2fc7ba825063 None None] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:57:07.482991 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7f57e1-d046-4eea-a43e-2fc7ba825063 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:57:07.482991 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7f57e1-d046-4eea-a43e-2fc7ba825063 None None] PATH_INFO: `/` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:57:07.484434 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 670/1329] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:57:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:57:07.489630 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc991bc-8c01-4ff6-9589-a9f02430a247 None None] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:57:07.490053 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc991bc-8c01-4ff6-9589-a9f02430a247 None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:57:07.490332 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc991bc-8c01-4ff6-9589-a9f02430a247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:57:07.556049 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.auth.core [None req-0dc991bc-8c01-4ff6-9589-a9f02430a247 None None] MFA Rules not processed for user `2950d06ae84d46c081f68af26db145d4`. Rule list: `[]` (Enabled: `True`). {{(pid=281786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:57:07.557270 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.fernet_utils [None req-0dc991bc-8c01-4ff6-9589-a9f02430a247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:57:07.622739 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:57:07.656201 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 660/1330] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:57:07 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:57:07.693935 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7f57e1-d046-4eea-a43e-2fc7ba825063 None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:57:07.695494 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-5b7f57e1-d046-4eea-a43e-2fc7ba825063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:57:07.800119 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7f57e1-d046-4eea-a43e-2fc7ba825063 None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:57:07.821876 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:57:07.875624 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ab5afd-9525-4629-bb0f-81543939c8a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8de553016ea641cdbf9c4a8e08c96a67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc99d742ae004954b6d841293b5fa79c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:57:07.876280 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ab5afd-9525-4629-bb0f-81543939c8a0 service neutron] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:57:07.876575 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ab5afd-9525-4629-bb0f-81543939c8a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:57:07.876861 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ab5afd-9525-4629-bb0f-81543939c8a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:57:07.878604 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-15ab5afd-9525-4629-bb0f-81543939c8a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:57:07.987941 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ab5afd-9525-4629-bb0f-81543939c8a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:57:07.989394 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ab5afd-9525-4629-bb0f-81543939c8a0 service neutron] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:57:08.008518 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 671/1331] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 14:57:07 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:57:10.456552 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e6c013-4a66-4097-80eb-63fa0822aad0 None None] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:57:10.457172 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e6c013-4a66-4097-80eb-63fa0822aad0 None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:57:10.457172 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e6c013-4a66-4097-80eb-63fa0822aad0 None None] PATH_INFO: `/` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:57:10.458576 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 661/1332] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:57:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:57:10.464581 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4be72dd-0c3c-4346-bb78-01d05373a13f None None] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:57:10.465063 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4be72dd-0c3c-4346-bb78-01d05373a13f None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:57:10.465175 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4be72dd-0c3c-4346-bb78-01d05373a13f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:57:10.527528 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:57:10.530258 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.auth.core [None req-b4be72dd-0c3c-4346-bb78-01d05373a13f None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:57:10.532088 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-b4be72dd-0c3c-4346-bb78-01d05373a13f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:57:10.629069 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 672/1333] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:57:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:57:10.666147 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e6c013-4a66-4097-80eb-63fa0822aad0 None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:57:10.667691 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.fernet_utils [None req-c1e6c013-4a66-4097-80eb-63fa0822aad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:57:10.772145 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e6c013-4a66-4097-80eb-63fa0822aad0 None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:57:10.845283 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93735f9f-c1a2-4059-bf6b-5449265ea958 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8de553016ea641cdbf9c4a8e08c96a67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc99d742ae004954b6d841293b5fa79c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:57:10.845984 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93735f9f-c1a2-4059-bf6b-5449265ea958 service neutron] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:57:10.846275 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93735f9f-c1a2-4059-bf6b-5449265ea958 service neutron] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:57:10.846554 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93735f9f-c1a2-4059-bf6b-5449265ea958 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:57:10.848213 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.fernet_utils [None req-93735f9f-c1a2-4059-bf6b-5449265ea958 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:57:10.874849 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:57:10.957752 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93735f9f-c1a2-4059-bf6b-5449265ea958 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:57:10.959233 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93735f9f-c1a2-4059-bf6b-5449265ea958 service neutron] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:57:10.977339 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 662/1334] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 14:57:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:57:13.252430 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e034c1-aefd-44a6-9d96-8086b866f145 None None] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:57:13.253011 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e034c1-aefd-44a6-9d96-8086b866f145 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:57:13.253011 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e034c1-aefd-44a6-9d96-8086b866f145 None None] PATH_INFO: `/` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:57:13.254428 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 673/1335] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:57:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:57:13.260266 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cffee8-51c5-4b5c-877e-b882ffcbd1c4 None None] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:57:13.260740 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cffee8-51c5-4b5c-877e-b882ffcbd1c4 None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:57:13.260866 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cffee8-51c5-4b5c-877e-b882ffcbd1c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:57:13.326637 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.auth.core [None req-a7cffee8-51c5-4b5c-877e-b882ffcbd1c4 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:57:13.327871 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.fernet_utils [None req-a7cffee8-51c5-4b5c-877e-b882ffcbd1c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:57:13.405441 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:57:13.431173 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 663/1336] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:57:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:57:13.470882 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e034c1-aefd-44a6-9d96-8086b866f145 None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:57:13.472500 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-a4e034c1-aefd-44a6-9d96-8086b866f145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:57:13.579076 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e034c1-aefd-44a6-9d96-8086b866f145 None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:57:13.607013 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:57:13.654519 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68659f60-40e8-45b9-91a5-0d3e2b11330d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8de553016ea641cdbf9c4a8e08c96a67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc99d742ae004954b6d841293b5fa79c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:57:13.655172 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68659f60-40e8-45b9-91a5-0d3e2b11330d service neutron] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:57:13.655461 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68659f60-40e8-45b9-91a5-0d3e2b11330d service neutron] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:57:13.655741 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68659f60-40e8-45b9-91a5-0d3e2b11330d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:57:13.657463 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-68659f60-40e8-45b9-91a5-0d3e2b11330d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:57:13.769789 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68659f60-40e8-45b9-91a5-0d3e2b11330d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:57:13.771885 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68659f60-40e8-45b9-91a5-0d3e2b11330d service neutron] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:57:13.790381 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 674/1337] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 14:57:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:57:16.401868 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aa4edb-7581-4b1e-a253-b64384c5f1c7 None None] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:57:16.402456 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aa4edb-7581-4b1e-a253-b64384c5f1c7 None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:57:16.402456 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aa4edb-7581-4b1e-a253-b64384c5f1c7 None None] PATH_INFO: `/` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:57:16.403852 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 664/1338] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:57:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:57:16.409067 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac92ff4b-17d4-4a2f-84e8-1245a54d2c73 None None] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:57:16.409498 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac92ff4b-17d4-4a2f-84e8-1245a54d2c73 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:57:16.409652 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac92ff4b-17d4-4a2f-84e8-1245a54d2c73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:57:16.475923 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.auth.core [None req-ac92ff4b-17d4-4a2f-84e8-1245a54d2c73 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:57:16.477134 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-ac92ff4b-17d4-4a2f-84e8-1245a54d2c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:57:16.485498 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:57:16.576822 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 675/1339] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:57:16 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:57:16.612437 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aa4edb-7581-4b1e-a253-b64384c5f1c7 None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:57:16.614029 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.fernet_utils [None req-03aa4edb-7581-4b1e-a253-b64384c5f1c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:57:16.720704 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aa4edb-7581-4b1e-a253-b64384c5f1c7 None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:57:16.793769 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0263e25c-eb12-4989-9565-528dadb48506 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8de553016ea641cdbf9c4a8e08c96a67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc99d742ae004954b6d841293b5fa79c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:57:16.794454 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0263e25c-eb12-4989-9565-528dadb48506 service neutron] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:57:16.794745 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0263e25c-eb12-4989-9565-528dadb48506 service neutron] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:57:16.795026 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0263e25c-eb12-4989-9565-528dadb48506 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:57:16.796697 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.fernet_utils [None req-0263e25c-eb12-4989-9565-528dadb48506 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:57:16.834558 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:57:16.907663 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0263e25c-eb12-4989-9565-528dadb48506 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:57:16.909105 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0263e25c-eb12-4989-9565-528dadb48506 service neutron] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:57:16.927436 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 665/1340] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 14:57:16 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:57:21.186632 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72bc168-432c-4de7-8f70-a4c59c634baa None None] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:57:21.187447 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72bc168-432c-4de7-8f70-a4c59c634baa None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:57:21.187447 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72bc168-432c-4de7-8f70-a4c59c634baa None None] PATH_INFO: `/` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:57:21.188848 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 676/1341] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:57:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:57:21.194711 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be68517-aa51-4f64-b881-3240c2a2b53d None None] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:57:21.195195 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be68517-aa51-4f64-b881-3240c2a2b53d None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:57:21.195308 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be68517-aa51-4f64-b881-3240c2a2b53d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:57:21.268959 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.auth.core [None req-2be68517-aa51-4f64-b881-3240c2a2b53d None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:57:21.270095 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.fernet_utils [None req-2be68517-aa51-4f64-b881-3240c2a2b53d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:57:21.375264 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=100 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:57:21.389342 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 666/1342] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:57:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:57:21.428092 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72bc168-432c-4de7-8f70-a4c59c634baa None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:57:21.430310 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-b72bc168-432c-4de7-8f70-a4c59c634baa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:57:21.528923 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72bc168-432c-4de7-8f70-a4c59c634baa None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:57:21.532609 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831812e9-85ce-49a5-a957-2ad21ad40cf4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8de553016ea641cdbf9c4a8e08c96a67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc99d742ae004954b6d841293b5fa79c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:57:21.533266 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831812e9-85ce-49a5-a957-2ad21ad40cf4 service neutron] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:57:21.533559 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831812e9-85ce-49a5-a957-2ad21ad40cf4 service neutron] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:57:21.533869 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831812e9-85ce-49a5-a957-2ad21ad40cf4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:57:21.543589 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831812e9-85ce-49a5-a957-2ad21ad40cf4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:57:21.545036 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831812e9-85ce-49a5-a957-2ad21ad40cf4 service neutron] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:57:21.555815 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 677/1343] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 14:57:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:57:24.807401 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65095976-7579-45e2-bd4f-5f8dea39096e None None] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:57:24.807967 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65095976-7579-45e2-bd4f-5f8dea39096e None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:57:24.808058 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65095976-7579-45e2-bd4f-5f8dea39096e None None] PATH_INFO: `/` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:57:24.809455 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 667/1344] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:57:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:57:24.815296 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfbb6da-f091-4d9d-a660-aa5c7fafaa51 None None] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:57:24.815825 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfbb6da-f091-4d9d-a660-aa5c7fafaa51 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:57:24.816002 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfbb6da-f091-4d9d-a660-aa5c7fafaa51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:57:24.844243 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.auth.core [None req-ecfbb6da-f091-4d9d-a660-aa5c7fafaa51 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:57:24.845450 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-ecfbb6da-f091-4d9d-a660-aa5c7fafaa51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:57:24.875422 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:57:24.926060 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 678/1345] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:57:24 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:57:24.963817 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65095976-7579-45e2-bd4f-5f8dea39096e None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:57:24.975518 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65095976-7579-45e2-bd4f-5f8dea39096e None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:57:24.979075 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d681c35-0e20-4dee-aaa4-aeb93a03eea0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8de553016ea641cdbf9c4a8e08c96a67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc99d742ae004954b6d841293b5fa79c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:57:24.979724 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d681c35-0e20-4dee-aaa4-aeb93a03eea0 service neutron] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:57:24.980015 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d681c35-0e20-4dee-aaa4-aeb93a03eea0 service neutron] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:57:24.980298 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d681c35-0e20-4dee-aaa4-aeb93a03eea0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:57:24.990107 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d681c35-0e20-4dee-aaa4-aeb93a03eea0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:57:24.991562 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d681c35-0e20-4dee-aaa4-aeb93a03eea0 service neutron] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:57:25.001191 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 668/1346] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 14:57:24 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:57:29.559433 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11999a6e-a5ed-4ba9-9a64-336275a0aad6 None None] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:57:29.559958 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11999a6e-a5ed-4ba9-9a64-336275a0aad6 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:57:29.560072 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11999a6e-a5ed-4ba9-9a64-336275a0aad6 None None] PATH_INFO: `/` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:57:29.561424 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 679/1347] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:57:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:57:29.568088 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e55b95c-e873-435f-a970-508b3534ce8a None None] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:57:29.568579 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e55b95c-e873-435f-a970-508b3534ce8a None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:57:29.568779 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e55b95c-e873-435f-a970-508b3534ce8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:57:29.588182 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.auth.core [None req-3e55b95c-e873-435f-a970-508b3534ce8a None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:57:29.589390 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.fernet_utils [None req-3e55b95c-e873-435f-a970-508b3534ce8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:57:29.597212 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 669/1348] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:57:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:57:29.636116 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11999a6e-a5ed-4ba9-9a64-336275a0aad6 None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:57:29.637831 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-11999a6e-a5ed-4ba9-9a64-336275a0aad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:57:29.656345 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11999a6e-a5ed-4ba9-9a64-336275a0aad6 None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:57:29.664262 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc830599-6033-4dd1-8d65-478f7cf02e37 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8de553016ea641cdbf9c4a8e08c96a67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc99d742ae004954b6d841293b5fa79c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:57:29.664913 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc830599-6033-4dd1-8d65-478f7cf02e37 service neutron] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:57:29.665201 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc830599-6033-4dd1-8d65-478f7cf02e37 service neutron] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:57:29.665481 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc830599-6033-4dd1-8d65-478f7cf02e37 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:57:29.675122 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc830599-6033-4dd1-8d65-478f7cf02e37 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:57:29.677332 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc830599-6033-4dd1-8d65-478f7cf02e37 service neutron] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:57:29.687504 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 680/1349] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 14:57:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:57:35.136324 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72b779f-a568-4e0e-bfb8-a94c1a3da7ea None None] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:57:35.136807 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72b779f-a568-4e0e-bfb8-a94c1a3da7ea None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:57:35.136901 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72b779f-a568-4e0e-bfb8-a94c1a3da7ea None None] PATH_INFO: `/` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:57:35.138586 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 670/1350] 10.222.0.57 () {54 vars in 921 bytes} [Thu May 19 14:57:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:57:35.143937 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dfc407-81e7-4e7c-91e7-e7408a92e24d None None] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:57:35.144404 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dfc407-81e7-4e7c-91e7-e7408a92e24d None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:57:35.144508 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dfc407-81e7-4e7c-91e7-e7408a92e24d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:57:35.297153 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.auth.core [None req-e9dfc407-81e7-4e7c-91e7-e7408a92e24d None None] MFA Rules not processed for user `e80cdca4cf964f308a7618f89cce7b1a`. Rule list: `[]` (Enabled: `True`). {{(pid=281785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:57:35.298996 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-e9dfc407-81e7-4e7c-91e7-e7408a92e24d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:57:35.412859 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 681/1351] 10.222.0.57 () {58 vars in 1018 bytes} [Thu May 19 14:57:35 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:57:39.582852 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=13 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:57:40.084705 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a7ef6d-9cbb-48e8-be37-f9d23e491bce None None] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:57:40.084994 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a7ef6d-9cbb-48e8-be37-f9d23e491bce None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:57:40.085518 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a7ef6d-9cbb-48e8-be37-f9d23e491bce None None] PATH_INFO: `/` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:57:40.086950 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 671/1352] 10.222.0.57 () {54 vars in 921 bytes} [Thu May 19 14:57:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:57:40.092527 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61cbded-35ad-4a76-8ad7-fe698423e7db None None] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:57:40.093059 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61cbded-35ad-4a76-8ad7-fe698423e7db None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:57:40.093178 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61cbded-35ad-4a76-8ad7-fe698423e7db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:57:40.115762 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.auth.core [None req-f61cbded-35ad-4a76-8ad7-fe698423e7db None None] MFA Rules not processed for user `e80cdca4cf964f308a7618f89cce7b1a`. Rule list: `[]` (Enabled: `True`). {{(pid=281785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:57:40.117022 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-f61cbded-35ad-4a76-8ad7-fe698423e7db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:57:40.125918 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 682/1353] 10.222.0.57 () {58 vars in 1018 bytes} [Thu May 19 14:57:40 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:57:41.693192 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838563a4-e772-4e76-9343-152672970d4e None None] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:57:41.693778 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838563a4-e772-4e76-9343-152672970d4e None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:57:41.693778 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838563a4-e772-4e76-9343-152672970d4e None None] PATH_INFO: `/` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:57:41.695190 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 672/1354] 10.222.0.57 () {54 vars in 921 bytes} [Thu May 19 14:57:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:57:41.700317 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e7a441-be49-4d93-a033-3d64b05a42af None None] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:57:41.700754 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e7a441-be49-4d93-a033-3d64b05a42af None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:57:41.700877 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e7a441-be49-4d93-a033-3d64b05a42af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:57:41.719461 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.auth.core [None req-77e7a441-be49-4d93-a033-3d64b05a42af None None] MFA Rules not processed for user `e80cdca4cf964f308a7618f89cce7b1a`. Rule list: `[]` (Enabled: `True`). {{(pid=281785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:57:41.720681 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-77e7a441-be49-4d93-a033-3d64b05a42af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:57:41.729810 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 683/1355] 10.222.0.57 () {58 vars in 1018 bytes} [Thu May 19 14:57:41 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:57:45.801669 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905e2a67-a5c4-48eb-ab9c-4fe4dca5831f None None] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:57:45.802256 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905e2a67-a5c4-48eb-ab9c-4fe4dca5831f None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:57:45.802256 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905e2a67-a5c4-48eb-ab9c-4fe4dca5831f None None] PATH_INFO: `/` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:57:45.803539 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 673/1356] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:57:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:57:45.809367 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d381c228-4be4-4550-a057-81912c49c478 None None] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:57:45.810300 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d381c228-4be4-4550-a057-81912c49c478 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:57:45.810599 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d381c228-4be4-4550-a057-81912c49c478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:57:45.850885 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.auth.core [None req-d381c228-4be4-4550-a057-81912c49c478 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:57:45.852146 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-d381c228-4be4-4550-a057-81912c49c478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:57:45.861668 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 684/1357] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:57:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:57:45.916414 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946d5b36-31e8-46bd-9a77-7872be8e9cc8 None None] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:57:45.917148 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946d5b36-31e8-46bd-9a77-7872be8e9cc8 None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:57:45.917714 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946d5b36-31e8-46bd-9a77-7872be8e9cc8 None None] PATH_INFO: `/` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:57:45.919581 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 674/1358] 10.222.0.57 () {54 vars in 961 bytes} [Thu May 19 14:57:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:57:45.929110 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49c73ae-28d6-4452-9916-f5b89d100071 None None] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:57:45.930124 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49c73ae-28d6-4452-9916-f5b89d100071 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:57:45.930637 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49c73ae-28d6-4452-9916-f5b89d100071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:57:45.979389 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=100 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:57:45.988482 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.auth.core [None req-c49c73ae-28d6-4452-9916-f5b89d100071 None None] MFA Rules not processed for user `5d2aa8c8870a49d98f23f26e036695ef`. Rule list: `[]` (Enabled: `True`). {{(pid=281785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:57:45.989772 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-c49c73ae-28d6-4452-9916-f5b89d100071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:57:46.077352 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 685/1359] 10.222.0.57 () {58 vars in 1058 bytes} [Thu May 19 14:57:45 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:57:46.086184 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946d5b36-31e8-46bd-9a77-7872be8e9cc8 None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:57:46.102407 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946d5b36-31e8-46bd-9a77-7872be8e9cc8 None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:57:46.108612 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7df6ec-b0a3-45ee-83cd-ac7f514a13b1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d2aa8c8870a49d98f23f26e036695ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc99d742ae004954b6d841293b5fa79c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:57:46.109671 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7df6ec-b0a3-45ee-83cd-ac7f514a13b1 service nova] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:57:46.110162 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7df6ec-b0a3-45ee-83cd-ac7f514a13b1 service nova] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:57:46.110603 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7df6ec-b0a3-45ee-83cd-ac7f514a13b1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:57:46.121317 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7df6ec-b0a3-45ee-83cd-ac7f514a13b1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:57:46.123263 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7df6ec-b0a3-45ee-83cd-ac7f514a13b1 service nova] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:57:46.134970 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 675/1360] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 14:57:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:57:48.610469 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7db42cf-a2f8-4ba8-abfd-91defcc0e200 None None] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:57:48.611058 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7db42cf-a2f8-4ba8-abfd-91defcc0e200 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:57:48.611058 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7db42cf-a2f8-4ba8-abfd-91defcc0e200 None None] PATH_INFO: `/` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:57:48.612594 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 686/1361] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:57:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:57:48.618985 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f30f581-3201-4e46-ad92-b0f1b5a22bdf None None] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:57:48.619459 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f30f581-3201-4e46-ad92-b0f1b5a22bdf None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:57:48.619459 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f30f581-3201-4e46-ad92-b0f1b5a22bdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:57:48.638384 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.auth.core [None req-4f30f581-3201-4e46-ad92-b0f1b5a22bdf None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:57:48.639602 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.fernet_utils [None req-4f30f581-3201-4e46-ad92-b0f1b5a22bdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:57:48.647872 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 676/1362] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:57:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:57:48.693552 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7db42cf-a2f8-4ba8-abfd-91defcc0e200 None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:57:48.708692 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7db42cf-a2f8-4ba8-abfd-91defcc0e200 None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:57:48.714447 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631f2381-9070-4679-9969-4e8ceeea856c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d2aa8c8870a49d98f23f26e036695ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc99d742ae004954b6d841293b5fa79c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:57:48.715097 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631f2381-9070-4679-9969-4e8ceeea856c service nova] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:57:48.715394 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631f2381-9070-4679-9969-4e8ceeea856c service nova] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:57:48.715685 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631f2381-9070-4679-9969-4e8ceeea856c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:57:48.725056 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631f2381-9070-4679-9969-4e8ceeea856c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:57:48.726737 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631f2381-9070-4679-9969-4e8ceeea856c service nova] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:57:48.736723 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 687/1363] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 14:57:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:57:51.378240 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909cd2f8-1514-4534-bb47-d067835a9699 None None] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:57:51.378776 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909cd2f8-1514-4534-bb47-d067835a9699 None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:57:51.379141 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909cd2f8-1514-4534-bb47-d067835a9699 None None] PATH_INFO: `/` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:57:51.381135 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 677/1364] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:57:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:57:51.386482 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830702b1-e794-4142-856c-1f5ccd23dcc8 None None] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:57:51.386974 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830702b1-e794-4142-856c-1f5ccd23dcc8 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:57:51.387062 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830702b1-e794-4142-856c-1f5ccd23dcc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:57:51.407544 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.auth.core [None req-830702b1-e794-4142-856c-1f5ccd23dcc8 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:57:51.408874 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-830702b1-e794-4142-856c-1f5ccd23dcc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:57:51.418102 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 688/1365] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:57:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:57:51.465522 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42519607-8d08-436c-a910-e2b27065ba8c None None] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:57:51.465812 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42519607-8d08-436c-a910-e2b27065ba8c None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:57:51.466349 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42519607-8d08-436c-a910-e2b27065ba8c None None] PATH_INFO: `/` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:57:51.467732 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 678/1366] 10.222.0.57 () {54 vars in 961 bytes} [Thu May 19 14:57:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:57:51.474035 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b165166a-aa51-4c5e-8ce9-d26614a5c466 None None] REQUEST_METHOD: `POST` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:57:51.474440 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b165166a-aa51-4c5e-8ce9-d26614a5c466 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:57:51.474795 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b165166a-aa51-4c5e-8ce9-d26614a5c466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:57:51.495208 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.auth.core [None req-b165166a-aa51-4c5e-8ce9-d26614a5c466 None None] MFA Rules not processed for user `5d2aa8c8870a49d98f23f26e036695ef`. Rule list: `[]` (Enabled: `True`). {{(pid=281785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:57:51.496531 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.fernet_utils [None req-b165166a-aa51-4c5e-8ce9-d26614a5c466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:57:51.507128 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 689/1367] 10.222.0.57 () {58 vars in 1058 bytes} [Thu May 19 14:57:51 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:57:51.514911 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42519607-8d08-436c-a910-e2b27065ba8c None None] Authenticating user token {{(pid=281786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:57:51.531088 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42519607-8d08-436c-a910-e2b27065ba8c None None] Validating token access rules against request {{(pid=281786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:57:51.537429 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb460d88-3ba3-482f-99ca-406e50bb2f2b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d2aa8c8870a49d98f23f26e036695ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc99d742ae004954b6d841293b5fa79c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:57:51.538206 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb460d88-3ba3-482f-99ca-406e50bb2f2b service nova] REQUEST_METHOD: `GET` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:57:51.538563 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb460d88-3ba3-482f-99ca-406e50bb2f2b service nova] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:57:51.538916 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb460d88-3ba3-482f-99ca-406e50bb2f2b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:57:51.547739 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb460d88-3ba3-482f-99ca-406e50bb2f2b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:57:51.549415 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb460d88-3ba3-482f-99ca-406e50bb2f2b service nova] RBAC: Authorization granted {{(pid=281786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:57:51.560572 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 679/1368] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 14:57:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:57:52.439602 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9af48d3-598d-4baa-9eec-f22e46f90906 None None] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:57:52.440150 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9af48d3-598d-4baa-9eec-f22e46f90906 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:57:52.440150 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9af48d3-598d-4baa-9eec-f22e46f90906 None None] PATH_INFO: `/` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:57:52.442574 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 690/1369] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:57:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:57:52.448434 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ebfefa-55d4-4312-906d-0a7100522bb4 None None] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:57:52.448996 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ebfefa-55d4-4312-906d-0a7100522bb4 None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:57:52.448996 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ebfefa-55d4-4312-906d-0a7100522bb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:57:52.468528 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.auth.core [None req-06ebfefa-55d4-4312-906d-0a7100522bb4 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:57:52.469756 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.fernet_utils [None req-06ebfefa-55d4-4312-906d-0a7100522bb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:57:52.478724 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 680/1370] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:57:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:57:53.927989 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7cd811-461c-4950-aabe-e69d9ddecca5 None None] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:57:53.928517 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7cd811-461c-4950-aabe-e69d9ddecca5 None None] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:57:53.928517 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7cd811-461c-4950-aabe-e69d9ddecca5 None None] PATH_INFO: `/` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:57:53.929904 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 691/1371] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 14:57:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:57:53.935317 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ab8cb2-2051-4934-90b8-69530e633f55 None None] REQUEST_METHOD: `POST` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:57:53.935799 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ab8cb2-2051-4934-90b8-69530e633f55 None None] SCRIPT_NAME: `/identity` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:57:53.935908 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ab8cb2-2051-4934-90b8-69530e633f55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=281786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:57:53.954619 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.auth.core [None req-62ab8cb2-2051-4934-90b8-69530e633f55 None None] MFA Rules not processed for user `8c487f3640744d568515b03847007a04`. Rule list: `[]` (Enabled: `True`). {{(pid=281786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:57:53.955834 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG keystone.common.fernet_utils [None req-62ab8cb2-2051-4934-90b8-69530e633f55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=281786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:57:53.964551 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: [pid: 281786|app: 0|req: 681/1372] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 14:57:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:57:54.009074 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7cd811-461c-4950-aabe-e69d9ddecca5 None None] Authenticating user token {{(pid=281785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:57:54.024455 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7cd811-461c-4950-aabe-e69d9ddecca5 None None] Validating token access rules against request {{(pid=281785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:57:54.030051 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ad8f19-07a8-4f90-8bfb-681799546984 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d2aa8c8870a49d98f23f26e036695ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc99d742ae004954b6d841293b5fa79c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=281785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:57:54.030698 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ad8f19-07a8-4f90-8bfb-681799546984 service nova] REQUEST_METHOD: `GET` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:57:54.030997 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ad8f19-07a8-4f90-8bfb-681799546984 service nova] SCRIPT_NAME: `/identity` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:57:54.031283 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ad8f19-07a8-4f90-8bfb-681799546984 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=281785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:57:54.039903 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ad8f19-07a8-4f90-8bfb-681799546984 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:57:54.041510 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ad8f19-07a8-4f90-8bfb-681799546984 service nova] RBAC: Authorization granted {{(pid=281785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:57:54.051052 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: [pid: 281785|app: 0|req: 692/1373] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 14:57:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:58:03.949349 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: DEBUG dbcounter [-] [281786] Writing DB stats keystone:SELECT=21 {{(pid=281786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:58:04.046802 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: DEBUG dbcounter [-] [281785] Writing DB stats keystone:SELECT=54 {{(pid=281785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:58:19.899931 dv-cinder-842541-1-iscsi devstack@keystone.service[281774]: Thu May 19 14:58:19 2022 - graceful shutdown triggered... May 19 14:58:19.900640 dv-cinder-842541-1-iscsi devstack@keystone.service[281786]: Gracefully killing worker 2 (pid: 281786)... May 19 14:58:19.900067 dv-cinder-842541-1-iscsi systemd[1]: Stopping Devstack devstack@keystone.service... May 19 14:58:19.901333 dv-cinder-842541-1-iscsi devstack@keystone.service[281785]: Gracefully killing worker 1 (pid: 281785)... May 19 14:58:20.901061 dv-cinder-842541-1-iscsi devstack@keystone.service[281774]: worker 1 buried after 1 seconds May 19 14:58:20.901061 dv-cinder-842541-1-iscsi devstack@keystone.service[281774]: worker 2 buried after 1 seconds May 19 14:58:20.901661 dv-cinder-842541-1-iscsi devstack@keystone.service[281774]: goodbye to uWSGI. May 19 14:58:20.902674 dv-cinder-842541-1-iscsi systemd[1]: devstack@keystone.service: Succeeded. May 19 14:58:20.903232 dv-cinder-842541-1-iscsi systemd[1]: Stopped Devstack devstack@keystone.service.